Editorial Evidence Based Library and Information Practice 2009, 4:2 120 Evidence Based Library and Information Practice Article Understanding the Information Research Process of Experienced Online Information Researchers to Inform Development of a Scholars Portal Martha Whitehead Associate University Librarian Queen's University Library Kingston, Ontario, Canada Email: martha.whitehead@queensu.ca Terry Costantino Principal and Information Architect Usability Matters Toronto, Ontario, Canada Email: terry@usabilitymatters.com Received: 04 March 2009 Accepted: 12 May 2009 © 2009 Whitehead and Costantino. This is an Open Access article distributed under the terms of the Creative Commons Attribution License, , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Objective - The main purpose of this study was to understand the information research process of experienced online information researchers in a variety of disciplines, gather their ideas for improvement and as part of this to validate a proposed research framework for use in future development of Ontario’s Scholars Portal. Methods - This was a qualitative research study in which sixty experienced online information researchers participated in face-to-face workshops that included a collaborative design component. The sessions were conducted and recorded by usability specialists who subsequently analyzed the data and identified patterns and themes. Results - Key themes included the similarities of the information research process across all disciplines, the impact of interdisciplinarity, the social aspect of research and opportunities for process improvement. There were many specific mailto:martha.whitehead@queensu.ca mailto:terry@usabilitymatters.com Evidence Based Library and Information Practice 2009, 4:2 121 observations regarding current and ideal processes. Implications for portal development and further research included: supporting a common process while accommodating user-defined differences; supporting citation chaining practices with new opportunities for data linkage and granularity; enhancing keyword searching with various types of intervention; exploring trusted social networks; exploring new mental models for data manipulation while retaining traditional objects; improving citation and document management. Conclusion – The majority of researchers in the study had almost no routine in their information research processes, had developed few techniques to assist themselves and had very little awareness of the tools available to help them. There are many opportunities to aid researchers in the research process that can be explored when developing scholarly research portals. That development will be well guided by the framework ‘discover, gather, synthesize, create, share.’ Introduction and Context for the Study The Ontario Council of University Libraries (OCUL), a twenty-one member consortium in Canada, is at an interesting stage in the development of its Scholars Portal. The vision for Scholars Portal is a sophisticated electronic environment that enables easy access to high quality scholarly resources and long term archiving of those resources. One of the benefits is a cost-effective infrastructure for centrally managing systems that libraries routinely purchase or develop to support the use of scholarly information resources. Additionally, it is something much more interesting: a vast collection of diverse resources completely under the control and stewardship of the consortium. Scholars Portal is perfectly positioned for the development of innovative, integrated services to support scholarly information research. Since its inception in 2002, Scholars Portal has had vendor permissions to locally store, permanently, the vast majority of e- journals purchased by OCUL members. These were initially stored on a local installation of Science Server and then moved to a local Mark Logic platform as XML-encoded files. With 14 million articles in 8400 journals, this is one of the largest e- journal archives in existence and is currently undergoing accreditation as a trusted digital repository. Since 2005, Scholars Portal has been using a local installation of CSA’s Illumina product to aggregate many of the abstracting and indexing databases licensed by OCUL members as well as the metadata of the e-journal archive. This provides the ability to perform a single search across multiple sources. In 2007, work began on two other significant projects: the development of an e-book platform to host content purchased by or digitized by member libraries and the development of ODESI (Ontario Data Documentation, Extraction Service and Infrastructure Initiative), a platform to provide access to data sets. The potential provided by local control of these various scholarly resources is exciting. The powerful search opportunities afforded by the full-text content and the ease of linkages between citations and sources can easily be imagined. It is interesting to consider the potential of XML-encoded digital objects beyond their traditional narrative linear form, and which familiar mental models may need to be retained as new innovative features are introduced. The development of Scholars Portal requires an understanding of the features of the ideal online research environment, from the scholars’ perspective. Evidence Based Library and Information Practice 2009, 4:2 122 In 2008, the Scholars Portal team initiated a user study to inform development of this online research environment. Working with external consultants in Toronto Canada (Usability Matters), the team formulated questions regarding users and their contexts, user research tasks, current and potential features of user interfaces and other relevant technologies and services. A research assistant was hired to explore the research literature and compile relevant observations1.. The largest gap in understanding related to the information research processes of experienced online information researchers. The team therefore decided that this would be an appropriate focus of its own user study. One of the most relevant studies the team considered in its literature review was conducted by the University of Minnesota, with support from the Andrew W. Mellon Foundation. The goal of this study was to develop a model for bringing greater coherence to wide ranging aspects of support for scholarship in the humanities and social sciences. In A Multi-Dimensional Framework for Academic Support: A Final Report, the authors presented a way to structure the analysis of the data they gathered regarding faculty and graduate students’ research needs and to frame possible future directions. They proposed the categories of ‘discover,’ ‘gather,’ ‘create’ and ‘share’ as phases in the research process, noting that these are not discrete or linear, but rather iterative and overlapping in multi-dimensional ways (University of Minnesota 38). This concept is rooted in the notion of ‘scholarly primitives’ presented in 2000 by John Unsworth, an internationally renowned scholar and leader in the field of digital humanities. Considering the work of humanities scholars, and comparing this to the work of other disciplines, Unsworth spoke of scholarly primitives as basic functions common to all scholarly activity and proposed several as a starting point for considering the tasks to be supported by our digital tools: discovering, annotating, comparing, referring, sampling, illustrating, representing (Unsworth 1). Each of Unsworth’s primitives can be considered to be elements of the categories proposed by the University of Minnesota. Although the project team did not set out to seek a framework to inform development, ‘discover, gather, create, share’ seemed to have the potential to guide Scholars Portal’s development of services. Because the framework resonated strongly with the team, it was decided to explore the value of this particular framework in the study rather than seek alternative approaches. The team recognized that the questions raised prior to the research review, and much of the selected research, was centred around the ‘discover’ phase and more needed to be learned about the other phases. Other projects have also seen the value in this framework (for example, it is cited on the Project Bamboo Planning Wiki) and it is hoped that the Scholars Portal study provides useful insights for those considering its application. Study Objectives and Methodology The main purpose of this study was to understand the current information research processes of experienced academic researchers in a variety of disciplines and to gather their ideas for improvement. As part of this, the team wanted to explore whether the framework ‘discover, gather, create, share’ (University of Minnesota 38) resonated with the participants. They also wanted to determine what tools and techniques the participants were currently using to aid the information research process. Through this study, the team sought to improve and enhance the Scholars Portal suite of tools in the near-term but, more significantly, to gain insights that Evidence Based Library and Information Practice 2009, 4:2 123 would inform the future vision of Scholars Portal. Based on the objectives, it was decided that a series of face-to-face workshops would provide the most valuable interaction with and amongst the participants. The first half of the workshop was considered as preparation and followed a traditional format with a facilitator asking questions, to which individuals responded. This format was used to discuss the current information research processes of the participants, to validate the University of Minnesota framework and to review a search interface prototype. In the second half of the workshop participants worked together in small groups to envision an idealized information research process. Because the design component was key, the workshops were referred to as ‘collaborative design sessions’. The term ‘collaborative design’ can be applied to any situation in which two or more people work together to design anything. Collaborative design is frequently used in the field of Human-Computer Interaction (HCI), specifically in its sub- domains of Computer Supported Cooperative Work (CSCW) and Participatory Design (PD), to refer specifically to collaborations between ‘designers’ and ‘end-users’ of computer systems and quite often these collaborations extend over time (Kyng 66). In this particular case, participants were asked to collaborate with one another on a single occasion to envision an idealized information research process. Study Participants To recruit study participants, members of the OCUL Public Services Advisory Group sent an email to Faculty and Graduate students at three Toronto-area universities. In the email, potential participants were asked to self-identify with one of three broad discipline areas: Arts and Humanities, Social Sciences and Sciences (Natural, Applied, Health, etc.), as well as one of four experience levels for conducting online information research: None, Novice, Intermediate and Advanced. As the study required participants with experience in online information research, those indicating ‘None’ were excluded from the study. Potential participants were asked to identify their role at the university (Faculty, Grad Student, Post Doc, Research Librarian, Other Researcher) as well as their age group, to ensure a reasonable mix of participants. A mixture of participants was chosen to meet a range of criteria with eight to ten participants in each of the six sessions. Most of the participants were graduate students; however 9 of 60 participants were Faculty members. There was a good mix of age and gender within each session. At the end of the session, participants received a small cash incentive for their participation in one 90 minute session. Study Procedures The six collaborative design sessions were held at the University of Toronto in May 2008. Two sessions were conducted in each of the three broad discipline areas (Arts and Humanities, Social Sciences and Science). As they arrived, participants were asked to complete a ‘warm-up’ questionnaire outlining the steps involved in their research process and three things that would make that process easier. Participants were encouraged to use this questionnaire as reference during the first part of the session in which the whole group outlined the steps involved in a ‘typical’ research process. The questionnaires were collected at the end of the session1. During the next part of the session, the Minnesota framework was introduced and Evidence Based Library and Information Practice 2009, 4:2 124 participants were asked, in general, if the steps the group had identified in their research process could be loosely organized by this framework and, if so, which steps would fit into each of the stages (i.e. discover, gather, create and share). A few moments of the workshop were spent reviewing the draft search and search results interfaces to aid development of a new Scholars Portal search interface. In the second half of each session participants envisioned an ‘ideal’ information research process, concentrating on the ‘gather’, ‘create’ and ‘share’ parts of the process. Participants were assigned to three small groups of three or four people each and given flip-chart paper and other materials with which to ‘storyboard’ their ideal research process. Each of the small groups presented their outcomes to the large group, answering clarification questions only. Finally, the entire group discussed similarities and differences in their approaches, elements that surprised them, and other reflections of interest. Overall, the sessions ran smoothly and provided the project team with a clearer understanding of the current research process of experienced online information researchers together with the strengths and weaknesses of the Minnesota framework. However, the collaborative design sessions did not yield the anticipated types of ‘storyboards’. Despite encouragement to the contrary, most of the groups focused on ‘discover’ and left little time for ‘gather’, ‘create’ and ‘share’. Very few groups mapped an ‘ideal’ process, continuing to focus on their current process. Nonetheless, there were some interesting insights gained from this design exercise and especially from the sessions as a whole. In hindsight, trying to cover the current and ideal research processes in one session was perhaps overambitious, since participants struggled to shift their focus between the two. The rationale had been that discussing the current approach would enable participants to envision improvements. A future approach could include a facilitator/designer in each small group to help the group focus on the specific task, to elicit more ideas and to create more useful ‘storyboards’. Data Collection and Analysis During each session, notes were taken on a laptop and the session was video-recorded. In addition, the pre-questionnaires and flip- chart notes were transcribed, as were the ‘storyboards’ produced by the small groups in each session. The session notes were the main basis for the data analysis, along with discussions between the facilitator and note- taker (both of whom are experienced usability specialists). Like most qualitative research, the analysis involved combing the data looking for patterns and themes (Creswell 203). Working together the consultants (i.e. the facilitator and note-taker) looked for similarities, differences and patterns between individuals, between groups and between the discipline areas. In the report, findings were organized task-by-task, in the order that these tasks were undertaken during the collaborative design sessions. The findings within each task were organized by themes that emerged from the data, for the earlier tasks, and organized by the Minnesota framework for the tasks that followed its introduction into the workshop. At the end of the report on each task, the consultants provided analysis and recommendations based on the findings and their expertise in interpreting the findings for the specific context of this organization (OCUL) and initiative (the Scholars Portal suite of tools). Evidence Based Library and Information Practice 2009, 4:2 125 The raw data (notes, videos, etc.) was provided to OCUL and further examination of the data was encouraged to better understand the findings, to reveal additional insights but, perhaps most importantly, to inspire design ideas from the Scholars Portal and OCUL team members. Results Validating the framework The ‘discover, gather, create, share’ framework was proposed by the University of Minnesota in the context of a broad range of research-related activities and services, whereas in this study the focus was somewhat narrower, examining ‘the information research process.’ This focus was made clear to the study participants in the opening warm-up task in which they were asked to ‚list the steps you take for doing information research for academic purposes, from when you recognize that you need information to when you use that information in one or more ways.‛ The framework resonated well with the participants in this context. All groups agreed that it provides a useful, high-level picture of the information research process. As noted in the Minnesota report, however, participants emphasized that the process is non-linear, that steps rarely happen in a specific order and that they are often repeated with differing levels of specificity at different stages of the process. In some cases, the terms themselves were problematic. For example, many participants felt that ‘discover’ wasn’t quite the right word in relation to the information research process because it relates to the result they are trying to achieve through their primary research. Most groups also believed that there was a step missing between ‘gather’ and ‘create,’ related to engaging with the materials and organizing one’s thoughts. Within the definitions used by the University of Minnesota, this falls within the category of ‘create,’ but most participants were insistent that ‘synthesize’ is distinct and so it is introduced below. Overarching observations Several themes emerged relating to all phases of the framework: Similarities across disciplines - the processes described in each discipline group were remarkably similar. Different sources were mentioned in the different groups, and in the Science sessions there was emphasis on preparation for and validation of bench research, but this did not reveal fundamental differences in the information research process. Interdisciplinarity - participants talked about the challenge of interdisciplinarity and the need to easily search across disciplines, but also the need for tools that can help them be selective about the disciplines included in a search and/or the ability to narrow the results to their areas of interest. The social aspect of research - several of the groups talked about ‚interaction,‛ ‚collaboration‛ or ‚conversation‛ as part of the framework. Ultimately they decided that these are not discrete steps in the process but, rather, are overarching throughout all phases. They were adamant, however, about the importance of this aspect of their information research process. Room for improvement - most participants believed that they should have a better process and Evidence Based Library and Information Practice 2009, 4:2 126 would like to improve their approach, but taking the time to learn was not a high priority. In answer to the question of ‚what would make your information research process easier,‛ one participant gave a telling response: ‚user friendly search engine; actually attending some of the different seminars on web research.‛ Discover The Process Participants associated many activities with ‘discover’: talk with colleagues, keep up with the field, attend conferences, observe, read, develop questions, consider one’s own personal knowledge and beliefs on a chosen subject, follow known sources, rediscover things you’ve found previously, search for literature. This phase generated the most discussion in all groups and several themes emerged: Web search engines and common internet tools appeared in most researchers’ steps, but these same people relied on research databases provided by libraries. One researcher’s succinct summary is representative: ‚Google, Wikipedia, JSTOR, Scholars Portal, LexisNexis.‛ Keywords, colleagues and the citation network are all important approaches. Most participants said that they start their search broadly and then narrow it, but there is no set routine. The process could involve, at various points: getting ideas for keywords from colleagues or overview sources, searching keywords, discovering known experts and searching for their publications, finding a literature review and following the references. The ultimate goal of the search effort is a resource that can be downloaded, ideally a PDF document. When shown a prototype interface that provided tabs for tables and figures, a few participants were intrigued by the thought that they could easily access these data elements, but they were puzzled by how these elements would be separated from their original context, which was assumed to be an article or book. Keeping up in one’s field is accomplished through a combination of methods, such as getting ideas and resources from listservs, RSS feeds and email alerts from key journals or news services. Often these sources were discovered by chance and the researchers had no memory of how they signed up for them. No one mentioned receiving alerts from their University library. The Ideal Participants easily articulated elements of their ideal ‘discover’ process, many of which relate to features common in current systems. The following were given the most emphasis: More electronic resources were mentioned in all of the groups. A ‘one-stop shop’ or single interface to search for all relevant material. Some participants expressed a need for narrower search engines, including a narrower Scholars Portal, and in talking about desired features they often identified ‚relevant results‛ and ‚ability to narrow results.‛ Assistance generating keywords, Evidence Based Library and Information Practice 2009, 4:2 127 synonyms or other related terms, to ensure the search is complete. An interesting discussion arose in one session about including one or more definitions of the search term at the top of the results. This would help researchers less familiar with the subject matter get a quick overview without having to click through and would also help all researchers focus their search by choosing the definition that applies to their current search. Several groups wanted the ability to find out the history of a topic, for example through a visual representation like a mind map. Expert advice. Participants wanted to see recommendations from ‚authorities‛ and they wanted to be able to identify ‚classics in the field.‛ It was interesting to note, however, that in the prototype design, use of the term ‚top journals‛ was very contentious because it was not clear to participants how ‚top‛ was derived. More intelligent refinement of results. Participants wanted the ability to say ‚don’t show me this item again‛ in subsequent searches together with an indication in subsequent searches of items already marked or downloaded. They also wanted the ability to start a completely new search within search results, for example a search box entitled ‚search within.‛ Easier citation. Enabling easy cut and paste of citations was more important to most participants than exporting to citation management software. Some participants wanted to display the full citation in each result, with the ability to choose citation style (APA, etc). Gather The Process Activities associated with ‘gather’ included: obtaining materials (downloading, printing, photocopying); weeding and sorting based on a brief review of table of contents, abstract or conclusions (often into ‘yes’, ‘maybe’, ‘no’ categories); filing materials; creating a bibliography; reading and annotating lightly. The process of borrowing from the library was mentioned only in terms of frustrations with missing materials. In terms of filing materials, most participants download PDFs and store them on their hard-drive in a self-styled folder/sub-folder system generally based on topics or author name. In addition to, or instead of PDFs, many participants print the papers they intend to use and physically file them, usually by topic. Very few participants are consistently using any bibliographic management tools. Many seem to use the bibliographies they produce for individual academic papers as their main organizing method, returning to these bibliographies when working on subsequent papers. Most participants create some sort of annotated bibliography/citation list, most often in MS Word, and one person mentioned creating a handwritten list. About half the participants were aware of the bibliographic management tool provided by Scholars Portal, however most of these were either not using it at all or not consistently. Of those who had tried it, many said they had abandoned it quite quickly, not willing to make the effort to learn how to use it effectively. A few said they were using other bibliographic management software. When asked if they had created any systems to manage their citations, few seemed to feel they have a system and only one person mentioned creating a database. Evidence Based Library and Information Practice 2009, 4:2 128 The Ideal Participants had several comments about the ideal ‘gather’ process: More electronic resources was a constant refrain. Easy, successive annotation. Annotations develop over time, being lighter at the beginning and more detailed in later stages on specific papers of interest, and this evolution should be retained and evident. It should be possible to annotate PDFs with the equivalent of post-it notes. A few participants said they annotate in PDF and then use the search function later to find their specific, pertinent notes. The ability to annotate directly in PDF was a surprise to many in those sessions and some were very intrigued by the possibilities. Ability to display, extract, and easily compare relevant sections of each paper, for example conclusions and methods. Synthesize The Process Activities noted in this phase related to organizing thoughts, a process that participants saw as distinct from ‘gather’ and ‘create.’ They included weeding further, validating the quality of sources, organizing and coding sources thematically, annotating further by hand or directly in the PDF, reading for detail, taking notes and extracting quotes. The latter might entail cutting and pasting text from sources into an email, a document, Excel, a table or index cards. Participants talked about the intellectual acts of summarizing, looking for patterns, mind-mapping (e.g. with Mindmeister ), digesting, fitting data to one’s purpose, formulating the research question, formulating a thesis sentence, determining the theoretical framework, outlining the paper and creating a bibliography. The Ideal Although this was a process that participants emphasized as important, few expressed any particular ideas for how it could be improved beyond those covered in ‘gather’ above. Create The Process In some sessions, the term ‘create’ was closely associated with the participants’ original research and less so with research output such as scholarly papers. However, all participants did easily identify activities related to ‘create’: sweat, clarify audience, outline, write findings and ideas, edit, refine, consider reviews and feedback, revise, discuss, collaborate. Almost all participants use MSWord but other tools are used as well, such as LaTex. Writing in groups was mentioned only briefly by faculty members who are working with their research assistants. In this context, a few concerns about tracking changes and version control were raised. There was little discussion of issues regarding illustrating papers and presentations in the sessions other than the Sciences. The Ideal Participants provided a few ideas for their ideal process: One small group dreamed of a personalized online whiteboard or light table, for organizing materials and, ultimately, the paper. It would include templates, the ability to Evidence Based Library and Information Practice 2009, 4:2 129 export to PowerPoint and the ability to attach references, documents and figures. One participant wanted the opportunity to run papers through ‘Turn it In’ in advance, so that adjustments could be made to the paper before making the final academic submission. One group suggested a timeline tool that would provide a schedule, tell you it’s time to take a break and prevent use of email if set to do so. Share The Process The activities associated with ‘share’ included: share with specific individuals (supervisor, colleagues, experts, authors of the papers you used), publish, submit to online archives (mentioned only in a Science group), teach, give presentations and participate in seminars, conferences, symposia. The Ideal Ideas for the ‘share’ process included: Submission process improvements, such as providing more standardized and more online processes. Tools for facilitating sharing with colleagues, students, advisors. As well as sharing folders and documents, sharing search strategies and results was suggested. Graduate students saw the value in identifying and getting in touch with authors and leading researchers, but said they rarely follow through. One suggestion was a network of researchers to facilitate communication between learners and experts. Help with identifying potential publishing venues and conferences. It was noted in one of the sessions that these opportunities arise and should be collected throughout the information research process, including during the ‘discover’ phase. Alerts regarding who has cited your article and links to those publications, and alerts to new research in your area. Discussion The goal of a portal is to provide unified access to diverse resources and services. For OCUL, Scholars Portal is an entry point for the information research process but its services can also be embedded in the learning and research workflow, in web spaces designed for different groups of users and different purposes at different schools. In developing a portal to support scholarship, the Minnesota framework appears useful for envisioning the information research process as a whole. It is not suggested that the research framework be made visible to end-users: experienced online information researchers may quibble over the words chosen and perceive it as a rigid categorization of a very fluid intellectual process. However, given researchers’ strong interest in finding better ways to manage the information research process, Scholars Portal developers will use the framework to consider how best to promote the availability and interrelation of a set of research tools that goes beyond the traditional portal focus of ‘discover.’ Considering the opportunities presented by Scholars Portal’s data repositories, several aspects of the study results are of particular Evidence Based Library and Information Practice 2009, 4:2 130 interest in informing portal development and future research; these are highlighted below. The overall information research process is similar across disciplines. It appears that Scholars Portal should support this common process while providing the flexibility necessary to accommodate differences in resources and tools by discipline. The single search interface, with both excellent precision and recall, would be the ideal outcome. However, as well as searching across all disciplines and types of resources it should be possible to search a user- defined subset of disciplines or resources. The relative importance of different search methods (i.e. browsing, citation chaining and directed searching) is known to vary by discipline and depend on various factors (Talja 1675). It seems clear from this study, however, that citation chaining is relied upon heavily by at least some researchers, and that the opportunities to provide linkages between the digital objects in Scholars Portal’s repositories will be of great benefit. It would be useful to turn any citation within a digital object into a link to the cited source, whether it is a journal article, a data set or some other type of resource. All disciplines also routinely rely on keyword searches, as confirmed in this study and by Vakkari and Talja, who state that ‚Keyword searching in journal and reference databases were clearly the most important access methods in all disciplines compared to browsing, chaining or obtaining material from colleagues‛ (Vakkari 1). In many cases, however, researchers are not confident about the vocabulary they are using. Participants in this study had some suggestions for features that would help with this, but more exploration is required. In a recent summary of research on end user searching, Markey describes different types of user searching difficulties that could cause systems to intervene with vocabulary assistance, and suggests a half dozen research questions to inform such development (Markey 1126). The potential of user generated content and social search – a search aided by trusted or expert opinion – was not discussed directly in this study, but the participants’ emphasis on the social aspect of research suggests it is relevant. While they stressed their reliance on colleagues and an interest in being able to identify sources recommended by experts, it was not clear what would make them trust others’ evaluations. This is an area that needs to be more clearly understood in the academic context. One of the opportunities provided by Scholars Portal’s repositories is the flexibility provided by the XML-encoded digital objects. It seems likely that researchers will take advantage of linkages provided between pieces of data, given current habits of following paths to find and verify information. The ability to manipulate particular pieces of data and use it for other purposes was not suggested by study participants, but there was nothing to indicate that they would not respond positively to those opportunities once presented. One clear message is that the mental model of the downloadable PDF replicating the traditional print object will need to be supported for some time to come. Participants in this study uniformly felt they could be doing a better job of managing the sources they used in their research. Some aspects of that could be helped with a citation management tool, yet few were interested in using one. This lack of uptake is not just a problem of awareness and the Scholars Portal team plans to investigate this issue in 2009. In general, there appear to be opportunities for improvement in all phases of the information research process, though the ones that engaged participants the most were ‘discover’ and ‘gather.’ Participants appeared very interested in developing Evidence Based Library and Information Practice 2009, 4:2 131 techniques to improve their currently haphazard approaches, and suggested some avenues to explore. In some cases it may be as simple as providing visibility for existing software, such as PDF annotation tools. In all cases, the challenge will be to provide tools with very low barriers and clear advantages over well-established current practices. Conclusion This study was intended to help the Scholars Portal team better understand the information research process of experienced online information researchers, the tools and techniques they currently employ and their vision for an ideal information research process. It validated the idea of approaching development from a framework of ‘discover, gather, synthesize, create, share’ and provided a variety of useful insights. Overall, it was apparent that participants have almost no routine in their processes, have developed few techniques to assist themselves and have very little awareness of the tools available to help them. The collaborative design sessions yielded fewer ideas for the ideal information process than hoped, instead focusing on improvements to current processes. A recommended modification to the methodology would be to include a facilitator or designer in each group to help the group focus and react to ideas and thus create more useful storyboards. The sessions yielded a wealth of observations about the information research process and experienced online information researchers’ needs. Based on this study, the Scholars Portal team has begun designing a new interface for the e- journal repository on the MarkLogic platform and will be conducting iterative usability testing in early 2009. Acknowledgements The authors gratefully acknowledge Stacy Allison-Cassin, Alan Darnell, Kate Davis, Amy Greenberg, Kathy Scardellato, Dana Thomas and Sarah Toy for their contributions to the design and execution of this study, Patricia Lawton for her research assistance and OCUL Directors for their support. Works Cited Creswell, John W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks: Sage, 2003. Kyng, Morten. ‚Designing for Cooperation: Cooperating in Design.‛ Communications of the ACM 34. (1991): 64-73. Markey, Karen. ‚Twenty-Five Years of End- User Searching, Part 2: Future Research Directions.‛ Journal of the American Society for Information Science and Technology 58.8. (2007):1123-30. Project Bamboo. 10 April 2009 . Talja, Sanna, Pertti Vakkari, Jenny Fry and Paul Wouters. ‚Impact of Research Cultures on the Use of Digital Library Resources.‛ Journal of the American Society for Information Science and Technology 58.11. (2007): 1674-85. University of Minnesota Libraries. ‚A Multi- Dimensional Framework for Academic Support: A Final Report Submitted to the Andrew W. Mellon Foundation, June 2006‛.10 April 2009 < http://search0.scholarsportal.info.myaccess.library.utoronto.ca/ids70/view_record.php?id=1&recnum=13&log=from_res&SID=02cad1cd4298a0079e64d7317d131b0e http://search0.scholarsportal.info.myaccess.library.utoronto.ca/ids70/view_record.php?id=1&recnum=13&log=from_res&SID=02cad1cd4298a0079e64d7317d131b0e http://search0.scholarsportal.info.myaccess.library.utoronto.ca/ids70/view_record.php?id=1&recnum=13&log=from_res&SID=02cad1cd4298a0079e64d7317d131b0e https://wiki.projectbamboo.org/display/BPUB/Discover,+Gather,+Create,+Share+-+A+Research+Lifecycle+View https://wiki.projectbamboo.org/display/BPUB/Discover,+Gather,+Create,+Share+-+A+Research+Lifecycle+View https://wiki.projectbamboo.org/display/BPUB/Discover,+Gather,+Create,+Share+-+A+Research+Lifecycle+View https://wiki.projectbamboo.org/display/BPUB/Discover,+Gather,+Create,+Share+-+A+Research+Lifecycle+View https://wiki.projectbamboo.org/display/BPUB/Discover,+Gather,+Create,+Share+-+A+Research+Lifecycle+View Evidence Based Library and Information Practice 2009, 4:2 132 http://www.lib.umn.edu/about/mellon/ UMN_Multi- dimensional_Framework_Final_Report. pdf>. Unsworth, John. ‚Scholarly Primitives: What Methods Do Humanities Researchers Have in Common and How Might Our Tools Reflect This?‛ Humanities Computing, Formal Methods, Experimental Practice Symposium, Kings College, London, 13 May 2000. 10 April 2009 . Vakkari, Pertti and Sanna Talja. "Searching for Electronic Journal Articles to Support Academic Tasks. A Case Study of the Use of the Finnish National Electronic Library (FinELib)." Information Research 12.1 (2006). 10 April 2009 . Notes 1 Documents related to this user study project, including an annotated bibliography compiled by Patricia Lawton and the full report by Usability Matters appended with research instruments and the raw data, are available on the project wiki: http://spotdocs.scholarsportal.info/display/P SWG/User+Study http://www.iath.virginia.edu/~jmu2m/Kings.5-00/primitives.html http://www.iath.virginia.edu/~jmu2m/Kings.5-00/primitives.html http://informationr.net/ir/12-1/paper285.html http://informationr.net/ir/12-1/paper285.html http://spotdocs.scholarsportal.info/display/PSWG/User+Study http://spotdocs.scholarsportal.info/display/PSWG/User+Study