USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK Using Concept Mapping for Needs Analysis for a Social Support System in Learning Network doi:10.3991/ijim.v5i1.1520 Danish Nadeem, Slavi Stoyanov and Rob Koper Center for Learning Sciences and Technologies (CELSTEC), Heerlen, The Netherlands Abstract—This paper describes the needs analysis of tool for connecting people in a Learning Network using Group Con- cept Mapping method. We conducted an empirical study to identify the most relevant features of such a tool. There were 11 experts from 7 different countries with technical and non-technical background who participated in the study. Each of the participants were invited by email and given a brief description about the scenario for the need of a tool in an online Learning Network to find people for help (social support system). This tool can be developed on mo- bile devices to search for peer learners. The participants generated 153 unique ideas about the need for social sup- port system in Learning Network. These ideas were sorted and rated, and further analysis with Concept Mapping pro- duced ten relevant clusters of interest. These clusters (con- cepts) suggest the features which are relevant for a tool that would support learners in connecting with others in a Learning Network. The concept mapping methodology de- scribed in this paper has also been applied in other projects; one of them is about the “investigation of pedagogic areas in which mobile learning is an appropriate solution”. We in- tend to publish the work in the future publication. Index Terms—concept mapping, learning network, social support system, software needs analysis I. INTRODUCTION In general, there are several contexts in which people look for other people, pursuing shared common interests or social conversation [7]. The tendency is also that quite often people tend to seek the tacit knowledge held by in- dividuals instead of explicit knowledge stored in formal documents [2, 9]. Research has claimed that technology could catalyze collaboration and knowledge sharing by enhancing the traceability and visibility of information, which resides in people's mind, especially the knowledge which is not well-documented, like best practices and les- sons learned [13]. Social support system in learning net- works would be an added value for a learner to know about people and decide whom and how to interact with, pursuing particular learning goals [5]. It would give sev- eral benefits to a learner: (1) A learner could connect and make social ties with other learners either for question in hand or for future con- tact in similar context. (2) It is possible that in connecting people to other people, one can share and exchange exper- tise about the information which is not documented like people's experiences, interests and also tacit knowledge. (3) A learner could share with others different perspec- tives on a topic, comparing and constructing meaning, which would lead to effective learning for her/him. (4) A learner would not only depend on experts but s/he can seek help from any suitable person available in learning networks, where people with different levels of expertise exist. II. METHODOLOGY In this empirical study we wanted to gather opinion of experts on the requirements of Social Support System (SoSuSy) in Learning Network. For this purpose we have chosen Concept Mapping [8] as appropriate method be- cause it includes both qualitative techniques and multi- variate analysis approaches [4]. The term is also known as Group Concept Mapping (GCM) in some of the related studies [6, 11]. Concept Mapping provides a structured approach to identify experts’ opinion on a given domain. We used Concept Mapping to collect, structure, analyze and interpret data from the experts of the domain on a particular project. The method identifies the experts’ opin- ion in the form of clusters on a given subject using ad- vanced statistical techniques as referred above. Group Concept Mapping differs from other approaches in the following ways, as reported in [6]: (a) there is only one round of structuring the data as the participants work in- dependently and anonymously to each other; (b) in data structuring, coding the information in different ways is not a problem as it is the advanced analysis included in the methodology that identifies common patterns in the data; (c) consensus is not forced, it emerges from the data; and (d) visualizations, as substantial part of the analysis, allow for grasping at once the emerging structures in the data and help their interpretation. These characteristics provide a good basis to choose this methodology. The Concept Mapping provides a visual map of useful and important concepts which can be used for the develop- ment of a project under hand. Traditionally, Concept Mapping has been used for getting direction in project developments that are related to business management, planning, evaluation and identifying health care policies [10, 8, 12]. In this study we report our first attempt to use Concept Mapping for the purpose of software systems needs analysis. We expect that the conceptual domains derived from the Concept Mapping process will be the guiding principles for the developers of the social support systems for desktop or mobile versions. The inputs taken from diverse people with varying expertise can be com- bined to understand the important features to implement while developing the system. Also, the ideas generated about the important concepts in the concept map will pro- vide a cross-validation for reliable and efficient systems that meet the expert opinion. Concept Mapping consists of: generation of statements by the experts (brainstorming); rating and sorting of iJIM – Volume 5, Issue 1, January 2011 41 http://dx.doi.org/ijim.v5i1.1520� USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK statements; and interpretation of the concept map (data analysis). While all three stages may be done in person, in this study the first two stages were done by mail due to participants' busy schedules and different locations. All concept mapping analyses were accomplished using The Concept System© (Version 1.75, Concept Systems Incor- porated, Ithaca, NY). A. Participants On the basis of research problem we selected a total of 11 different participants from 7 different countries across different continents. Each participant came from a variety of educational backgrounds, namely: computer sciences, business management, human resources, education sci- ences and journalism. The idea was to gather varied expert opinion on the topic of social support system, from a di- verse group of people. An email invite was sent to each of the participants with the brief description of the require- ments of social support systems in Learning Network. They were asked to think-aloud about the need for such a system in Learning Network that searches for people. Fur- thermore, a trigger statement was given to help them in generating the ideas: "What defines a good Social Support System in Learning Network". This trigger statement helped them to generate statements about the features or characteristics, that they thought are necessary to develop such a system in Learning Network. B. Procedure Idea generation requires the participants to individually generate ideas in respond to a focus (trigger) statement. We gave them a scenario description and the following focus statement: “What makes a good social support sys- tem?” We asked the participants to consider a scenario: “Imagine you are interested in learning Piano. You al- ready have some skills in playing piano, like the rules for building major and minor piano scales from any note on the piano keyboard and you also learned how to combine these scales to form major and minor chords. Now you want to improvise, let’s say you are interested in playing Arpeggios, a technique in which the notes of chord are played independently rather than together. As you are learning by yourself, it will be helpful if you knew who else is learning similar things in Piano. You can ask him to give you advice, share his experiences of learning Pi- ano techniques etc, may be you already have a friend or a friend of your friends or somebody totally out of your per- sonal/professional network who is interested in learning Piano. How will you find that person? How will you know who is suitable person for you to contact and share knowledge about Arpeggios technique. Let’s say you are on Facebook, where you have your network of friends, you could ask there. What if the person you are looking for is not on Facebook, but say Orkut, LinkedIn or other social or professional networking sites What you need is a ser- vice to search people in different networks (profes- sional/social) available over the web that recommends the people with their contact details, knowledge and interests so you can choose whom to contact for your pur- pose”. This service (basically a software application for mobile phones or desktop computers) facilitates search for people associated with Arpeggios in the network. The ser- vice looks for others who have interests, knowledge and skills in playing Arpeggios and gives you a list of people suitable for you. It may happen that you already know one of the recommended people through your social net- works), but never knew s/he could also play Arpeggios as well. The need for such a system is to find sufficiently qualified people who have interest to help others in learn- ing. To accomplish this, the system needs to first "know- about" people's interests and competence, then rank them according to their levels of expertise and finally recom- mend them to the learners. To fulfill the requirements of finding people, a "social support system" has to be pro- vided in a Learning Network. When you are asked to think-aloud of what you expect from such a system that searches people based on your problems or interests, what are the features or characteristics, you think are neces- sary to develop such an application. Then some hint statements were given to the partici- pants and they were asked to think in similar lines to gen- erate their own statements.  assists learners in finding suitable people.  allows knowledgeable people to answer learner's question on a given topic.  supports socializing of learners with other learners.  matches people according to their common learning interests.  affects the productivity of individuals and groups in a learning scenario. C. Sorting and Rating of ideas The participants generated a total of 153 statements These generated ideas in the first phase, were sent back to the experts for first sorting and then rating according to separate instructions for the two idea structuring activities. With regard to sorting, we asked the participants to use the following guidelines provided with a sorting recording sheet: (a) group the statements for similarity in meaning; (a) place each statement in one group only; (c) place each statement somewhere (not one pile comprising all state- ments); (d) place a statement in own group if it seems unrelated to the other statements; do not make groups of statements called miscellaneous. We also suggested the participants to give a name of each of the group of state- ments. For the next activity in this phase, we asked the experts to rate each of the statements on a 1-to-5 scale of impor- tance and feasibility (for importance, 1 = Relatively Un- important; 5 = Very Important; for feasibility, 1 = Least Feasible; 5 = Most Feasible). The meaning of importance is straightforward: how important is the each feature of social support system? Feasibility is about how likely those features are implemented? In the rating instruction, we advised the participants to use the full range of rating values, emphasizing on relative rather than absolute rat- ing. III. RESULTS A. Data Analysis In Figure. 1, there are 10 different clusters shown, which depicts how they were grouped together by hierar- chical cluster analysis [1, 8]. The hierarchical cluster analyses, which are contiguous but not overlapping with each other [3] gives the number of clusters indicating the experts’ opinion (see Figure. 1). To determine the right 42 http://www.i-jim.org USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK number of clusters we modified the heuristic called “20- to-5”to the heuristic “12-to-5”, because the experts in this study produced 12 clusters in average. The classical “20- to-5” heuristics is based on the fact that most of the par- ticipants in Group Concept Mapping (GCM) projects make between 5 and 20 clusters. We began the analysis in this study with the 12-cluster solution checking at each step whether the solution from the merging of clusters makes sense until we arrived at the 10-cluster solution. Other criteria were bridging or anchoring values of the statement in a particular cluster. The bridging/anchoring value is between 0 and 1. A low bridging/anchoring value means that more people have grouped the statement to- gether with others in its vicinity. Statements with low bridging / anchoring value better represent the meaning of a particular cluster’s content than those with a higher value. Statements with a relatively high bridging value within a cluster play more the role of bridges to other clus- ters, while statements with relatively low bringing value play the role of anchors for the statements within a cluster. The main point of interpretation of the cluster map is that all participants come to figure out well the interrela- tionships among the clustered statements. It is aimed that everyone in the group has a clear picture of the project through the concept map. The clusters shown are depict- ing the expert opinion on various features that a Social Support System should have with varied degree of impor- tance (shown as multiple layers on each cluster). 1. Technical features: Focusing the state-of-art tech- nologies for development like web 2.0 applications and Social applications. 2. Showing search similarity: The feature focuses on the search options where people would like to know the search results similar to what they are looking for. 3. Visibility: The feature focuses on the how social help system provides visibility of learners in the Learning Network. 4. Business application: The social help system as a tool for collaboration for further interests and com- mercial interests among people. 5. Communication among learners: The tool sup- ports effective sharing of knowledge among learners facilitating communication via email, instant mes- sengers or phones. 6. Learning community and connection among var- ied people: The tool supports forming learning communities and enhancing communication among the participants. 7. Facilitating learning and engagement: The tool provides support for bringing people to engage on a learning task and enhance learning by social en- gagement 8. Interface design: Easy to use features and user- interface to search for suitable people in a network of learners. 9. Effects on society: The overall effects on society as a useful feature for supporting learning by socializa- tion. 10. Output and Solution execution: The feature focus- ing on additional support like finding not only peo- ple but also relevant learning resources, supporting external collaboration, other embedded support. Apart from the traditional descriptive statistics, the GCM approach applies some specific types of analyses such as multidimensional scaling (MDS) and hierarchical cluster analysis (HCA). The input for the multidimen- sional scaling is the creation of a total square similarity matrix from the sorting data across the participants. The matrix is binary and symmetric, showing the number of people that group together in their sorts each pair of statements. The combined matrix sums individual matri- ces of the participants. An individual matrix consists of as many rows and columns as are the statements. A cell indi- cates whether two statements are grouped together or not. ‘1’ in the cell indicates that the two statements are sorted together by the participant, ‘0’that they are not. MDS transforms the total square similarity matrix into a map depicted as a coordinated matrix. From the coordi- nates, MDS can compute the distances between all pair of statements (points) and can show this as matrix of dis- tances between points. The closer the statements to each other are, the more similar in meaning they are, which also mean that more people sorted these statements to- gether. The extent to which each of the distances between the statements on the map deviate from the values of the total similarity matrix which is used as input to the map is measured with the “stress index” [4]. In principle, the lower the value of the stress index is, the better the overall fit between the map and the input matrix is. A meta- analytical study across a broad range of concept mapping projects indicated that around 95% of concept mapping projects would produce value of the stress index in the range between 0.205 and 0.365. There is a weak relationship between the two values (see Figure 2.). It is also interesting to see how end-users and experts view the different features of the Social Sup- port System. The clusters ‘showing search similarity,’ Facilitating learning and engagement’ and ‘Interface de- sign’ represent the largest margins in scores on the two scales. There are relatively small differences in scores of clusters such as, ‘Technical features’, Communication among learner, ‘visibility’ and ‘output and solution execu- tion’. Figure 1. Cluster Map: eliciting the focus on 10 required features of a social support system iJIM – Volume 5, Issue 1, January 2011 43 USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK Figure 2. The ladder graph: Also termed as pattern match in Figure 2 provides a visual comparison of the clusters on importance and feasibility between the end-users and experts view A specific analysis that compares the statements on im- portance and feasibility within a particular cluster is “go- zone” (Figure 3.). Go-zone identifies statements that score high on both importance and feasibility, thus suggesting where we should look first when planning the implemen- tation of changes in education and training. Go-zone is a bi-variate graph that shows the average ratings for impor- tance and feasibility on each statement within a specific cluster. For example, something that is considered to be very important could be deemed not to be very feasible. The graph is divided into quadrants based upon the mean rating values of importance and feasibility. The upper right quadrant represents issues that are above average on both variables (“go to” – very important and very feasi- ble). IV. DISCUSSION AND CONCLUSION The results of this study suggested 10 different features that can be developed for Social Support system (figure 1.). The system itself can be developed for mobile devices or desktop learning environment. The cluster “Showing search similarity” is differently rated on the scale of im- portance and feasibility (figure 2.) by the experts and end- users, this also indicates how the different users, view the need of this feature in Social Support System. Group Concept Mapping (GCM) in the SoSuSy project proved to be an efficient way and appealing to the partici- pants approach for data collection, structuring and analy- sis. Idea generation, sorting and rating are activities that the participants are used to. Eleven participants generated 153 unique ideas (statements). Sorting and Rating were time consuming activities, but not difficult for the partici- pants from conceptual point of view. In contrast to some other similar techniques (e.g affinity diagram or card sort- ing), GCM applies some rigorous statistical techniques for data analysis such as multidimensional scaling and hierar- chical cluster analysis to identify emerging structures in the data. The visualizations provided by the software (Concept System, 2010) support the interpretation of the results. The clusters indicates the possible features for the social support system that can be developed and later vali- dated according to their importance as we found in the ladder graph in figure 2. The main point of interpretation of the cluster map is that all participants come to figure out well the interrelationships among the clustered state- ments. It is aimed that everyone in the group has a clear picture of the project through the concept map. The clus- ters shown are depicting the expert opinion on various features that a Social Support System should have with varied degree of importance (shown as multiple layers on each cluster). The development these features as depicted in cluster map, will be the guideline for developing the Social Support System for a Learning Network. The clus- ters identified, will also provide a way to evaluate any such system which needs and early evaluation to meet the user needs. 44 http://www.i-jim.org USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK Figure 3. An example of a go-zone comparing the statements in the cluster Social Support system on importance and feasibility. ACKNOWLEDGMENT This paper has been (partly) sponsored by the TEN- Competence Integrated Project that is funded by the Euro- pean Commission's 6th Framework Programme, priority IST/Technology Enhanced Learning. Contract 027087 (www.tencompetence.org). REFERENCES [1] Everitt, B. (1980). Cluster Analysis (2nd Edition). New York, NY: Halsted Press, A Division of John Wiley and Sons [2] Hertzum M, Pejtersen AM. The Information-Seeking Practices of Engineers: Searching for Documents as well as for People. Infor- mation Processing & Management. 2000;36(5):761-778. doi:10.1016/S0306-4573(00)00011-X [3] Kane, M., & Trochim, W. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publications. [4] Kruskal, J.B. and Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage Publications. [5] Nadeem D, Stoyanov S, Koper R. Social support system in learn- ing network for lifelong learners: a conceptual framework. Inter- national Journal of Continuing Engineering Education and Life- long Learning. 2009;Vol. 19, N(4/5/6):337–351. [6] Stoyanov S, Kirschner P. Expert concept mapping method for defining the characteristics of adaptive e-learning: Alfanet project case. ,. Educational technology research and development. 2004;52(2):41-56. doi:10.1007/BF02504838 [7] Terveen L, McDonald DW. Social matching: A framework and research agenda. ACM Trans. Computer-Human Interaction. 2005;12(3):401-434. doi:10.1145/1096737.1096740 [8] Trochim W. An introduction to concept mapping for planning and evaluation. In W. Trochim (Ed.) A Special Issue of Evaluation and Program Planning. 1989;12:1-16. [9] Wilson T. Information-seeking Behaviour: Designing Information Systems to meet our Clients’ Needs. ACURIL: Association of Caribbean University, Research and Institutional Libraries. XXV Conference, San Juan, Puerto Rico. 1995. [10] Wolf . J, Boevink W. Mapping the quality of life of people with severe and enduring mental health problems. Ment Health Care. 1999;2:228–231. [11] Wopereis, I.G.J.H., Kirschner, P.A., Paas, F., Stoyanov, S., & Hendriks, M. (2005). Failure and success factors of educational ICT projects: a group concept mapping approach. British Journal of Educational Technology, 36, 681-684. doi:10.1111/j.1467- 8535.2005.00546.x [12] Watson GR. What is…concept mapping? Med Teach. 1989;11:265–269. doi:10.3109/01421598909146411 iJIM – Volume 5, Issue 1, January 2011 45 http://dx.doi.org/10.1016/S0306-4573%2800%2900011-X� http://dx.doi.org/10.1007/BF02504838� http://dx.doi.org/10.1145/1096737.1096740� http://dx.doi.org/10.1111/j.1467-8535.2005.00546.x� http://dx.doi.org/10.1111/j.1467-8535.2005.00546.x� http://dx.doi.org/10.3109/01421598909146411� USING CONCEPT MAPPING FOR NEEDS ANALYSIS FOR A SOCIAL SUPPORT SYSTEM IN LEARNING NETWORK [13] Yimam, D. & Kobsa, A. (2000). DEMOIR: A Hybrid architecture for expertise modeling and recommender systems. In Proceedings of the 9th IEEE international Workshops on Enabling Technolo- gies: infrastructure For Collaborative Enterprises, WETICE. pp. 67-74. IEEE Computer Society, Washington, DC. AUTHORS Danish Nadeem works at Centre for Learning Sciences and Technologies (CELSTEC), Open University of the Netherlands as researcher and technology developer. He obtained his M.Sc. in Cognitive Science from University of Osnabrueck, Germany. He has a solid background in computer science, artificial intelligence (AI) and computa- tional linguistics. His research interests are in the fields of cognitive science, AI, human computer interaction (HCI) and semantic web/desktop technologies. (e-mail: danzinde@gmail.com). Slavi Stoyanov is a Senior Researcher at the Centre for Learning Sciences and Technologies, Open University of the Netherlands. He holds a PhD on Instructional Tech- nology from the University of Twente, The Netherlands. His major research interests include technology-enhanced learning, individual differences in learning, learning to solve ill-structured problems and cognitive mapping. (e- mail: slavi.stoyanov@ou.nl). Rob Koper holds a PhD in Educational Technology from the OUNL. Since 1998 he has been a Full Professor in Educational Technology, specifically in e-learning technologies. He was responsible for the development of educational modelling language (the predecessor of IMS learning design). As the Director of RTD in learning tech- nologies, he led the OUNL’s contribution to the IMSLD specification activities. His research focuses on self- organised distributed learning networks for lifelong learn- ing, including RTD into software agents, educational se- mantic web, interoperability specifications and stan- dards.(e-mail:rob.koper@ou.nl) A short version of this paper was presented in the IEEE EDUCON 2010 conference in Madrid and selected for submission to this journal. It received a co-award as a Best Student Paper of the conference. Submitted November 20th, 2010. Published as resubmitted by the autors December 11th, 2010. 46 http://www.i-jim.org 1st_Announcement_ICL2011.pdf