83 https://www.designforsocialchange.org/journal/index.php/DISCERN-J ISSN 2184-6995 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Social impact measurement in Cyprus and other European countries Eleni Zenonos Published online: May 2022 To cite this article: Zenonos, E. (2022). Social impact measurement in Cyprus and other European countries. Discern: International Journal of Design for Social Change, Sustainable Innovation and Entrepreneurship, 3(1), 83-94. 84 Social impact measurement in Cyprus and other European countries Eleni Zenonos a aCenter for Social Innovation, Nicosia 1010, Cyprus. Eleni.Zenonos@csicy.com Abstract Although non-profit and other organisations related to social issues in Europe have increased, there is currently a gap when it comes to measuring their performance and social impact. This can cause mistrust and may have negative implications on their efficiency and sustainability. As part of the Erasmus+ project Social Impact Measurement for Civil Society Organizations (SIM4CSOs), which aims to create a common methodology and approach civil society organisations can apply to measure their social impact (in the form of a methodological manual and online resources), a survey and focus groups were conducted with representatives of civil organisations, corporate social responsibility departments and policymakers to identify existing skill gaps and training needs. The research took place in Croatia, Cyprus, Greece, Italy, Norway and Portugal. The results showed that social change practitioners, whether they work for non- profit organisations or the private sector, are keenly aware of the benefits of impact measurement. However, they expressed a need for buy-in and participation from organisational management, as well as identifying the main stakeholders who benefit from their activities. Additionally, many organisations indicated a lack of resources (financial and technical) to carry out a full-impact measurement process and evaluation. Finally, there was a clear need to consider the skill levels of practitioners, which varied. Therefore, most organisations suggested that the methodology developed should be practical, adaptable and non-time-consuming and that any platform used should be accessible and easy to use. Keywords: Social impact measurement, Civil society organisations Introduction The Social Impact Measurement for Civil Society Organisations (SIM4CSOs) is an Erasmus+ funded programme which aims to empower non-profit and other civil society organisations by creating a set of tools and methodologies they can use to assess their social impact. The programme is led by Higher Incubator Giving Growth and Sustainability from Greece, a non-profit organisation that aims to reinforce non-profit organisations through educational and supportive programmes. The other consortium members are NOVA Institute, which is part of Oslo Metropolitan University in Norway, the Center for Social Innovation in Cyprus, ACT Group from Croatia, CESIE in Italy, INOVA+ in Portugal and MIDOT in Israel. All participating organisations have experience in social research and social science, and their collective knowledge and experience have been instrumental to the success of this project. The aims of the SIM4CSOs project are: “…to improve the effectiveness of the Third Sector, increase the fundraising possibilities of civil society organisations (CSOs), enhance the overall sector and the protection of the reputation of CSOs from bad practice cases, create synergies with organisations active in the social field and equip adult staff members with strategic planning of their professional development.” (SIM4CSOs Consortium, 2022) 85 As part of the project, several best practices concerning social impact measurement were collected from each partner country. Furthermore, to explore and assess the state of the art of social impact measurement of CSOs in each partner country, desk and field research was conducted, including a survey and focus groups. In this paper, the results of the research in Cyprus and the SIM4CSOs partner countries will be presented as well as contextual information about Cyprus and brief information about the other countries. Current state of art in Cyprus There are currently 6,300 registered associations, although most of them are inactive, which gives a false impression of the size of civil society in Cyprus, 330 registered non-profit organisations, more than 400 charity foundations (Office of the Commissioner for Volunteering and NGOs et al., 2019) and about 190 social enterprises (Isaias, 2019) which operate in a range of areas – from the environment, gender, education and culture to disabilities and health. Nevertheless, reports and studies indicate that organised civil society in Cyprus still has weak foundations for a variety of reasons, including limited participation. In July 2017, progressive amendments to the Law on Associations, Foundations and Clubs to improve the regulation of CSOs in Cyprus were approved by the Parliament, but there are still obstacles to overcome. For example, the legislation does not have specific provisions for the legal sources of income or property for any type of NGO, nor does it set any prerequisites regulating their immediate involvement in financial operations, which hinders the need for transparency. Based on the most recent assessment of civil society nationally (CIVICUS, 2011): “Accountability and transparency rates of CSOs need to be encouraged and enhanced, particularly for the ones receiving public funding. This will help to create a culture of openness and transparency and will help to counter abuses and corruption in the long run.” This highlights the importance of measuring social impact, which can enhance accountability and transparency of CSOs. Current state of the art in other European countries Based on the comprehensive report published by the SIM4CSOs consortium (2021), research conducted in partner countries, namely Greece, Portugal, Croatia, Norway, Italy and Poland, only a few had established policies or regulations in place regarding the definition of national standards for impact measurement. In Italy, social impact measurement became more relevant with the reform of the Third Sector (Law 106, 2016), in which social impact measurement was introduced and presented as a “key element for the legal recognition of social enterprises”. Despite the lack of formal governmental guidance and official regulations regarding social impact measurement in most partner countries, it is evident that the situation has started changing, as CSOs, social enterprises and even private companies, including their corporate social responsibility (CSR) departments, from all partner countries have indicated that they realised the importance of social impact measurement and identified several external and internal drivers which further stress the crucial role social impact measurement plays in their sector. Methodology To explore and assess the state of the art of social impact measurement of CSOs in Cyprus, first, several best practices were collected from each partner country, which consisted of examples of organisations 86 or projects which successfully implemented social impact measurement. Second, a survey developed by the SIM4CSOs consortium and set up on Google forms was sent to over 160 non-governmental non-profit organisations, social enterprises and corporate responsibility departments in Cyprus in February 2021. A total of 32 organisations from Cyprus responded to the survey (215 responses came from the other consortium countries, 247 responses in total). Finally, to further explore the topic of social impact measurement and the training needs related to it by civil society representatives, two focus groups were conducted in each partner country. The participants were recruited through email contact or had already expressed interest to take part in the focus groups via the survey. Twelve participants took part in the Cyprus focus groups, from a variety of non-governmental organisations in Cyprus, CSR departments and policymakers. Cyprus research results Demographics Most survey respondents 78% (25) worked for non-profit organisations but participants from other types of organisations also took part, including a private company (1), higher education institutions (2), a public organisation dealing with social issues (1), a company limited by guarantee (1), a grassroots initiative (1) and an informal network (1). The respondents worked in a variety of areas: youth development (7), arts, culture and humanities (5), environmental and animal (3), environmental, climate and social justice (1), educational institutions (3), health and human services (1), intercommunal communication (1), hospitals and care (1), community capacity (1), business incubator (1), gender issues and human rights research (1), gender and women’s rights inequality (1), accounting (1), advocacy for civic space (1), business network (1), sustainable development goals (1), social care (1) and economics and public policy (1). Figure 1: Type of organisation the respondents worked for. 87 In addition, most participants had more than three years of experience in the non-profit sector (22, 69%) and only one had less than one year of experience. Figure 2. Years of work experience in the non-profit sector. Figure 2: Years of experience in the non-profit sector. The main funding channels for most of the organisations were EU funds (9), government grants (8), corporate donations (4) and individual donations (3). Other sources of funding reported were a combination of the ones mentioned (1), membership fees (1) and sales of products (1). Three (3) organisations reported they received no funding. Figure 3: Main funding channel of organisations. Regarding the focus groups, 12 representatives of NGOs, policymakers and CSR departments in Cyprus took part. Most participants were 40–50 years old (5) and had a Master's degree (9/12). The rest were 20–30 (3), 30–40 (2) and 50–60 (2) years old. Also, two participants had a BA and one had a PhD. Measuring social impact When asked how measuring social impact could help their organisations, 8 participants stated that it could help improve their current processes and their organisation in general (e.g. “It is an important element of understanding the role and importance of our activities. This allows us to better understand how to improve 0 1 2 3 4 5 6 7 8 9 28% 69% 3% 1-3 years Above 3 years Less than 1 year 88 our activities in a way that will hopefully achieve the desired impact”). In addition, 6 participants believed that measuring their social impact could assist in their planning and strategy (e.g. “It will help with formulating future activities based on the impact, feedback and needs of our target group.”) Other ways in which measuring social impact could help their organisations that were mentioned were assessing their efficiency (5 responses, e.g. “It can help us make sure we are creating the change that the community needs”), securing funding (4 responses, e.g. “To secure more funding, more collaboration with other CSOs, better public awareness of our work”), proving their effectiveness (2 responses, e.g. “…People respond/understand better when you present your impact instead of anything else…”) and raising awareness (2 responses, e.g. “Promote the identity and values of the organisation. Raise brand awareness”). Furthermore, 41% (13) of the respondents reported that they measured their impact, with most measuring both qualitative and quantitative data (92%, 12) and one participant only measuring quantitative data. Figure 4: Percentage of organisations measuring their social impact. The reasons the participants measured their impact were to improve the services they provide (10), due to internal process requirements (7), donor requirements (5) and to reach more people (1). Regarding the methods used to measure their impact, most used records/files (9), questionnaires (7) and interviews (6), followed by focus groups (4) and a combination of social media, emails and face-to-face conversations (1). Figure 5: Reasons for measuring social impact. 89 Just over half (53%, 7) of the respondents used a logical model for these measures. The logical models used varied. Some of the models mentioned were based on the organisation’s inputs, activities, outputs and outcomes, their goals (1), the organisation’s rules (1) and according to European criteria (1). Of those who did not use a logical model, the main reason was lack of knowledge (1), lack of structured activities (1) and use of a theory (Theory of Change) similar to a logic model. In addition, 10 of the 13 organisations that measure their social impact had defined success metrics. Some of those metrics were the satisfaction of employees and volunteers, financial income, quality of services- activities provided and/or implemented, participation in activities, an annual target set by their Council, the organisation’s projects and day-to-day actions, annual improvement, achievement of goals of projects and specific targets set and measured by the organisation. Of those not using defined success metrics, the reason was lack of data. Regarding the frequency of social impact reporting, most participants (5/13) report about it annually, every six months (2/13) or after each activity/project (2). Figure 6. Frequency of social impact reporting. The organisations that did not measure their impact reported that it was due to a lack of human resources (11/19), lack of financial resources (10/19), lack of time (10/19) and lack of knowledge (9/19). Figure 7: Reasons for not measuring social impact. 90 The most common reasons they reported they would like to start measuring their impact were to be able to improve their services (16, 84%) and due to internal process requirements (6, 32%) or donor requirements (2, 11%). Figure 8: Reasons organisations would like to start measuring their impact. Obstacles in measuring impact The main obstacles to measuring impact that the participants responded with were lack of human resources (19, 59%), lack of an established system (17, 53%), lack of trained personnel (13, 41%), lack of know-how (12, 37%) and difficulty accessing data (11, 34%). Figure 9: Obstacles in measuring social impact. When asked how many hours a month they could allocate to measuring social impact, most participants stated that they could dedicate up to 5 hours (14, 44%), followed by up to 10 hours (8, 25%) and up to 20 hours (3, 9%). In terms of the amount of money they could give, most stated that they could spend up to 100 euros (10, 31%), followed by up to 500 euros (5, 16%) and up to 1000 euros (2, 6%). Seven participants said they had no funds available to allocate for measuring their social impact. Eighteen respondents claimed that they would also like to measure other aspects of their work, e.g. indirect impact on society in general (3 respondents), not just the immediate target group, people’s opinions/general public opinions (3 participants), comparing their impact with their stakeholders versus people who do not take part in the organisation’s programmes, the impact of scientific and cultural events and the impact of CSR actions. 91 Most respondents (27) agreed (agree and mostly agree combined) that social impact measurement influences donors’ decisions. They also agreed (25) that social impact measurement is an advantage for large-scale organisations and that in the long term these can lead to better, more informed decisions and become a management tool for the organisation (29). Most participants also agreed that (29) social impact is more than numbers and that it has a long-term impact on the organisation (22). Figure 10: The number of hours organisations could allocate to social impact measurement. Skills analysis Most participants (combined scores of 3 and 4) felt confident in their ability to apply questionnaire research techniques to a target audience (23), present the results on the monitoring plan (22), compile a report with the results of the monitoring plan (21) and successfully research target audiences (20), but felt less confident in their ability to set/define key performance indicators (KPIs) (17), measure those indicators (16), apply focus group and interview techniques (16), design a monitoring plan (14), and apply social return on investment (SROI) methodology (5). Finally, in terms of their training needs, most respondents reported that they need training in applying SROI methodology (26), applying focus groups research techniques (24), interview research techniques (23), questionnaire research techniques for a target audience (20), successfully reaching target audiences (24), setting/defining KPIs (23), measuring those KPIs (24), designing a monitoring plan to measure KPIs (26), compiling a report on the monitoring plan (23) and presenting the results of that monitoring plan (21). Focus groups The groups were asked about their experience in social impact measurement (if any), its importance for them, challenges they might face in implementing such methodology, what a good methodological tool would look like and what training or other needs they may have related to this area. All the participants agreed that measuring social impact was of high importance for them, but most participants either did not measure social impact or measured it occasionally with no specific methodology. The CSR representatives and the Commissioner of Volunteering in Cyprus emphasised how vital it is for NGOs to be transparent and be able to measure and report their social impact, as funders will trust them more and will be more willing to donate. 3%3%3% 3% 3% 25% 3% 10% 44% 3% 2 hours per month can't estimate it depends See answer above. Surely more than 20 but do not know how many up to 10 hours Up to 2 hours up to 20 hours 92 In terms of the main difficulties faced in measuring social impact, most respondents replied, similarly to the survey, that lack of time, money and professional staff were the main reasons. Furthermore, for a tool to work for them, it must be easy to use, user-friendly, with minimal resources needed and should be adaptable to their organisational needs. All the participants agreed that some form of training will be necessary for them to be able to use such a methodological tool. Results in other European countries The results of the research conducted in the other partner countries were strikingly like the Cyprus results for both the survey and the focus groups. Conclusions Social change practitioners in Cyprus and the other consortium countries are aware of the benefits of impact measurement, the fact that it provides organisational infrastructure and is a way to attract and communicate with funders etc., but they also recognise the lack of impact measurement and the need for a more methodological approach. It is also obvious that buy-in and participation from organisational management are crucial to the implementation of social impact measurement. However, there was some ambiguity about how the organisations should measure social impact. There was also a strong desire for professional support concerning the methods and skills necessary to conduct impact measurement. Considering basic skills levels and limited resources, most, if not all, organisations suggested that the methods developed should be practical, adaptable, not time-consuming and that any platform should be accessible and easy to use. Next steps The next stage of the project is the development of a methodological manual and an online platform that will supply information and resources to CSOs about impact measurement. Based on the conclusions above: “…this tool would be developed to help practitioners of all skill levels achieve a common baseline. Specifically, it would provide guidance on how to create a logic model, identify impact KPIs, and develop a logic-driven measurement plan which includes clear steps and phases. More advanced content might include links to measurement tools specific to certain target groups and intervention types. It might also link up with national and regional indicators and longitudinal data and would help organizations understand how to incorporate country or region-level data into their impact assessments” (SIM4CSOs consortium, 2022) Find out more about this project and the tool that will soon be developed on our website www.measuringimpact.eu References CIVICUS (2011). The Assessment of Civil Society in Cyprus. A Map for The Future-2011. CIVICUS https://tinyurl.com/2p9cav4f Isaias, G. (2019). Social Enterprises and their Ecosystems in Europe. Country report: Cyprus. Luxembourg: Publications Office of the European Union. https://tinyurl.com/4furvfpx https://measuringimpact.eu/ https://tinyurl.com/2p9cav4f https://tinyurl.com/4furvfpx 93 Office of the Commissioner for Volunteering and NGOs, Expert Council for the Legislation on Non- Governmental Organizations (NGO) of the Conference of International NGOs of the Council of Europe, NGO Initiative Group (2019). The organized Civil Society in Cyprus. Press and Information Office. SIM4CSOS Consortium (2021). Comprehensive Report. Best Practices and Needs Regarding Social Impact Measurement in CSOs. SIM4CSOS consortium. https://measuringimpact.eu/wp-content/uploads/2021/10/SIM4CSOs-IO1-Comprehensive-report_EN.pdf SIM4CSOs Consortium (2022). The Contributions of The Project. SIM4CSOs consortium. https://measuringimpact.eu/ https://measuringimpact.eu/