Measuring and Comparing Civil Societies Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 71 Measuring and Comparing Civil Societies Mark Lyons Abstract: In most countries (Australia is an exception) the term ‘civil society’ is widely used by social researchers, activists and government officials. However its analytical value is limited by its great variety of meanings. The article proposes that if ‘civil society’ is to be of any use analytically it should be capable of being measured. It outlines two related efforts to measure civil society and to compare it cross nationally and identifies several problems with those approaches. The recent popularity of ‘civil society’ The term ‘civil society’ has had a long history, used at different times and with different meanings by, among others, Ferguson, Hegel, Marx and Gramsci (Ehrenberg 1999). However, until the 1980s the term was not widely used. That is no longer the case. No longer is ‘civil society’ only a topic for philosophical discussion; it is widely invoked by activists and appears in the policy documents of many national governments and international organisations. This current popularity is largely a product its adoption in the 1980s by activists in Eastern Europe, Latin America and Asia who were struggling against oppressive communist regimes or military dictatorships. They referred to ‘civil society’ to describe what they were and what they were trying to enlarge(Cohen & Arato 1992). Civil society was people acting as citizens, freely, collectively and for a wider good; it was also the sphere where such citizen activity took place. The success of these activists, especially in Eastern Europe, led to the popularity of the term not only in those societies but in the United States, both among social science academics and many US foundations. In this process, civil society also acquired a normative dimension; it became something to be encouraged. It became entangled with the discovery of social capital and tied in with wider projects to extend tolerance, democracy and good governance and to end corruption. For a time, it was adopted by the World Bank. It came to be seen as a pre-requisite for successful social and economic development. For some it acquired a global dimension, representing a vision of a less violent world and the organisations, institutions and processes that might bring it about (Keene 2003). 72 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 Eva Cox first introduced Australians to ‘civil society’ in her 1995 Boyer Lectures, entitled A Truly Civil Society (1995). As her title indicates, she gave the term a normative dimension; identifying the characteristics of a ‘truly civil society’. However, the term has never achieved wide recognition or use in Australia. Almost a decade later, at the conclusion of a major ten year study of Australian institutions, Geoff Brennan and Frank Castles regretted that they excluded the institutions of civil society. They attributed this exclusion to the reluctance of Australian scholars to study these institutions, though more accurately it was their failure to discover and incorporate the research that was being conducted (Brennan & Castles 2002). Elsewhere the popularity of the term among social science researchers, activists and government officials has created a major problem. As frequently happens in politics and the social (but not economic) sciences, concepts that acquire popularity also acquire many and varied meanings. This has led some scholars to warn that the term will soon fall into disuse unless some agreement over meaning can be reached. Michael Edwards was one who embraced the term but has more recently sounded the alarm. A development practitioner and writer, he had been in charge of research for the UK based development charity Save the Children and then, for a short while in the 1990s was senior civil society specialist at the World Bank with the responsibility for helping the Bank understand civil society (and social capital). He then moved to the Ford Foundation where until late 2008 he ran their governance and civil society unit. In 2004 he published a book Civil Society (Edwards 2004) in which he addressed the proliferation of uses of the term and suggested that as a working definition civil society might be thought of as having three dimensions: • associational life, • the ‘good society’ and • the public sphere or an arena for public deliberation. The first dimension refers to the private nonprofit associations that most people belong to and support, through which they cooperate with others and, in some cases, Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 73 participate in the political system (or societal governance). This is the most common use of ‘civil society’. The second dimension refers to the features of the good society. Generally it is taken that this encompasses a core of values shared by everyone living in a particular society and polity, but where there are arguments about interpreting and prioritising these values. This is what people have in mind when they talk about, when they enumerate the features of, a civil society. Generally such features include values such as tolerance, social justice and equality. The third dimension of civil society refers to the spaces and media where views about the features of a good society and related arguments about the policies to be pursued by a country are advocated and resisted. This includes the various publicly available media and parliament itself. Concerns that ‘civil society’ may soon fade from use may be misplaced. Some terms have a long and virtuous life without ever being clearly defined. Indeed their vagueness and flexibility seems essential to their continuing popularity. ‘Community’ is an example. Whatever collective we might describe as a ‘community’, wherever we might put the boundaries to encompass ‘community’, it is clearly a ‘good thing’. So too is ‘democracy’. But ‘civil society’ is not in their league. Its entry into the language of advocacy and governance is new. Its wide acceptance is hindered by its ambiguities. If its use is to become more than mere rhetoric or claim making; if it to become embedded in the discourse of government and the media it will need to be capable of measurement. But what is measured will need to encompass much of the present variety in uses of the term. Perhaps measurement and conceptual clarification can advance together, in a dialectical way. Measurement The search for ways to measure a phenomenon is both an interesting scholarly exercise and an essential aspect of advocacy. Measurement is important if actors want to use a term to summarise, to publicise and to advocate for more of the reality it is describing – in this case for a more or for a stronger civil society. It is often said that for governments these days if a phenomenon cannot be measured it does not exist. 74 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 These days, in the international arena, jockeying for attention from the world media and from international agencies, advocates for causes such as poverty relief or press freedom appear to believe that their cause is helped by constructing an index. The huge growth of indexes (eg the United Nations Development Program’s Human Development Index, Freedom House’s Freedom Index, Transparency International’s Corruption Perceptions Index or Reporters Without Borders’ Worldwide Press Freedom Index) is clearly an interesting product, or sign, of globalisation, but for people who wish to champion civil society, it provides another incentive to get an agreed meaning, and thus some measurements for ‘civil society’. Only thus can claims that civil society is growing or declining or that civil society is stronger in certain countries or under certain types of economic or political regimes be verified and in that way legitimised. This paper reviews two efforts to provide such measurement, one originating within the activist community, the other in academia. They are: • the Civil Society Index (CSI) developed by Civicus, an international organisation that describes itself as a world alliance for citizen participation, with its main offices in South Africa (Heinrich 2007) and • the Global Civil Society Index (GCSI) developed by the Center for Civil Society Studies at the Johns Hopkins University in the United States (Salamon & Sokolowski 2004). Previous efforts to measure civil society Before turning to the development of the two indexes, a little history is necessary. It illustrates the influence (some would say the distorting influence) of the popularity of ‘civil society’, in its ‘associational life’ dimension among US and European foundations prepared to fund social science research. Around 1990 Lester Salamon at the Johns Hopkins University assisted by Helmut Anheier, then at Rutgers, gathered together researchers from 12 countries in Europe, Latin America and Asia and began the Comparative Nonprofit Project (CNP), a project designed to develop and apply in these various countries a common definition and ways of measuring nonprofit organisations (or nonprofit sector) in those countries. Salamon, a political scientist, was one of a growing group of United States scholars (mainly economists, lawyers Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 75 and sociologists) that since the 1980s had been seeking to understand the extent but also the aetiology and dynamics of nonprofit organisations – ie private organisations that were prohibited from distributing profit to members or other stakeholders. The CNP achieved a common definition, an industry based classification system and set of measures, mainly of the economic dimensions of the sector (such as employment, income, sources of income and contribution to GDP). It also developed an agreed way of taking account of the contribution of volunteers to nonprofit organisations. The CNP went through several phases (Australia was part of the second phase) and Salamon subsequently persuaded the United Nations Statistical Division to endorse a set of rules for the preparation of a nonprofit institutions satellite account to a country’s set of national accounts (Department of Economic and Social Affairs Statistics Division 2003). Much of this work has been funded by US and European foundations. But in the late 1990s, as the term civil society became more and more popular, Salamon and Anheier began talking of the project as a civil society project. This is clearly indicated in the title of the book that reported the results of the second phase of the CNP: Global Civil Society: Dimensions of the Nonprofit Sector (Salamon et al. 1999). It appears that that the change was a response to the changing interests of funders: foundations were now much more interested in funding research into civil society rather than the nonprofit sector. This was especially true of research being conducted in former Eastern bloc countries or less developed nations: to keep the CNP on the road required a name change. This conflation of civil society with nonprofit sector was viewed sceptically by associates from many participating countries1 . In 1998, Anheier left the project to become director of the Centre for Voluntary Organisations at LSE. He soon renamed it the Centre for Civil Society and began work on a project designed to conceptualise and measure civil society. One of the products of this work, finally published in 2004 as Civil Society: Measurement, Evaluation, Policy (Anheier 2004), was a proposal that there were many indicators for the different aspects of civil society and that these could be individually measured and grouped into four dimensions which could in turn be portrayed as a diamond. Thus 1 The author of this article was Australian Associate for the CNP and along with other country associates was present at a conference where the GCSI was revealed. 76 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 although he did not propose that the state of civil society in any one country could be reduced to a single indicator, he did argue that it could be compared graphically as a diamond with different shapes according to the strength of civil society along each dimension. It also left open the prospect that the area of each country’s civil society diamond could be used as a proxy for a single index number. Most of the indicators he specified were similar to the data collected for the CNP, but went further to encompass other national accounting type data and survey data similar to but more extensive than that undertaken by the World Values Survey. Civil Society Index (CSI) At the same time, Civicus was looking for a way of lifting the profile of civil society within each of the over 60 countries from whence it drew its membership and globally. In the early 1990s, not long after its formation, Civicus had published an overview of civil society in each of the world’s major regions (De Oliveira & Tandon 1994). Interestingly, although the book was titled Citizens: strengthening global civil society and the first chapter spoke of an emerging global civil society, most chapters described the nonprofit sector or independent sector or third sector (the exception was that on civil society in Asia and the Pacific written by a Philippine activist). Then in the mid-1990s, Civicus commissioned and published a set of profiles of civil society in 60 countries (Civicus 1997). Finally, and responding to a growing trend by international government agencies and advocacy groups to publicise their cause by producing and publishing an annual index, Civicus leaders decided that it would be useful to develop a civil society indicator or index (CSI) for each country that could be released each year with considerable publicity. They gave initial encouragement to Anheier’s work on the civil society diamond, but then were persuaded by others that such a methodology was too ambitious and would be impossible to apply in most countries. What emerged was a methodology for discussing civil society within a country and for raising awareness of it. Civil society was defined as ‘the arena outside the family, the state and the market where people associate to advance common interests’(Heinrich 2007, p. 4). In practice this meant nonprofit organisations, conventionally the organisational form used to advance common interests, their membership and activities. The methodology was structured around the four Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 77 dimensions of Anheier’s diamond: structure, values, environment and impact. Within each dimension there were a number of indicators of civil society (structural and values), of the support or hostility of its environment and of its impact. In the pilot stage these indicators were mostly used to structure discussion (and for consciousness raising) among groups of leaders of nonprofit organisations or NGOs (described as civil society leaders): charities, advocacy groups, unionists. In some cases, where data was available for some of the indicators (eg percentage of population who were members of associations or who volunteered or took part in protest rallies or boycotts) this was presented to groups to shape their deliberations. No attempt was made to generate any aggregated numerical indicators along the various dimensions. However, after reflections on these pilots, in 2003 it was decided to launch a more ambitious project which since then has been operationalised across 53 countries. The primary goal of the project was to ‘generate a contextually valid assessment of the state of civil society in a given country’ (Heinrich 2007, p. 3). However, responding to the interests of many groups, including the foundations that funded most of the work, ‘the CSI also seeks to achieve cross-country comparability of its findings’ (Heinrich 2007, p. 3). Altogether there are 74 indicators that are grouped into 25 subdivisions which are in turn grouped into the four original dimensions. Subdivisions in the Structural dimension include strength and depth of citizen participation, diversity and resources. The Values dimension tries to measure the extent of commitment and practice of certain values within civil society organisations; subdivisions include democracy, transparency, gender equity and poverty eradication. Subdivisions in the Environment dimension include political context, basic freedoms and state-civil society relations. The Impact dimension tries to measure the level of civil society influence on public policy, responsiveness to social needs and empowering of citizens. The process works thus: a researcher pulls together what can be found from secondary data; primary research is then carried out consisting of a specialist population survey, media analysis and regional consultations. Results from these are then drafted into a report which is then reviewed by a national advisory group of civil society leaders from across the variety of fields where civil society is to be found. Indicators are 78 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 scored on a 0-3 scale by the advisory group using a citizen jury methodology and aggregated into subdivisions and then the four dimensions, allowing the construction of a diamond. These scores or ranks are then reviewed by a national workshop which also uses the analysis of civil society to identify potential activities to strengthen civil society (Heinrich 2007, pp. 4 - 10). Below, the diamonds prepared for two rather different counties illustrate the end result of the project (Heinrich 2007 p. 271 and p. 369 respectively). Global Civil Society Index (GCSI) Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 79 The GCSI was first developed by Lester Salamon and Wojciech Sokolowski in 2002 as a response to the pilot phase of the Civicus project. Basically it involved utilising data already collected via the ongoing CNP. It justified this by noting that while ‘disputes rage about the outer boundaries of the civil society concept’, there was ‘reasonable consensus’ about its central core: ‘the basic private associational life of a society’(Salamon & Sokolowski 2004, p. 65). The GCSI is formally more rigorous than the CSI. Arguing that their approach was ‘theory driven’ Salamon and Sokolowski focussed attention on three dimensions, not of civil society but of what they called ‘the civil society sector’: its capacity, its sustainability and its impact (2004, pp. 67 - 77). For Capacity they identified four indicators, including employees of civil society organisations (read nonprofit organisations) as a percentage of the economically active population; the volunteer employment (expressed as equivalent full time employees) as a percentage of the economically active population; the level of giving by individuals companies and foundations as a percentage of GDP and the degree of diversification of the civil society sector, measured as the distribution of civil society workforce (including volunteers) in different fields of activity. For Sustainability they used another four measures: self generated income; government support; popular support reflected by numbers volunteering (as percentage of adult population) and legal environment. The last was an amalgam of twenty measures; each measure being the presence or absence of particular legal requirements that either gave confidence in the transparency of civil society organisations or affected the ease with which they could form and operate. These raw scores were then weighted by country scores from two of the World Bank’s governance measures: government effectiveness and rule of law. This was done in recognition of the possibility that the presence of legal provisions in a particular country did not mean that they were enforced. The popular support score was double weighted ‘to counteract the effect of having several indicators in the index that reflected the size of the formal components of the civil society sector’. 80 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 The Impact dimension also contained four indicators that were thought to provide an adequate proxy of the impact of civil society organisations. Once again, these indicators reflected the data that were available. One was the overall value added by the civil society organisations to the economy for which they use the wages paid to employees of civil society organisations together with the imputed wages of their volunteers, as a percentage of GDP. A second measure was the contribution by civil society organisations to human services, measured as the percentage of total employment in health, education, social services and culture and recreation. A third sought to measure the contribution of civil society to advocacy and expression, measured as the number of employees and volunteers mobilised by civil society organisations primarily engaged in what it describes as expressive activities: advocacy groups, professional associations, unions, environmental protection and culture and recreation. This number was expressed as a proportion of the adult population. A fourth indicator was popular commitment, measured as the percentage of the adult population claiming membership of voluntary associations (as reported by World Values Surveys). At the end of this process for each country there were a set of 12 numbers. Whilst these were comparable for each indicator (most were expressed as a percentage of a country’s GDP or adult population), to derive an index, the Hopkins’ researchers had to aggregate all the indicators, and to do that they had to be normalised. For this process, for each country, each indicator was expressed as a percentage of the highest score achieved by one country. At the end of this process each country had a twelve scores ranging between 0 and 100% (in reality between around 30 and 100%). These were then averaged for each dimension and then the scores attained within each dimension were totalled and averaged to give a single index. For the record the Netherlands had the highest score of 74, followed by Norway (65) and the United States (61). Sweden scored 60 and the UK 58. Australia came in at 49, dead heating with France. South Korea scored 35. Pakistan had the lowest score: 19. Much of the data was drawn from that collected for the Stages 2a and 2b of the CNP and applied to various years in the last half of the 1990s. To illustrate the sensitivity Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 81 of indicators to choices of data and points of comparison (and to add a parochial note), Australia was marginally held back by the use of the “economically active population” as a denominator in several indicators. Especially in the mid 1990s, Australia’s workforce participation rate was among the lowest of the OECD and this meant that when nonprofit employment was expressed as a percentage of the labour force, Australia was higher than say the UK and almost on par with the US, but when expressed as a percentage of the economically active population (in effect of the entire population of work force age), Australia slipped behind. More importantly, Australian volunteering data was drawn from the ABS 1995 voluntary work survey, which the ABS later acknowledged significantly underestimated the percentage of the population volunteering and the hours they contributed2 . Evaluation Despite its methodological sophistication and use of “objective” measures that can be obtained in almost any country, the GCSI can only be accepted as a measure of civil society if one accepts that civil society is synonymous with nonprofit organisations. This means accepting that civil society is captured by only one of the three dimensions that Michael Edwards outlines; that of associational life. Even then, it gives only a limited account of that, omitting variables which can be derived from the World Values Surveys, such as the percentage of associational members that participate in political activities of varying kinds. More, it overemphasises the contribution of highly professionalised charities such as Mission Australia, St Vincent’s Hospital, the Kings School or the Walter and Eliza Hall Institute compared with neighbourhood associations, local footy clubs or environmental protest groups. This is because it counts employment in nonprofit organisations and wages in two of its indicators. These are important measures of economic impact, but are not so clearly measures of civil society and while it counts numbers of volunteers (expressed as full time employment equivalents) and the imputed value of volunteer wages as well, this reduces volunteers to being members of a workforce, which is an economic measure rather than a civil society measure. This conflation of civil society with the economy illustrates the absence of a readily agreed theory of civil society that focuses on measures that are unique to civil society rather than inappropriate substitutes. The 2 The author of this article, as country associate for Stage 2a of the CNP contributed the data for Australia, but several years before the data was used to create the GCSI. 82 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 GCSI indices offer only one measure, the legal environment, which might be viewed as part of Mike Edwards’ public sphere dimension. There are no measures of the good society dimension. For example, to take account of that dimension would require identifying and allocating a negative score to those associations that encourage intolerance and inequality. It would require much else as well. On the other hand, while Civicus’s CSI attempts to cast a wider net, it appears to rely overmuch on the impressions and judgements of groups of people viewed by the compilers in each country as leaders of, or knowledgeable about civil society. To be sure, in rating performance on many of the 74 indicators, they are presented with independently generated data, including survey data, but as the Civicus team acknowledged, despite training, many of the advisory group found it difficult to score objectively, ‘without letting their preconceptions and views influence their scores’(Heinrich 2007, p. 10). It should be acknowledged that many other global indexes, for example the Freedom Index, rely on the judgements of a few selected experts. In a paper published before the results of the Civicus CSI survey were published, but written after the methodology for that had been put in place, Finn Heinrich, the leader of the Civicus CSI team, proposed measuring civil society along two dimensions: the structural and cultural (Heinrich 2005). These approximate to the structural dimension and the values dimension in the original Civicus CSI. In this iteration, the structural dimension seeks to describe the make up of collective citizen action in terms of individual actions and organisational presence: using measures such as organisational membership, volunteering, demonstrating, and the existence of networks, of inter-organisational cooperation and the resource base of civil society organisations. He wants the cultural dimension to capture civil society as a ‘public sphere where a plurality of social norms are nurtured, practiced and promoted’(2005, p. 218). This would presumably include the way in which civil society organisations practice norms of participation and transparency. This still omits Edwards ‘good society’ dimension, though the values component could presumably encompass that. It also omits the environmental and impact dimensions (such as the extent to which a nations laws and their enforcement facilitate of discourage citizen action) which are important in both the CSI and the GCSI. It also continues to omit a manifestation of Cosmopolitan Civil Societies Journal, Vol.1, No.1, 2009 83 civil society as public sphere that many would consider important: the media of mass communication. At the very least, the media is a central component of the public sphere – the place where ideas and opinions are expressed (or excluded). The media is also the vehicle whereby actions and views expressed in other places – eg a demonstration outside parliament are selected and communicated to a wider public. Its controllers thus significantly shape the issues that are debated in the public sphere and the way they are debated. In conclusion, while neither of these efforts to measure and rate civil society comparatively are convincing, the goal is an important one. Those activists who claim an importance for ‘civil society’ are pointing to an important reality. But more effort needs to be devoted to discerning and obtaining agreement on the dimensions of civil society, so as to give it a clear identity within the plethora of other important (but vague) concepts such as democracy, governance and social capital. That process will require theorising but also recognition that measurement is important. If a concept cannot be measured, albeit with what are clearly understood and accepted to be proxy measures, then it is unlikely to survive in the world of fast changing intellectual fashions. Bibliography Anheier, H. 2004, Civil Society: Measurement, Evaluation, Policy, Earthscan, London. Brennan, G. & Castles, F.G. (eds) 2002, Australia Reshaped: 200 Years of Institutional Transformation, Cambridge University Press, Melbourne. Civicus 1997, The New Civic Atlas: Profiles of Civil Society in 60 Countries, Civicus, Washington. Cohen, J.L. & Arato, A. 1992, Civil Society and Political Theory, MIT Press, Cambridge, Mass. Cox, E. 1995, A Truly Civil Society, ABC Books, Sydney. De Oliveira, M.D. & Tandon, R. (eds) 1994, Citizens: Strengthening Global Civil Society, Civicus, Washington. Department of Economic and Social Affairs Statistics Division 2003, Handbook on Non-Profit Institutions in the System of National Accounts, United Nations, New York. Edwards, M. 2004, Civil Society, Polity Press, Cambridge. Ehrenberg, J. 1999, Civil Society: The Critical History of an Idea, New York University Press, New York. Heinrich, V.F. 2005, 'Studying Civil Society Across the World: Exploring the Thorny Issues of Conceptualisation and Measurement', Journal of Civil Society, vol. 1, no. 3, pp. 211-228. 84 Cosmopolitan Civil Societies Journal Vol.1, No.1, 2009 Heinrich, V.F. (ed.) 2007, Civicus Global Survey of the State of Civil Society, Volume 1 Country Profiles, vol. 1, Kumarian Press, Bloomfield Conn. Keene, J. 2003, Global Civil Society?, Cambridge University Press, Cambridge. Salamon, L.M., Anheier, H.K., List, R., Toepler, S., Sokolowski, S.W. & Associates 1999, Global Civil Society: Dimensions of the Nonprofit Sector, The Johns Hopkins Center for Civil Society Studies, Baltimore. Salamon, L.M. & Sokolowski, S.W. 2004, 'Measuring Civil Society: the Johns Hopkins Global Civil Society Index', in L.M. Salamon, S.W. Sokolowski & Associates (eds), Global Civil Society: Dimensions of the Nonprofit Sector, Volume 2, Kumarian Press, Bloomfield Conn.