







































1 

 

 

Stumbling on/in Evaluation Anthropology 

Review by Imola Püsök 

 

Evaluation: A Cultural Systems Approach  

by Mary Odell Butler 

Left Coast Press, 2015 

 

Culture and International History Series seems to promise a great deal to our understanding of 

international relations, and despite the fact that the editor did not try to include an analysis of culture or 

of the specificity of Western culture, the seventh volume of this series fits perfectly with the editor's 

intended goals. “Throughout my career I have told people that I landed on my feet as an anthropologist 

when I became an evaluator.” (p. 76) In the phrase that seems to be not only the summary of her life 

and career, but also the leitmotif of her book, Mary Odell Butler expresses all that Evaluation is about: 

the meeting of the two disciplines on one hand, and their entanglement with her career path. In its eight 

chapters, the book ambitiously aims to introduce evaluation to anthropologists, anthropology to 

evaluators, and to map the difficulties, controversies, challenges and possible rewards of what doing 

evaluation anthropology means. 

 

In her endeavor to fit together the two disciplines and introduce evaluation anthropology to beginners 

in either one of the two parts – evaluation or anthropology (p. 22) – the author gives all the details to 

understand the overarching picture. The eight chapters follow a clear order, where several threads 

(What is evaluation? What is anthropology? Who are the evaluators, and what do anthropologists do?) 

are woven into the re-emerging web of what evaluation anthropology is, how it can be done, and what 

important aspects to take into account, should one consider a career – academic or not – in or with 

evaluation anthropology. The “figure in the carpet” is re-emerging, after the author skillfully (albeit 

perhaps necessarily artificially) delimits the two approaches: the one of the anthropologist and the one 



2 

 

of the evaluator. The book, in its clear linear structure can, thus be separated into two parts. The first 

part, which can be summarized as “understanding evaluation and anthropology”, comprises the first 

four chapters. It is a theoretical-introductory part, with some overview of the history and paradigms of 

the two fields, careful definitions and explanations of how she uses and defines the core concepts, such 

as evaluation, ethnography, culture, community, system and contexts. It is admirable that in her writing 

Butler maintains a reflexive stance and allows for the many other possible approaches, while in the 

same time is able to provide a conceptual and theoretical framework for beginners to work with. She 

thus stresses the necessity for evaluations to be scientific, but also opens paths for young researchers to 

find their own methods and theoretical stances through learning and unlearning some of the 

disciplinary certainties anthropologists (or, in fact, evaluators) might be biased towards (p. 19).   

 

The next four chapters are within the framework of “how to do evaluation anthropology”, and discuss 

very practical methodological and ethical perspectives, while providing the reader not only with the 

know-how of doing research, but also that of building a career in evaluation anthropology. These 

general guidelines become especially lifelike through the author’s candid accounts of her own career 

experiences. 

 

Evaluation being defined as “a study that seeks to scientifically determine the value of something for 

some kind of utilization” (p. 25), and anthropology as the “study of culture and its role in articulating 

what humans do as members of societies” (p. 72), the overlap that evaluation anthropology becomes 

(p.71), is inevitably a discipline laden with ideas and relations of power. The two chapters on ethics and 

methodology raise important questions that anthropologists and evaluators have to engage with 

throughout their research. Whose interests are served? How do we protect vulnerable groups and 

people from being compromised by our research data? When do we have informed consent and how do 

we honor the trust of our informants? What are the conflicts between the necessity for scientific 

objectivity and negotiating through the needs and wants of stakeholders or participants? How do we 

deal with these? Who decides about how to use the data and about the consequences of research? 

Perhaps one of the most important statements in the book is that ethics “are not common sense. It 

requires training, experience, and judgment to make ethical decisions” (p. 108). Indeed, our 

professional training is (still) lacking in engaging discussions, seminars, and informative courses with 

regards to ethical guidelines, something the author also laments. The need for such discussions is, 

however, there – demonstrated by the multitude of online forums and conference discussions. One of 



3 

 

the greatest merits of Evaluation is the way it further invites conversation about the relationship of the 

discipline and the solving of social problems, of evaluators and stakeholders, of decision-making and 

scientific research. Good research in evaluation, as in anthropology, depends on building trust with 

stakeholders (p. 12), and evaluators “have ethical obligations to stakeholders seeking scientific ‘truth’ 

or verisimilitude” (Harklau and Norwood 2005, 279). It is, therefore, vital to reflect on how 

subjectivities change in the evaluation process, how and with whom power-relations are negotiated, 

and the role and influence of the evaluator-anthropologist in decision-making processes, considering 

that “program evaluators hold the power to affect the very nature and future of the phenomena they 

investigate” (ibid., 278). 

 

Considering all this, why evaluation anthropology, though? What new does it bring to the table? The 

author herself questions in the introduction the necessity to create a subfield by merging the two 

disciplines. Why not just evaluation? Or why not anthropology? Her answer is, that evaluation 

anthropology is stronger than its parts (p. 18), as academics and practitioners from both disciplines 

come together to mutually share and learn from each other’s expertise. Qualitative and quantitative data 

are both necessary for research that is on the one hand scientific enough to serve as the base for policy-

change and to influence stakeholders in positions of power to make decisions in the right directions. On 

the other hand, such research is attentive to the sensibilities of wider social contexts and advocates for 

the needs of vulnerable groups. It is in this sense that there is a necessity for the two disciplines to 

converge, and bring ethnography as much as possible to the forefront of research. Constantly being in 

positions of negotiation not only with vulnerable research participants, but with powerful stakeholders 

as well, evaluation anthropology becomes one of the rare subfields that not only hold the possibility to 

investigate social problems and the effects of government and NGO programs on marginalized 

populations, but it also has a “playing area” that allows for researchers to “study up” (Hopson citing 

Nader 2005, 293), i.e. to study and ask questions of the groups who hold power, who make the all-

important decisions. In reading Butler’s work, one gets the feeling that this tool and focus of evaluation 

anthropology is not yet applied. Practicing evaluators, and in most cases academic anthropologists also, 

depend on governmental or non-profit agencies for their research. In Evaluation, the author somewhat 

balances this moral dilemma in expressing her views on ethical best practice. Given, that “our work is 

neither scientifically, nor politically neutral” (Harklau and Norwood 2005, 281), picking up the thread 

of conversation on this matter is necessary in order for future practice not to end up compromising the 

needs of vulnerable groups, or indeed, researchers. 



4 

 

 

Compared to the 2005 NAPA Bulletin special issue, Creating Evaluation Anthropology: Introducing an 

Emerging Subfield, which she co-edited with Jaqueline Copeland-Carson, in Evaluation Butler focuses 

somewhat more on program evaluation – the area she has most experience in. In both cases, however, 

we only read about evaluation and evaluation anthropology done in the US or as commissioned in the 

US. As it turns out, the reader learns more about American society, programs and policy-making, than 

about the applicability and usefulness of doing evaluation anthropology to assess government and civil 

programs anywhere. The strong focus on US evaluation anthropology alone leaves the reader wanting 

more, even though this is a somewhat understandable bias, given the history and context of emergence 

of the subdiscipline. However, is there really no need for such research, or are there no similar 

endeavors elsewhere? Is evaluation anthropology not applicable outside the American social and 

political contexts? Perhaps being more concerned with ethics and methodology, rather than a complete 

review of the theories and history of the discipline, there is no room for such detours in Evaluation, it 

would, however, have been interesting if the book entered into conversations on this level too. 

 

Mary Odell Butler proposes her book to be “about how to link evaluation and anthropology into a 

dynamic and flexible approach that brings out the vitally human nature of the ways people organize 

themselves to accomplish their goals” (p. 9). While an ambitious objective, she does manage to shed 

new light on how the two disciplines can inform and challenge each other, even subtly suggesting that 

evaluation anthropology is a necessary, but necessarily evolving transdisciplinary process. She does not 

intend to give a complete theoretical overview of the two parent-disciplines, or their hybrid, nor a 

complete methodological or ethical guideline, yet, through mapping possible approaches and career 

paths, and – often painfully honestly – narrating her own experiences, she provides the reader with the 

necessary starting points to understand anthropological and evaluation theory, and to develop their own 

methodologies. In doing so, she also advises about very practical issues, such as agencies, 

organizations, websites and tips on how to move on in their career. All in all, Evaluation is both an 

interesting and a useful read, where one is able to enjoy the personal narratives, gain an understanding 

of the basic conceptual tools and frameworks, and come back to the useful links and tips whenever they 

are needed. 

 

 

 



5 

 

References Cited 

Butler, Mary Odell, and Jaqueline Copeland-Carson (eds.) 2005.  Creating Evaluation Anthropology: 

 Introducing an Emerging Subfield. NAPA Bulletin 24 (1). Berkeley, CA: University of 

 California Press. 

Harklau, Linda, and Rachel Norwood. 2005. Negotiating Researcher Roles in Ethnographic Program 

 Evaluation: A Postmodern Lens. Anthropology and Education Quarterly, 36 (3), pp. 278-288. 

Hopson, Rodney K. 2005. Reinventing Evaluation. Anthropology and Education Quarterly, 36 (3), pp. 

 289-295. 

 

Imola Püsök graduated from the University of Aberdeen and the Babeș-Bolyai University in Cluj 

Napoca. Her areas of interest are educational anthropology, ecological anthropology and anthropology 

of post-socialism. 

 

 

© 2017 Imola Püsök 


