TS I Eleven Ways to Critique an Article (Part II) MIKE METCALFE University a/South Australia Adelaide, South Australia, 5000 mike. metcalfe@unisa.edu.au Part I presented 6 ways to look at articles, six sets of criteria against which to think about the authors approach and underlying assumptions. In Part II another 5 perspectives are presented, the last of which pulls together the previous 10 perspectives. The perspectives are not intended to be a definitive set. They are all merely samples in need of improvement and modification. The implicit learning is that when you decide to question something, you ask yourself what underlying assumptions is driving your questions. For definitions and purpose see Part 1. Root-Metaphoric Perspective Morgan [1986], in his "Images of Organisations" , classified the literature on organi- sational theory by root metaphor, a concept discussed earlier by Pepper [1942]. His book suggests that, when people wrote about organisations, it was possible to identify certain implicit assumed "visions" that was driving their thinking. For example, some writers had the root metaphor of an organisation being like a "ma- chine" so they discuss employees as being like cogs in that machine. Another example, those in education often complain about, is the assumed root metaphor that knowledge is an object that lecturers pass over to students for money. Social scientists often use a root metaphor of their research subjects being less intelligent than them. Lakoffin Ortney [1992] provided the connection between metaphor and thinking. Therefore, it is suggested that one way to critique an article may be to try and spot root metaphor. There have been a flood of papers that appeared in the 1980' s and early 1990' s trying to identify root metaphors in various human activities but it is hard to provide exact direction on how to so this. All I can think to suggest is that you read through the article and write down any metaphors you can identify. From this list ask yourself if there is a collective name, theme or root metaphor that could be used to sum up the list. For example, I recently asked some fellow academics for metaphors of a literature review. Their suggestions are listed below. I have separated them into two as I felt there were two root metaphors. Group! These metaphors of a literature review have been allocated to the root metaphor of "investigation" (seeking out, discovery, detection, journey). Undertaking a litera- © Informal Logic Vol. 24, No.1 (2004) Teaching Supplement #10, pp.l-l0. TS 2 ture review is a search for (in) the unknown, going out to find out something you don't know. - map ("go where you like") - detective (seeking clues for new research directions) - beachcomber (sifting through others' castoffs) - prospector (seeking gems or nuggets). - lens (to focus readers), - pathfinders (layout an area and point to key sources) - sifting and winnowing - funnel (funnelling in) - signpost (to fmd your research amidst the literature) - concertina (narrowing and enlarging your search, like a concertinawindbag) - focus mechanism (you are talking about this, not that) - releasing the imagination Group 2 These metaphors of a literature review have been allocated to the root metaphor of "prosecution" Gustification, evidence, proof, argumentation). The literature is used to support a reasoned argument, justify a conjecture, or validate a hunch. - expert witnesses (courtroom evidence ... ). - currency (to buy credibility) - building blocks, - concrete foundation, - the history, - mirror (to see oneself in context) - requirements document - credibility filter (whose work do you draw on, what is the source of your ideas?) - due diligence (have you acknowledged others whose ideas you draw on?) - social courtesy (can you be relied on not to steal ideas?) - situational context (what conversation is this paper a part of?) - family tree - foundation (you are educated and building on a tradition) - puzzle (your idea fits into this larger puzzle) Metaphor Questions: Having listed the metaphors in the article ask yourself the following questions about the article in order to find a root metaphor that suits you. ... People: - Are the people in the article treated like "things" that need to be organised or as sources of knowledge? - Is any human activity treated as though it occurred in a linear sequence (from A to B to C) rather than in a recursive, or interactive one? - Are the suppliers of capital assumed to be superior to people investing their time (labour)? - Is gender considered relevant? - Does the article take a personal perspective or organisational one? - Is technology assumed superior to humans? Organisations - Are any organisations presented as machine like? - Are any organisations similar to adapting, evolving organisms interacting with their environment? - Are any organisations liberating or oppressive to stakeholders? - Is there any indication of the dependency or organisations on the broader community? - Are any organisations depicted as flows of information rather than people interacting? - Are the interpersonal and/or interdepartmental power issues made explicit? - Is efficiency and effectiveness a communal problem, not related to the desires of particular stakeholders? - Is Government treated as more powerful than the market place? Things TS 3 - Do any machines have gender, or are they analogous to any body parts? - Do problems exist separately from the people who have the problem? - Is information and/or education treated as a commodity? - Is equal access to information assumed? Dialectic Perspective The word "dialectic" is being used here in the same way it is used in the organisa- tional change literature [Nielsen, 1996; Mason 1996; Morgan 1986] to refer to "change that emerges from the interplay of conflict and among differences and affirmation of areas of agreement". Nielsen [1996] identifies five types of dialectic processes which he explains using terms like "iteration", "spiralling", "up building", and "trans- formation" which can be involved when there is a social reconstruction of human activities. He argues (in a dialectic with the reader?) that these change processes differ in terms of emotionality, they do not have to be quarrelsome, aggressive or conflictive but they can be. Identifying these types of dialectic can form a further method of critiquing articles. TS4 In "interaction dialectic", an idea (conjecture) is constructively refuted with the intent of developing an improved idea. The action science dialectic is analogous to Argyris and Schon's [1983] idea of experiment, reflect, and experiment learning loops. The up-building dialectic is similar to the "not invented here syndrome" where one human group develops a culture of how change should occur that an outsider does not respect. There is then tension between these two groups. For strategic dialectic, Nielsen draws on the work of Mason [1969] who suggested that alternative plans of the future can be used to develop a synthesis or improved third plan. The transformation dialectic is about the underlying social and political forces between group vying for resources. It is hard to mention this version of "dialectic" process without mentioning the hugely influential work of Karl Marx. Sowell [1985], a Marxist economist, argues that their inquiry approach was to consider the "underlying dialectic forces". In their case, this was mainly the underlying political and social forces associated with the class struggle. Nielsen identifies more "localised" struggles that go between participants of human groups engaged in a common activity. For article critique, this can be translated to suggest you think about the underlying tensions inherent in the situation outlined by the article. Dialectic Questions Ask yourself the following questions about the article. Iterative Dialectic - Does the author systematically question the article's conclusion? - Is the author suitably suspicious of his or her own conclusion? - Is there a specific perspective guiding this suspicion? - Are improvements to the ideas under suspicion being suggested? Action-Learning - Was there any action undertaken simply as a lets 'try it and see what happens'? - Was any empirical research seen as a learning-from-doing exercise? - Was there a full and systematically post-mortem of all actions? - Was there any response to unanticipated results? If so, did it lead to learning? Up-building -Is there any suggestion of tradition or culture about how the article was developed? - Was there any consideration of how "radicals" or "a devil's advocate" would have responded to the activities depicted in the article? - Have all the possible responses to the article been given a voice? Strategic - Does the article set up any sort of dialectic between two alternatives? - Do they get equal consideration? - Can you think of an alternative that might (also) be set to what is used in the article? - Is there any evidence of the article trying to learn by setting up an interaction between two alternatives? Transformation - Can you identify any underlying tensions between human groups vying for resources? Evolution-Metaphor Perspective TS 5 This perspective draws on the metaphor of an article being a species, a biological entity. More specifically, an article is to be thought of as the reproductive young of some species; something's baby. The perspective is inspired by the work of Darwin, Dawkins [1989] and Dennett [1995]. The basic tenet of evolution is that the young are a random genetic variation on the genetic possibilities of the parents. The environment at the time of its growing to maturity determines whether the variation is able to reproduce its particular attributes to another generation. In most species, many more are born than reproduce. The appropriate level of analysis is the gene pool or the survival of the species not the individual. Birth and death is seen as being essential to improving adaptability to the environment. The article can be thought of as the newborn. The parents being the major influences on the author, the genus being the school of thought the article comes from. The degree of variation is how different the article is from the parents, who mayor may not admire their offspring's features, but the environment will decide success, whether later authors are influenced by or cite it. Publication may be analogous to birth; working papers and drafts can be seen as failures to reach birth and the environment the audience. Some species can alter their environment in order to survive, others cannot. So, when critiquing an article, examine the references etc. Evolution Questions Ask yourself the following questions about the article. - From the references can you identify the genus of the paper? - From the references can you identify the parents? - Would the parents have admired this article or not? - Checks the citations index, do later authors use this article? - Were the authors influential enough to alter their environment? - Would the environment in which this article was published be hostile, friendly or manipulated by the article? - What environment might have treated the article very differently? Power Perspective Boulding [1954] talks of the three faces of interpersonal power: economic, destruc- tive and integrative. Economic power comes from being able to allocate resources, destructive power comes from being willing to not participate in some activity, and TS 6 integrative power comes from manipulating the human need to belong to social groups. Critical Social Theory [Alvesson and Skodberg, 2000] focuses on institu- tional power. Members of institutions have power from their formal positions. Journal editors accept and reject certain research methods, politicians have the power to influence legislation, the police from having to process law-breakers, doctors by having patients seen as victims, scientists by determining what is good knowledge and teachers by claiming to be more knowledgeable than their student. Markus [1986] identifies power from technical knowledge that can be extended to knowledge over regulations and processes (bureaucratic power). The feminist movement has been very effective in increasing awareness of the implicit power given to certain groups through language, and Kuhn talks of the power of para- digms inside which people act while not appreciating many of their own assump- tions. These perspectives can be used to critique articles: look for the implicit power and persuasion in an article both between the author and the reader but also be- tween any participants in the article. Be emancipatory: help the disadvantaged to see the causes of their stress, look behind the curtain of institutionalised power, assume there are no tall poppies, but rather some groups of people exploiting another group of people. Power Questions Ask yourself the following questions about the article. - Who determined that the topic of the article is important? - Do the authors use their position in an organisation, or superior insight, to convince rather than using reasoning or empirical data? - Is the journal pushing any particular topics of research or research methods? - Is science assumed to be the only reliable source of knowledge? - Are all contributors to the article's conclusion acknowledged? - Does the field of study have a small group of' experts' who gatekeep on journal article acceptances? - Are there any rival journals that intentionally take an alternative approach? Within the Article - Does the article reveal any struggle for resources from the point of view of all those involved? - Does every point of view get a voice? - Is the language exclusive, acting as a barrier to outsiders and the application of common experience? - Are there any appeals to authority? - Are references justified in terms of what the author's studies or on the basis of quality of journal? - Are all the counter arguments fully explored? - How would low-income people view the article and its contents? TS 7 Fuller Argumentation Perspective The author has a dominant perspective that argument is the basis of human com- munication, the creation and testing of knowledge. This perspective has been well argued by Pereleman [1969], van Emmeren [1987], Walton [1998] and Rehg [1999]. Moreover, I need a way to wrap up all the diverse critique perspectives presented in this essay! Therefore, this last section will attempt to combine all the perspectives presented. The umbrella perspective will be a simple argument, with the others providing detail to expand this critique perspective. Argumentation Questions Arguers - Who are the authors? - What is the basis of their expertise? - What else have they published? Audience - What is the intended audience? - What audience would it not work for? - Is the paper explicitly persuasive to this audience? Object or Thing Under Study - What is the object or 'thing' the article is studying? - Did they discuss, define this 'thing' in system thinking terms; its boundaries, relationships (context), how it changes over time, and its purpose? Concern - What is the authors' concern (perspective, lens, frame) on the objects under study? - Did 'they scope this concern or perspective? - What is the problem they seek to address? Argument - What is the explicit or implicit argument (conclusion) of the article, was it stated upfront? - What was their fresh insight, i.e. was the argument novel? - Can you draw a picture of the article? Definitions - Are all key words well defined (described, bounded, scoped)? Motivation - Is the importance of the study fully explained? - What is the purpose of the authors writing the article? Evidence - How were they (implicitly) defining knowledge? - How did they know things? TS 8 - What evidence is brought to support the argument/conclusion? - Was this evidence convincing, novel, insightful? - Was the counter argument fully considered? - Where there any empirical data if so why, if not why not? Collecting Empirics as Learning - Were they treating the act of collecting their empirical data as an exercise in precise measurement to produce objective knowledge or as action to learn? - Did they build and test something, what and how? - Was the testing integrated in the building and evaluated using learning loops and multiple stakeholders' perspectives? - Was the evaluation convincing? - Were all stakeholders given a voice? - Were they searching for 'the truth' or a range of perspectives? Implications - Were the implications (the so-what) of accepting their argument/ conclusion fully explained? - What good knowledge was created? - What actionable knowledge (rules of thumb, recommendations) was created to aid future decision-making? Lessons Learnt - What did you learn from the article? - What purpose do you give to the article, how could you use it in your own studies? An Active Critique The critique perspectives presented so far have tried to encourage the seeking of some insight about an article through questioning it. A much more demanding approach is to draft your own argument in response to an article. This was touched upon in the introduction as something that could follow the questioning process. It is not easy to provide very detailed guidelines on how to go about drafting a response-argument, but some very general comments may help. Having asked yourself some questions about the article, attempt to think of some insight. For example, that it is based on some underlying assumption that can usefully be made explicit or that it presents a different perspective to some other article you have read. Having thought of the insight put it in the form of a claim (one line argument or conclusion), reflect upon whether you feel able to justify it with evidence. Assume a cynical audience. You might also reflect on your possible argument using Popper's [1963] advice of asking yourself if it is falsifiable. Is there a possible counter argument to yours, which you can anticipate and respond to? Also does your argument have any surprise value, is it novel? Having set your argument, use the article and any other reference material you have to write an essay of your own. The length of this essay is a matter between TS 9 you and your audience. It may help to draft a diagram of your supporting evidence, any possible counter arguments and your response to those. Now remember that your essay is now the article that others will critique so ask of it the type of ques- tions listed in the earlier chapters of this book. In Summation These two parts have tried to illustrate how critique questions, including critiquing your own writing, is so much more considered if you know where your questions are coming from. Please do not see the perspectives provided as rigid, bounded steps but rather as illustrative of critiquing from a explicit perspective. I find it a useful exercise for those embarking on a large thesis to first identify the perspective inherent in the argument (conclusion) of their thesis (see the multiple perspective section) and to use this to critique the literature. For example if a thesis is, "That the Islam needs a 20th Century interpretation", then the perspective (on Islam) is "a 20th century interpretation". Having researched (scoped, defined, contrasted) what that means, then it can be used to critique all 'articles'. It is even often possible to be reflexive and use that perspective back on that perspective. Critical thinking, if it is to provide real insight, is a skill that needs to be improved through exercise, through a search for new depths. One perspective is often not enough, but the skill of separating them may well be good exercise for the brain. References Ackoff, R. (2000). Making a Difference, Systems Thinking/Systems Change. Retrieved 1217/05, from http://a2j.kentlaw.edu/Presentations/GirlsLink/index.html Argyris, C., & Schon, D.A. (1996). Organisational Learning fl. Boston, Massachusetts: Addison-Wesley. Arygris, C. (1996). Actionable Knowledge. Journal of Applied Behavioural Science, 32(4),390-406. Alvesson. M. & Skoldberg, K. (2000). Reflexive Methodology, Thousand Oaks, CA: Sage. Boulding, K. (1956). The Image: Knowledge in Life and Society. Ann Arbor: University of Michigan Press. Boulding, K .(1961). General Systems Theory: The Skeleton of Science. Management Science, 2 (3), 197-208. Bronte-Stewart, M. (1999). Regarding Rich Pictures as Tools for Communication in Information Systems Development. Computing and Information Systems, 6, 83- 102. Chalmers, A.F. (1982). What Is This Thing Called Science? Queensland: University of Queensland Press. Checkland, P. (2000). Soft Systems Methodology: A Thirty Year Retrospective. Systems Research and Behavioural Science, 17 (1), SII-S58. Churchman, C. W. (1971). The Design of Inquiring Systems. New York: Wiley. Churchman, C.W. (1979). The Systems Approach and Its Enemies. New York: Basic Books. TS 10 Cohen, H.F. (1994). The Scientific Revolution. Chicago: University of Chicago Press. Dawkins, R. (1989). The Selfish Gene. Oxford: Oxford University Press. Dennett, D. C. (1996). Darwin's Dangerous Idea. New York: Simon & Shuster. Dewey, J. (19lO). How We Think. Lexington: D.C. Heath. Dillon, 1. (1988). Questioning In Science, in Meyer, J. (Ed.) Questions And Questioning. Berlin: Walter de Gruyter & Co. Eemeren, van F. H., Grootendorst, R. & Kruiger, T. (1987). Handbook of Argumentation Theory. Dordrecht: Foris Publications. Einstein, A. (1995). Relativity: The Special and The General Theory. New York: Crown Publications. Haynes, J. (2000). Perspectival Thinking. New Zealand: OneBook and Co. Kuhn, T.S. (1970). The Structure of Scientific Revolutions. 2nd Ed. Chicago: University of Chicago Press. Lakoff, G. (1992). The Contemporary Theory Of Metaphor, in Ortney, A. (Ed.) Metaphor and Thought. Cambridge: Cambridge University Press. Linstone, H. (1984). Multiple Perspectives For Decision Making: Bridging the Gap Between Analysis and Action. New York: Elsevier Science Publishing Co. Linstone, H A. (1999). Decision Makingfor Technology Executives: Using Multiple Perspectives. Boston: Artech House Publishing. Markus, M. Lynne (1983). Power, Politics, and MIS Implementation. Communications of the ACM, 26(6), 430-444. Mason, R. O. (1969). A Dialectical Approach to Strategic Planning. Management Science, 15, B-403 - B-414. Mason, R.O., and Mitroff, I.I. (1981). Challenging Strategic Planning Assumption. New York: John Wiley and Sons. Mitroff, I., & Linstone, H. (1993). The Unbounded Mind. Oxford: Oxford University Press. Morgan, G., (1986). Images of Organisations. Thousand Oaks, CA: Sage Publications. Nielsen, R. P. (1996). Varieties Of Dialectic Change Processes. Journal of Management Inquiry, 5(4), 276-292. Perelman, Ch. & Olbrechts-Tyteca, L. (1969). The New Rhetoric: A Treatise On Argumentation. Notre Dame: University of Notre Dame. Pepper, S.c. (1967). Concept and Quality: A World Hypothesis. LaSalle: Open Court. Popper, K.R. (1963). Conjectures and Refutations. London: Routledge. Rehg, W. (1999). Argumentation Theory and the Philosophy of Science Since Kuhn. Inquiry, Summer. Schon, D. (1983). The Reflective Practioner. New York: Basic Books. Sowell, T. (1985). Marxism. London: Unwin. Ulrich, W. (1983), Critical Heuristics of Social Planning. New York: Wiley. Walton, D. (1998). The New Dialectic. Toronto: Toronto University Press. Weick, K.E. (1983). Managerial Thought In The Context Of Action, in Bennis, W., Mason, R.O., & Mitroff, I.I. (eds.), The Executive Mind. Hoboken, NJ: Jossey Bass.