Research Quality Assessment and Planning
Journals.

IJPP ­ Italian Journal of Planning Practice 96Vol. II, issue 2 ­ 2012

Bruno Zanon

ISSN: 2239­267X

The Italian Perspective

Via Mesiano, 77 ­ 38123 ­ Trento, Italy ­ bruno.zanon@unitn.it
Dipartimento di Ingegneria Civile e Ambientale ­ Università degli Studi di Trento

ABSTRACT
Assessment of research products is a crucial issue for universities and
research institutions faced with internationalization and competition.
Disciplines are reacting differently to this challenge, and planning, in
its various forms – from urban design to process­oriented sectors – is
under strain because the increasingly common assessment procedures
based on the number of articles published in ranked journals and on
citation data are not generally accepted.
The reputation of journals, the impact of publications, and the profiles
of scholars are increasingly defined by means of indexes such as
impact factor and citations counts, but these metrics are questioned
because they do not take account of all journals and magazines – in
particular those published in languages other than English – and they
do not consider teaching and other activities typical of academics and
which have a real impact on planning practices at the local level.

Associate Professor



IJPP ­ Italian Journal of Planning Practice 97Vol. II, issue 2 ­ 2012

CHANGING METRICS FOR ASSESSING ACADEMIC PLANNING RESEARCH
A heated debate is in progress in different countries and in diverse journals
on the changing metrics used to assess planning research products. It relates
to the general adoption in academia of formalized methods to evaluate
articles and rank journals by using internationally recognized criteria and
procedures. Scientific disciplines, social sciences and the humanities have
diverse traditions and different levels of international integration. But a
common trend is emerging in the selection of articles to be published and the
evaluation of their impact. ‘Citation index’ and ‘impact factor’ have become
the buzzwords of research assessment.
The reasons for the change are clear: to improve the quality of research by
adopting methods which are comparable across disciplinary sectors and
consolidated at the international level. In a competitive scientific
environment, universities and research institutions must use sound criteria
and reliable indicators for personnel selection and promotion, and to assess
the results of research projects. In particular, this means considering, among
the criteria used to rank institutions and for academic career progression, the
number of articles published in authoritative journals fulfilling quality
requirements, and considered of interest by the scientific community (ones
that are cited, in other words). The key assessment method is peer­review,
which is the selection of papers by academics and researchers working in the
same or a similar sector. The purpose of peer­review is to choose
contributions that present the results of research work, are based on a sound
methodology, and are attuned to the international disciplinary debate. This is
not a generic procedure, but rather a formalized process which must be
appropriately managed by a well­organized publishing system which makes

In Italy the discussion is particularly heated because assessment
procedures are recent, the disciplinary community is not used to
publishing in ranked international journals, and the Italian literature
is not attuned to the international quality criteria.
The paper reviews the recent debate on planning journals and
research assessment. It focuses on the Italian case from the
perspective of improving current practices.

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 98Vol. II, issue 2 ­ 2012

it possible, once the article has been published, to count how many times it
is cited, and thus define its “impact”.
This change has had important effects, ranging from the growing use of
English as the dominant language to the increasing power of a small number
of publishers, to the debatable role of certification agencies providing data
and indicators, and of the criteria and methods used. Some disciplines,
planning included, are under strain because this change means replacing
traditional ways of doing research, assessing results, evaluating articles and
publications, and organising career progression procedures. In particular,
many of the journals and magazines, as well as books, in which it is usual to
publish in countries such as Italy, do not meet the new requirements, and the
older generations of scholars run the risk of being considered unfit for their
current positions.
The specificities of planning – consisting of inter­disciplinarity and action
orientation – are constantly cited and used as arguments to claim that
research in this field is mostly coincident with practical experiences and
projects (or even professional products) and that publications are of a
different nature from those of scientific disciplines. But new trends are
consolidating in this field as well.
The procedure followed to evaluate the impact of articles – in terms of the
reputation of the journal in which they are published, and of the citations
generated – is much debated, in particular as regards the appropriateness of
the criteria and indicators used and the role of the agencies and companies
that certify journals and provide citations and impact factors data. Thomson
Reuters’ ISI Web of Knowledge/Web of Science, founded by Emile Garfield
(Garfield, 2006) is the most famous of them, but others that are well known
are Elsevier’s SciVerse Scopus, Scimago (using Scopus data) and Publish or
Perish, based on the Google Scholar database, which is more inclusive than
the others (Harzing, 2008).
Besides the discussion on the indicators used, there are side­effects to be
considered. Old­fashioned paper journals (and many books and local
publications), regardless of their intrinsic quality, are excluded by
certification mechanisms, and this affects their influence. Moreover, a digital
divide is being created by the high subscription rates for electronic journals
and citation databases which are not always affordable for institutions and

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 99Vol. II, issue 2 ­ 2012

individuals. Furthermore, a large body of “grey literature”, as well as the
growing Open Access literature – which have a major impact at the local
level or through diffusion via the Internet – are not considered by formalized
assessment indicators.
These developments are taking place within a more general change affecting
research and university institutions. At European level, there is increasing
homogenization among universities due to the Bologna process and the
exchange of students and scholars, while access to EU research funds and
growing competition tend to disseminate similar methods and to make
institutions in different countries more similar (with drawbacks which will
be commented on later in light of Klaus Kunzmann’s warnings). Moreover,
results in terms of impact factors and access to international research funds
differentiate between research and teaching universities, although in many
countries there is no official distinction. This is the case of Italy, where – in
principle – all universities are equal, and where some disciplinary groups
–planning included – are reluctant to use formalized assessment methods, on
the grounds that they are unable accurately to describe the quality of
research and the qualification of scholars.
This article, after analysis of the emerging issues relating to the assessment
of research products in the light of the recent international literature,
conducts a short survey of Italian journals and the debate ongoing in Italy.
The aim is not to discuss indicators and ranking methods, let alone propose
new ones, but rather to report on the discussion concerning the evaluation of
planning research and the qualification of Italian journals, drawing some
conclusions with a view to improving current practices.
AN EVOLVING RESEARCH ENVIRONMENT
The growing competition in education and research is well represented by
the rankings periodically reported by the media and stating which are the
best institutions in the world. As usual, together with the use of common
criteria, diverse aspects are considered by the specific indicators adopted by
rankings such as Quacquarelli Symonds, Academic Ranking of World
Universities (ARWU – by the Shanghai Jiao Tong University), Times Higher
Education World University Rankings, Webometrics and similar. Most

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 100Vol. II, issue 2 ­ 2012

university rankings are primarily based on research productivity data and
therefore they furnish an “oversimplified picture” of university missions, as
stressed by a recent survey conducted by the European University
Association (Rauhvargers, 2011, p.7) which provides deep insights into the
diverse methodologies adopted, together with recommendations on how the
results should be used. Some years ago in Italy, the La Repubblica
newspaper started to publish a ranking compiled by the Censis research
institute. This gave rise to a debate and stimulated other inquiries using
different indicators (Il Sole 24 Ore, Campus, Vision). Depending on the
aspects considered more important, diverse results are produced; some say
that depending on the results wanted, different aspects are considered.
Assessment has become a routine exercise in the UK (government­
sponsored) and the USA (association­based) (Tewdwr­Jones, 2005; Stiftel et
al, 2004a; Punter and Campbell, 2009; Stiftel et al. 2009), and institutions
which perform well in rankings are quick to display their scores in
promotional materials, whereas when results are based on controversial
measures, harsh criticisms are made (Stiftel et al., 2004a, p.6; Stiftel et al.,
2009, p. 323). Planning schools in any case require specific kinds of
assessment, and in the USA recommendations have been made to consider
not only research but also “design, outreach and teaching”, thus respecting
the specific mission of each university (Stiftel et al., 2009, p. 330). The
outcomes are indeed important, because in recent years both academics and
potential students have made their decisions also on the basis of ranking data
(Stiftel et al., 2004a, p.6).
It is accepted that there is no single way to rank research and educational
institutions, because of the intrinsic differences among the activities and
products considered, and scores are generally “a synthesis of a multitude of
parameters attached to measurable variables” (Geraci and Degli Esposti,
2011, p. 668). Still unresolved, however, is “the problem of which variables
should be measured and how” (Geraci and Degli Esposti, 2011, p. 668).
The complexity of the assessment of institutions and research is
demonstrated by the literature generated by it – which includes specialized
journals (such as Scientometrics) – as well as by dedicated research strands
(at the University of Pescara, Italy, there is a PhD course in research
assessment). Also disciplinary publications devote increasing attention to

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice Vol. II, issue 2 ­ 2012

assessment, showing the extent to which different sectors are worried by the
application of the new methods. These procedures are not new in scientific
disciplines, but they are a relative novelty for the humanities and some of
the social sciences. Planning is a discipline whose boundaries are loosely
defined, and which is influenced on the one hand by urban design and the
methods and traditions of architecture – focused on projects – and on the
other by a variety of social science methods and process­oriented theories
and practices. Moreover, urban planning publications are often oriented to
practitioners, and they contain illustrated presentations of projects and plans
selected by the editors, not papers submitted by authors and undergoing a
peer­review process.
In Italy a sharp distinction is traditionally drawn among hard sciences, social
sciences, and the humanities, and the assessment of academic research usually
uses this distinction (Abramo et al., 2011). Nevertheless, there is a strong push
for the introduction of formalized methods. In particular, new criteria for
academic career progression – in planning as well – have been recently issued,
and they require the use of indicators (CUN, 2011; ANVUR, 2011a).
Concerning scientific literature, in recent years a major change has taken
place in the way that publications are produced, consulted and cited, owing
to the diffusion of electronic journals and the availability online of huge
amounts of documents and bibliographic data. The access to all the relevant
published literature at websites where searches can be made on the basis of
key­words has changed both bibliographic research and the nature and
organization of journals – especially in the way that papers are written and
quoted: “This greater ease of access and use is leading to increased rates of
citation to prior work” (Stiftel and Mogg, 2007, p. 73). In fact, the electronic
version of a journal is currently the most important one. It activates cross­
referral links and supports all the statistics connected to citation and impact
factor indexes. Planning journals are also involved in this change, and nearly
all of the most prestigious ones are currently on­line (Stiftel and Mogg,
2007, pp. 71­72).
The use of rankings and indicators is facilitated by the growing
homogenization among European educational institutions and academic
courses. This trend, affirms Klaus Kunzmann (2004) in a paper echoed by a
number of subsequent articles, reflects the Anglo­American system and is

Zanon ­ Research Quality Assessment and Planning Journals

101



IJPP ­ Italian Journal of Planning Practice 102Vol. II, issue 2 ­ 2012

undoubtedly positive. But there is the risk that some will win (those who
comply with such academic “rules and rituals”), and some will lose: those
who “cannot easily adapt to the global model, or who refuse to throw 100 or
more years of local academic tradition over board” (Kunzmann, 2004, p.1).
The consequences may be severe, changing long­established traditions well
rooted in institutional systems (with a key role of the public) in cultural
environments and socio­economic milieus. In particular, Kunzmann claims
that planning schools will be squeezed into smaller units and shorter
courses, thus endangering the teaching specificity based on a balance
between theory and practice, and favouring lectures and seminars. It should
be added that the strengthening of research assessment procedures
(discouraging low­profile activities) will exacerbate such effects.
A key aspect, as said, regards the diffusion of the use of English, which has
become the lingua franca for the academic world (Kunzmann, 2004). This
does not simply concern the use of a neutral communication medium; it also
gives more space to the approaches, views and experiences of the English­
speaking world. It is a problem that relates, more or less, to all disciplines,
but some are more sensitive to it because they are connected to local
conditions and mindsets, specific legal frameworks, communication
methods and media, etc. Planning is certainly affected by this trend, together
with other disciplines, such as human geography (Schuermans et al., 2009;
Derudder, 2010). For Kunzmann, the use of English will have consequences
on universities (forced to offer courses in that language) and on the literature
market, which will be divided into two tiers: the upper one formed by a small
number of English language publishers, the second one by regional publishers,
with a lower impact (Kunzmann, 2004, p. 6). Planners wanting to be up­to­
date and to build a career must refer to papers in English and publish in that
language, with obvious consequences on “the relationship to the regional
socio­political environment, such as alienation from local milieus, little
influence on local development” (Kunzmann, 2004, p. 6). As a consequence,
local issues run the risk of being neglected by the academic world and of
being left to practitioners, often not so dynamic and less well prepared.
In the case of Geography (Stiftel and Mukhopadhyay, 2007), the contrasting
issues regard the cultural as well as the economic hegemony of the Anglo­
American countries, on the one hand, and opportunities to facilitate

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 103Vol. II, issue 2 ­ 2012

international discussion on the other. Many considerations can be made on
economic and political power, as well as cultural influences, over weaker
countries, but also a pragmatic attitude should be encouraged: the use of a
common language facilitates communication. Stiftel and Mukhopadhyay
(2007, p. 545) have stressed that “English is the global language of scientific
communication and English­language publishers have positioned themselves
as the arbiters of scientific progress and the profiteers of knowledge
distribution”. Indeed, the use of English has created an editorial system
centred on Anglo­American institutions and scholars, with a partial coverage
of the rest of the world. Currently, most articles are written by English
mother­tongue scholars. As far as planning journals are concerned, only a
small number of those indexed in the ISI’s Web of Science are not printed in
English, and citations of papers written in languages other than English
(LOTE) are few.
Among the drawbacks of monolingual communication especially to be
stressed is the risk that important points of view and experiences will be
excluded because “there is not enough cross­movement of ideas among the
planning literatures in different languages” (Stiftel and Mukhopadhyay,
2007, p. 566). The prevalent use of English also produces unexpected
effects. Small countries and the developing ones have long been oriented
towards using English in academic communication, while other countries
with long cultural and scientific traditions have resisted, thus running the
risk of lagging behind the newcomers in the global competition.
PLANNING PRACTICE AND PLANNING RESEARCH
Planning is an operational discipline and its main goal is “the development
of practical outputs” (Goldstein and Carmin, 2006, p. 68) in order to provide
benefits for the communities involved. From this perspective, the discipline
is engaged in providing operational methods and tools, and it accompanies
political actions. But theoretical, explanatory knowledge is also needed to
build effective know­how and to validate the outputs. It has been noted that
“planning scholarship has a history of drawing from many disciplines” and
this interdisciplinarity has produced a variety of “approaches to problem
definition and solution generation but also to notable ontological,

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 104Vol. II, issue 2 ­ 2012

epistemological, and methodological variations in planning scholarship”
(Goldstein and Carmin, 2006, p. 68).
The composite nature of planning is a long­debated issue in the disciplinary
literature, and only some recent references to the effects in terms of
publications are considered here. “While some consider this
‘interdisciplinary’ basis as a weakness ­ making it difficult for planners to
know exactly what belongs to planning ­ others see it as a key strength”
(Davoudi, 2010, p. 33). In fact, planning shares methods and tools with
social sciences, on the one hand, and with more technical sectors – in
particular urban design – on the other (Goldstein and Carmin, 2006, p. 67).
It concerns policies and practices, and its scientific bases are related to
sociology, economics, environmental sciences, law and administration,
geography and GIS. It is therefore difficult to define the homogeneous areas
within which to make comparisons, because a variety of publishing
traditions are present, including those coherent with scientific disciplines.
It is not only planning that has two sides: social science and design, but also
the progress of urban design can have different bases: “there are two primary
locations for innovation in urban design: practice; and university­based
research innovations” (Forsyth, 2007a, p. 461). In particular, “In architecture
schools, success is based more on success in the design world ­
competitions, exhibitions ­ than on peer­reviewed publications and citations”
(Anselin et al., 2011, p. 199). In order to counter competition, academics
must strengthen their positions, also making use of methods for measuring
productivity, in order to improve research effectiveness and to increase
funding (Forsyth, 2007a, p. 461).
Planning is also intrinsically locally connected, so that a great deal of low­
level literature is produced (mostly of a descriptive kind: experiences, plans,
projects) and it is not possible to consider only high­profile products as
important. Academic career progression in some countries – and this is the
case of Italy – was long based also on professional products. Much has
changed in recent decades, also in Italy, differentiating between physical
planning and decision­making, between practice and research. This confirms
that “[u]rban design is not equivalent to urban research” (Forsyth, 2007a, p.
467) and that research products must undergo assessment procedures.
In the 1960s, planning “became firmly ensconced as a public sector activity

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 105Vol. II, issue 2 ­ 2012

that drew increasingly on formal social­scientific theories and rational
decision­making to solve problems” (Forsyth, 2007a, p. 464). This is the
basis on which the discipline has developed, requiring specific research and
debate on its background, goals and methods, assessing contributions, and
accumulating knowledge (Forsyth, 2007a, p. 466, table 1). In short, the
multidisciplinarity of planning requires more, not less, methodological
reflection and entails strict assessment of hypotheses and proposals. It thus
pushes towards formalized research methods and an appropriate assessment
of contributions (Forsyth, 2007a, 2007b). The result is the progressive
consolidation of specific approaches and methods. For academic institutions
it is therefore important – and often vital in terms of economic support ­ the
“reflection with practice”, that is the engagement of scholars in applied
experiences, which allow to develop a “reflection on practice” (Balducci and
Bertolini, 2007; Balducci, 2007).
A survey on the articles published in the Journal of the American Planners
Association – JAPA – found evidence “that academic planning is developing
along the lines of a compact scientific discipline” (Goldstein and Carmin,
2006, p. 74). Another survey of a large number of journals confirmed that
the discipline is not “fragmented among different areas of specialization and
that there is a common body of journal literature that binds us together as a
scholarly community” (Goldstein and Maier, 2010, pp. 69). These
conclusions, which cannot be thoroughly extended to the European case, as
it will be stressed below on the basis of the analysis of planning journals,
concern in particular the specific areas regarding the decision process, while
the connections with a number of hard sciences have recently multiplied,
thus requiring the use of formal methods, both qualitative and quantitative.
Apart from transport, which is traditionally linked with planning, issues such
as environment protection, ecosystem defence, climate change effects, as
well as sustainable development, require the use of measures and indicators,
and the application of techniques like GIS. All these disciplinary fields share
the same international standards and publishing traditions. As a
consequence, important sectors of planning research are more attuned to
scientific paper writing.
However, there are differences in planning trends and practices among
countries. In recent years, planners in the USA and the UK “have seen

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 106Vol. II, issue 2 ­ 2012

themselves as applied social scientists” (Stiftel and Mukhopadhyay, 2007, p.
565), after a struggle with urban design and architecture (Anselin et al.,
2011), whereas “[d]esign approaches to planning appear to be more common
in continental European schools and in schools in developing regions”.
These two approaches “have clear differences in the frequency and pattern
of publication that results” (Stiftel Mukhopadhyay, 2007, p. 565).
In Italy, the delivery of planning courses mainly at the schools of
Architecture and Civil Engineering and a professional practice still
dominated by such technical practitioners keep the discipline connected to
urban design, while research is oriented to physical planning or the planning
process, but mainly from a legal point of view. In recent decades, however,
much has changed, in particular because new challenges, apart from the
physical organisation of the urban space, have emerged and require the use
of a variety of disciplinary methods and tools (sociological inquiries,
decision theory, economic programming, environmental protection, etc.),
and more scientific approaches have entered the composite field of planning.
CITATION INDEX, IMPACT FACTOR AND THE LIKE
Assessment criteria and indexes are much debated, and even the agencies
managing such tools warn against their misuse. The problems regard the
focus of the assessment, whether the ranking concerns the journal or the
qualifications of its contributors, the number and type of journals covered,
and the time extension of data collected. It is clear that one indicator cannot
be used to measure different aspects, that only part of the literature is
covered, and that not all citations can be considered. As said, there is much
discussion on these issues, and a large body of literature has been produced
to propose new criteria and indicators.
A caveat has been issued in order to prevent evaluators, in particular for
academic career progression, from simply considering the prestige of the
journals in which articles are published instead of reading the articles and
evaluating them (Goldstein and Maier, 2010, p. 67). Also the European
Association of Science Editors has approved a “statement on inappropriate
use of impact factors” (EASE statement) which stresses that impact factor
scores regard journals, not single articles or scholars.

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 107Vol. II, issue 2 ­ 2012

The problem considered here is whether it is useful or indispensable to rank
planning journals, and to consider, in the evaluation of scholars, indicators
such as citation index, impact factor, h index, etc. The consideration that in
the USA “[a]mong planning educators, there has been a long standing
reluctance to the publication of comparative performance measurements”
(Stiftel et al., 2004a, p.6) can be extended to most countries.
Research can deal with a variety of issues and involve different end­users. In
particular, final products (and in particular publications) may be of interest to
academia (and the impact may be restricted to a small community of scholars),
to practitioners and civil servants (for whom more practice­oriented articles
are obviously of interest), and to politicians. In some countries, as in Italy,
English language is not so well known by practitioners and public officers, so
that publications in Italian have far more practical results.
The effects of assessment procedures are evident, but they depend on how
the evaluation is performed. When the 1992 UK research assessment
exercises “placed emphasis on the quantitative aspect of scientific
production the response was an increase in publication numbers. However in
1996, when the focus shifted from ‘output counts’ to ‘quality’, there was a
greater propensity to publish in journals with a higher impact factor”
(Abramo et al., 2011, p. 230).
In general, it is not easy to compare articles in different journals on the basis
of indicators because the numbers cannot be taken as they are, and in any
case an insight into the publication is required. But it has been observed that
there is no strict correlation between the impact factor of a journal and its
reputation in the scientific community. Therefore “university administrators
should hesitate to evaluate the scholarly output of individual faculty
members by the impact factors of the journals in which they publish”
(Goldstein and Maier, 2010, p. 73).
Peer­review, as mentioned, is the key means to select papers for publication.
It can effectively choose good products, but it can also be a self­referential
procedure: the members of a small group of scholars select articles which
are attuned to their shared background, address concerted issues, and cite
materials produced within that group. Alongside the validation of key topics
and methodologies, there is the risk of creating short­circuits among a small
number of scholars, thereby reducing creativity and preventing risk­taking

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 108Vol. II, issue 2 ­ 2012

research strands (CGC, 2011, p. 191). It has been observed that impact
factors “pursue a circular logic ­ measuring the impact of academic
publication activities on academic publication activities ­ and reflect a
conflict of interest on the side of the publishing industry, which develops
indices that centre its own publications in the assessment of academic
work”. (CGC, 2011, p. 190). From this perspective, scientific production can
be “production of paper by means of paper”, to rephrase Piero Sraffa’s
definition of capitalist production.
As regards indexes, a journal’s impact factor represents “a ratio of the
number of citations to the number of articles published in that journal”. The
higher the ratio, the greater is the journal’s presumed reputation. One
consequence is that this value is used to assess scholarly production:
“[j]ournal impact factors can then be used as de facto ‘weights’ for each
article published by individual faculty members” (Goldstein and Maier,
2010, p. 67), with the connected risk that a paper is rated without specific
evaluation of its contents.
The most widely used index is Thomson­Reuters’ ISI Web of
Knowledge/Web of Science impact factor, which was first proposed by
Eugene Garfield in 1955 (Garfield, 2006). It is calculated, for a given
journal and a specific year, as the mean of citations per paper in the two
preceding years. “An impact factor of 2.0 means that, on average, papers
published one or two years ago have been cited twice in the current year”
(Webster, 2006, p. 485).
The journals must satisfy three criteria to be included in ISI Web of Science:
“1) works in the journal must be cited regularly in other journals; 2) the
journal must provide full bibliographic information in English for each work
included and for works cited; and 3) the journal must come out on time…”
(Stiftel and Mogg, 2007, p. 81).
Apart from the impact factor, other statistics take account of the immediacy
of citations, the average number of references per article (citation density),
the life­time of articles (cited half­time), etc. There are also specific
indicators regarding the features of an author’s production. Increasingly
popular among these indicators is the “h index” proposed in 2005 by J.E.
Hirsch. Its value measures the number of articles that have received a
number of citations equal to or greater than the same number of articles.

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 109Vol. II, issue 2 ­ 2012

What is counted is the major problem in the compiling of indexes because
peer­reviewed articles can be related to citations which include different
materials (reviews, comments, etc.), thus over­ or under­estimating the real
impact (Goldstein and Maier, 2010, pp. 67­68). Moreover, citations must be
considered carefully because they are made for a number of different and
contrasting reasons (provide background, praise, confutation, etc.) and
therefore do not directly furnish an indication of the article’s value
(Goldstein and Maier, 2010, p. 67).
Other problems regard the number of journals listed in the ISI Web of
Knowledge, which does not include either low­quality ones or many of
those written in languages other than English, or more recent journals or
niche publications. It has been stressed, in particular, that some prestigious
planning journals are not ranked (Webster, 2006, p. 485).
Another issue is that “[p]lanning papers tend to cite eclectically”, referring
to “reports and plans as well as academic papers”, thus lowering impact
factors for more compact disciplines. The result is that “[p]lanning has an
average of about ten citations per article” (Webster, 2006, p. 488).
Apart from a few key journals, there is a large spread among the
publications consulted by planners, which highlights that a number of
specific interests are involved (Goldstein and Maier, 2010, p. 69). In fact,
articles in certified journals, professional journals and magazines, and in
particular books, are produced and cited by planners. Books can be
published without quality control by local publishers, and in some cases they
are home­made publications. But everybody knows how challenging it is to
write a book, be it a handbook, a textbook, or a collection of essays. Books
are indispensable for teaching, and they have a long­lasting educational
impact which is not certified by the usual procedures.
Finally, the complexity of territorial disciplines and the variety of
publications involved have prompted warnings against the use (and abuse)
of the impact factor, because it may recognise the academic fame but not the
real “social impact” of research. Two scholars representative of radical
geography (Cynical Geographers Collective, S.E. Di Mauro and H. Bauder;
CGC, 2011) have advanced severe criticisms and proposed a “social impact
factor”. They argue that the impact factor “stifles risk­taking scholarship and
creative thinking, rewarding a production­line mentality that undermines

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 110Vol. II, issue 2 ­ 2012

academic work quality”. (CGC, 2011, p. 190). The proposed “social impact
factor” should track not only publications but also teaching, public outreach,
and knowledge production. These considerations are of particular interest to
academic planners, who are involved in all these activities and encounter
difficulties in making progress in one strand without having experience in
the others. For the above­cited authors, the real impact to consider should be
the number and the quality of students educated, at the different levels, and
the ability to stimulate innovation in planning practices at the local level.
INTERNATIONAL JOURNALS OF PLANNING
A number of assessments have been performed on the reputation of planning
journals, and controversial results have emerged (Stiftel and Mukhopadhyay
2007; Goldstein and Maier, 2010; Salet and Boer, 2010, Anselin et al.,
2011). The mixed academic/professional nature of planning is reflected in
the variety of research interests, as well as in scholars’ perceptions of the
relevance of journals (Goldstein and Maier, 2010, p. 73). Papers therefore
spread over a number of journals of diverse disciplines, and in order to find
titles in ranking databases, one must look through subject categories
concerning environmental studies, geography, planning and development,
transportation, urban studies, or sociology.
The composite nature of the discipline “makes it difficult to devise
meaningful rankings of planning journals” (Webster, 2006, p. 485), and not
all high­quality journals are considered by rankings, whose requirements
imply costs and management burdens that editors and publishers may be
unwilling to assume1.

Zanon ­ Research Quality Assessment and Planning Journals

results:
‘Urban’­ 28 items, among which: European Urban and Regional Studies; International

Journal of Urban and Regional Research; Journal of Urban History; Journal of Urban
Planning and Development; Landscape and Urban Planning; Urban Design
International; Urban Geography; Urban History; Urban Morphology; Urban Studies.

‘Planning’ ­ 29 journals, among which: Environment and Planning A/B/C/D; European
Planning Studies; Journal of Environmental Planning and Management; Journal of
Environmental Policy & Planning; Journal of Planning Education and Research;
Journal of Planning Literature; Journal of the American Planning Association;
Planning Theory; Progress in Planning.

A search in the ISI database using keywords contained in journal titles yielded the following1



IJPP ­ Italian Journal of Planning Practice 111Vol. II, issue 2 ­ 2012

While scholars of architecture and urban design do not frequently publish in
peer­reviewed papers, they produce a variety of “design work,
commentaries, edited collections, and research with a humanistic bent”, with
the result that “the scholarly production of each group may be invisible to
the other” (Forsyth, 2007b, p. 179). Moreover, “teaching is seen as a
peripheral issue in the research university”, while “architectural programs
spend a great deal of time and energy defending a relatively unique and
time­consuming teaching approach…” (Forsyth, 2007b, p. 179).
The questions of where planners publish and how to assess the quality of
journals and magazines and the impact of production are still unanswered.
More objective measures of scholars’ qualification are certainly needed, in
particular by adopting the procedures commonly used in other disciplines, in
order to judge the productivity of faculty members and to take decisions on
career promotion. But it has been found that “there is no statistically
significant relationship between the relative value planning faculty
subjectively place on journals and their impact factors” (Goldstein and
Maier, 2010, p.71).
An important analysis of international planning journals has been conducted
by Stiftel and Mukhopadhyay (2007), whose results are now briefly
summarized. Only 4 out of the 25 journals considered were published
outside the USA and the UK, specifically in the Netherlands by Elsevier
B.V. More than 63% of authors came from the USA, UK or Canada. Italian
authors do not rank badly, occupying eighth place, but they accounted for
only 1.51% of the articles. Moreover, Italian members of editorial boards
accounted only for 1.88% of the total.
In general the rate of internationalization is very low, with few journals having
a board of which one­third of members are non­English mother­tongue. There

Zanon ­ Research Quality Assessment and Planning Journals

‘Regional’ ­ 19 journals, among which: Annals of Regional Science; Journal of Regional
Science; Regional Science and Urban Economics.

‘Landscape’ ­ 8 journals, among which: Landscape Architecture; Landscape Architecture
Magazine; Landscape Ecology; Landscape Research.

‘Assessment’ ­ 28 journals of different disciplines, among which: Environmental Impact
Assessment Review.

Some prestigious journals, not yet ranked, are the following: Planning Theory and
Practice; Town Planning Review; DISP; Planning Practice and Research; Journal of
Urban Design.



IJPP ­ Italian Journal of Planning Practice 112Vol. II, issue 2 ­ 2012

is also a difference between American journals and the rest, the former being
less internationalized than the latter. This has important consequences
because there is a correlation between the level of internationalization of an
editorial board and that of authorship, with some journals comprising one­
third of articles written by non Anglo­American authors.
Another survey on the diffusion and reputation of journals has been
conducted among the ACSP schools (Association of Collegiate Schools of
Planning) (Goldstein and Maier, 2010), and it has highlighted a difference
between American and European planners. The survey showed that in the
former case planners make use of a high number of journals (191), but the
most important ones are considered to be the Journal of the American
Planning Association and the Journal of Planning Education and Research.
Urban Studies follows at a certain distance, while “[t]he large majority of
the 191 journals were listed by only 1 respondent” (Goldstein and Maier,
2010, p. 69).
For European planners, a similar inquiry involved AESOP schools (Salet
and Boer, 2010; Salet and Boer, n.d.). Scholars cited 235 different journals.
While for American planners the first two journals focused on planning, for
the European ones “the highest­ranked journals are not typically specialized
planning journals” (Salet and Boer, 2010, p. 95), namely Urban Studies,
Environment and Planning A and European Planning Studies, which are
open to a variety of contributions in diverse fields. “So, contrary to the USA,
there appears to be no cohesion in pure planning studies in Europe” (Salet
and Boer, 2010, p. 95). The difference between European and American
planners’ interests is highlighted by the fact that American journals obtain a
second row classification.
Other inquiries have considered the qualifications of planning schools also
on the basis of publication data. In particular, the American urban planning
schools have been analyzed by considering the qualification of faculties
(total number of faculty members, percentage of full professors, ratio of
members publishing in ISI journals) and faculty productivity and impact
(number of articles in ISI journals and citations) (Stiftel et al., 2004a; for
comments: Teitz, 2004; Forsyth, 2004; Myers, 2004; Albrechts, 2004; a
reply: Stiftel et al., 2004b). To be stressed among the results is that 45% of
faculty members published at least one ISI article in the interval 1998­2002.

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 113Vol. II, issue 2 ­ 2012

Another finding is the concentration of publications in the best­ranking
schools: the top 9 schools out of the 84 considered (34 of which offering
PhD programmes) account for 38.8% of all publications. Also citations (on
the basis of the ISI database) are concentrated: the top 3 schools account for
38.2% of total citations, the top 5 for 50.7%, one half of the schools for
95.7%. Faculty members are even more differentiated in terms of citations,
because the top 4 members account for 27.4% of all citations, and the top 19
for 50.5% (Stiftel et al., 2004a, pp. 10­13).
THE PRESTIGE AND IMPACT OF ITALIAN JUORNALS
In Italy, the “urbanism tradition” (CEC, 1997; Espon, 2007) implies that
planning consists mostly of urban design and the professional practice of
drafting master plans. In general, disciplinary interests regard the legal
framework (more precisely, its criticism), urban history, and the analysis of
case studies that can be considered good practices or innovative experiences
because of their theoretical assumptions or methodological contents. For
these reasons, articles in Italian planning journals reflect the tradition of
architecture reviews. They are usually descriptive, with few or no
bibliographic references to the international literature, while they often take
positions in the political debate, with a focus on innovation of the legal
framework or on governmental provisions.
The Italian approach to urban planning began to be innovated in the 1960s,
in particular by Giovanni Astengo, who founded the first planning school in
Italy at the Faculty of Architecture in Venice. The approach viewed planning
as an instrument for “rational” public intervention which required
competencies in social sciences, economics, and public decisions methods.
In the 1970s and 1980s the disciplinary bases of planning were scrutinized
(Vettoretto, 2009), creating space on the one hand for more traditionally
design­oriented approaches, while on the other pushing for more research
work on planning foundations and methods. Regional laws began to
structure a precise – albeit bureaucratic – framework. Thereafter, other
sectoral planning practices, innovative decision­making processes (public­
private partnerships) and assessment procedures (Environmental Impact
Assessment, Strategic Impact Assessment), started to be used and required

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 114Vol. II, issue 2 ­ 2012

an expertise different from urban design. More recently, the need to address
the new forms of the urbanized space, as well as the emerging
environmental issues, the challenge of energy and climate change, the quest
for sustainable development, together with emerging social problems and the
availability of new technical instruments (GIS, in particular), have opened
new research strands, and articles increasingly appear in academic journals.
The Italian academic tradition in evaluating publications for academic career
progression is rather loose. It used to be based on an outdated law defining
“publication” as everything submitted for legal deposit, that is, presentation
of the publication (or manuscript) to a court office. This implied that, for
career progression, evaluators could consider also products of minor quality,
even self­produced ones. Subsequently, ISBN numbers were requested; and
recently, more precise criteria have been established by a national academic
body (“Consiglio Universitario Nazionale”, CUN, 2010) and the research
and university evaluation agency ("Agenzia Nazionale di Valutazione del
sistema Universitario e della Ricerca", ANVUR, 2011a).
In the 1980s, doctoral programmes were introduced and educated a new
generation of scholars (Balducci, 2005, p.250) with sounder methodological
bases attuned to the international debate. Recently, the pressure to use
formalized assessment methods has become strong. The growing
competition among universities and among diverse sectors within the same
institutions has required assessment procedures for Departments (the
university institutions organising research), disciplinary sectors, and
individual scholars (in terms of productivity). In recent years, some legal
provisions have established assessment procedures for the universities, and
the above­mentioned evaluation agency (ANVUR), after the experience of
previous committees, was set up in 2008 (Piazzini, 2010).
The national academic authorities have issued guidelines defining criteria
and parameters for the next competitions for academic positions which will
be conducted at the national level (CUN, 2011) and a database on
productivity has been created by the Ministry for University and Research.
Following the directives of the CUN document, the selection of a scholar for
a full professorship entails that s/he must have authored at least 10 ISI – or
comparable level – articles or books. Also innovative projects or plans
critically presented by other authors on authoritative publications can be

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 115Vol. II, issue 2 ­ 2012

considered. An adequate working time span must be covered, and recent
productivity must be demonstrated.
Comparison among different disciplines and scholars working in diverse
fields has therefore become inevitable. However, the results may be very
problematic because scientific sectors which have long introduced
international assessment models easily outweigh more traditional ones, in
particular in the humanities, but also in architecture and planning.
The discussion on the ANVUR document (ANVUR, 2001a, 2001b), which
insists on the consideration of ranked journal papers, has influenced the final
guidelines issued by the Ministry for the next academic competitions. A
decree (n° 76, June 7th, 2012) has defined such criteria and parameters,
which make a distinction between disciplines using ‘bibliometric’ indicators
and disciplines using ‘non bibliometric indicators’, the latter including
architecture and urban planning. Anyhow, a different weigh is given to
publications, according to the prestige of the journal or the publisher of the
books. And this classification has implied discussion, as well as the criterion
that both evaluators and candidates must rank, in terms of publications,
above the median of the disciplinary sector.
An overview of Italian journals shows that planning occupies a niche in
architecture and urban design publications, and that only a few of the latter
are focused on the discipline. In general, they are not organized as
disciplinary journals usually are: that is, publishing papers submitted by
authors and selected through peer­review procedures. Urbanistica, the most
prestigious journal, published by the Istituto Nazionale di Urbanistica, and
which has recently provided a full English translation of articles, is focused
on the description of plans and experiences. Other journals, such as
Territorio, published by the Dipartimento di Architettura e Pianificazione of
the Politecnico di Milano (with an international editorial board) and Archivio
di Studi Urbani e Regionali, to cite two of them (but there are very few
others) have a mixed system whereby some papers are selected but not
through a formal peer­review process.
The “Conference of the Deans of the Italian Schools of Architecture” has
recently drawn up a list of journals and magazines of interest to scholars in
the disciplines of architecture and urban planning (personal communication,
internet documents). The list ranks the journals in four classes on the basis

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 116Vol. II, issue 2 ­ 2012

of a number of parameters, taking the following into account: English
abstract, article in English, editorial board, blind peer­review, peer­review,
international diffusion, article selection procedures, ISI certification.
In all, 215 journals and magazines are listed, some of which are in electronic
format. There are items from different countries and in diverse languages
(English, Italian, German, French, in particular).

Zanon ­ Research Quality Assessment and Planning Journals

Table 1 – Journal classification by the Deans of the Italian Schools of Architecture
The upper class counts 44 items, none of them Italian. In the second class
there are 60 items, 14 of which are Italian (23.3%). Publications include
journals and magazines of architecture (some open to urban design, such as
Casabella, Domus, Lotus International), monument restoration, history of
architecture, steel constructions, drawing, industrial design, regional sciences,
urban sociology. Also Urbanistica is included in this class. In the third class
there are 78 items, 39 of which are Italian (50.0%). Some important planning
journals (Archivio di Studi Urbani e Regionali, Territorio) and urban design
and architecture magazines (Parametro), are included here. Last, in the fourth
class there are 33 items, 28 of which Italian (84.9%).
It is evident that the Italian editorial landscape is defined mostly by
magazines oriented to professionals rather than by academic journals, but it
must be stressed that many low­ranking items are in fact good quality
publications but lack the features required, such as a prestigious editorial
board, a reliable publisher able to cover different markets and to guarantee
the continuity of publication, and, most importantly, peer­review procedures
and inclusion in certification mechanisms. In fact, most magazines contain
descriptive articles not organized in the tradition of academic papers
(definition of the problem addressed and the methodology used, overview of



IJPP ­ Italian Journal of Planning Practice 117Vol. II, issue 2 ­ 2012

current literature, empirical analysis, discussion and conclusions).
The reasons for drawing up the list of journals regard its use in scholar
selection and promotion (in particular providing guidelines for the ranking
of Italian publications), but they also concern the endeavour to improve
research products. Some journals have started to use peer­review procedures
and to provide translations or at least abstracts in English.
Another disciplinary group, comprising planners at the schools of
engineering (Urbing), has issued some statistics on publications by its
associates and proposed an integrative list of journals and magazines
(personal communication, web­site materials). In regard to the results, first
to be stressed is that articles represent around 24% of the total publications
by the scholars involved, the rest being books, proceedings, and local
publications. A major outcome is the variety of issues addressed and the
number of journals and magazines in which this disciplinary community
publishes. Together with ISI journals, there are national but also local
publications, which are important in terms of the dissemination of research
results but do not provide scientific feedback in terms of citations. It is
obvious, however, that a picture of where planners are actually publishing is
not enough to consider the publications listed as important for the discipline.
CONCLUSIONS: THE RISK TO BECOME INVISIBLE
Formalized methods to assess the effects of research products are spreading.
Their role is crucial because they stimulate competition at higher levels. Yet
impact factors and citation data cannot be used as the sole instruments with
which to evaluate the importance of a paper or the qualification of a scholar.
Not only is the real importance of journals not correlated with impact factors
(Goldstein and Maier, 2010), and cross­citation in planning articles is not as
frequent as in other disciplines (Webster, 2006, p. 488), but other
requirements are needed to qualify a scholar or to appreciate the real effects
of his/her activity, apart from those on the academic environment, on what
really counts: planning practice.
Some risks of the new methods are well known: “There is an old saying in
relation to the promotion process which goes: if you want to promote a lecturer
you weigh his/her publications, and if you don’t, you read them” (Punter and

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 118Vol. II, issue 2 ­ 2012

Campbell, 2009, p. 52). Hence there is “no entirely satisfactory substitute for
peer review in terms of metrics” (Punter and Campbell, 2009, p. 52).
Planning is a niche discipline. Scholars are few and schools are small, even
in the USA (Stiftel et al., 2004a, p. 13). As already noted, the discipline has
no visibility in ranking databases, and relevant journals must be found under
a variety of different headings.
Besides legitimate criticism of assessment criteria and indicators, there is a
resistance against the new methods which has clear consequences: “urban
planning programs may be losing visibility and resources because they do
not participate in comparative performance measurement” (Stiftel et al.,
2004a, p.6). It is therefore evident also that planning scholars must adopt, or
at least stay tuned with, the research methods of the hard sciences and accept
the predominant assessment criteria. This does not mean neglect of the
discipline’s usual topics or abandoning its specific methods; rather, it implies
accepting the challenge of writing in peer­reviewed journals considering the
impact effects. Moreover, given that most qualified journals are in English,
this also means accepting to write in that language.
Most European journals – also among those published by large companies –
are the initiatives of associations or university departments, and they usually
have international editorial boards which allow the integration of different
interests and sensitivities. They are therefore open to a variety of topics and
appreciate contributions presenting results of applied research. Moreover,
some on­line journals are available – among them the Italian Journal of
Planning Practice – offering additional publishing opportunities. Planners
can make use of these tools to give wider visibility to their research work,
but they must fulfil the fundamental requirements of methodological rigour
and connection with the international debate.
Two aspects must be stressed in relation to the Italian case: the presence of
Italian authors and topics in international journals, and the increasing role of
research assessment procedures. Not only is a more frequent presence of
Italian planners in international journals needed, but also new journals
should be promoted in order to fill the gaps in the current literature
landscape by better representing Mediterranean issues and experiences. This
should be part of “a twin strategy of increasing non­anglophone authorship
in English­language journals coupled with improved mechanisms for

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 119Vol. II, issue 2 ­ 2012

sharing of ideas across literatures of different languages” (Stiftel and
Mukhopadhyay, 2007, p. 566).
In Italy a first assessment exercise has been conducted on research products
(CIVR, 2006). It “produced performance rankings of universities based on
an evaluation of a share of their 2001–2003 product that was equal in
number to 25% of each university’s research staff complement in each of the
18 disciplines considered” (Abramo et al., 2011, p. 231).
Another assessment is under way on the basis of the quoted guidelines
(MIUR, 2011). “The evaluation criteria for the first VTR [triennial
assessment of research] clearly directed research institutions to concentrate
their resources on top scientists, while the new VQR [assessment of the
quality of research] will offer reward on the basis of average performance of
their research staff” (Abramo et al., 2011, p. 231).
To date, such assessments have not had much effect in terms of the
allocation of public resources, because other parameters (number of students
and staff, consolidated budget, etc.) are considered; but competition will be
tough in the near future, and the image of institutions and academic staff will
be more important in attracting students and obtaining research contracts.
To conclude, planning runs the risk of becoming an invisible academic
discipline. In regard to design­oriented sectors, it is necessary to develop
reliable assessment methods appropriate to a variety of products, often marked
by creativity rather than research (Forsyth, 2007b). For process­oriented
sectors, there is much space for improving the quality of research, publishing
in international journals, without abandoning contacts with the local planning
environment and publications. To be avoided is the risk that scholars involved
in publishing in high­ranking journals “do not see a need or do not find the
time to write for local journals” (Schuermans et al., 2007, p. 422), thus
reducing the impact of research on local society. In fact, “a deeper level of
reflection is beneficial, or even necessary, for the [planning] process to be
effective” (Balducci and Bertolini, 2007, p. 532). In short, a planner’s CV
should not contain only well­ranking articles because s/he must demonstrate
an ability to put his/her knowledge into practice and show commitment to the
discipline’s social effects. On the other hand, it is no longer acceptable that a
scholar be promoted on the basis only of local (or parochial) publications
because s/he is a member of a self­ protecting circle of academicians.

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 120Vol. II, issue 2 ­ 2012

REFERENCES

ABRAMO, G., D’ANGELO, C.A. and DI COSTA, F. (2011), "National research
assessment exercises: the effects of changing the rules of the game during the
game", in Scientometrics, 88, pp. 229–238.

ALBRECHTS, L. (2004) "A challenge for the global planning community?", in Journal
of Planning Education and Research, 24(1), pp. 26­27.

ANSELIN, L., NASAR, J.L., and TALEN, E. (2001) "Where Do Planners Belong?
Assessing the Relationship between Planning and Design in American
Universities", in Journal of Planning Education and Research, 31(2), pp. 196­207.

AGENZIA NAZIONALE DI VALUTAZIONE DEL SISTEMA UNIVERSITARIO
E DELLA RICERCA – ANVUR (2011a), Criteri e parametri di valutazione
dei candidati e dei commissari dell’abilitazione scientifica nazionale.
Approvato dal Consiglio Direttivo il 22 giugno 2011.
(Criteria and parameters for the evaluation of candidates and committee members
of the national scientific qualification)
http://www.anvur.org/sites/anvur­miur/files/documento01_11.pdf.

AGENZIA NAZIONALE DI VALUTAZIONE DEL SISTEMA UNIVERSITARIO
E DELLA RICERCA – ANVUR (2011b), Sul documento ANVUR relativo ai
criteri di abilitazione scientifica nazionale. Commenti. Osservazioni critiche e
proposte di soluzione. Approvato dal Consiglio Direttivo il 25 luglio 2011.
(Critiques and proposals on the ANVUR document on criteria and parameters for
the national scientific qualification)
http://www.anvur.org/sites/anvur­miur/files/Documento%2002_11.pdf.

BALDUCCI, A. (2005), "Collegiality within a Mass University System: Reflections
from Italy", in Planning Practice and Research, 6(2), pp. 249­251.

BALDUCCI, A. (2007), A View from Italy, Planning Theory & Practice, 8(4), pp.
533­534.

BALDUCCI, A. and BERTOLINI, L. (2007), "Reflecting on Practice or Reflecting
with Practice?", in Planning Theory & Practice, 8(4), pp. 532–533.

COMITATO DI INDIRIZZO PER LA VALUTAZIONE DELLA RICERCA ­ CIVR
(2006) Ministero dell’Istruzione, dell’Università e della Ricerca, VTR 2001­2003,
Risultati delle valutazioni dei Panel di Area.

(Italian Triennial Research Evaluation 2001­2003, Results of Area Panels), Rome.
http://vtr2006.cineca.it/

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 121Vol. II, issue 2 ­ 2012

COMMISSION OF THE EUROPEAN COMMUNITIES ­ CEC (1997) The EU
Compendium of Spatial Planning Systems and policies, Regional
development studies, 28, Luxembourg, European Commission.

CONSIGLIO UNIVERSITARIO NAZIONALE ­ CUN (2010) Ministero
dell’Istruzione, dell’Università e della Ricerca, Consiglio Universitario
Nazionale, “Criteri identificanti il carattere scientifici delle pubblicazioni, ai
sensi dell’art. 3­ter, comma 2, del decreto legge 10 novembre 2008, n. 180,
convertito dalla legge 9 gennaio 2009, n.1”, Roma.

CONSIGLIO UNIVERSITARIO NAZIONALE ­ CUN (2011) Ministero
dell’Istruzione, dell’Università e della Ricerca, “Proposta su ‘criteri e parametri
per la valutazione’ ai fini di cui all’Art 16 comma 3 lettere a) e h) della Legge
30 dicembre 2010, n. 240”, Roma.

CYNICAL GEOGRAPHERS COLLECTIVE ­ CGC (Salvatore Engel Di Mauro and
Harald Bauder) (2011) "Measuring Impact Beyond Academic Fame: An
Alternative Social Impact Factor", in Antipode, 43(2), pp. 190­194.

DAVOUDI, S. (2010) "Planning and Interdisciplinarity", in: Geppert, and Cotella, G.,
Quality Issues in a Consolidating European Higher Education Area, Planning
Education, N°2, July 2010, Reims, AESOP, pp. 33­35.

DERUDDER, B. (2011) "Some reflections on the ‘problematic’ dominance of ‘Web
of Science’ journals in academic human geography", in Area, 43(1), pp. 110–112.

ESPON (2007), Project 2.3.2, Governance of Territorial and Urban Policies from
EU to Local Level. Final Report, Luxembourg, Espon.
http://www.espon.eu/export/sites/default/Documents/Projects/ESPON2006
Projects/PolicyImpactProjects/Governance/fr­2.3.2_final_feb2007.pdf

FORSYTH, A. (2004) "The view from design", in Journal of Planning Education and
Research, 24(1), pp. 24­25.

FORSYTH, A. (2007a) "Innovation in Urban Design: Does Research Help?", in
Journal of Urban Design, 12(3), pp. 461–473.

FORSYTH, A. (2007b) "The Rise of the Nerds? Interdisciplinary Research and
Architecture, Archnet­IJAR", in International Journal of Architectural
Research, 1(3), pp. 177­182.

GARFIELD, E. (2006) "The History and Meaning of the Journal Impact Factor.", in
JAMA: The Journal of the American Medical Association, 295(1), pp. 90 ­93.

GERACI, M. and DEGLI ESPOSTI, M. (2011) "Where do Italian universities stand?
An in­depth statistical analysis of national and international rankings", in

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 122Vol. II, issue 2 ­ 2012

Scientometrics, 87(3), pp. 667­681.
GOLDSTEIN, H. and MAIER, G. (2010) "The Use and Valuation of Journals in

Planning Scholarship: Peer Assessment versus Impact Factors", in Journal of
Planning Education and Research, 30(1), pp. 66 – 75.

GOLDSTEIN, H.A. and CARMIN, J.A. (2006) "Compact, Diffuse, or Would­be
Discipline?: Assessing Cohesion in Planning Scholarship, 1963­2002", in
Journal of Planning Education and Research, 26(1), pp. 66­79.

HARZING, A.W. (2008) “Google Scholar ­ a new data source for citation analysis."
www.harzing.com

KUNZMANN, K.R. (2004) “Unconditional Surrender: The Gradual Demise of
European Diversity in Planning”, key note paper presented to the 18th AESOP
Congress in Grenoble, France on 03­07­2004.

MINISTERO DELL’ISTRUZIONE, DELL’UNIVERSITA’ E DELLA RICERCA ­
MIUR (2011) Linee guida VQR 2004­2008 (Guidelines for the evaluation of
the quality of research), Rome.
http://www.civr.miur.it/vqr_decreto.html

MYERS, D. (2004) "How can planning schools be usefully compared?", in Journal
of Planning Education and Research, 24(1), pp. 25­26.

PIAZZINI, T., 2010, "Gli indicatori bibliometrici: riflessioni sparse per un uso
attento e consapevole", JLIS.it, Vol. 1, n. 1, Giugno/June, pp. 63–86.

PUNTER, J. and CAMPBELL, H. (2009) "Reflections on the 2008 United Kingdom
Research Assessment Exercise for Town and Country Planning and closely
associated disciplines", in Town Planning Review, 80(1), pp. 31­54.

RAUHVARGERS, A (2011) Global University Rankings and their Impact,
Bruxelles, European University Association.
www.eua.be

SALET, W. and BOER S. (2011) "Comparing the Use and Valuation of Journals
between U.S. and European Planning", in Journal of Planning Education and
Research, 31(1), pp. 95­97.

SALET, W. and BOER, S. (no date) “How European Planning Scholars Rank the
Importance and Reputation of Journals: Summary of findings”, Summary of
the research.
http://www.aesop­planning.eu/uploads/news/2010_surveyjournals_
europeanplanning.pdf

Zanon ­ Research Quality Assessment and Planning Journals



IJPP ­ Italian Journal of Planning Practice 123Vol. II, issue 2 ­ 2012

SCHUERMANS, N., MEEUS, B. and DE MAESSCHALCK, F. (2010) "Is there a
world beyond the Web of Science? Publication practices outside the heartland
of academic geography", in Area, 42(4), pp. 417– 424.

STIFTEL, B. and MOGG, R. (2007) "A Planner's Guide to the Digital Bibliographic
Revolution", in Journal of the American Planning Association, 73(1), pp. 68­85.

STIFTEL, B. and MUKHOPADHYAY, C. (2007). "Thoughts on Anglo­American
hegemony in planning scholarship: Do we read each other's work?", in Town
Planning Review, 78 (5), 545­572.

STIFTEL, B., FORSYTH, A., DALTON, L. and STEINER F. (2009), "Assessing
Planning School Performance. Multiple Paths, Multiple Measures", in Journal
of Planning Education and Research, 28(3), pp. 323­335.

STIFTEL, B., RUKMANA, D. and ALAM, B. (2004a) "A National Research
Council­Style Study", in Journal of Planning Education and Research, 24(1),
pp. 6­22.

STIFTEL, B., RUKMANA, D. and ALAM, B. (2004b) "A Reply to Graduate
Planning School Study Responses", in Journal of Planning Education and
Research, 24(2), pp. 128­130.

TEITZ, M.B. (2004) "Assessing the quality of U.S. planning schools: A comment",
in Journal of Planning Education and Research 22(1), pp. 23­24.

TEWDWR­JONES, M. (2005) "Return of the indicators: the Research Assessment
Exercise", in Environment and Planning B: Planning and Design, 32(3), pp.
317­322.

VETTORETTO, L. (2009) "Planning cultures in Italy – Reformism, Laissez­faire
and Contemporary Trends", in: Knieling, J. Othengrafen, F. (ed.), Planning
Cultures in Europe. Decoding Cultural Phenomena in Urban and Regional
Planning, Ashgate, pp. 189­203.

WEBSTER, C. (2006) Editorial, in Environment and Planning B: Planning and Design,
33(4), pp. 485­490.

Zanon ­ Research Quality Assessment and Planning Journals