CET Volume 86


 
 

 

                                                             DOI: 10.3303/CET2186061 
 

 
 

 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Paper Received: 17 September 2020; Revised: 6 March 2021; Accepted: 24 April 2021 
Please cite this article as: Mazri C., 2021, “You Don’t Manage What You Don’t Measure” … But Do You Really Manage What You Measure?, 
Chemical Engineering Transactions, 86, 361-366  DOI:10.3303/CET2186061 

 CHEMICAL ENGINEERING TRANSACTIONS 
VOL. 86, 2021 

A publication of 

The Italian Association 
of Chemical Engineering 
Online at www.cetjournal.it 

Guest Editors: Sauro Pierucci, Jiří Jaromír Klemeš
Copyright © 2021, AIDIC Servizi S.r.l. 
ISBN 978-88-95608-84-6; ISSN 2283-9216

“You Don’t Manage What You Don’t Measure” … But Do You 
Really Manage What You Measure? 

Chabane Mazri 
INERIS, Parc Technologique ALATA, BP2, 60550 Verneuil en Halatte, France. 
chabane.mazri@ineris.fr 

The imperious need to measure safety performances through indicators is now a well-established tradition in 
the process safety community. You don’t improve what you don’t measure has consequently been the mantra 
that drove important efforts of the community towards not only building reliable lagging representations of 
safety evolutions but also designing leading indicators providing decision makers with anticipation abilities. 
Taking a step back in order to gaining perspective allows us to understand the origin and foundations of this 
situation. P.Drucker’s “If you can’t measure it, you can’t manage it” seems to be the original starting point that 
indeed has spread throughout the management community in times where quality management was 
promising zero default products and conformity to standards believed as the key to successful management. 
However, management science has moved away towards far more modest precepts thanks to large efforts 
dedicated to observing real life decision making and organizations appropriation of management tools. 
Consequently, it is required for the safety community to deconstruct this precept and explore its validity given 
process safety specificities. This paper is a contribution to this effort. We will discuss accordingly this precept 
through two complementary angles. The first (measuring but not managing) questions the extent to which 
measuring paves the road for good management. This first angle provides the reader with the key traps 
separating measurement from management. The second angle (managing without measuring) shed lights on 
successful management practices not necessarily relying on anterior measurement.  
One should not throw the baby with the bathwater. This work is not be understood as a proposal to put aside 
or discredit safety measurement efforts which remain necessary. It however invites safety managers to put 
these approaches in perspective and question their validity scope so to make the best out of them without 
being imprisoned by their limits or sometimes, misleading character.  

1. Introduction

You don’t manage what you don’t measure is one of these assertions that each safety management scientist 
or practitioner has heard or read at least once in his life without questioning its validity or exploring its origin. It 
just appears as an unquestionable dogma, an axiomatic foundation on which solid and rational safety 
management can be built. Its impacts are visible in numerous ways. First, safety management models are 
widely described as control loops where expected objectives are compared to actual ones in order to detect 
adjustment requirements or demonstrate objectives achievements (Hollnagel, 2006) (Giraud & Zarlowski, 
2011). The measurement of objectives plays here the central role without which no loop can be closed. 
Further on, the CCPS chose this assertion as the frontline of its guideline on process safety metrics (CCPS, 
2011) to emphasize the importance of leading and lagging safety metrics in every process safety 
management. This vision has since then been widely shared in a large set of professional guidelines (EPSC, 
2011)  (CEFIC, 2011) and a safety science special issue on process safety leading and lagging indicators has 
extensively discussed the safety measurement issue without questioning this assertion (safety science special 
issue 47 (4), 2009). To be fair, safety managers are nothing more than one link in a long chain of diffusion of 
management theories where a mix of white and grey literature (Adams et al, 2017) as well as a strong 
influence of consultancy  (Engwall et al, 2016) allow for precepts like this to widely diffuse within the 
management practitioner community before allowing scientists to provide a critical analysis of their strengths 
and weaknesses.  

361



Given this context, this paper questions the fitness of this precept in the specific case of process safety. 
Indeed, if its emergence in the quality management domain where conformity to predefined standards was 
deemed to be the key of success can be understandable, its validity in the process safety area needs to be 
questioned for at least two reasons: 

• First, realistic decision-making and descriptive management literatures unveiled the gap between the
classic rational-normative theory of decision (Scott, 1999) (Uzonwanne, 2016) and the actual real life 
of organizational decision making. In the first instance, decisions are based on a structured process 
throughout which managers examine all available options and select those who fit a predefined and 
stable structure of preferences. In the second, decisions are the results of complex mechanisms 
where opportunities, uncertainties and poor information combine in chaotic manner to shape 
decisions. The garbage can model (Cohen, March, & Olsen, 1972) for instance define organisations 
as organised anarchies, where priorities are shifting, preferences uncertain and, more importantly for 
our purpose, relationships between information availability and decision making is highly ambiguous.   

• Second, safety science literature has made its own path towards describing and justifying the need
for a shift from a pure conformity paradigm to a more realistic vision where the complexity and 
unpredictability of reality impose to place workers’ skills, competences and adaptation capabilities  at 
the heart of safety models. In doing so, quality-based models of management appear outdated and 
all the practices that thrived under this model are now to be reviewed and deconstructed in order to 
reassess their fitness for safety management. 

This paper is a modest brick in this deconstruction project. More precisely, we suggest examining the You 
can’t manage what you can’t measure myth through two complementary angles: 

a. First, there is nothing linear or straightforward in the measurement-decision sequence. Indeed, the
process of (i) defining what deserves to be measured, (ii) collecting required data, (iii) analyse and 
interpret results to finally (iv) act and communicate bears a variety of complex mechanisms that may 
hinder the capacity of managers to capture the reality of their organisation. In such a case, indicators 
become misleading instead of leading. Said in simpler terms, what is measured is not necessarily 
managed. 

b. Second, organisations are actually able to manage key aspects of their performance without
necessarily measuring them. Several replacement mechanisms can be put in place: Informal 
practices, cultural control or incentives/rewards systems will be discussed as means to manage 
complex sociotechnical systems out of the measuring-deciding sequence. Indeed, you can manage 
what you can’t/don’t measure. 

2. Measuring but not managing

Measuring can be defined as the formal process of gathering data and building synthesis meant for action. 
The underlying paradigm on which this process relies in the field of safety is commonly labeled as cybernetic 
(Figure1). In the terms of his creator (Wiener, 1948), cybernetics is the field of control and communication 
theory whether in machine or animal. Structurally based on systems regulation through negative loops 
(Hofstede, 1978), cybernetics describe and structure the way a large variety of goal-oriented systems create 
and maintain order by adjusting their behavior thanks to the comparison of their outputs to a predefined 
reference state(s). The term “negative loop”, usually named feedback, refers to the function of negating or 
reducing the sensed system’s deviation from its reference state. The ability of cybernetics to focus on 
system’s functions regardless from its composition (Ashby, 1961) made it easily transposable to almost all 
type of systems, ranging from purely technical (Shinskey, 1967) to human (Carver & Scheier, 1982), 
economical (Balakrishnan, 1973) and more broadly social/organizational (Geyer, 2001) 

Figure1 – The cybernetic paradigm 

Before engaging in our deconstruction project, it is fair to say that cybernetics have developed a large set of 
concepts that go far beyond the one of first order feedback on which we will further focus. Accordingly, this 
work is much less a criticism of cybernetics than it is of the way one of its mechanisms, being first order 
negative feedback, has been used and understood in the field of management globally and safety 
management more particularly. Our deconstruction project starts with the observation that the cybernetic 
structure of this management model described a straightforward relationship between the system’s outputs on 
hand and the ability to manage accordingly its safety on the other hand. For such a relationship to operate, we 

362



identify a set of reasonable conditions without which the validity of such a relationship is strongly undermined. 
These conditions are: 

i. Availability: there is no feedback process without data. The availability condition here points to the
natural availability of data required for decision making or the possibility, given acceptable conditions of
technical and economic feasibility, to recover them.

ii. Reliability: Data need to be reliable in the sense that their collecting/recovering process does not alter
or bias them intentionally or unintentionally.

iii. Sufficiency: All data required for sound decision making are believed to be furnished by the system’s
outputs.

iv. Rationality: when all available data are finally captured, we expect managers to base their decisions
primarily on insights provided through the feedback channel. In other words, it is reasonable to expect a
strong correlation between data and decisions.

We will discuss in the followings all logically based or empirically observed arguments questioning the 
existence of such a straightforward relationship. Our point here is not to say that it does not exist, it is rather to 
point all the traps and complex mechanisms laying between the act of measuring and the one of deciding. 
i. Availability: Comparatively to quality management where conformity to standards can be assessed

using hundreds if not thousands of products, process safety managers deal with an extremely reduced
set of events. Major accidents are hopefully extremely rare and incidents exploitation require a well-
organized learning from incidents process combining detection, analysis treatment and communication of
lessons learnt (Lukic, Margaryan, & Littlejohn, 2010) (Stemn, Bofinger, Cliff, & Hassall, 2018). The
visibility of these incidents is negatively correlated to their number and consequence levels. In other
words, the large majority of safety related events belong to the category of weak signals (Ansoff, 1975)
(Brizon & Wybo, 2009) which timely detection and interpretation is still an important challenge for all
organizations.

ii. Reliability: Amongst the variety of data required to operate and control complex sociotechnical systems,
those relating to human and organizational performances are of paramount importance. Behaviors,
competences, conformity to procedures, collective work dynamics, adhesion to values, management of
change are only a few of safety performance determinants that require the monitoring of human and
organizational aspects. However, monitoring humans and organizations is fundamentally different from
monitoring hardware in the sense that human’s behavior is likely to be altered by observation. Snowden
(2002) provides an insightful example of this specificity. Imagine a rumor spreads within an organization
that new management rules and objectives will be applied soon. The unpredictability of each human
behavior and their interactions within the organization according to psychological, social, and cultural
laws may lead to the emergence of unknown and unpredicted states in the organization. Comparatively,
approaching a plane with a maintenance toolbox will not generate any reaction from plane components.
Measuring is sociotechnical systems is therefore a non neutral act which consequences may alter the
reliability of collected data. At least two mechanisms of altering data reliability can be considered here:
Conformity to standards and sanctions avoidance. Conformity to standards refers to the willingness of
people to meet the objectives assigned to them through a reporting tool, even if this requires altering the
raw data or leave unnoticed important issues as long as they are not considered in the tool. In his work
on audits approaches, (Power, 1997) wisely observes that the first impact of auditing tools is that it
makes systems more easily auditable. In other words, people tend to improve in a sense that allows
them to better fit auditing schemes instead of really thinking about what deserves to be improved. The
second mechanisms (sanctions avoidance) refers to the tendency of people to limit or avoid reporting of
incidents if they believe this will lead to sanctions, additional work or being perceived as incompetent.
These social mechanisms can highly alter the ability of measuring tools to capture what really happens
on the field. Direct consequences of this may be the lack of data for measurement or, worse, a false
sense of safety if data collected defines a positive although imaginary safety performance.

iii. Sufficiency: the terms Key Performance Indicators (KPIs) have widely spread in management practices
and literature to point the need to select what deserves to be measured. This selection process reflects
two things. First, it is impossible to measure all aspects of systems involving interaction between
technical and social components. Important choices are thusly required to select the key determinants
that one expects to have the most important impacts on final results. Second, the “s” of “indicators”
reminds us that considered individually, no indicator is able to embrace all the facets of a topic. We
therefore need several of them. Pushing this logic forward, and considering the individual flaws and
biases possibly introduced by each indicator, it becomes clear that what is required is a structured
system of complementary indicators which combination makes it possible for managers to reconstruct a
global and hopefully accurate representation of their system. This systemic way of addressing the issue
of measurement is key if we want for the measurement to provide managers with the required

363



information at an acceptable cost. Accordingly, providing managers with the sufficient grounds for action 
requires a carefully designed architecture of complementary indicators. Without it, there is no possibility 
to expect evidence-based decision making. 

iv. Rationality:  Let’s suppose that all the previous challenges listed above are successfully addressed. We
now have a set of data that is reliable, sufficient and collected despite their weak signal properties. Still,
there is a big chance that managers won’t pay sufficient attention to them, if not at all, or not relate their
decisions to the lights provided by these data. This somber prophecy is unfortunately quite well based
empirically and formalized in various descriptive theories of organizational decision frameworks. For
instance, Lindblom (1959) (1979) questioned the existence of a rational decision making process where
alternatives are carefully screened and evaluated according to a stable set of preferences. His empirical
work shed light on a different reality where managers confronted to their bounded cognitive capabilities
on one hand and limited resources and time on the other hand limit their reasoning to a reduced set of
alternatives. Moreover, their decisions tend to be incremental in the sense that they avoid “big jumps” to
focus on marginal and reversible improvements. This incremental approach offers a sense of safety as
past errors can be fairly quickly corrected (Lindblom, 1959). Further on this path, the garbage can model
(Cohen, March, & Olsen, 1972) describes organizations as anarchies. Decisions are here described as
the results of aleatory convergence of four different and mutually independent flows: solutions, problems,
participants and opportunities. In these anarchies, it is not unusual for problems to be formulated in a
way that fits preexisting sets of solutions; participants that helped bringing up an issue may not be there
when it is addressed, preferences required to define a course of action are shifting…All these aspects
reflect a reality where, instead of the straightforward relation between information and decision assumed
by the cybernetic paradigm, one may observe ambiguous, shifting and opportunistic mechanisms
partially involving information alongside with various other contextual determinants.
These empirical evidences question the centrality of rationality and the correlation between information
and decision. An available information is not necessarily a used or a useful one. Accordingly,
measurement should not be focusing on selecting and acquiring relevant data, it is foremost the process
through which these data are made available at the right time for the right managers in order to help
them make sense of their reality and foster their abilities to rationalize their decisions.

The various hurdles described above prove the extent to which the recursive loop assumed by the cybernetic 
paradigm may reveal ineffective. Its very existence requires a carefully designed process of data collection 
and communication alongside with a strong organizational commitment to enhance data-driven decisions.  

3. Managing without measuring: The capabilities approach

Managing is about inspiring, steering and constantly improving collective dynamics towards predefined 
objectives. Assuming that this activity can be conducted and legitimized only through data-driven approach 
reduces organizations to mechanistic systems. In his Images of organizations, Morgan (2006) invites us to 
consider the multifaceted character of organizations which can be analyzed through multiples angles : 
organizations as learning, cultural or political systems. Each angle shed light on specific mechanisms through 
which interactions are mediated and terms of collaboration specified without necessarily requiring an infinite 
iteration of comparison and adjustment through measurement.Inspired by social sciences, safety science has 
also done its share in exploring the various ways organizations may structure and maintain the levers through 
which it produces safety at a daily basis. One of the most inspiring theoretical frameworks which 
developments can be retrieved in safety sciences is the one of capabilities approach. Based upon the theory 
of Resource Based view of the firm (Wernerfelt, 1984), capability theory suggest analyzing organizations from 
the resource side rather than from the product one. It aims at identifying and strengthening what is believed as 
key resources in the journey towards achieving objectives. The resources evoked here can be both tangible 
and intangible to encompass a large array of organizational aspects: brand names, in-house knowledge of 
technology, personnel skills, efficient procedures….Accordingly, a capability is the capacity to utilize resources 
to perform a task or an activity, against the opposition of circumstances (Teece, 2014). In other words, it is the 
capacity of an organization to select, through a learning process (Andreu & Ciborra, 1996), the resources to 
be considered as key, and thusly, addressed as core capabilities of the organization. To be considered as 
core, capabilities need to reveal (i) valuable, (ii) rare, (iii) imperfectly imitable by other organizations and finally 
(iv) with no strategically equivalent substitutes (Barney, 1991). Finally, a representative image of this approach 
cannot be completed without emphasizing the unescapably dynamic character of these capabilities. Indeed, 
what is a core capability in a given market, technological or regulatory context can become useless if one or 
several of these contextual variables change. Organizations are therefore required to develop the routines and 
processes by which they achieve new resources reconfigurations out of which more suitable and potentially 
new core capabilities emerge. 

364



According to this extremely brief introduction to this theoretical framework, we can already emphasize two key 
differences with the cybernetic paradigm discussed above: 

1. Philosophically, the core capability approach focuses on resources and their combinations required
to achieve strategic objectives. To be operationalized, a thorough analysis of the organization’s
environment as well as the building of a comprehensive model linking available or to be purchased
resources with final strategic orientations is required. Comparatively, this focus on developing a
comprehensive model of the organization’s value creation process is totally left aside by the
cybernetic paradigm where the system is seen as a black box which corrects its trajectory with
respect to predefined objectives or standards.

2. Operationally, actions at the individual and collective levels induced by capability approaches are
meant to create or reinforce a specific way of using resources. It requires a qualitative model, namely
architecture, linking resources configuration to developed capabilities and capabilities to strategic
objectives. Building such an architecture is a process around which may revolve a large variety of
expertise in the organization using natural phrasing or accessible modeling approaches. The
inclusiveness of such a process can be a key asset in ensuring organizational alignment and
commitment to achieve strategic objectives. Here again, the cybernetic approach on the other hand
may rely on cold and top-down objectives and metrics which relevance and initial aim may remain
hidden to those impacted by them. Such a situation can worsen if the development of indicators is
done in silos leading, at the operational level, to potential conflictual objectives.

Thinking in terms of core capabilities is not a stranger in the field of safety sciences. One of the key 
frameworks that has largely developed in the last decade is the one of High Reliability Organizations (HRO). 
HRO is an empirical-based corpus of literature interested in identifying the properties of organizations 
displaying excellent safety records despite operating in high risk, uncertain and volatile environments. 
Amongst the most decisive contributions in this field, the work of Weick and Sutcliff (2001) distinguishes 5 
hallmarks repeatedly identified in this type of organizations and believed decisive in achieving such high 
performances: 

a. Pre-occupation with failure spots the organizational ability to continually look for existing failures,
even the smallest, or anticipating the emergence of new ones. Such an individual and collective state
of mind is built upon the shared awareness that knowledge available is incomplete and attention to
what the smallest incident may reveal is vital to safety.

b. Reluctance to simplify is built upon the need to overcome oversimplifications introduced by the use of
predefined models, categories or synthesized information. A higher granularity in information
acquisition and more inclusiveness and argumentation in their processing is required to grasp what
may appear as details but which tend to reflect the complexity of real situations. Such a statement is
of particular importance with regard to indicators tendency to synthetize and sometimes oversimplify
complex realities.

c. Sensitivity to operations reflects the managers and staff’s ability to display real-time/immediate
capabilities for errors identification and improvements identification. This capability is highly
operational and anchored in the present while risky activities are being carried.

d. Commitment to resilience. Here again, with individuals being conscious of the incomplete character
of available knowledge, anticipation needs to be completed with adaptation. Accordingly, resilience,
defined as the combination of elasticity and recovery, becomes a highly valuable capability based not
only on individual competencies but also en collective cooperation.

e. Deference to expertise refers to the collective agreement to hand over, in specific situations, decision
making to individuals with the required expertise regardless of their hierarchical status. Such a
capability cannot be improvised. It implies collective commitment to this value as well as a careful
anticipation and development of the required competencies in the organization.

Comparatively to the cybernetic paradigm discussed earlier, the following key differences are to be 
emphasized: 

- The cybernetic focus is on results whilst capability-based approaches are oriented towards shaping 
and combining what are believed as key resources. 

- The cybernetic paradigm is strongly anchored in a quantitative mindset inherited from its technical 
systems background whereas capability approaches are more qualitative and rely on inclusiveness 
and sharing of mental representations using natural language. 

Finally, the reference model used in the cybernetic paradigm focuses on what the organization should achieve 
whereas capabilities require further investigation to question the means through which objectives are to be 
achieved. In doing so, the capabilities mindset requires a deeper dive in organizational environment, identity 
and finalities allowing managers to reduce discrepancies between their representations and reality.  

365



4. Conclusions

By discussing the extent to which measurement may serve as shaky basis for decisions in one hand and the 
existence of alternative and highly valuable approaches on the other hand, this work does not suggest 
avoiding management measurement. Indeed, the two approaches discussed have their own limitations but 
should not be seen as incompatible. On the contrary, given the challenge of grasping the reality of highly 
complex organizations in one hand and the high value of diversity in management practices on the other hand, 
practitioners and scientists should further investigate how these approaches may be combined in order to 
reinforce their strengths whilst compensating their weaknesses. More practically, the use of indicators can be 
explored to help assessing implementation of HRO capabilities discussed earlier. At the same time, managers 
should go beyond the cybernetic paradigm to develop more explicative models of the capabilities required to 
achieve their safety objectives. 

Reference 

Adams, R. J., Smart, P., & Huff, A. S. (2017). Shades of Grey: Guidelines for Working with the Grey Literature in 
Systematic Reviews for Management and Organizational Studies. International Journal of Management Reviews 
19, 432-454. 

Andreu, R., & Ciborra, C. (1996). Organisational learning and core capabilities development: The role of IT. Journal 
of Strategic Information System 5, 111-127. 

Ansoff, H. I. (1975). Managing strategic surprise by response to weak signals. Californian Management Review 
18(2), 21-33. 

Ashby, R. (1961). An Introduction to Cybernetics. London: Chapman & Hall. 
Balakrishnan, A. V. (1973). Stochastic differential systems. New York: Springer Verlag. 
Barney, J. (1991). Firm resources and sustained competitive advantage. Journal of Management 17(1), 99-120. 
Brizon, A., & Wybo, J. L. (2009). The life cycle of weak signals related to safety. International Journal of Emergency 

Management 6(2), 117-135. 
Carver, C. S., & Scheier, M. F. (1982). Control Theory: A Useful Conceptual Framework for Personality-Social, 

Clinical, and Health Psychology. Psychological bulletin 92(1), 111-135. 
CEFIC. (2011). Guidance on process safety performance indicators.  
Cohen, M. D., March, J. G., & Olsen, J. P. (1972). A Garbage Can Model of Organizational Choice. Administrative 

Science Quarterly 17(1), 1-25. 
Engwall, L., Kipping, M., & Usdiken, B. (2016). Defining management. Business Schools, Consultants, Medias. New 

York and London: ROUTLEDGE. 
EPSC. (2011). Process safety leading and lagging metrics.  
Geyer, F. (2001). Sociocybernetics. Dans J. Smelser, & P. B. Baltes, International Encyclopedia of the Social & 

Behavioral Sciences (pp. 14549-14554). Elsevier Ltd. 
Giraud, F., & Zarlowski, P. (2011). Fundamentals of Management Control. France: Pearsons Education. 
Hofstede, G. (1978). The Poverty of Management Control Philosophy. The academy of Management Review (3), 

450-461. 
Lindblom, C. E. (1959). The science of "Muddling Through". Public Administration review 19(2), 79-88. 
Lindblom, C. E. (1979). Still Muddling, Not Yet Through. Public Administration Review 39(6), 517-526. 
Lukic, D., Margaryan, A., & Littlejohn, A. (2010). How organisations learn from safety incidents: a multifaceted 

problem. The Journal of Workplace Learning 22(7), 428-450. 
Morgan, G. (2006). Images of Organizations. London: Sage Publication. 
Scott, J. (1999). Rational choice theory. Dans G. Browning, A. Halcli, & F. Webster, Understanding Contemporary 

Society: Theories of the Present (pp. 126-138). London: Sage Publications. 
Shinskey, F. G. (1967). process Control System. New York: Mc Graw Hill. 
Snowden, D. (2002). Complex act of knowing: Paradox and descriptive self awareness. Journal of Knowledge 

Management 6(2), 100-111. 
Stemn, E., Bofinger, C., Cliff, D., & Hassall, M. E. (2018). Failure to learn from safety incidents: Status, challenges 

and opportunities. Safety Science 101, 313-325. 
Teece, D. J. (2014). A dynamic capabilities-based theory of entrepreneurial theory of the multinational entreprise. 

Journal of International Business Studies 45, 8-37. 
Uzonwanne, F. C. (2016). Rational Model of Decision Making. Dans A. Farazmand, Global Encyclopedia of Public 

Administration, Public Policy, and Governance (pp. 73-88). Springer. 
Weick, K. E., & Sutcliffe, K. M. (2001). Managing the Unexpected: Assuring High Performance in an Age of 

Complexity. San Francisco: John Wiley and Sons. 
Wernerfelt, B. (1984). A resource-based view of the firm. Strategic Management Journal 5(2), 171-180. 
Wiener, N. (1948). Cybernetics: Or control and Communication in the Animal and the Machine. Cambridge: MIT 

Press. 

366