DOI: 10.3303/CET2290124 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Paper Received: 14 February 2022; Revised: 30 March 2022; Accepted: 12 May 2022 
Please cite this article as: Secheppet M., Bony-Dandrieux A., Bouillet P., Tixier J., 2022, Improving an Observation Tool of crisis management 
teams by involving Crisis Managers, Chemical Engineering Transactions, 90, 739-744  DOI:10.3303/CET2290124 
  

 CHEMICAL ENGINEERING TRANSACTIONS  
 

VOL. 90, 2022 

A publication of 

 
The Italian Association 

of Chemical Engineering 
Online at www.cetjournal.it 

Guest Editors: Aleš Bernatík, Bruno Fabiano 
Copyright © 2022, AIDIC Servizi S.r.l. 
ISBN 978-88-95608-88-4; ISSN 2283-9216 

Improving an Observation Tool of Crisis Management Teams 
by involving Crisis Managers 

Mélanie Secheppet*, Aurélia Bony-Dandrieux, Philippe Bouillet, Jérôme Tixier 
Laboratoire des Sciences des Risques (LSR), IMT Mines Ales, Ales, France 
Melanie.Secheppet@mines-ales.fr 

In the frame of the CRIZ’INNOV project, IMT Mines Ales is in charge of developing an observation tool in order 
to enhance vocational training of crisis managers. Using a research approach articulating research-design-
training, the cooperation between Researchers, Engineers and Crisis Managers is fundamental. With co-design 
goal, the first step was to meet potential users and integrate their in everyday work practices. To reach this goal, 
interviews and observations were realised. Results allowed to identify needs, practices and to assess 
observation’s tool feasibility. These first outcomes have oriented characteristics of observation tool. 

1. Introduction 
Training crisis managers is a major challenge in optimizing crisis management [Direction générale de la sécurité 
civile et de la gestion des crises, 2020]. Ongoing training of crisis managers regularly takes the form of a 
workplace exercise in which crisis actors act as if the simulated phenomena were real. This kind of exercise, 
called functional (Trnka and Jenvald, 2006), allows to replicate the actions and interactions that a crisis unit 
would experience in the event of real crisis management (Tena-Chollet et al., 2017). 

1.1 Crisis exercises challenges 

To prepare for the management of crises such as those that may result from technological accidents or natural 
disasters, decision-making teams can be immersed in a simulated crisis situation. The use of simulation in 
vocational training is commonly used in high-risk areas such as health (Horcik et al., 2014), military (Bisschop 
and Leblanc, 2022) or public safety (Sauvagnargues, 2018). This pedagogical format is based on a principle of 
fictitious experience that encourages participants to act as if the simulated crisis were real. Therefore, on the 
basis of a crisis scenario, the decision-making team can be activated to play its own role and train. (Fleming et 
al., 2020). 
Civil or industrial authorities propose frameworks to guide the implementation of exercises (e.g. Federal Office 
of Civil Protection and Disaster Assistance, 2011). These scoping texts highlight the planning, preparation, 
conduct and analysis phases of exercises. They also present the different roles played by exercise facilitators, 
players, observers and evaluators. While these texts provide an overview of the implementation of the exercises, 
they are not sufficient to develop and implement observation activities. 
Research on vocational training through simulation highlights different aspects of this pedagogical approach: 
the design of scenarios (Limousin et al., 2018), the development of team skills (Crichton, 2009), the place of 
debriefing in the simulation (Fanning and Gaba, 2007; Gardner, 2013). This researchs focuses on the 
relationship between the exercise phase and the debriefing phase to support the development of participants' 
skills. This concern for the relationship between play and debriefing is also present in health training (Abelsson 
and Bisholt, 2017). It raises the question of the relationship between observations and discursive reflections. 
The debriefing should allow to review the actions carried out during the simulation in order to discuss them and 
improve the team’s performance. The content of the discussion is therefore based on the different observations 
and their memorization. 

739



1.2 Importance of observation 

To provide a common frame of reference for the analysis of exercises, Jenvald and Morin (2004) propose an 
impartial observation method based on multiple data sources including observation data. Audio-video recordings 
and observation records are used to compile data for the purpose of constructing a global view of the exercise 
to support debriefing (Jenvald and Morin, 2004). This systemic approach is in line with the idea of defining the 
objectivity of observation by “an interpretive movement that develops in the intersubjective relationship between 
the observer and the observed in an observation situation” (Bonnemain et al., 2015, p. 102). Therefore, 
objectivity of observation is not a position of overhang of an expert view but rather a dialogue between the 
different sources of observation and construction of traces of activity. This approach to objectivity is linked to 
the notion of interpretation of giving meaning to what is observed (Norros, 2004). 
Referring to different observation tools, Lapierre (2016) developed observation grids for evaluation and 
debriefing assistance. In order to do this, the author created a reference framework for the work of crisis 
management teams, classified into 3 axes, 17 missions, 64 objectives, 192 learning objectives, 6 categories of 
knowledge and 15 skills. One of the interests of his research is to conceive the observation within an approach 
that integrates and links exercise preparation, observation and exercise implementation. Thus, the observation 
is divided between players, facilitators and observers in order to foster the reconstruction of a global vision of 
the experienced exercise. However, one of the limitations of this tool is related to the segmentation of actions 
which makes them lose visibility in their context of appearance. 

2. Theoretical and methodological framework 
In order to offer a complementary entry to the work already carried out in vocational training through simulation 
and in the conceptualisation of observation tools, an anthropo-technological approach had been chosen, as 
defined in the research program for the chosen ‘course of action’ (Theureau, 2002). 

2.1 The consequences of an “activity entry” 

This approach considers activity as the dynamics of the actor’s interactions with his environment. According to 
enaction paradigm (Varela and Maturana cited by Theureau, 2002), these interactions emerge of couple actor-
environment in the situation. 
Although it is the learning of the crisis management team that is targeted, the hypothesis put forward is that this 
development goes through the activity of observing the crisis exercises. Thus the crisis exercise observation 
activity was considered as a work to be paid attention to. This activity is considered both individual and social. 
As such, the activity of the observer is intrinsically linked to the activity of the crisis management team observed 
during the exercise. This observation activity is also finalized and inscribed in a culture: the observer is expected 
to report on his observation in order to participate in the improvement of the collective. Following advices of 
Flandin et al. (2018), “support participatory-sensemaking and collective sensemaking” is an adopted principle 
to drive the development of the observation tool. So if the focus is on the observer’s activity, this is always 
considered in the exercise situation and toward it enhancing. 

2.2 Interview methodology and analysis 

Following an anthropo-technological perspective such as that proposed by Poizat et al. (2016), the research 
carried out is intended to be cooperative and not only applicative. Thus, the first study, presented here, consisted 
of meetings with the organisers of crisis exercises of local major risk prevention and civil security departments. 
The construction of the data was based on an interview method inspired by comprehensive interviews 
(Kaufmann, 2016) while incorporating the situational character of the ordinary activity of the exercise organisers 
in accordance with the ontological principles of the ‘action course’ program (Poizat et al., 2016). The interviews 
had three aims: first, to mobilize exercise organisers, then to take the ordinary constraints of the work, finally, to 
build on existing actual practices. 
Each recorded interview was transcribed. Interview notes and note taking were then classified according to 
emerging categories: work constraints, observer posture (Figure 1), observer-observed relationship, 
observable, observation tools. This allows a view of all interviews at the same time according to a common 
category. 
The results were based on these categories. The analysis was performed manually by a researcher. The results 
were compared with the eyes of the CRIZ'INNOV research team and the participants of the interviews. 
 

740



Table 1: Four interviews 

Interview 1 Interview 2 Interview 3 Interview 4 

Head of Prevention and 
Major Risks 

City of about 40000 
inhabitants 

 

Exercise organiser 
Interdepartmental Civil 
Defence and Protection 

Department 
Occitanie Region 

 

Head of Interdepartmental 
Civil Defence and 

Protection Department 
Occitanie Region 

 

Exercise organiser 
Interdepartmental Civil 
Defence and Protection 

Department 
Occitanie Region 

 
Interview in face to face Interviews in face to face Videoconference Videoconference 

Date 31-05-2021 
Date 03-06-2021 and 21-

06-2021 
Date 29-06-2021 Date 20-07-2021 

Duration: 83 minutes 
First not recorded; Second, 

duration: 47 minutes 
Duration: 81 minutes 

Not recorded; notes-taking 
in real time by two 

researchers (13 pages 
word) 

 

 

Figure 1: Extract from the results analysis table 

3. Results 
These interviews allowed us to better understand the needs and constraints of the end users of the observation 
tool in their daily work. These elements are important to build a tool adapted to their expectations and thus 
facilitate its appropriation and use. 

3.1 Towards an user-friendly and scalable tool 

The analysis of the interviews reveals two ordinary constraints experienced and evoked by the managers of the 
exercises of local authorities and public departments: the multiplicity of the missions of their department and the 
difficult mobilization of the teams for the exercises. These two constraints are related to time management and 
human resource management concerns. They also show a need for continuous training for the organization of 
the exercises as a whole and for observation of exercises in particular. 
Having to deal with the multiplicity of missions of the department is a constraint mentioned during the four 
interviews with the organisers of crisis management exercises. This aspect of the job describes a work 
environment constrained by urgency and time constraints. This has an impact on the the time spent for the 
preparation and implementation of exercises. One interviewee estimated that 1/5 of the time is spent on crisis 
exercises. The remaining 4/5 being devoted to the other missions of the Prevention and Major Risk department 
of this municipality of almost 40,000 inhabitants. Another interviewee explained that the other tasks of the 
department add up when they spend time organising and carrying out the exercises. This means that these 
other tasks will have to be done later. There are also times when the exercise is no longer a priority due to 
unforeseen events that need to be dealt with urgently. These testimonies of the interviewees are corroborated 
by the observation of the researcher during her intervention in these depatments. For example, during an 
exercise, an unforeseen event took over an exercise forcing one of the exercise organisers to manage the event 

741



and temporarily withdraw from the organisation of the exercise. Thus, the unexpected and the emergency 
characterize the departments and constrain the organisation of exercises. 
Another constraint mentioned in the four interviews with crisis management exercise organisers was the amount 
of human resources needed to carry out the exercise. The difficulty in mobilizing teams affects the priority of 
allocating human resources to the roles of players and facilitators. This constraint impacts the observation by 
relegating it to the background or even abandoning it. The allocation of human resources to observation is then 
seen as a deprivation: “Some services make the sacrifice and put observers” says one of interviewees. Another 
stressed that it is difficult to gather all the people needed for the exercises. These exercise organisers also 
express regret at having to abandon certain aspects of exercises such as observation: “I don’t have an observer 
like I should have, […] [who] is there to observe, […] [who] has his little booklet and he writes”. At the same 
time, they indicate the interest to set up observers: the benefit an external point of view or the added value of 
the crossed-views. 
Following this initial analysis of the interviews on the service’s work constraints, two ergonomic components 
should be considered in the design of the observation tool: 

• Offer a tool that is easy to use and can be used without special training 
• Provide a scalable tool for both basic and in-depth observation 

The interviews also made it possible to highlight the practices used when observation is possible as well as the 
obstacles and difficulties encountered when carrying out the observation. 

3.2 Towards a shared culture of observation 

Questions oriented towards the practice of observation reveal a type of observation qualified as intuitive. 
Observers report comfort and discomfort areas in their usual exercise observation activity. 
Crisis exercise organizers speaking about their observation practices all referred to an intuitive form of 
observation. This type of practice is “hands-on learning” based on personal experience in crisis management 
and exercises. One of the interviewees retrospectively analyses the development of his observation skills: “my 
observation when I started was totally different from the one I did when I finished”. The interviewee continued 
his speech by highlighting the fundamental role of the observer’s experience. The practice of observation and 
knowledge built on crisis management transforms the activity of the observer. Another interviewee talks about 
empirical observation based on their experience and what they know how to do. He adds that “it is without 
technique”. 
In the practices mentioned by the interviewees note-taking is done in real time and is based on the observer’s 
experience. His experience in crisis management and his participation in crisis exercises allow him to implicitly 
identify the different observation criteria that he considers important with regard to the exercise’s objectives. 
Two different ways of taking notes are described by people: either a note-taking of time-stamped events, or a 
note-taking of the thematic categories identified at the time of observation. For observers, the interest of the first 
approach is the search for factual precision; the interest of the second one is to obtain a synthetic overview of 
the crisis unit and its actions. 
Furthermore, interviewees expressed areas of comfort and discomfort in their experiences of crisis exercise 
observation. These areas of comfort and discomfort are characterized by: occupied posture (observer or 
observed), the relationship between observer and observed, the identification and interpretation of elements to 
be observed (Table 2). 

Table 2: Areas of comfort and discomfort expressed by interviewed observers 

Areas of comfort Areas of discomfort 

• Identify factual and technical observables: • Search for completeness in the note-taking 

-“It is easy to observe something very factual, the phone 
call was made at such and such a time” 

• Posture of seeing others and being seen to be 
recording observations (posture associated 
with the idea of spying or controlling) 

- “I note the physical organization. This is most obvious. 
(…) material organization, knowledge of each person’s 
role and (…) knowledge of the contingency plan” 

• Hierarchical positioning (associated with an 
unwanted overhang position) 

 • Difficulty in observing behaviours 

742



The analysis of this result on the practice of observation reveals a solitary and isolated form of observation 
activity. It therefore seems necessary to develop the culture of crisis exercises by sharing observation 
experiences within a community. Following the identification of this second category of analysis, two aspects 
emerge that need be taken into account in the development of observation through an observation tool: 

• To design a tool that that opens up possible spaces and allows the sharing of observation experiences 
• To develop an observation tool and methodology for sharing didactic and pedagogical knowledge on 

crisis observation 

4. Conclusions 
The results put forward are the consideration of the daily work and constraints of the field actors, the practice of 
an intuitive observation and the development of a working community consisting of exercise organizers, crisis 
manager, engineers and researchers. These results allow discussion of the observation methods already 
existing in the scientific literature and based on technical and non-technical skills grids. This discussion 
contributes to the creation of a set of requirements for the crisis exercise observation tool. Linking the scientific 
literature with the experiences of the field actors helps to guide both the existing and relevant resources to be 
retained as well as their transformation for co-development of an appropriate tool by exercise organizers and 
crisis managers. 

Acknowledgments 

We would like to thank the Prefecture of the Occitanie Region, which is the funder of the Criz’Innov project. 
We would also like to thank those responsible for the crisis exercises who agreed to share their experience 
through the interviews. 

References 

Abelsson, A., & Bisholt, B. (2017). Nurse students learning acute care by simulation – Focus on observation 
and debriefing. Nurse Education in Pratice, 24, 6-13. https://doi.org/10.1016/j.nepr.2017.03.001 

Bonnemain, A., Perrot, E. & Kostulski, K. (2015). Le processus d’observation, son développement et ses effets 
dans la méthode des autoconfrontations croisées en clinique de l’activité. Activités [en ligne], 12(2). 

Crichton, M.T. (2009) Improving team effectiveness using tactical decision games. Safety Science, vol. 47, 
p. 330–336. DOI: 10.1016/j.ssci.2008.07.036. 

De Bisschop, H. & Leblanc, S. (2022). How do simulated high-intensity situations train leaders to maintain their 
ability to act in unfamiliar, unforeseen or uncertain environments? Cases from crisis preparedness training 
at the École Militaire Interarmes. In S. Flandin., C. Vidal-Gomel & R. Becerril Ortega (Coord.), Activity-based 
simulation training: informing the design of simulation programs for professional development. Springer 
International Publishing. https://www.springer.com/gp/book/9783030895662#aboutBook 

Direction générale de la sécurité civile et de la gestion des crises (DGSCGC). (2020). Rapport d’activité 2019. 
Ministère de l’intérieur. Paris, France. https://www.interieur.gouv.fr/Le-ministere/Securite-
civile/Documentation-technique/Rapport-d-activite-de-la-DGSCGC 

Fanning, R.M. & Gaba, D.M. (2007). The role of debriefing in simulation based learning. Simulation in 
Healthcare, 2(2), 115-125. 

Federal Office of Civil Protection and Disaster Assistance. (2011). Guideline for Strategic Crisis Management 
Exercises. BBK. Germany. URL: Crisis Exercise | Crisis Management | Software Development (scribd.com) 

Flandin, S., Poizat, G. and Durand, M. (2018), "Improving resilience in high-risk organizations: principles for the 
design of innovative training situations", Development and Learning in Organizations, Vol. 32 No. 2, pp. 9-
12. https://doi.org/10.1108/DLO-03-2017-0027 

Gardner, R. (2013). Introduction to debriefing. Seminars in Perinatology, 37, 166-174. 
Horcik, Z., Savoldelli, G., Poizat, G., & Durand, M. (2014). A phenomenological approach to novice nurse 

anesthetists’ experience during simulation based training sessions. Simulation in Healthcare, 9, 94-101. doi: 
10.1097/ SIH.0000000000000021 

Jenvald, J. & Morin, M. 2004. Simulation-supported live training for emergency response in hazardous 
environments, SIMULATION & GAMING, Vol. 35 No. 3, September 2004 363-377 DOI: 
10.1177/1046878104266223 © 2004 Sage Publications 

Kaufmann, J.-C. (2016). L’entretien compréhensif. Paris : Armand Colin. 
Lapierre, Dimitri (2016) Méthode EVADE : Une approche intégrée pour l'EValuation et l'Aide au DEbriefing. 

Thèse de doctorat. Université de Nîmes, Nîmes. 
Limousin, P., Bony-Dandrieux, A., Tixier, J., Sauvagnargues, S. (2018). Improving Crisis Exercises and 

Managers’ Skills through the Development of Scenario Design. In S. Sauvagnargues, Decision-making in 

743

https://doi.org/10.1016/j.nepr.2017.03.001
https://www.springer.com/gp/book/9783030895662#aboutBook
https://fr.scribd.com/document/386228229/Crisis-Excersice


Crisis Situations: Research and Innovation for Optimal Training, ISTE, pp.59-78. 978-1-119-55782-1. 
ff10.1002/9781119557869.ch3ff. ffhal-02923807f 

Morin, Jenvald & Crissey (2004a). Using Simulation Modeling and Visualization to Prepare First Responders for 
Homeland Defense. Proceedings of the Second Swedish, American Workshop on Modeling and Simulation 
(SAWMAS 2004). 

Morin, M., Jenvald, J. & Thorstensson, M. (2004b). Training First Responders for Public Safety Using Modeling, 
Simulation and Visualization. SIMSafe 2004, June 15-17, Karlskoga, Sweden. 

Norros, L. (2004) Acting under uncertainty. The core-task analysis in ecological study of work. VTT publications, 
546. En ligne : https://www.vttresearch.com/sites/default/files/pdf/publications/2004/P546.pdf, consulté le 4 
février 2021. 

Poizat, G., Durand, M. & Theureau, J. (2016). The challenges of activity analysis for training objectives. Le 
travail humain, 3(3), 233-258. https://doi.org/10.3917/th.793.0233 

Sauvagnargues, S. (Ed.). (2018). Decision-making in Crisis Situations. Research and Innovation for Optimal 
Training. London: ISTE Editions. 

Tena-Chollet, F., Tixier, J., Dandrieux, A. & Slangen, P. (2017). Training decision-makers: Existing strategies 
for natural and technological crisis management and specifications of an improved simulation-based tool, 
Safety Science, 97, 144-153. 

Theureau J. (2002) First version fully developed of a chapter « course of action centred design », proposed for 
E. Hollnagel ed., Handbook of cognitive task design, Lawrence Erlbaum Ass. Access on Microsoft Word - 
2002-JT-T15 (coursdaction.fr) 

Trnka, J. & Jenvald, J. (2006). Role-Playing Exercise – A Real-Time Approach to Study Collaborative Command 
and Control, International Journal of Intelligent Control and Systems, 11(4),218-228. 

744

https://doi.org/10.3917/th.793.0233
http://www.coursdaction.fr/08-nonpublies/2002-JT-T15.pdf
http://www.coursdaction.fr/08-nonpublies/2002-JT-T15.pdf

	lp-2022-abstract-134.pdf
	Improving an Observation Tool of Crisis Management Teams by involving Crisis Managers