Running head: AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

Emergency Management After-Action Review Processes to Influence Ontario Municipalities
Final Report
Amy Leggate J0048935
Bachelor of Emergency and Security Management Studies
Justice Institute of British Columbia
ESMS 4900 19WI
Instructor: Beth Larcombe
Advisor: Amy Severson
April 14, 2019

1

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

2

Abstract
Organizational agility and learning from lessons play an important role in enhancing emergency
management programs and the after-action review (AAR) facilitates this process. This paper
discusses the role of the Office of the Fire Marshal and Emergency Management Ontario
(OFMEM) and how it can better encourage the standardized use of the AAR by municipalities in
Ontario. This paper reviews different methods and essential elements of the AAR and makes
recommendations for the OFMEM to consider.
Keywords: After action review/report, after-action process, hot-wash, lessons learned,
debrief.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

3

Table of Contents
Abstract .......................................................................................................................................... 2
Keywords ....................................................................................................................................... 2
Background ................................................................................................................................... 4
Definition of the Question and Rationale................................................................................ 5
Problem ...................................................................................................................................... 5
Intervention ............................................................................................................................... 6
Comparison ............................................................................................................................... 6
Outcome of Interest .................................................................................................................. 7
Literature Review ......................................................................................................................... 7
Search Methodology ................................................................................................................. 7
Selecting Articles to Review ..................................................................................................... 8
Description ................................................................................................................................. 9
Critical Appraisal........................................................................................................................ 13
The OFMEM ........................................................................................................................... 13
AAR Background Information .............................................................................................. 14
Best Practices ........................................................................................................................... 15
Recommended AAR Facilitation Processes .......................................................................... 16
Active and Reflective Learning.............................................................................................. 17
Inter-organizational AAR Sharing........................................................................................ 18
Essential Elements Supported by Evidence.......................................................................... 18
Municipal Surveys .................................................................................................................. 19
Discussion..................................................................................................................................... 20
Evaluation and Recommendations ............................................................................................ 21
Conclusion ................................................................................................................................... 23
References .................................................................................................................................... 25
Appendix A .................................................................................................................................. 27
Appendix B .................................................................................................................................. 28
Appendix C .................................................................................................................................. 29
Appendix D .................................................................................................................................. 30
Appendix E .................................................................................................................................. 35

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

4

Emergency Management After-Action Review
Processes to Influence Ontario Municipalities

Background
The after-action review (AAR) process is an important one that allows organizations and
municipalities to address organizational limitations, enhance strengths, and to learn from
experiences. The AAR is designed to promote employee and team development and is founded
on several areas of science, including information feedback, performance measurement,
cognition and memory, group processes, communications theory, and instructional science
(Tannenbaum & Cerasoli, 2013). The AAR is a structured opportunity for a shared retrospective
learning, innovative development, and continuous improvement (Scott, Dunn, Williams, &
Allen, 2015). Despite the encouragement found in emergency management literature, there is no
governance that mandates municipalities in Ontario to record or address lessons learned from
AAR’s after an incident or an exercise.
At a Federal level, the policy on emergency management (EM) articulates the need for
continuous improvement, lessons learned, and best practices for each of the four pillars of EM;
prevention/mitigation, planning, response, and recovery (Friesen, Kelsey, & Legere, 2017). The
identification of lessons learned at a Federal level across the EM pillars provides a basis for the
requirement at a provincial level.
At the provincial level, the Office of the Fire Marshall and Emergency Management
Ontario (OFMEM) is responsible for supporting municipalities to achieve the requirements of
the Provincial Emergency Management and Civil Protection Act (EMCPA), R.S.O. c.E.9 (1990),
EM program requirements. The OFMEM fulfills this responsibility by supporting the

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

5

implementation of municipal and provincial EM programs with suggested tools, developing
training, providing advice, and guidelines.
Another method of supporting municipalities was the OFMEM’s development of the
incident management system (IMS) for Ontario. By encouraging this system, OFMEM provided
a standardized organizational structure with functions, processes and terminology that can be
assumed by all levels of emergency management governance in Ontario (Ontario, 2017).
Although, federal governance and the full implementation of IMS encourages the practice of
documenting lessons learned in AARs when evaluating exercises and incident or event response
(Ontario, 2018), the doctrine does not provide AAR templates or clear procedures. This could
explain the reason that there is no clear evidence that the AAR process is utilized, or consistency
of the practice across Ontario municipalities.
Definition of the Question and Rationale
The research question is what literature is available to assist the OFMEM when making
AAR process suggestions to municipalities in Ontario? To define the question and the rationale,
this study utilizes the PICO method, identifying the problem, intervention, comparison, and the
outcome of interest.
Problem
Without proper guidance from the OFMEM in Ontario, the lessons learned process may
be avoided when a municipality experiences an incident or exercise. This is unfortunate because
lessons learned will increase organizational agility and play an important role in EM (Friesen et
al., 2017). Municipalities may not build well-rounded, broader, and strategic approaches to
lessons learned without proper steps taken after an incident or event (Friesen et al., 2017). Errors
may be repeated in the future if there is no organized system that considers their rectification.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

6

Further, if municipalities do complete the AAR process, what method do they use to input and
track the changes they have identified? What recommendations are there for sharing the
information with others inside and outside of the organization? And what research studies are
available to assist with these recommendations?
Intervention
Establishing an AAR process could allow municipalities to learn from what went well
and areas for improvement as a result of an incident or event. There are multiple approaches to
the AAR that may be considered and will be outlined in this study. The method for completing
an AAR can be verbal or written and should be followed up to ensure the lessons learned process
is functional. The amount of time taken, best practices, culture and learning environment are
considerations to take during the AAR initialization. Continuous improvement and encouraging
teams to work collaboratively are important goals to highlight when implementing an AAR
system (Scott et al., 2015).
Comparison
The main alternative to completing an AAR would be to partially complete or not
complete one. The choice ultimately will depend on the nature of the incident or event. When
considering the AAR process, there are many identified methods within the literature that should
be considered. Certain aspects such as communication, openness of the environment,
participants, and leadership styles should be considered. These considerations will be important
for the OFMEM when creating a thorough AAR process. The comparison process for
municipalities that do or do not complete an AAR is a good consideration for future research.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

7

Outcome of Interest
The outcome and desired effect of this study is to identify and articulate some AAR
processes that may be of use to municipalities in Ontario. The process of surveying
municipalities to begin research on how they manipulate the AAR process now will allow a
small amount of insight to the common practices in Ontario. The literature search will provide
multiple levels of AAR involvement at different agencies and organizations. This insight may
lend itself to future research and whether or not the AAR, when done completely and
appropriately, has an impact on community safety and resilience in Ontario.
Literature Review
This literature review begins with a detailed explanation of the search methodology and
selection process conducted to narrow the research used. Next, a review and analysis of each
selected article is conducted through a description, followed by a critical analysis of the
literature. Finally, a conclusion will bring together all of the sections of this literature review in a
comprehensive manner.
Search Methodology
Multiple electronic databases were searched for literature containing variants of the
keywords which included: after action review/report, after-action process, hot-wash, lessons
learned and debrief. The choice for these keywords relied on their meaning and
interchangeability within the emergency management field. The initial search for resources was
completed through the Justice Institute of British Columbia’s (JIBC) library system, followed by
Google Scholar, ResearchGate and other websites related to the topic.
The initial search for “after action review” recommended over 17,000 unique nonduplicated references with a filter for only scholarly journals. Further, with an inclusion of only

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

8

“full text” articles, the results provided was within the 10,000 range. Similar results were found
when searching for the various other combinations of keywords identified above. When
searching “after action review” and “process” together the results were fewer at approximately
1900, by simply adding a dash between after-action in this search, the recommendations were
then limited to 180. Finally, with the use of a Boolean search, 130 hits were found when
searching for: “after-action reviews” OR “after action reviews” AND “process” NOT “health.”
Health related articles were found to use too much jargon. This final search was the most
successful and held the most relevant articles related to this topic.
Selecting Articles to Review
Twenty-one articles were chosen for abstract review. These were selected for their
reliable source and their clear method or process of AAR’s. When reading the titles, they were
broad enough to encompass the research focus of this paper and detailed enough to inform the
content of the article would be beneficial to the research of the study topic. When scanning the
abstracts of these twenty-one articles, eleven articles that did not fit the scope of this paper were
excluded due to their use of jargon, lack of AAR method or redundant information. Ten clear and
concise articles that included some of the keywords within their abstracts were selected for full
review.
At a minimum, articles selected for full review included information regarding an AAR
and its importance to training or work environments; lessons learned and the future; the model or
process used for an AAR that is well defined; the definition of an AAR and its characteristics;
best practices, and some statistics supporting or not supporting the AAR. From the 10 articles
fully reviewed, one article was excluded due to a significant focus on the industry and job title it
was written about. Finally, nine articles were carefully chosen for final review and analysis.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

9

Description
The following is a review and analysis of nine selected articles including the title and
author(s), a brief description of the content, quotes, concepts or ideas that help define and
describe the research problem, and an explanation of how the article relates to this research.
The first article discussed is Do Team and Individual Debriefs Enhance Performance? A
Meta-Analysis by Tannenbaum and Cerasoli (2012). Tannenbaum and Cerasoli (2012)
completed their study using quantitative meta-analysis across published and unpublished
research on team and individual-level debriefs to conclude their effectiveness. It was found that
organizations can improve individual and team performance by approximately 20% to 25% by
using properly conducted debriefs. They further go on to discuss how the learning technique of
AAR’s can push learners to engage in further experimentation with ideas and actions. In
summary, Tannenbaum and Cerasoli (2012) discuss the effectiveness of debriefs and support the
research problem of this paper by stating that “self-discovery has been shown to be a key factor
in effective developmental experiences” (p. 232).
Reflections Today Prevent Failures Tomorrow, (Brock, McManus & Hale, 2009) is an
article written to discuss the use of the AAR effort to improve processes at the organizational
level, specifically to develop computer software. This article compares the U.S. Army AAR
process and the AAR steps that would be used on a software project. This distinction will allow
readers the ability to think outside the realm of using AAR’s in an army context and provides an
example of an AAR process within an organization. The following table was excerpted from this
article (Brock et al., 2009) and gives a brief example of how the process between the two can be
distinct.
Table 1.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

10

Parallels between AAR’s conducted for military and software projects

(Brock et al., 2009, p. 141).
In addition to this, the article details the process of using an AAR, which includes when to
conduct them, how to ensure success, expected leader mindset, accountability, owning the
process, and who can use the AAR.
Defining a Risk-Informed Framework for Whole-of-Government Lessons Learned: A
Canadian Perspective” (Friesen et al., 2017) discusses how lessons learned can play an important
role in organizational agility and that virtually all aspects of emergency management can derive a
benefit from a lessons learned program. This article provides insight for implementing an AAR
program from a Canadian perspective. Friesen et al. (2017) suggest a robust lessons learned
process and methodology that provides an evidence base used to inform decisions, guide
emergency management plans, strengthen prevention/mitigation strategies, as well as assist in
the development of tools for exercises and responses to “real operations” (Friesen et al., 2017, p.
219). This quote reinforces the argument for the need of guidance in the area of AAR program
development for Ontario municipalities.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

11

Implementing After Action Review Systems in Organizations: Key Principles and
Practical Considerations, a chapter written by Scott et al., (2015) within the edited book named
The Cambridge Handbook of Meeting Science, presents an input-process-output model of AAR
systems. The model shared in this chapter outlines a process that includes rules of engagement,
cultural inclusions, facilitation, and best practices. Scott et al. (2015) suggest that employees in
bona fide groups will find that they are more successful when they effectively capture and
process information and knowledge from their environments through discussion with their peers.
This model can be applied to the AAR conversations as it offers considerable opportunities for
employees to develop and refine what they know through discussion (Scott et al., 2015).
After-Action Reviews: Linking Reflection and Planning in a Learning Practice, (Darling
& Parry, 2001) describes the basics of AAR’s and best practices. The Darling and Parry (2001)
five step AAR model is modular and gives a basis for comparison in this research paper.
Examples of deeper questions are discussed such as “…[w]hat was our actual performance
compared to our intent, and what do we think caused our actual results…” (Darling & Parry,
2001, p. 67). This quality of dialogue offered in an AAR is just as important to their research as
the method or process itself.
The U.S. Army’s After Action Reviews: Seizing the Chance to Learn is an excerpt from
the book Learning in Action, a Guide to Putting the Learning Organization to Work written by
Garvin (2000), which describes the AAR process used by the U. S. Army in great detail. This
process, and the reasons they adopted it are advantageous as it provided examples to influence
the AAR suggestions by OFMEM. Garvin details the discussion points that an AAR would
revolve around as: “What did we set out to do? What actually happened? Why did it happen and
what are we going to do next time?” (2000, p.106). In the example, each question is broken

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

12

down into greater detail. This encourages a comprehensive understanding to the process and
concludes with the valuable advice that participants must reflect on their experiences to fully
understand the reasons for success or failures. The guidance shared in this book could provide
insight to the emergency management professionals of Ontario.
Emergency Management Ontario (2017) shares fifty-two recommendations on their
webpage Emergency Management Review from an independent consultant that encourages the
OFMEM to include a goal of ongoing improvement that identifies gaps in Ontario’s Emergency
Management program. One of these recommendations is to implement the AAR process. An
AAR process template is shared (Emergency Management Ontario, 2016) within the area of
exercise templates on the webpage Phase 4: After Action Report in the resource section of their
website but is not shared within their newest version of the community emergency management
coordinator (CEMC) handbook that is dated 2014.
Finally, an article written by Bolton (2016) called Use of the After-Action Review to
Improve Learning suggests the adoption of the U.S. Army method of AAR’s to improve
classroom quality of learning. This article lends itself to this research as many emergency
management professionals at the municipal level conduct training and exercises and this
approach would enhance their efforts to improve emergency management programing. For
example, Bolton (2016) suggests the first step of the U.S. Army AAR (review what was
supposed to happen) is based on the desired training objective and overall intent of the
organization commander (Bolton, 2016, p. 3). This equates in the classroom to the identification
of student learning objectives and outcomes as identified by the instructor and articulated in the
course syllabus (or training manual, etc.). This model also suggests a discussion format for
engaging students in course evaluation, which moves away from the traditional feedback form.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

13

The previous literature reviewed provides formidable evidence for this research paper.
Multiple methods or processes from different sources will be discussed. A significant focus on
the U.S. Army AAR process is discussed within the literature as it has been adopted after many
years and serves to be the basis for AAR models. The following section of this literature review
will provide a critical analysis.
Critical Appraisal
This critical review of the literature begins with the report provided to the OFMEM and
background information. Next, it includes the themes that emerged within the analysis: the issues
that connected the sources together including some different views and solutions identified.
Identified themes within the literature include culture, best practices, active learning, and
reflective learning.
The OFMEM
As mentioned in the background, in Ontario, the OFMEM is responsible for overseeing
the EMCPA and assisting municipalities and provincial ministries in implementing their
emergency management programs by providing tools, training, advice and guidelines. The
Minister of Community Safety and Correctional Services was asked by the Premier of Ontario to
conduct a review of Ontario’s emergency management systems (Emergency Management
Ontario, 2017). A report was released in August of 2017 by the independent consultant,
Lansdowne Technologies Inc. The consultant suggested the following in their report:
“Continue to promote the sharing of lessons learned and best practices. After Action
Reports (AARs) and tracking of lessons identified and corrective actions are important
for continuous improvement and fiscal planning. Re-energize the Provincial Emergency
Management Coordinating Committee (PEMCC) and regularize meetings. Utilize social

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

14

media and e-forums when possible” (Emergency Management Ontario, 2017,
Recommendation number 52, no page number available).
As of February 2019, there is only one document provided from the OFMEM that
provides a template for AARs, it is located deep within the website suggesting the use of a sixstep AAR process to be completed after exercises (Emergency Management Ontario, 2016). The
six steps consist of: executive summary, exercise overview, exercise goals and objectives,
exercise events synopsis, objective-based evaluations, and conclusions. Although, the
methodology of an AAR program is outlined in the OFMEM website, there is no mention of
AARs within the newest version (updated in 2014) of the CEMC handbook.
AAR Background Information
In order to guide AAR use in Ontario municipalities, a thorough understanding of AARs
and processes mentioned within the literature is required and included below. The after-action
review can also be called debrief, lessons learned, or a hot-wash. They are relatively inexpensive
and can be a quick intervention for enhancing performance (Tannenbaum & Cerasoli, 2013). The
U.S. Army has been effectively utilizing the AAR since the 1980’s, embraced by soldiers, it has
become an integral part of a mission plan (Brock et al., 2009).
Culture
It is critical the AAR process be integrated into the culture of the organization (Brock et
al., 2009; Darling & Parry, 2001), and supported by senior leadership and employees alike. A
sure way to do this is to take the feedback of an AAR and implement it, as participants will see
that the energy they spend conducting the AAR is beneficial and productive. The process of the
AAR “is truly a [culturalized] phenomena used to maintain a focus on continuous process
improvement (Brock et al., 2009, p.141). It is found that when organizations facilitate AAR’s on

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

15

an ongoing basis (post event/incident/exercise) individuals learn as a result, therefore growing
the organization (Brock et. al., 2009).
The people who facilitate and participate in an AAR are not just individuals but are also
representatives of various cultural groups (Scott et al., 2015) and an important cultural factor to
consider. Different risk perceptions and interpretations will be found across the spectrum of
individuals. An open sharing environment that allows individuals to express candid information
is imperative.
Best Practices
AAR industry or organizational best practices have been outlined by several studies
included in this literature review, for example, ensuring someone is assigned to facilitate who is
skilled and able to keep the group on track, and encouraging an open and sharing environment
that leaves rank and status at the door (Bolton, 2016; Brock et al., 2009; Garvin, 2000; Scott et
al., 2015).
When to complete the AAR will depend on the situation, but in most situations, it is
recommended to complete an AAR shortly after the completion of an activity, or as soon as
possible to minimize memory losses (Garvin, 2000). A different point of view suggests the AAR
should provide continuous reflection after event milestones and be completed during and after
the event to maximize planning for the next event (Brock et al., 2009; Darling & Parry, 2001).
The differences in this approach and efficiencies is dependent on the event being planned.
The discussion processes that occur during an AAR are recommended to include the core
elements of enactment, selection, and retention (Scott et al., 2015). Enactment: early portions of
the AAR discussion generally focus on bracketing off (enacting) relevant portions of the event
for further attention, in other words the most important elements of the incident (Scott et al.,

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

16

2015, p. 1105). It is necessary to pause here as a facilitator and remind participants of the
opportunity to make suggestions about other issues they would like to discuss to ensure that
nothing is left out (Scott et al., 2015, p. 1105).
Selection: facilitators will notice the inputs that the group chooses to enact may be
interpreted a certain way. During selection, the information should be organized in a way that
discovers what it means and what should be done about it. Facilitators should understand that
this process is not linear but can occur through ongoing trial and error (Scott et al., 2015, p.
1105). The facilitator plays a critical role in the selection process in two ways, they manage the
flow of discussion to make sure that a wide range of individuals have the opportunity to
participate. Secondly, the facilitator must control their influence on the direction of the
discussion by maintaining a neutral physical and verbal response to conversations (Scott et al.,
2015, p. 1105).
Retaining: to influence retention of information and lessons learned in the AAR process
for participants, the facilitator must summarize repeatedly the key lessons learned and carryout
any follow-up that the participants collectively deemed necessary (Scott et al., 2015, p. 1106).
In addition to best practices, there are some practices to avoid, including: misdirecting
and subversion within the AAR to promote a false representation of the event, stifling
communication by responding in a way that prematurely disconfirms their viewpoint, assigning
blame and inhibiting the processes that help to synthesize explanations (Scott et al., 2015).
Recommended AAR Facilitation Processes
The best AAR’s follow a well-defined path, where timing and discussion points are predetermined (Garvin, 2000). Roughly 25 percent of the time allotted to complete the AAR should
be devoted to the objective and actual events log. Following this, another 25 percent of the time

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

17

allotted should be used to discuss the “why” or the reason it happened that way. The last half of
the AAR should be used to discuss future actions and what should be expected next time
(Garvin, 2000). A facilitator should be prepared and accepting of criticism and should begin a
discussion regarding future actions with a statement explaining what they did wrong and how
they plan to rectify it in the future. The facilitation process should be seen as an art form and
regarded as such. The facilitator should have the ability to guide a conversation without
dominating it, ensure that all necessary discussion points are achieved, and possess the ability to
read and understand non-verbal cues from the crowd through their body language to ensure a
successful AAR is achieved (Garvin, 2000).
Active and Reflective Learning
Active learning is a consistent theme in the AAR process that encourages participants to
engage in experimentation with ideas and actions (Tannenbaum & Cerasoli, 2013, p. 232). “A
true AAR practice pays attention to future actions, not just reflections on what has happened to
date” (Darling & Parry, 2001, p. 65). This quote gives a deeper meaning to Tannenbaum and
Cerasoli’s (2013, p. 232) mention of experimentation with ideas and actions. By experimenting
with new ideas and actions, enhancements are being implemented. This process of active
learning allows the participants to have “a state of mind where everybody is continuously
assessing themselves, their units, and their organizations and asking how they can improve”
(Garvin, 2000, p. 106). Reflective learning is a product of the AAR, as it provides the
opportunity for both individuals and groups to engage in a multi-dimensional learning activity
which allows continuous improvement (Bolton, 2016; Brock et al., 2009; Friesen et al., 2017;
Tannenbaum & Cerasoli, 2013).

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

18

Inter-organizational AAR Sharing
Finally, the process of what to do with the AAR after it has been completed has been
identified as a gap in operationalizing AAR’s in some of the literature, for example, adding them
to an online database (Friesen et al., 2017). Without follow-up or sharing of AARs with others in
the organization or possibly other organizations, some lessons learned may be lost which leads to
the same errors being repeated again in the future. Sharing knowledge and best practices with
other organizations can be completed face-to-face, voice/video-conference, or electronic postings
(Brock et al., 2009). The submission process should be clear and concise and should provide the
opportunity to serve the larger EM community. Table 3 (Appendix C) outlines a process or
method for incentivizing EM professionals to share their completed AAR’s, and provides an
overview of general issues and recommendations for the lessons learned submission process
(Friesen et al., 2017, p. 229).
The right reporting forms, templates, and instructions that enable a variety of end users to
complete and submit an AAR is an important component (Friesen et al., 2017). Artifacts must be
easily accessible, such as checklists, a common submission template and tools for process
reporting to senior management that allow recommendations, lessons learned and AAR
information from stakeholders to be captured and reviewed (Friesen et al., 2017).
Essential Elements Supported by Evidence
Five hypotheses were tested in the study completed by Tannenbaum and Cerasoli (2013).
Of these five, three were supported. The information found in this study should be helpful for the
OFMEM when determining the direction and goals of the AAR program. The following
hypotheses were supported (Tannenbaum & Cerasoli, 2013, p. 233): “Hypothesis 1: Individuals
and teams that use debriefing are more effective than individuals and teams that do not use

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

19

debriefing.” This shows the AAR is a systematic and credible method for improving
performance. “Hypothesis 2: Debriefs are more effective when levels are aligned” (Tannenbaum
& Cerasoli, 2013, p. 234). The debrief effectiveness hinges on the goals that are set, and that
they are the same for everyone involved. For example, the focus of the AAR is team
performance rather than the performance of each individual team member in a group setting. The
last supported hypothesis is: “Hypothesis 3: Facilitated debriefs are more effective than
[nonfacilitated] debriefs (Tannenbaum & Cerasoli, 2013, p. 234). Structure during an AAR is
more effective than without, but as seen in the next unsupported hypothesis, the higher the level
of structure does not determine a more effective debrief. “Hypothesis 4: A higher level of
structure is associated with a more effective debrief” is unsupported (Tannenbaum & Cerasoli,
2013, p.234). Finally, “Hypothesis 5: Debriefs using multimedia aids are more effective than
debriefs not using multimedia” (Tannenbaum & Cerasoli, 2013, p. 234) is also unsupported.
The results provide empirical support for the use of team and individual debriefs. A lowcost option that has the potential to increase up to 25% improvements through proper structure
and effective facilitation (Tannenbaum & Cerasoli, 2013). A table detailing essential elements
(Appendix B) describes the definitions and exclusions for active self-learning, developmental
intent, specific events, and multiple information sources. Self-discovery and active involvement
are encouraged as long as there is a clear intent for improvement. Additionally, external sources
of information should be considered, such as observers. Finally, it is recommended that negative
or passive feedback should be avoided (Tannenbaum & Cerasoli, 2013).
Municipal Surveys
To gain a deeper understanding of the AAR methods used in Ontario and whether or not
there is any continuity between municipalities, survey questions were sent to four different

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

20

emergency management professionals in southern Ontario. Those were: the CEMC for the City
of Sarnia, the CEMC for the County of Lambton, the CEMC for Guelph Wellington, and the
alternate CEMC for the City of London. Three surveys were returned completed from the City of
Sarnia, City of London, and the County of Lambton. The following survey questions were asked:
1. Have you ever completed an after-action review?
2. What was the process?
3. What was the outcome?
4. What terminology do you use to define the after-action review?
5. Do you have an after-action review that you can share with me? Completed and
template?
6. Have you implemented lessons learned?
The survey questions were each designed to allow answers based on fact and not opinion. All
emergency managers answered the questions completely but one (one question left unanswered).
Appendix D contains Table 4 with each question and answer included and Table 5 with the AAR
steps.
With the OFMEM’s suggestion of an AAR process that promotes active and reflective
learning, emergency management divisions within Ontario municipalities will be able to grow
their programs. The OFMEM should consider outlining AAR best practices, and how to
introduce and facilitate AAR’s within a municipality. The next section provides insight to the
AAR process used by the municipalities.
Discussion
The AAR is a powerful, yet simple tool to improve the effectiveness of individuals and
teams (Tannenbaum & Cerasoli, 2013), when completed with proper guidance. There are

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

21

essential elements and best practices that should be followed to ensure success. The literature
shows that the process of completing an AAR is just as important as the procedure itself.
Facilitation and participation are the grounding foundation of the AAR and provide a truthful and
influential AAR that allows for lessons learned to be considered.
The AAR processes found between the three municipalities surveyed were not identical
to each other or the process outlined by the OFMEM. The processes do, however, involve mostly
the same steps (Table 5, Appendix D) in a slightly different order. Each municipal AAR process
included: background information and overview, event chronology, recommendations, and
conclusions. Additionally, one municipality also included consequences and two municipalities
included an analysis. The survey results show that all three municipalities have completed an
AAR, their templates are slightly different, but outcomes and lessons learned were noted.
For any developmental interventions to work, lessons learned must be integrated into
everyday practice (Tannenbaum & Cerasoli, 2013). The AAR process and methodology provide
a base in which to inform decisions, guide emergency management plans, strengthen mitigation
and prevention strategies, and assist in the development of tools for response to real operations
and exercises (Friesen et al., 2017). Building this framework will assist to prevent mistakes from
being made in the future building of a structure to learn from experiences that focus on
challenges (Darling & Parry, 2001). The following section will discuss evaluations and
recommendations for the OFMEM and provide options to consider when creating an AAR
process for implementation.
Evaluation and Recommendations
A benefit can be derived from an AAR whether it is a major event, or an exercise event.
A lessons learned program should be built into the culture of the organization to ensure that it

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

22

becomes second nature. Eventually, a new mind-set will develop in the organization and learning
can be incorporated into daily work (Garvin, 2000).
The recommendation for the OFMEM to implement an AAR process within their
organization and structure would provide additional guidance to municipalities in Ontario. In
order to maximize the AAR process, the OFMEM should consider the essential elements as well
as what to do with the finished product.
The following recommendations were made by Friesen et al., (2017):
1. develop a database, to allow structure, organization and accountability, and to track
lessons learned and implemented,
2. establish a policy and standard, using existing governance for the completion and
collection of AARs
3. create a shared accountability model, this will encourage municipalities and the
OFMEM to share pertinent information with a larger audience,
4. validate forms and templates, ensure ease of use, usability and effectiveness are
embedded in the design,
5. validate project methodology and risk-informed prioritization, provide ongoing
validation to ensure continued suitability for implementing a robust, flexible and
adaptable lessons learned processes,
6. involve lessons learning in training and exercises, this will allow further continuous
improvement and will build the AAR process into the culture of the organization,
7. conduct training for working group/secretariat members, develop training and
orientation courses for the key members of the emergency operations centre,

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

23

8. and, create partnerships, partner with academic or community members to build
relationships and learn from each other’s experiences during the same incidents or
exercises (pp. 230-231)
Each of these items will assist to providing a general framework for an AAR program.
Further research should be completed to determine the best approach that the OFMEM could
take to appeal to different size municipalities that would also encompass the different needs and
hazards that each community is faced with.
In order to address hazards in a community, the OFMEM should consider a revision of
their hazard identification and risk assessment (HIRA) table that would allow emergency
managers to determine which lessons learned should be implemented first. Friesen et al., (2017,
p. 226) recommend an impact-likelihood risk matrix and risk rating factor (Appendix E).
Weighting the risks this way would allow factors such as people affected, the cost or the level of
public confidence to be considered (Friesen et al., 2017). The AAR can set out an organizational
roadmap for initiating, planning and implementing process improvements for the future (Brock
et. al., 2009).
Conclusion
Organizational capacity can be expanded by implementing a comprehensive and welldeveloped lessons learned program following incidents or disaster exercises, known as the afteraction review. According to Tannenbaum and Cerasoli (2013), organizations can improve
performance up to 25% by using properly conducted after-action reviews. The themes discussed
in this paper; best practices, active learning, reflective learning and culture provide an
informative approach to creating an improved AAR process for the OFMEM. This inexpensive
process can deliver great results if completed properly. The benefits of the AAR are found to

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

24

enhance strengths, address limitations and learn from experiences within organizations. These
benefits should be maximized to ensure municipalities are completing their due diligence within
their communities. The OFMEM may play a large part of the AAR process that municipalities
choose to use within their emergency management divisions to promote active and reflective
learning.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

25

References
Bolton, F. C. (2016). Use of the after-action review to improve learning. Assessment
Update, 28(2), 3–15. https://doi.org/10.1002/au.30051
Brock, G. W., McManus, D. J., & Hale, J. E. (2009). Reflections today prevent failures
tomorrow. Communications of the ACM, 52(5), 140–144.
https://doi.org/10.1145/1506409.1506443
Darling, M. J. & Parry, C. S. (2001). After-action reviews: Linking reflection and planning in a
learning practice. Reflections, 3(2), 64–72. https://doi.org/10.1162/15241730152695252
Emergency Management and Civil Protection Act, R.S.O. c. E.9. (1990). Retrieved from
https://www.ontario.ca/laws/statute/90e09
Emergency Management Ontario. (2016). Exercise. Retrieved from
https://www.emergencymanagementontario.ca/english/emcommunity/program_resources
/exercise/exercise_aar.html
Emergency Management Ontario. (2017). Emergency management review. Retrieved from
https://www.emergencymanagementontario.ca/english/insideemo/EmergencyManagemen
tReview.html
Friesen, S. K., Kelsey, S., & Legere, J. A. (2017). Defining a risk-informed framework for
whole-of-government lessons learned: A Canadian perspective. Journal of Emergency
Management, 15(4), 219–232. https://doi.org/10.5055/jem.2017.0331
Garvin, D. A. (2000). Learning in action: A guide to putting the learning organization to work.
Boston, MA: Harvard Business School Press Books.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

26

Scott, C., Dunn, A. M., Williams, E. B., & Allen, J. A. (2015). Implementing after-action review
systems in organizations: Key principles and practical considerations. In J. A. Allen, N.
Lehmann-Willenbrock, & S. G. Rogelberg (Eds.), The Cambridge handbook of meeting
science (pp. 634–659). New York, NY: Cambridge University Press.
Tannenbaum, S. I., & Cerasoli, C. P. (2013). Do team and individual debriefs enhance
performance? A meta-analysis. Human Factors, 55(1), 231–245.
https://doi.org/10.1177/0018720812448394

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

27

Appendix A
Graphic Chart.

Initial Search:
130 Articles

Abstract Review:
21 Articles

Title Review: 21
Articles

Excluded Articles: 11
Articles - Duplicate
information and did
not fit the scope

Excluded Articles:
109 Articles - Many
focused on
healthcare industry.

Full Review:
10 Articles

Excluded Articles: 1
Articles

Included in
Study: 9 Articles

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT
Appendix B
Table 2.
Debriefs: Essential Elements.

(Tannenbaum & Cerasoli, 2013, p.233).

28

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT
Appendix C
Table 3.
General issues and recommendations for the lessons learned submission process.

(Friesen et al., 2017, p. 229).

29

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

30

Appendix D
Table 4.
Comparison table of municipal survey questions.
Question Number
1. Have you ever
completed an
AAR?

Municipality
City of Sarnia

Answer
For incidents, yes. For exercises, not formally, no documentation, discussion-based
hotwash/debrief only.
City of London
The City of London completes an After-Action Review (process) annual after our EOC
exercises. Also after other training events and smaller scale exercises we conduct a less
formal review.
County of Lambton Yes, we have conducted a few after-incident reviews at the County, however the review
following Snowmageddon in December 2010 was the most in-depth and involved the largest
number of stakeholders. The County held three reviews (EMO organized and held a fourth
review by teleconference shortly after the event, which also included representatives from
abutting counties).
An internal review was conducted within a week of the termination of the emergency. An
external review was held with local stakeholders and external agencies within the month. A
final report was prepared and shared with all stakeholders for comments and review prior to
being submitted to County Council. A third review took place in November, prior to the oneyear anniversary of the blizzard and included municipal representatives who were not
necessarily involved in the 2010 emergency and response. This third review was held as a
joint, discussion-based exercise with local municipalities and stakeholders.
Formal reviews involving relevant stakeholders were also conducted following the Central
Lambton Tornado of July 2011 and following the sinking of the Tug Madison and Authur J
barge in July 2012. We also have a protocol to review all significant incidents on Highway
#402 or the County's Emergency Detour Route with our stakeholder group, which includes

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

31

the MTO, local municipal emergency services that respond to #402 and the EDR, Public
Works Departments and the MTO maintenance contractor (and others as required).

2. What was the
process?

3. What was the
outcome?

County personnel also participate in local municipal after-actions reviews (i.e. 2014 Boil
Water Advisory, 2015 Indian Hills' emergency declaration) and chemical valley emergency
coordination organization (CVECO) incident reviews.
City of Sarnia
CVECO uses a process, uses Codes/time/timelines and who was called in. Conclusions and
recommendations are done for response.
Kenwick Fire – no AAR form completed, everyone submitted their notes. A debrief was
done at County level 2 weeks after, and at City Hall 3 weeks after the incident.
City of London
After the EOC exercise we conduct a “Hot Wash” review immediately after the “end X”.
Where we look at the exercise outcome, strong points, points to improve on the exercise
objectives and exercise design. Minutes/notes are taken from the “Hot Wash” process and
added to our exercise participates critique and exercise evaluator reports. Our CEMPC
committee also reports on an After Action Review process with the exercise design team.
Using the SWOT format or pre designated questions, or check lists, we prepare an After
Action Report from that exercise. The AAR is presented to the Community Emergency
Management Program Committee (CEMPC).
County of Lambton For Snowmageddon, stakeholders were asked to attend the reviews in person. Given the size
of the group, the review took place in Council Chambers. The first internal and external
reviews were chaired by the County Emergency Coordinator and the third review was led by
a hired facilitator. The discussion generally followed the attached Incident Review Format.
High-level comments (not attributed) and recommendations were captured in the final report.
City of Sarnia
Reaffirming obvious issues – such as human error and communications errors. CVECO
completed AAR’s go to community awareness emergency response (CAER) office, the
response review committee reviews, then the CAER board shares to all industries.
City of London
For planning purposes for the next years exercise design team we review the AAR to make
the next exercise better. The AAR as a CEMPC we try to remedy the recommendations of
that report by training education of for funding for the next year.
County of Lambton Although the stated goals of an incident review are to identify what went well and what did
not, identifying problems is usually more constructive. Identifying issues, which can be
worked on and hopefully resolved prior to the next incident, improves future emergency

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

4. What
terminology do
you use to
define the
AAR?

5. Do you have an
AAR that you
can share with
me?
Completed or
template
6. Have you
implemented
lessons
learned?

32

responses. Following the Snowmageddon reviews, a final report was provided to County
Council (attached to original email) listing many of the actions taken for improvement.
City of Sarnia
Learnings are reviewed, recommendations are made and follow-up takes place. This process
only works if participants are truthful and feel comfortable to speak their mind creating a
positive experience.
City of London
The After Action Review is the process used to complete an After Action Report. The review
process of a hot wash, participant critiques or evaluations, the exercise evaluators reports and
AAReview of the exercise design team to review lessons learned from the exercise objectives
and how to make the exercise better to finalizes it in an AAReport.
County of Lambton Sorry, I don’t know what this question is asking.
City of Sarnia
Just the CVECO template.
City of London
Yes
County of Lambton Attached is a template that I first saw used at a CVECO incident review (reviews are
conducted by the Response Review Committee following all major industrial incidents). I
retained a copy of the template and modified it slightly so it could be used for municipal
purposes.
City of Sarnia
With CVECO, each industry receives a copy for their own implementation.
With the Kenwick Fire, no implementations completed yet.
City of London
Yes. We implement the LL to better the exercise itself and what training or courses, our
participates can take or do. The formal EOC process or documentation or Planning P can be
implemented to work more smoothly in a revised operating procedure.
County of Lambton Yes. For one, a Pre-Winter Traffic Meeting is now held each fall to meet with various
stakeholders involved in #402 or EDR response or maintenance. Internal protocols were
adopted to improve communications between stakeholders and readiness to respond to
snowsqualls (i.e. relocation and purchase of resources such as cots and blankets), MTO
implemented a number of actions (i.e. planting living snow fences, portable and permanent
message signs, access ramp swing gates, etc.), etc. Many of these details are contained in the
attached Council report.
In other reviews, communications were often identified as an issue for emergency response.
Even with continuing improvements, it has been necessary to revise or expand contact lists or
reassess how individuals or agencies are contacted. More robust protocols have been
developed to ensure relevant stakeholders are contacted, but on-going meetings (even if only

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

33

annually) help to ensure contacts are updated and new people in positions are introduced. A
maxim of emergency management is, "make your friends before you need them"… the value
of "putting a face to a name" cannot be overstated.
It should be noted, with the adoption of MyCNN for public alerting and internal notification,
sending emergency notifications has been greatly improved.

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

34

Table 5.
AAR Processes provided by municipalities.
Step
City of Sarnia
1

2
3

4

5

6
7

Municipality
City of London
County of Lambton

Aim: Background
Background event
chronology: Industrial
event chronology:
Sarnia dispatch event
chronology:
Discussion Comments: Objectives: Strengths
and weaknesses
Conclusion:
Analysis: Capability:
Activity observation:
Reference:
Recommendations:
Conclusion:
Recommendations &
Follow-up:
Recommendations
Follow-up/action items:
Adjourned:

The OFMEM

Background: Describe the location, actions
and conditions preceding the event.

Executive summary

Events: List events in chronological order.

Exercise overview:
details of exercise
Exercise goals and
objectives

Incident: Describe the incident.

Consequences: Describe consequences
resulting directly from the incident – provide
facts not conjecture.

Exercise events
synopsis: overview of
what happened.

Analysis: Using stated facts obtained from all
parties, evaluate the timeliness and level of the
response/actions. The analysis should provide
the framework for the resulting conclusions.
Conclusions: Formulate specific conclusions
as a result of the analysis of facts.
Recommendations: Each recommendation
must be assigned for follow-up within a
specified timeframe.

Objectives-based
evaluations

Conclusions

AFTER ACTION REVIEWS IN EMERGENCY MANAGEMENT

Appendix E
Table 6.
Impact-Likelihood Risk Matrix

(Friesen et al., 2017, p. 229).

35