Running head: AFTER-ACTION REVIEWS After-Action Reviews - Can They be Used Effectively for Learning Purposes? Ed Moffat Emergency Management Division Justice Institute of British Columbia March 24, 2019 Instructor- Beth Larcombe Advisor- David Smulders February 24, 2019 1 AFTER-ACTION REVIEWS 2 Abstract After-action reviews have been in use for over 40 years, first by the US Military, and then adopted by many organizations or agencies to improve the learning capabilities of emergency responders to reflect, act and learn in real time. The after-action review is known by many different names and may actually be a tool that is under-utilized to convey important lessons learned as it appears that many recommendations are either being repeated or modified in the next event as they have not been institutionalized. After-action reviews are utilized by many organizations in emergency management as a way to promote a lessons-learned approach to promote organizational learning. In the realm of emergency management and preparedness the usage of after-action reviews is a structured review of the following: 1) what went well? 2) What did not go well? 3) Where are the gaps? and 4) what can be done differently next time to improve performance? This paper will examine how after-action reviews are conducted following disasters to ensure that lessons-learned and the responses to real-incidents or exercises are implemented to minimize avoidable deaths or negative economic and social consequences. Keywords: After-action Reviews, After-action Reports, Emergency Management, Disaster Preparedness, Lessons Learned AFTER-ACTION REVIEWS 3 TABLE OF CONTENTS Abstract ............................................................................................................................................2 Background and Purpose .................................................................................................................4 Methodology ........................................................................................................................5 List of key words and subject terms………………………………………………………6 Final Selection of Literature ……………………………………………………………...7 Discussion of Findings ...................................................................................................................12 Conducting AARs ..............................................................................................................13 Effectiveness of AARs………………………………………………………………….. 14 Analysis of Two Canadian AARs ……………………………………………………….15 Critical Analysis …………………………………………………………………………………17 Conclusion……………………………………………………………………………………….20 AFTER-ACTION REVIEWS 4 Background and Purpose After-Action Reviews (AARs), also known by other terms, are a detailed critical summary of a past event conducted and documented for the purpose of re-assessing decisions and considering alternatives for future reference. The usage of AARs began over 40 years ago by the US military, and are now being used by those responsible for emergency preparedness and response recovery to convey important lessons learned to promote individual and organizational learning post event(s). With the number of disaster events increasing along with the complexity and magnitude of such events, how can after-action reviews be performed in a timely fashion to convey lessons learned and implemented for improving response in the next event. What are after-action reviews and how are they conducted post event? What processes are currently utilized to document what happened, why it happened and how it can be done better for the next event? This paper will examine what are after-action reviews? How after-actions reviews are conducted to ensure that lessons-learned are conveyed in a meaningful manner? What standard is there for AARs so all information presented can be easily disseminated to all parties that need to know? The Emergency Management Division (EMD) of the Justice Institute of BC is interested in determining how after-action reviews are conducted in Canada following disasters including resulting rationale for processes. The information gathered on after-action reviews will be utilized to support further research on this topic as well as support curriculum development in the JIBC Certificate of Emergency Management program. AFTER-ACTION REVIEWS 5 Methodology Search criteria utilized to identify relevant literature for study were difficult to pinpoint as there are many ways to describe after-action reviews (AARs) as terminology used varies between organizations as well as businesses who conduct them. Phrases such as after-action reviews or after-action reports were the most prevalent terms used; however other terms such as team huddles (used in medical facilities) as well as debriefs, critiques, post-mortems and hot washes are also utilized which complicated the search. Determination of the definition of afteraction reviews shows that it is different from after-action reports. The after-action review is conducted and participated in by responders immediately following the response to an emergency whereas, the after-action report is generated and released weeks later, once the incident data and review data are synthesized for use as a training tool to achieve recommendations for best practice. When the term emergency management was incorporated into the search the results were narrowed and even further reduced when the term disaster preparedness was utilized. All of the terms were utilized within quotations to improve the chances of narrowing the search to existing literature within the Emergency Management studies. Resources used for the search included Google Scholar, JIBC Library, and Google. Google Scholar was the primary source and netted the best relevant results for evidence of effective usage of AARs, as well as when and how they should be performed. The JIBC Library was mainly used to obtain articles through Google Scholar where payment was required to access them. Google was utilized to download relevant PDF’s of articles as well as relevant media sources; along with JIBC Library that were relevant to the topic. AFTER-ACTION REVIEWS 6 List of Keywords and Subject Terms Keywords: After-action Reviews, After-action Reports, Emergency Management, Disaster Preparedness, Lessons Learned The list below is representative of the samples of terms used to focus the subject matter. The number of results is represented in parenthesis received from Google Scholar. The results from utilizing the JIBC Library are represented in [ ]. • After-action reviews (2,050,000) [37,039] • After-action Reports ( 3,650,000) [59,309] • After-action reviews and Emergency Management (1,260,000) [1,116] • After-action reports and Emergency Management (1, 840,000) [1,226] • After-action reviews and Emergency Management or Disaster Preparedness(177, 000) [1,157] • After-action reports and Emergency Management or Disaster Preparedness (433,000) [1263] • “After-action reviews” and “ Emergency Management” or “Disaster Preparedness” (603) [10] • “After-action reports” and “Emergency Management” or “Disaster” (1,680) [85] • “After-action reviews” or “After action reports” and “Emergency Management” or “Disaster Preparedness” (135) [1335] For this literature review, 30 articles were initially selected for abstract review including two actual after-action reports that reviewed the Lesser Slave Lake Wildfire (2011) and AFTER-ACTION REVIEWS 7 Wood Buffalo Wildfire (2016) from Alberta. These articles were selected for their direct relevance to after-action reviews in Canada and were analyzed to determine if lessonslearned had been adopted for the latter and to determine if they were implemented and if not why not. Final Selection of Literature At the conclusion of reviewing the initial article abstracts, the final number of selected articles was reduced to 20, which was further reduced to 15 after examining the content to determine relevance to the chosen subject. The final articles selected for full review and use for research was limited to peer reviewed papers that studied the purpose of performing AARs, the challenges of performing AARs post event as well as the relevance of the lessons learned being incorporated in future training exercises and events. Although the initial intention was to limit the research of articles specifically to Canada, this was not feasible as the writer was only able to locate two articles that were Canadian based. The search for existing articles was expanded next to the United States and then Australia as they have similar experiences with emergency management as well as a variety of disaster events such as Hurricane Katrina (2005), BP Oil Spill (2010), and Wildland- Urban fires. The articles selected for literature review were chosen for their focus on performing afteraction reviews as well as the structure and what should be included when AARs are conducted and when they should be conducted. The following six peer reviewed articles were selected for review: 1) Make no mistake: the effectiveness of the lessons-learned approach to emergency management in Canada, by Adam Rostis, published in 2007. The article questions the AFTER-ACTION REVIEWS 8 usage of after-action reviews and the level of commitment to adopting a learning environment for emergency management professionals within Canada. A qualitative and exploratory survey was utilized and sent to provincial and territory emergency management agencies in Canada to determine what type of knowledge management was prevalent and being utilized to improve practice. This is important to understand the usage of AARs within the Canadian context and how they are utilized to convey lessons learned in Canada. Rostis indicates that the results from his study indicated that organizations dedicate non-financial resources to detecting and capturing knowledge from outside organizations. Exercises or simulations are knowledge management activities insofar as they are used to proactively acquire knowledge about how the organisation will respond to situations that they may not have encountered (Rostis, 2017, p. 207). Additionally, in a successful lessons-learned process, all those involved are responsible for learning (Rostis, 2017, p. 207). 2) When will we learn? The after action review, lessons learned and the next steps in training and educating the Homeland Security enterprise for the 21st century, by Jeffrey Kaliner, published 2013. In his thesis, Kaliner states, post exercise activity is not practiced regularly and after-action reviews (AARs) are not being distributed or implemented effectively. The knowledge that is gained from an AAR is not being transferred and applied which means that the AAR is not serving the intended purpose. This article is important as it questions why AARs are being used in the United States if the information is not being utilized to increase learning. AFTER-ACTION REVIEWS 9 According to Kaliner, the need to learn both during and after a crisis suggests that the traditional way of learning reflected by the current use of AAR must be re-examined (Kaliner, 2013, p5). Kaliner also states that most AAR practices are characterized by a facilitative technique that relies upon three or four open-ended questions (Kaliner, 2013, p14). Kaliner also states that what is known basically on this subject is that currently prescribed theoretical underpinnings have not been adequately expressed, explored, or assessed ( Kaliner, 2013, p21). 3) Near-real-time analysis of publicly communicated disaster response information by T. Girard, F. Wenzel, B. Khazai, T. Kunz-Plapp, J. E. Daniell and S. A. Brink, published in 2014. This article is important as to the when an after-action review is to be conducted and the information that should be included. The authors advocate a methodology by doing an after-action review near-real-time analysis as it is based on information that is current while the response is still in operation. This makes the feedback more pertinent as well as keeps it fresh in the minds for those that are participating in the response. According to the authors the quality of response is a large factor in its ability to limit the impacts of the disaster on the local population (Girard, Wenzel, Kunz-Plapp, Daniel & Brink, 2014, p. 165). A common method for analyzing disaster response is to carry out post-response evaluations, often referred to as “lesson-learned” or “after-action” reports. The main purpose of such reports is to identify what changes should be implemented in order to improve future responses (Girard et al., 2014, p.166). A valuable strength of post response evaluation is that it is based on actual disaster events (Girard et al., 2014, p. 166). AFTER-ACTION REVIEWS 10 4) Use of after-action reports (AARs) to promote organizational and systems learning in emergency preparedness by E. Savoia, F. Agboola and P.D. Biddinger published 2012. This article indicates that many health and healthcare organizations use formal knowledge management practices to identify and disseminate experiences. Emergency preparedness uses the lessons-learned approach on the basis that learning from past experiences improves practice and minimizes impact on economic and social consequences, including death. According to the authors, the lessons learned approach is one such example of knowledge management practice applied to the wider concept of organizational learning (Savoia, Agboola, & Biddinger, 2012, p. 2950). Also stated in the article is that despite voluminous attempts to document and learn from prior emergency preparedness system response failures, the challenges experienced in planning and responding to disasters seem to be “learned” over and over again in disaster after disaster (Savoia et al., 2012, p. 2950). The authors also state that if the identified insights and experiences have recurring themes across different types of threats and across multiple systems, they also present a direct mandate for responders, organizations, and systems to address challenges and create a requirement to test them in future exercises to ensure that planned improvements are successful (Savoia et al., 2012, p. 2959). 5) Debriefs: teams learning from doing in context, by J.A. Allen, R. Reier-Palmon, J. Crowe and C. Scott, published in 2018. This article introduces different concepts in regards to different names for after-action reviews including critiques, after event reviews, huddles, post-mortems and debriefs. To be effective debriefs need to identify ways to improve and ensure a supportive learning environment. AFTER-ACTION REVIEWS 11 According to the authors several meta-analysis evaluated the effectiveness of debriefs, and they have all concluded that having a debrief results in improved learning and team performance (Allen, Reier-Palmon, Crowe, & Scott, 2018, p508). To ensure effective facilitation that is less dependent on the skills of team members, most selfguided debriefs utilize some form of an aid such as checklist, list of questions, and detailed instructions and are as effective as facilitator-led debriefs (Allen et al., 2018, p508). Debriefs are a meaningful type of workplace intervention, deployed across contexts for a variety of purposes, and their effectiveness is essential to the accomplishment of the purpose identified (Allen et al., 2018, p509). 6) Do team and individual debriefs enhance performance? A meta-analysis, by S.I Tannenbaum and C. P Cerasoli published in 2013. To have an effective debrief there are three criteria that make it useful. The first being facilitation, second is structure and the third is multimedia. This is important for the research on AAR as it discusses different methods of performing the AAR and how it should be conducted overall to facilitate learning. The authors discuss that debriefs are a potentially powerful yet simple tool to improve the effectiveness of teams and individuals but research and theory have been scattered across multiple disciplines (Tannenbaum & Cerasoli, 2013, p240). The authors also argue that despite more than 30 empirical studies that examined debriefing, no quantitative integration of research existed with which to gauge debriefing efficacy (Tannenbaum & Cerasoli, 2013, p240). Their findings indicate that aligning participants, intent, and measurement yield the greatest effects (Tannenbaum& Cerasoli, 2013, p240). AFTER-ACTION REVIEWS 12 Discussion of Findings After-action reviews were first introduced 40 years ago by the US military and have been adopted by many organizations to improve the learning capabilities after an event as well as to improve performance during future emergencies. AARs are used as a tool by organizations to learn but they may not be practiced regularly and that the lessons learned from the completed AAR’s are not distributed or implemented effectively (Kaliner, 2013, p. 1). The lessons learned play an important (if not vital) role in emergency management and organizational agility (Friesen, Kelsey & Legere, 2017, p. 219). For emergency preparedness, the lessons-learned approach stands on the assumption that learning from experience, whether it be our own or others, and whether it be from real events or simulations, improves practice and minimizes avoidable deaths and negative economic and social consequences of disasters (Savoia, Agboola, & Biddinger, 2012, p. 2950). It is important to understand the role of the after-action review so that emergency managers can implement new strategies and solutions to previously identified problems to continue being prepared and adapting to what are normally fast changing situations. Instead of producing static knowledge assets to file away in a management report or repository, AARs generate raw material that feeds back into the execution cycle (Darling, Parry, & Moore, 2005). There is a profound and distinct difference between after-action reviews and after-action reports. After-action reviews are performed immediately after an emergency whereas the afteraction report is generated and released weeks later, once the incident data and review data are AFTER-ACTION REVIEWS 13 amalgamated to use as a training tool to achieve recommendations for best practice. Agencies that treat AARs as after-action reports instead of after-action reviews tend to not learn as they do not have a personal stake in the process and may only participate as they have been told to or out of loyalty (Darling et al, 2005). Conducting AARs Organizational learning is described as a process of strategic organizational renewal involving intuition, interpretation, integration and institutionalism (Rotsis, 2007, p. 199). As such, it is important to note that within Canada the lessons-learned approach is the primary method used for knowledge management. The main purpose of performing AARs is to identify what changes should be implemented in order to improve future responses (Girard, Khazai & Brink 2014, p. 166). After-action reviews are distinct from debriefs in that they begin with a clear comparison of intended versus actual results achieved (Dufty 2013, p. 15).What appears to be in debate is when the AAR should be performed. Traditional wisdom has been after the event or exercise is when an after-action review should be performed, however according to Girard, Khazi and Brink (2014), the time-lag between a disaster and when a post-event report is issued therefore presents an opportunity for other forms of analysis to be carried out (p 166). Girard, Khazi and Brink, argue that many of the issues raised post event can be identified within days rather than months after an event when a full report is usually completed (Girard et al., 2014, p. 167). After-action reviews need to be effective in conveying the information from security events and exercises to improve “best practice”, otherwise they become problematic as they will reinforce a narrative of the event that perhaps may not be accurate, may diffuse responsibility for AFTER-ACTION REVIEWS 14 the problems contained therein, and may ultimately lead to groupthink (Allen, Reiter-Palmon, Crowe, & Scott, 2018, p. 508). To ensure the AARs are effective, Allen et al advocate that a 12 step process should be used to ensure effectiveness as follows: 1) Debriefs must be diagnostic (identify specific ways to improve), 2) Ensure that the organization creates a supportive learning environment for debriefs, 3) Encourage team leaders to be attentive during the emergency regarding what they may want to discuss later, 4) Educate team leaders on the science of leading team debriefs (facilitation process), 5) Ensure that team members feel comfortable in debriefs, 6) Focus on critical performance issues during the debrief, 7) Describe specific teamwork interactions and processes involved in the team performance, 8) Support feedback with objective data, 9) Provide outcome feedback later (ie. not during the debrief) and less frequently than process feedback, 10) Provide both individual and team oriented feedback at appropriate times, 11) Shorten time delay between task performance and debriefing, and finally 12) Record conclusions made and goals set during debrief and follow-up. To conduct a successful AAR, a strong leader is needed who can create open, honest and safe environments in which people can speak honestly and with confidence; AARs should not be undertaken to simply fix problems or allocate blame (Cronin and Andrews, 2009, p. 32). Three characteristics that are widely thought to improve the quality of debriefs are facilitation, structure and multimedia aids; although the usage of multimedia may not show a meaningful improvement in effectiveness (Tannenbaum & Cerasoli, 2013, p. 240). Effectiveness of AARs The effectiveness of after-action reviews has been questioned as previous incidents issues and recommendations have not been actioned as agencies are not event taking the time to elicit AFTER-ACTION REVIEWS 15 and capture the knowledge that might be gleaned from an after-action review, and they are having trouble transferring and applying this knowledge for future events (Kaliner, 2013, p. 1) Planning to perform an AAR is absolutely critical to the effectiveness of AARs (Bryan, 2007, p. 45). According to Knox (2013), in Analyzing after-action reports from Hurricane Andrew and Katrina: Repeated, modified, and newly created recommendations, the issues that were identified after Hurricane Andrew should have increased the mitigation, preparation, response and recovery efforts in subsequent disasters (p. 160). However, the same issues that were identified in Hurricane Andrew were still occurring in the same type of disaster years later in Hurricane Katrina. After-action reviews are meant to instill an atmosphere of learning among individuals and organizations however, researchers are unable to answer basic questions about how individuals learn during exercises or the actual AAR process (Kaliner, 2013, p. 2). This was also evident in reviewing two finalized reports prepared by Klynveld Peat Marwick and Goerdeler, (KPMG) after the 2011 and 2016 wildfires that occurred in Alberta. Issues that were identified had not been fully implemented in the five years between the two events. There were several that had been implemented and completed but many had not been. If performing an AAR is not planned then this usually does not happen, according to Dufty (2013), post event emergency management evaluations tend to be done on an ad hoc basis because they are not an integral part of agency preparedness planning and are open to the vagaries of funding and political will (p. 19). This finding is supported by Kaliner (2013) as their thesis paper found that FEMA does not follow its own program policy which stipulates AARs are to be conducted at or near the conclusion of emergency or disaster operations (p. 9). Reasons for the lapse in practice vary, and include budget, timing and personnel issues (Kaliner, 2013, p. 10). Analysis of two Canadian AARs AFTER-ACTION REVIEWS 16 In May 2011, Alberta experienced a wildfire that started burning near Canyon Creek, Widewater and Wagner. A second wildfire started near the town of Slave Lake and a third wildfire started approximately 15 kilometers north of town (Kulig & Watson, 2015, p. 4). Approximately 8,000 residents were displaced when wind gusts of about 100 km/hour forced them to flee after the fires pushed past the barriers. The Slave Lake wildfire of 2011 was the most catastrophic event that the Albertan province had experienced to date. In 2016, Alberta experienced another major forest fire which began Southwest of Fort McMurray and displaced 88,000 residents (KPMG, 2016). The Fort McMurray wildfire of 2016 was now the largest disaster that was experienced by the province of Alberta to date. A review of two after-action reviews, also known as post incident assessment reports, that were written by Klynveld Peat Marwick Goerdeler (KPMG) shows that lessons learned in 2011 were not completely adopted for the event in 2016. The areas that still needed to be implemented were the following: 1) Strengthening emergency management across all stakeholders 2) Strengthening initiatives for Community Emergency Management Program (CEMP), indigenous programs and Alberta Environmental Management Panel (AEMP), 3) Enhance programs, resources and initiatives for inter-municipal cooperation 4) Work with local communities to improve preparedness 5) Improve provincial capacity and infrastructure to manage personal information during an emergency 6) Improve local and provincial preparedness for the possibility of evacuation 7) Ensure Incident Management teams are available to quickly deploy AFTER-ACTION REVIEWS 17 8) Fully implement Incident Command System (ICS) so roles and mandates are firmly established 9) Develop an approach for Government of Alberta to coordinate across ministries. 10) Build capacity to ensure timely distribution of funds, effective financial management 11) Create guidelines to plan and execute re-entry following an evacuation 12) Develop a comprehensive and coordinated approach for funding of disaster recovery plans 13) Develop and implement a province-wide approach to manage donations following a disaster. The reports from 2012 and 2016 were very informative and lengthy in evaluation and suggests that Alberta has a strong foundation for effective emergency management, and in many ways their response met or exceeded standards and leading practices from around the globe (KPMG, 2012, p. 1). The report for 2016 states that implementation will require the AEMA to plan and evaluate any changes to make sure that satisfactory results are realized. (KPMG, 2016, p. i) Based on the analysis of the two reports from KPMG it is unclear as to why the recommendations from 2011 were not fully implemented by 2016; however, it would appear that although there was support for the implementation of the recommendations made, time, money and lack of resources may have played a role in not having the ability to fully implement the recommendations from 2011 prior to the event of 2016. Critical Analysis and Recommendations There is a distinct difference between the two terms after-action reviews and after-action reports. After-action reviews are performed by individuals within an organization to determine AFTER-ACTION REVIEWS 18 what the intended accomplishment was, what actually occurred, what might have caused the event to occur as it did, articulates lessons learned, and finally establishes expectations for the next event. The after-action report is the final document which is created once all of the incident data is combined with review data and produced in a document by senior leadership to be actioned. Improving emergency management response to events is vital as the complexity, magnitude and the number of events is increasing. Although after-action reviews are supposed to be a lessons learned approach which is to be shared among agencies as well as government departments, recommendations made are not being actioned which hampers future mitigation strategies. Based on the articles reviewed, the usage of AARs are mandated by many governmental organizations such as the Canadian government, Homeland Security and FEMA in the United States however, those responsible for responding to events do not always follow the requirement to submit such documents for review. The performing of AARs are a post event activity and may only be completed when government inquiries are demanded by agencies as well as citizens. After-action reviews go by many other names depending on which organization is performing them and this may be confusing. Some of the names include after-action reports, debriefs, critiques, post-mortems as well as huddles and hot-washes. A consistent definition for the term of after-action reviews needs to be established to ensure consistency of its application across organizations as well as agencies responsible for emergency management. When an effective AAR is performed and the lessons learned have been adopted then and only then will the learning improve disaster response and resiliency. According to the majority of articles reviewed many organizations repeat the same issues over and over again as they have not adopted the recommended lessons learned from previous events. According to Savoia et al. AFTER-ACTION REVIEWS 19 (2012), there are common challenges that have consistently emerged during responses to different types of incidents that do not need to be learned again but rather they need to be addressed in the planning, testing, measuring and implementation cycle of emergency management (p. 2961). If lessons learned are not being adopted into new policies, procedures or response plans then the effectiveness of the whole process needs to be further researched to determine why so the same identified issues are not repeated. The closer to the event that an analysis is conducted of what the achievements were as well as understand the deficiencies by the organizations that responded to the event then the better the review will be. If near-real-time analysis, as advocated by Girard et al (2014), is used, then the likelihood of incomplete or inaccurate information moving forward into policy and process is diminished, and practice deficiencies identified can be addressed quickly. The performing of near-real-time assessment during the event will improve the learning capabilities of the organization and agencies responding and the issues identified in post response reports can be identified and potentially actioned during future response phases (Girard et. al, 2014, p. 166). The performing of after-action reviews is vital to the success of future events. As such it is important that those facilitating the process are trained and competent to lead participants through an open discussion that reflect on the planned goals, and successes, as well as areas in need for improvement for future responses. Currently most after-action reviews, when they occur, are performed after time has elapsed post event. This makes it difficult to elicit and capture the necessary data as it becomes more problematic to remember what occurred accurately the longer between the actual event and when a review is actually held. The original design of the after-action review was to receive instant feedback on one specific or particular action and when you are able to interrupt the event so it can be analyzed, discussed and then AFTER-ACTION REVIEWS 20 acted upon. In emergency management, when an event occurs there are many actions taking place all at once which may hamper the ability to stop, analyze, discuss and correct which makes it complex and the terminology of after-action review may actually be a misnomer and/ or misleading as time elapses between the actual event and a discussion is held. Reviewing and analysing the existing literature it was difficult to determine the best practice of how an AAR is to be written, supported by Friesen, Kelsey & Legere (2017), who were able to determine that there is a lack of standardization of policies surrounding AAR/After Incident Reports, and the requirement for collection, analysis, sharing and archiving of the information (p. 222). By not having a standard way of documenting after-action reviews and the lessons learned emergency management practitioners are prone to make the same mistakes event after event. Conclusion The search of existing literature shows that the usage of after-action reviews is a good tool to perform “lessons- learned” knowledge management after an event. This literature is consistent that once an after-action report is conducted they are able to convey lessons learned however, the after-action review is not being utilized to its full extent as “lessons-learned” and those responsible for emergency management are doomed to repeat the same challenges over and over again disaster after disaster. This would seem that the actual lessons learned are not being included when updating policies, procedures or actioned into response plans and those responsible for emergency response are repeating the same mistakes again as a result. Although after-action reviews have been practiced for over 40 years, the research indicates that current practice may not lead to increased individual or organizational improved practice. This is AFTER-ACTION REVIEWS 21 concerning in that AARs and usage of such documents can lead to improvements in planning and response to events. The term AAR is used simultaneously between after-action reports and afteraction reviews which is not helpful, and could lead to confusion. The term after-action review is about taking a look at the next similar event and to complete the learning cycle by applying the lessons-learned from the previous incident. After-action reviews are about changing behaviours through reflection and learning from meaningful discussions during events or incidents and not waiting until the event has finalized. The existing literature is consistent in advocating more research needs to be conducted on after-action reviews to ensure its efficacy. Existing research is spread across a broad spectrum of research fields, each with its own varying findings. The examination of after-action reviews needs to be explored further to determine the best methods of conducting and reporting as well as when and how they should be performed to improve overall emergency management response to events so that the identified issues are not reoccurring time and time again. The initial implementation by the US Military of after-action reviews was for the implementation of a new training strategy so they could adjust on a daily basis to get the intended results. The need for standardized AAR reporting structures, including reports forms that validate and record their efficacy needs to be incorporated within emergency management administration and training to ensure that a database can be centralized for unlimited emergency management access. A more consistent approach is necessary to ensure that after-action reviews are performed consistently, are comprehensive in nature as well as performed in a timely manner to ensure improvement of emergency management for future events and building overall resiliency. AFTER-ACTION REVIEWS 22 References Allen, J. A., Reiter-Palmon, R., Crowe, J., & Scott, C. (2018). Debriefs: Teams learning from doing in context. American Psychologist, 73(4), 504-516. http://dx.doi.org/10.1037/amp0000246 Bryan, P. L. (2017, January-March). AAR Considerations during multinational operations. Infantry, 45-48. Retrieved from https://www.benning.army.mil/infantry/Magazine/issues/2017/JANMAr/pdf/12)Bryan_AAR.pdf Cronin, G., & Andrews, S. (2009). After action reviews: A new model for learning. Emergency Nurse, 17(3), 32-35. Darling, M., Parry, C., & Moore, J. (2005). Learning in the thick of it. Harvard Business Review, 83(7/8). Retrieved from https://hbr.org/2005/07/learning-in-the-thick-of-it Dufty, N. (2013). Evaluating emergency management after an event: Gaps and suggestions. Australian Journal of Emergency Management, 28(4), 15-19. Retrieved from https://works.bepress.com/neil_dufty/31/ Friesen, S. K., Kelsey, S., & Legere, J. A. (2017) Defining a risk-informed framework for wholeof-government lessons learned: A Canadian perspective. Journal of Emergency Management, 15(4), 219-231. https://doi.org/10.5055/jem.2017.0331 Girard, T., Wenzel, F., Khanzai, B., Kunz-Plapp, T., Daniell, J. E., & Brink, S. A. (2014). Near real time analysis of publicly communicated disaster response information. International Journal of Disaster Risk, 5(3), 165-175. https://doi.org/10.1007/s13753-014-0024-3 AFTER-ACTION REVIEWS 23 Kaliner, J. (2013). When will we ever learn? The after action review, lessons learned and the next steps in training and educating the Homeland Security enterprise for the 21st century. (Master’s thesis, Calhoun: The Naval Postgraduate School Institutional Archive). Monterey, CA. Retrieved from https://apps.dtic.mil/dtic/tr/fulltext/u2/a585764.pdf Klynveld, Peat, Marwick and Goerdeler (KPMG). (2012). Lesser Slave Lake regional urban interface wildfire-lessons learned: Final report. Retrieved from http://www.aema.alberta.ca/documents/0426-lessons-learned-final-report.pdf Klynveld, Peat, Marwick and Goerdeler (KPMG). (2016). Wood Buffalo wildfire: Post-incident assessment report. Retrieved from https://www.alberta.ca/assets/documents/WildfireKPMG-Report.pdf Knox, C. C. (2013). Analyzing after-action reports from Hurricanes Andrew and Katrina: Repeated, modified, and newly created recommendations. Journal of Emergency Management, 11(2), 160-168. https://doi.org/10.5055/jem.2012.0135 Kulig, J. C., & Westlund, R. (2015). Linking research findings and decision makers: Insights and recommendations from a wildfire study. Society & Natural Resources, 28(8), 908-917. https://doi.org/10.1080/08941920.2015.1037876 Rostis, A. (2007). Make no mistake: The effectiveness of the lessons-learned approach to emergency management in Canada. International Journal of Emergency Management, 4(2), 197-210. https://doi.org/10.1504/IJEM.2007.013990 AFTER-ACTION REVIEWS 24 Savoia, E., Agboola, F., & Biddinger, P. D. (2012). Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. International Journal of Environmental Research and Public Health, 9(8), 2949– 2963. https://doi.org/10.3390/ijerph9082949 Tannenbaum, S. I., & Cerasoli, C. P. (2013). Do team and individual debriefs enhance performance? A meta-analysis. Human Factors, 55(1), 231245. https://doi.org/10.1177/0018720812448394