There is currently a lot of work being completed around knowledge and lessons management in the Australian emergency management sector. Since the release of the National Strategy for Disaster Resilience in 2011 (Attorney-General’s Department 2011), emergency services organisations have focused on a multi-agency approach to knowledge sharing. This multi-agency focus is working towards bridging a gap, where a lessons management process is yet to be proven to work for a sustainable period in emergency management and government contexts. (Boin et al. 2005, Gilpin & Murphy 2008). In addition, there are few pre-requisites required to enter into the emergency management sector. This has resulted in a dependence on an individual’s previous knowledge rather than sector-wide understanding.
The influence of an organisation’s culture on how it records, analyses and builds knowledge and manages its lessons can explain why organisations often struggle with implementing effective lessons management. There is a need for a holistic approach to ensure lessons can be captured and shared across emergency services organisations, government departments, businesses, industry, and the community. There is a sector-wide appetite to learn, improve performance, and support innovation in this area but siloes will remain if emergency services organisations continue to develop individual processes.
This paper presents a model for lessons management for the emergency management sector based on research conducted in 2014. The research investigated lessons management methodologies and their application in emergency services organisations. The outcomes of this research were to provide the Country Fire Authority Victoria with recommendations on how to incorporate a lessons management life cycle to support behaviour change, future service delivery planning, and improve organisational performance. The research included:
There is limited literature relating to lessons management implementation for the emergency management sector. The scope of this literature review included general lessons management literature and supporting organisational theories. The literature review focused on the most applicable and common elements of lessons management success. The purpose of the literature review was to identify key aspects of lessons management that are important for implementation, particularly for emergency services organisations.
Existing literature was selected based on criteria of success, relevance and significance. David Garvin’s work in learning organisations was chosen because of its popularity and recognition within the emergency management sector, in particular, through its adoption by the United States Wildland Fire Lessons Learned Centre, one of the most widely known and used lessons management centres in emergency management. Many leading authors adopt, or are influenced by, Garvin’s definitions and the learning organisation process of knowledge transfer (Huber 1991, Garvin 1993, Nasiatka & Christenson 2005).
Nick Milton’s work in lessons learned and knowledge management supports Garvin’s work and was chosen because it is highly influential in the emergency management context. Milton’s ‘learning loop’ of identification, action and institutionalisation (Milton 2010) fits Garvin’s definition of a learning organisation (Garvin, Edmonmson & Gino 2008). Milton has conducted extensive research and work in the lessons management field. His ability to translate complex concepts into digestible terms was desirable to build understanding and applicability for all CFA members.
The literature review provided an overview of what needs consideration and also the process to be undertaken including defining lessons management, cultural requirements, collection of lessons, analysis of lessons, and actioning lessons. The literature review identified some gaps particularly in areas that appear to be sticking points for lessons management implementation. These gaps are:
The literature review was supported by the analysis of emergency services organisations that have, or are in the process of, implementing lessons management. Emergency services organisations that were evaluated include:
The evaluation of these emergency services organisations identified and supported a realistic process to be developed and implemented within a supportive organisational culture and governance. The lack of available literature has resulted in emergency services organisations adapting a range of literature, theories and learnings to build their own models. Organisations may also be trying to fix larger organisational issues by implementing lessons management. Therefore, inconsistencies exist in lessons management nationally and internationally.
The outcomes of the literature review and emergency services organisation evaluation established the characteristics of successful lessons management. These outcomes influenced the case study research and focus group discussions and were evaluated against the needs and requirements of the CFA members. The CFA is a statutory authority made up of approximately 60 000 volunteers and 2000 staff distributed across Victoria. The sampling undertaken to establish the focus groups was purposeful to ensure a cross section representation of the organisation and to allow the findings to represent the organisation as a whole. The three focus groups represented different sections of the organisation being corporate (CFA headquarters), metropolitan and rural.
The approach to forming the focus groups was consistent with Rabiee (2004) where three focus groups of six to eight participants each allows for meaningful discussion of simple research questions. Each focus group meeting lasted 1–2 hours. Questions addressed a range of topics including definitions, prior knowledge, processes, implementation and roles. The collected data was analysed using an inductive approach, based on Rabiee’s (2004) key stages of the analysis continuum and Wilkinson’s (2000) coding and clustering process. The data was interpreted using the seven criteria outlined in Rabiee (2004), which allowed the interpretation of relationships between quotes, ideas and links in the data. The focus group results were combined with the literature review and emergency services organisation evaluation to establish links between all the data and to identify themes. The validity and integrity of the findings were assessed through triangulation of multiple data sources and peer review. The most consistent themes were selected and built into the Lessons Management Life Cycle (see Figure 1).
The research investigated lessons management methodologies and their application by emergency services organisations. Three key areas were scope, culture, and process. These three areas are represented in the Lessons Management Life Cycle in Figure 1. The scope of lessons management is displayed in the life cycle by the use of a magnifying glass around the process. Cultural characteristics are identified in the life cycle as people pushing cogs. The components that make up the lessons management process are displayed on the cogs.
The literature, emergency services organisation evaluations and focus group discussions identified the need for a comprehensive understanding of the role of lessons management within an organisation and awareness of the definitions, objectives and the scope. Consistent with the existing literature, this helps establish clarity of roles, responsibilities and accountability (Attorney-General’s Department 2013, Elliot 2009, Newman 2007). Due to the lack of definitions relating to lessons management in emergency services organisations, definitions from key lessons management literature were used to inform research findings and the resulting recommendations for implementing the life cycle.
Clear definitions of terms are critical to successful lessons management. A definition of lessons management applicable to CFA was developed using literature review data and tested during this research. Focus group members provided descriptive words and statements representing their understanding of the term. Focus group discussions indicated that the researcher’s definition captured the participant definitions and was a good representation of lessons management for the CFA. As a result, the definition of lessons management was confirmed as ‘the management of a continuous learning cycle where capturing, analysing and implementing lessons, occurs without barriers and results in measurable behaviour modification’ (Jackson 2014).
In addition to definitions, the research highlighted that successful lessons management requires connections to be identified between all areas within organisations and clear responsibilities in organisational structures identified. Lessons management requires a team and a ‘home’ within organisation structures. One focus group participant said,
‘it needs a team and it needs a home, there’s a lot of people doing a lot of stuff on goodwill with organisational value and stuff like that, but it doesn’t live anywhere. So if you are going to do this then do it and this is your job’.
Establishing these structures and connections early will assist the implementation of lessons management by encouraging engagement, ownership and accountability.
The literature review identified that organisational culture is the foundation of an organisation. The literature, the evaluation of the emergency services organisations, and the focus group discussions highlighted a number of cultural characteristics within an organisation required for successful lessons management. These included ‘just/fair’, ‘leadership’, ‘accountability’, ‘communication’ and being ‘learning-focused’. Without all of these characteristics working together within the process outlined in the Lessons Management Life Cycle, lessons management will not be successful. The life cycle includes these cultural characteristics that support lessons management as the words on the people pushing the process cogs. These are not in any particular order but are all required for lessons management to be a success.
‘Just/fair’ was a significant cultural characteristic identified in the literature, in the emergency services organisations evaluation, and was discussed in the focus groups (Attorney-General’s Department 2013, Bennett 2000, Bos et al. 2008, Elliott 2009, Garvin 1993, Milton 2014). The focus groups discussed the need for a no-blame or just/fair culture. One focus group participant said,
‘I think we are scared to put stuff out there and I think potentially, staff or volunteers, are worried they are going to upset someone so I think that’s why we don’t put it out there’.
Consistent with the literature (Attorney-General’s Department 2013, Elliott 2009, Safety Institute of Australia 2014), the term ‘just/fair’ has been adopted to describe this cultural characteristic rather than ‘no-blame’. This was due to organisations trying to implement no-blame based on face value. ‘Just/fair’ describes the balance between no-blame and accountability in that the organisation accepts the fact that people may make mistakes but those who participate in reckless behaviour are held accountable. This characteristic requires training, education and endorsement from senior leadership to support the adoption of a ‘just/fair’ culture.
The focus group discussions and the literature highlighted that leadership within lessons management requires accountability, ownership and appropriate behaviour modeling by senior management. People in leadership need to promote, enforce and prioritise lessons management throughout their organisation as a business-as-usual function (Attorney-General’s Department 2013, Milton 2009).
The lack of accountability was an issue experienced by a number of focus group participants. Accountability must include actions to be allocated to individuals or teams and for these actions to be prioritised (Garvin 2008, Milton 2009). The focus group discussions highlighted the importance of all members being aware they have a role to play in lessons management, even if it is primarily submitting observations to ensure the organisation maximises learning opportunities. The literature identified that leadership and accountability are complimentary characteristics and if one is not successfully occurring the other will be absent. These two critical components are both represented in the life cycle to ensure they are included in implementation.
Communication was identified as a key theme throughout the focus group discussions and within the literature. One of the main issues experienced by the participants was the lack of communication once they had contributed to lessons management activities, particularly in relation to what had happened with their information and if any changes had occurred. The ‘black hole effect’ was discussed frequently by focus group participants,
‘…closing the loop is such a simple concept. We never ever close the loop. It leads to a lack of interest. There’s nothing worse than putting a lot of time into something and it going into a big black hole and never getting anything out, its demoralising’.
There is a need for transparency and communication throughout the process. By specifying exactly what people should expect throughout a lessons management life cycle, particularly in regard to their contributions, people will be informed, can better understand, and have balanced expectations.
A learning-focused organisation requires active involvement in continuous improvement. Garvin (2008) identified a supportive environment, clear processes and leadership support as essential for a learning organisation. Part of this focus on learning is the importance of improvement. The purpose of lessons management is to improve the knowledge base of the organisation and ensure the organisation makes evidence-based decisions, minimises mistakes, and promotes positive behaviours and initiatives (Glassey 2015).
The Lessons Management Life Cycle in Figure 1 is a process of capturing observations, analysing insights, identifying lessons, assessing action, implementating and disseminating, monitoring and measuring, and lessons learned. The most important aspect of the cycle is that it is continuous and includes two sub-cycles:
The first stage of the life cycle involves the capturing of observations. Although this stage heavily involves the use of tools and techniques, the most important aspect for CFA members was a clear, consistent, understandable and accessible process to ensure members know what is expected and what will occur once the information is submitted (Garvin 1993, Garvin 2008, Milton 2010). The focus groups reported that the level of involvement in learning processes is slowly decreasing due to lack of motivation. This is due to it appearing as though the information is not being used or, if it is resulting in change, this is not being communicated to them
The second stage of the life cycle involves analysing the captured observations and identifying insights. Analysis can involve root cause, theming, and identifying trends. This stage is especially important because it identifies that not every observation will become an insight and every insight may not become a lesson identified. Therefore, the information gathering cycle will continue until a lesson is identified, also providing a filter for low-risk observations. Communication is important to ensure the members contributing the data are aware of what has happened with it. By managing expectations, members understand that some observations may remain as a single observation for a significant amount of time and may never become an insight or lesson identified (Attorney-General’s Department 2013, Newman 2007).
Once a lesson is identified, the next step is to establish what action will occur. There are two options at this point. The organisation accepts the risk of the identified lesson because there is not enough reward for implementing change, or an action is allocated to an individual or team. Accountability and leadership are important at this stage to guarantee that when actions are identified they are allocated, prioritised and implemented within a reasonable timeframe. The literature highlights the need for establishing responsibilities for actions and the process for escalating actions that are not progressing (Milton 2009, Milton 2010, Attorney-General’s Department 2013).
Implementing identified actions and disseminating the outcomes is the next component of the life cycle. This ultimately requires communication to ensure that members are informed of the outcomes of their contributions and can access any learning products. The literature identified that a clear and holistic process is required at this point to allow the organisation to implement the outcomes (Attorney-General’s Department 2013, Newman 2007).
Monitoring and measuring was the least detailed area within the literature and during the focus groups. This is a significant gap because it is here that a lesson identified can become a lesson learned (a term often used incorrectly and interchangeably with lesson identified). Although this area appears neglected within the research, the small amount of literature specifies that monitoring and measuring activities need to be built in from the outset to ensure it is a key component of lessons management. It also states monitoring and measuring should include regular reporting and communication on compliance, activity and output. In particular, reporting against success criteria identified in an implementation report and results being shared widely to ensure communication and engagement. In addition to regular reporting, the literature and focus group discussions identified exercising, training and operational activity as also providing valuable information about behaviour change as a result of implementation (Attorney-General’s Department 2013, Milton 2010).
The research identified components and characteristics that ensure findings are transferrable to other emergency services organisations. The identified barriers and issues experienced by CFA members in this study, the findings and the life cycle itself are applicable for all emergency services organisations. The cohort of CFA members in this research is broadly representative of many volunteer-based emergency services organisations across Australia. Findings from this case study could be extended to other non-volunteer emergency services organisations, government departments and emergency management partner organisations. Further research may find application for non-emergency services organisation contexts (e.g. military domains). Commonwealth documents, including the Attorney-General’s Department Handbook (2013) have a direct influence on the research and findings, ensuring that the life cycle and recommendations are consistent with these documents and applicable to other Australian emergency services organisations.
The research identified a lack of successful models for lessons management within emergency management internationally that are sustainable and proven over time to address social and institutional memory loss. There is currently an opportunity to progress the way emergency services organisations learn from events, improve practices and change behaviour improving safety and capturing knowledge. Emergency Management Victoria is using the research described in this paper to develop lessons management within the Victorian emergency management sector. The project deliverables include a lessons management framework, process and IT system that support the implementation of a successful lessons management life cycle.
In November 2015, Victoria’s first lessons management framework was approved for discussion. The Emergency Management – Lessons, Evaluation and Review Network (EM-LEARN) Framework establishes a model for lessons management, based on the research in this paper and extensive stakeholder engagement. The framework includes a life cycle that provides cultural characteristics and a lessons-management process for implementation. The framework was developed through stakeholder engagement over 12 months, including over 70 meetings with 25 agencies to identify good practise, understand the sector’s requirements and share the lessons management research. The framework supports the Monitoring and Assurance Framework for Emergency Management developed by the Inspector General – Emergency Management by detailing a culture and common process for continuous improvement.
How lessons management is being implemented for operational activities as an initial test of concept is outlined within the framework. The framework will also be applied to non-operational activity in the future, including project management. The operational lessons management process was piloted during the development of the Emergency Management Operational Review 2014-15 (now on the website at www.emv.vic.gov.au/our-work/review/emergency-management-operational-review/).
This report is a summary of the operational activities undertaken by emergency management personnel over 2014–15 supports the continuous improvement of the sector by sharing lessons. This provides a broader focus of year-round, multi-hazard, all-phases, and multi-agency. Part 1 is an overview of the weather and emergency management activities carried out during the 2014–15 financial year and a set of 11 case studies that demonstrate the variety of emergencies managed by emergency management personnel. Part 2 provides insights based on observations from emergency management personnel regarding the management of Class 1 emergencies.
An implementation plan accompanies the framework focused on five areas for action, being governance, communication, process, training and technology.
Due to the complexity of the multi-agency emergency management environment in Victoria, there is a need to establish a lesson-sharing platform to support learning and the lessons management life cycle. This technology is the last component of a comprehensive change process to facilitate learning and improvement across the sector.
The long-term vision of the project is for lessons management to support processes and activities in all hazards, all phases, all agencies and all levels. Throughout implementation of the project, the life cycle will be evaluated and adjusted to meet the needs and requirements of the sector. This will ensure lesson capture and that analysis and implementation is sustainable and contributes to continuous improvement in the sector therefore supporting the vision of ‘safer and more resilient communities’.
Attorney-General’s Department 2011, National Strategy for Disaster Resilience. At: www.ag.gov.au/EmergencyManagement/About-us-emergency-management/Pages/National-strategy-for-disaster-resilience.aspx.
Attorney-General’s Department 2013, Australian Emergency Handbook Series: lessons management Handbook 8. Barton: Commonwealth of Australia.
Bennett S 2000, Learning from ‘near-misses’: a case study. Australian Journal of Emergency Management, vol. 15, no. 3, pp. 7-9.
Boin A, ‘t Hart P, Stern E & Sundelius B 2005, The Politics of Crisis Management: Public Leadership under Pressure, Cambridge, UK: Cambridge University Press.
Bos P, Peters T, Ryan S & Billinghurst A 2008, Can the inclusion of the ‘learning’ aspects of High Reliability Theory add value to the control of risk in Emergency Organisations? (Unpublished Dissertation). University of Ballarat, Victoria, Australia.
Elliott D 2009, The Failure of Organizational Learning from Crisis – A Matter of Life and Death? Journal of Contingencies and Crisis Management, vol. 17, no. 3, pp. 158-167.
Garvin D A 1993, Building a learning organization. Harvard Business Review, vol. 71, no. 4, pp. 78-91.
Garvin DA, Edmondson AC & Gino F 2008, Is yours a learning organization? Harvard Business Review, vol. 86, no. 3, pp. 109-116.
Gilpin DR & Murphy PJ 2008, Crisis management in a complex world. Oxford: University Press.
Glassey S 2015, Preventing ‘lessons lost’: is evidence-based dynamic doctrine the answer? Australia Journal of Emergency Management, vol. 30, no. 3, pp. 11-14.
Huber GP 1991, Organizational learning: the contributing process and the literatures. Organizational Science, vol. 2, no. 1, pp. 88-115.
Jackson LM 2014, Implementing a lessons management life cycle for Country Fire Authority, Victoria (unpublished study report).
Milton N 2009, The status of lessons learning in organisations. At: www.knoco.com/Knoco%20White%20Paper%20-%20Lessons%20Learned%20survey.pdf [1 August 2014].
Milton N 2010, The Lessons Learned Handbook: Practical Approaches to Learning from Experience (1st ed.) Oxford, UK: Chandos Publishing.
Milton N 2014, Knoco stories: From the knowledge management front-line. At: www.nickmilton.com/search/label/lessons%20learned [1 July 2014].
Nasiatka P & Christenson D 2005, Organization Learning in Wildland Fire: Moving Toward a Learning Culture. Paper presented at 2005 8th Wildland Fire Safety Summit (1-4).
Newman V 2007, Integrating Learning into your Processes. KM Review, vol. 10, no. 2, pp. 14-19.
Rabiee F 2004, Focus group interview and data analysis. Proceedings of the Nutrition Society. vol. 63, pp. 655-660.
Safety Institute of Australia Ltd 2014, Shortfalls of no-blame approach to safety culture. OHS Professional enews, (217).
Wilkinson D 2000, Chapter 6: Analysing data. In D. Wilkinson (Ed.), The Researcher’s Toolkit, London: Routledge, pp. 77-96.
Lisa Marie Jackson completed a Master of Emergency Management through Charles Sturt University. She has worked in emergency management since 2009 and is project manager of the Lessons Management Project at Emergency Management Victoria.