Volume 27 Issue 2, 2012

Mainstreaming fire and emergency management across legal and policy sectors: Preliminary findings on measures of success

Dr Michael Eburn, Professor Stephen Dovers

Peer-reviewed Article


Abstract

This paper reports on a research project, funded by the Bushfire CRC and conducted at the Australian National University, jointly by the ANU College of Law and the Fenner School of Environment and Society. 'Mainstreaming Fire and Emergency Management across Legal and Policy Sectors: Joint Research and Policy Learning' is looking at the impact of law and policy on emergency and fire management. This paper argues that an absence of clearly defined goals in emergency management policy inhibits our ability to make decisions on what are acceptable trade-offs and makes it impossible to know when Emergency Management goals have been achieved. The paper reports on the research conducted so far and identifies the next steps in the research program.

Article

Introduction

This paper will report on the law and policy project being conducted at the Australian National University, jointly by the ANU College of Law and the Fenner School of Environment and Society. It will report on this research conducted so far, and identify the next steps in the research program. Other projects are also reporting in this special issue of AJEM.

Is emergency management mainstreamed into law?

Emergency management has traditionally been seen as the responsibility of the emergency services, such as fire brigades and the state emergency services. Vulnerability, and the ability to protect life, property and other assets, is, however, largely defined by activities and policy settings in other sectors. This interplay of policy means that fire and emergency management should be seen as a whole-of-government and cross-sectoral issue. To mainstream emergency management is to consider how other policy sectors impact upon the community’s ability to prepare for and respond to various hazards.

Our research has identified that:

… fire management considerations are relevant in many policy sectors, suggesting that emergency management is a mainstream consideration but the strength of emergency management mainstreaming is not clear (Eburn and Jackman, 2011, p. 74).

In order to further ‘mainstream’ emergency management into broader policy and law, and to determine the priority of emergency management, or in other words, the ‘strength’ of emergency management considerations, clear objectives have to be identified:

Policy interventions are intended to achieve goals in relation to identified problems, and it would be expected that goals would be clearly expressed in policy statements, and form the reference point for later implementation, monitoring and review (Dovers, 2005, p. 101).

Australian governments have not, however, given a clear statement of what emergency management, across the Prevent, Prepare, Respond and Recover spectrum, is meant to achieve. The National Strategy for Disaster Resilience says a ‘disaster resilient community is one that works together to understand and manage the risks that it confronts’ (COAG, 2009) but that does not give any measure by which we can determine whether or not resilience has been achieved.

Australians will always be subject to natural hazards – floods, bushfires, cyclones and the like. As the fires from 2003 and 2009, the floods and storms of 2010/11 and floods again in 2011/12 show, Australian communities are very resilient to natural hazards having the resources to prepare for, and recover from such devastating events. Making ourselves more resilient, more flood and fire proof may be possible, but not without cost. As we strive for further evidence of resilience it may be reasonable to ask ‘How prepared is prepared enough?’ (Jongejan et al, 2011), and how much are we prepared to pay, and forsake, for extra gains in community resilience? Costs we will have to pay are:

  • Economic – every dollar spent on hazard reduction is a dollar not spent on some other social good and with diminishing returns, at some point we will get better returns from spending the money on the other community good, but at what point? Within the emergencies sector, there are trade-offs between different activities (e.g. fuel reduction, building improvements, community education, response capacities). How those trade-offs are to be made depends on what the objective is.
  • Social – ensuring that all the bush is cleared around high risk communities, or that people live away from the coast and the threat of storm surge, or that people are limited in what and how they build on their properties will change the nature of communities and society generally; and
  • Environmental – undertaking hazard reduction activities such as fuel reduction burning or building coastal protection infrastructure can have dramatic impacts on the natural environment.

Our research concluded that:

Deciding how competing demands will be assessed and balanced requires a clear view on what are the policy objectives; that is what is emergency management policy meant to achieve? … Until there is a clear and specific goal or objective of emergency management policy, it is impossible to identify how that policy can be mainstreamed or the success (or otherwise) of the policy measured. Whatever objectives are selected, different legal and policy tools will be required to achieve them. A clear, specific and measurable goal may be “No one will die in a bushfire” but that will lead to a very different policy response than if the goal is to ensure that “There will be no bushfires.” (Eburn and Jackman, 2011, p. 74).

The need for clear policy objectives was also referred to by former Australian Federal Police Commissioner, Mick Keelty, in his review of the 2011 Perth Hills Bushfire. As a result of that fire, no lives were lost but many people were evacuated and homes were lost. Keelty said:

There remains one question the answer to which eluded the Special Inquiry but it is an answer that requires further examination and that is: What is the measure of success of the outcome of a bushfire? Is the loss of no lives the only performance measure? If so, how many houses is an acceptable number to lose? Does one performance indicator have the potential to cloud the ‘Shared Responsibility’ of all to build resilience of our community? (Keelty, 2011, Transmission Letter p. 3).
Photograph of firefighters with hoses in a bushfire

©Newspix / Megan Lewis

December 3, 2001: Firefighters in the Perth suburb of Trigg tackle flames metres from residential houses.

Measures of success

The next stage of our research will consider the question posed by Mr Keelty: ‘What is the measure of success of the outcome of a bushfire (or any hazard)?’ This will bring our research back to the question asked when we first began to consider if, and how, emergency management can be mainstreamed into law. The critical issue remains ‘what are the objectives of emergency management policy?’ Only when the objectives are identified can we determine what success looks like and what measures may be used to determine if there is, or is not, a successful outcome. It is not apparent that the social, political and thereby broader social goals of emergency management are clear or widely-understood. In an area of intense community, media and political scrutiny and interest, a lack of clarity can lead to unproductive argument and an inability to agree on necessary improvements to current policy and management arrangements.

To gain insight into these issues we interviewed fire and emergency service chief officers from nearly all Australian States and Territories and asked them ‘what do they understand is the measure of success that should be used in Australia?’ The interviews covered a number of related issues that influence this question.1 At the time of writing, interviews with sixteen emergency service leaders have been conducted. The results of those interviews are still being analysed. The discussion, below, offers some preliminary findings: more detailed analysis will be reported at a later date.

Photograph of a Perth hillside at night showing flames from fires

©Newspix / Jon Hewson

February 7, 2011: Perth, WA. Bushfires seen burning at night close to houses in the Perth suburb of Kelmscott in Western Australia.

It is recognised that emergency management is much more than emergency response, it covers the entire Prevent, Prepare, Respond and Recover spectrum. It is understood that vulnerability to natural hazards including bushfires and flooding, is affected by decisions made across government sectors, not just emergency responders, so for example, decisions by local governments and state planning agencies affect our ability to prevent the impact of floods on homes or to prepare to face the fires that will come out of the Australian bush. Some aspects of these cross-sectoral decisions and their impact on living with hazards are being addressed by colleagues at the University of Canberra (on urban and regional planning) and RMIT University, Melbourne (on shared responsibility). Although recognising that the voice of the fire and emergency service chiefs are but one voice, representing as they do the ‘response’ agencies, it is believed that their views on law and policy and the objectives that can be realistically achieved, will help inform the broader public policy debate.

The chief officers recognise that preserving life is a fundamental objective of the emergency services but it remains an aspirational goal. The emergency environment is dynamic, fast moving and unpredictable. An unpredictable variable is human behaviour. People will make decisions that will turn out to be wrong in the circumstances that occur, and deaths will follow. It follows that loss of life is a tragedy that the services seek to avoid, but the fact that people die during a bushfire or flood does not necessarily represent a failure by the emergency services:

… the zero death rate should always be aspirational. It should always be an aspirational goal. So you always push towards it but accept the inevitability of the event as well and then try and narrow the gap.
… that loss of life is tragic but at times unavoidable in these operations… I think if everyone has gone above and beyond their limits and really done everything they could practically and conceivably do in the circumstance, I don’t think that’s failure. I think there’s space to learn or improve or change, but is it failure? No. I don’t think so. I just don’t think it is.

We can then ask ‘If the answer to the question ‘Is the loss of no lives the only performance measure?’ is ‘no’, what might be the measure of success?’ One suggested measure is to measure the emergency response against plans and procedures. The argument would be that if the emergency services did all that they had planned to do, if they had responded in accordance with standards and procedures that had been developed in the calm before the storm, with appropriate community consultation and taking into account important community considerations including impact on life, economic well-being and the environment, then they would have had a successful response even if, tragically, some people died.

Most interviewees indicated a disjunct however, between a lack of clarity in overarching goals (e.g. to preserve life) and detailed operational procedures and plans:

… lawyers are writing the plans … They are so prescriptive as to be almost irrelevant … what we’re trying to do is manage these environments legally by saying, well, you need to tighten up this and tweak that, and write a procedure for that, and close that list off, and make sure all the documentation completely minimises any exposure to risk.
Well, that’s great, but then you’ve got the documents and then you’ve got the environment which you’ve got to operate it within. I’m yet to be convinced that the two will ever align. So, you know, tighter document control or more prescription … probably protects the minister and protects the government … But is that helping me as an incident controller? Probably not one little bit. It’s probably forcing me to be so paranoid about the doctrine that I won’t be able to use my initiative and my experience and my intuition in an operation.

The South Australian Deputy State Coroner has noted:

… one can always find fault in a setting of such complexity. The temptation to criticise the minutiae of every decision that was taken by a group of individuals or by the individuals themselves is sometimes difficult to resist. (Schapel, 2007, p. xiv).

Or, in other words, ‘The best laid schemes of mice and men; Go oft astray’ (Burns, 1785), or ‘… no plan survives contact with the enemy’ (von Moltke, u.d).

In a setting as complex as a developing fire, flood or storm event, errors must be made, even if they are only identified as errors when the consequences have been identified and the presence of hindsight bias makes what was merely a possible outcome, look as if it was always inevitable and foreseeable (Maguire and Albright, 2005, p. 53). Accordingly if one were to set up ‘compliance with operational procedures and plans’ as a measure of success, then an agency is doomed to fail.

Further, as many interviewees noted, set procedures must be interpreted by experienced staff and sometimes varied with good reason. However justified, this would allow criticism of not following procedures at some later point. If emergency responders slavishly obey procedures rather than adapt to changing circumstances, in order to avoid later scrutiny and criticism, the results in terms of lost lives and property may well at times be worse than it otherwise would be. If we aim to have procedures manuals that are completely prescriptive, there will be no room for professional judgment or experience; the only skill incident controllers will need is the ability to read (Eburn, 2012).

Photograph of an emergency services member looking at a map, with other members in the background

©Newspix / Ross Swanborough

January 17, 2005. The search investigating the deliberately lit fires on Mount Dale in Pickering Brook, Perth.

It is often tempting, when designing a system to measure success, to focus on measuring what is easy to measure. Annually, the Productivity Commission provides a Report on Government Services. In the context of the emergency services they measure such things as response times, number of fires contained to the room of origin, the number of deaths and the dollar value of property losses, and the number of households with smoke alarms and emergency plans and service staffing and funding levels (Productivity Commission, 2012, pp 9.2-9.32). These measures focus attention on response agencies and local communities and households. However, in the context of mainstreaming and the issue of success measures, other actors play an important role. These include local and state agencies that determine planning and development, agencies that place infrastructure and other assets in the landscape that require protection, and communications and health services providers. Should the Productivity Commission also measure such things as the ‘number of houses built in at-risk locations per year’?

Many chief officers took the view that it would be appropriate to measure success by comparing the actual losses with potential losses, that is to measure the ‘outputs’ rather than the ‘inputs’. The Commission does provide some ‘output measures’ (the value of property losses, the number of people killed etc) but these are raw numbers, without context. As one chief officer said ‘…we are very scant on outcome measures. That’s largely because … the outcome measures are quite difficult to measure.’

In terms of the 2009 Black Saturday fires, 173 people tragically died, but as a non-Victorian chief officer said:

… we need to get better at measuring what we save. … if you look at the extent of the impact and how many people could have potentially died, and how many people were in that area and didn’t die… That to me is probably quite successful...
So that 173 people dead sounds big, and it is, but … if they said, look, 15,000 people were directly impacted by this fire and tragically we lost 173. It still sounds bad but at least there’s a bit of context around it.

Another, also a non-Victorian, said:

… five million people or something in Victoria and you’d say 173 from five million is … as good as it gets and nobody is prepared to stand up and say that and I don’t think anybody could actually stand up and say 173, what a really good outcome that was. But the reality of it is, is it probably might have been as good as it gets because it’s about what are people prepared to sacrifice … we might be able to prevent fires in the rural landscape from occurring, all we have to do is concrete over everything. But are people going to be prepared to accept that, are they going to be prepared to live in that environment? No, they’re not. So we have to say there is a risk associated with doing anything, and the risk is that there be lives and properties put at risk.

Excellence, or perfection, cannot be the measure of success; ‘Whilst one always strives for excellence, excellence is not to be equated with absolute perfection’ (Schapel, 2007, p. xiv). Perhaps the only measure of success is:

… passing the scrutiny of an intelligent and reasonable critic…. all you can hope for is that reasonable people, reasonable, intelligent people and dispassionate people are fine that it went reasonably well under the circumstances and there were no large systemic failures.

Not that there was a perfect outcome, not that it could not have been better, but that ‘it went reasonably well under the circumstances and there were no large systemic failures.’

Photograph of Marysville Patisserie owners carrying the shop’s sign

©Newspix / David Caird

July 15, 2009: Marysville, VIC. Marysville Patisserie reopening in Glenferrie Road, Malvern after the original was destroyed during the bushfires. Owners Ashraf and Christine Doos carry out the original sign from the destroyed shop still covered in ash. The sign is the only thing they managed to save from the fire-razed patisserie. Pic. David Caird. The Marysville Patisserie recently re-opened in Marysville.

Conclusion

This paper has reported on research being undertaken at the Australian National University on Mainstreaming Fire and Emergency Management across Legal and Policy Sectors: Joint Research and Policy Learning.

We have determined that there is a depth of emergency management mainstreaming, that is emergency management considerations are relevant in a number of areas of law, but the strength of that mainstreaming, how important those considerations are and whether they take priority over other factors is unclear, suggesting that mainstreaming is ‘weak’.

We are now returning to our first question about mainstreaming emergency management. We cannot effectively incorporate emergency management considerations into other areas of law and policy until we determine what we are trying to achieve. We will, with further analysis of our research data and supported by reference to legal and political principle and theory, make further contribution to the discussion that the community has to have about what it is we are trying to achieve. Are we trying to meet the aspirational goal of zero deaths, or the goal of communities, and agencies and interests outside the emergency sector, that understand and appreciate their risks and understand that, at the end of the day, they need to answer the question ‘What are you going to do about it?’

Photograph of firefighters with hoses extinguishing spot fires in bushland

©Newspix / Kerris Berrington

November 26, 2011: Margaret River, WA. Cowaramup volunteer firefighters spot extinguishing fires in Prevelly after bushfires swept through the Margaret River region of Western Australia.

References

Burns R., 1785, To a Mouse, On Turning Her up in Her Nest with the Plough, English translation <http://www.robertburns.org>.

COAG (Council of Australian Governments), 2009, National Strategy for Disaster Resilience: Building our nation’s resilience to disasters, Commonwealth of Australia, Canberra.

Dovers, S., 2005. Environment and Sustainability Policy: Creation, Implementation, Evaluation. Sydney: Federation Press.

Eburn, M. and Jackman B., 2011, Mainstreaming fire and emergency management into law, Environmental and Planning Law Journal, Vol 28, pp 59-76.

Eburn, M., 2012, Margaret River Bushfire Inquiry, <http://emergencylaw.wordpress.com/2012/02/>, 24 February 2012.

Jongejan R.B., Helsloot I., Beerens R.J.J and Vrijling J.K., 2011, How prepared is prepared enough, Vol. 35, No. 1 Disasters pp. 130-142.

Keelty, M.J., A Shared Responsibility: The Report of the Perth Hills Bushfire February 2011 Review, Government of Western Australia, Perth.

Maguire, L.A. and Albright, E.A., 2005, Can behavioural decision theory explain risk-averse fire management decisions Vol. 211 Forest Ecology and Management 47-58.

Productivity Commission, 2012, Report on Government Services 2012, Commonwealth of Australia, Canberra.

Schapel, A.E., 2007, Inquest into the Deaths of Star Ellen Borlase, Jack Morley Borlase, Helen Kald Castle, Judith Maud Griffith, Jody Maria Kay, Graham Joseph Russell, Zoe Russell-Kay, Trent Alan Murnane And Neil George Richardson , South Australian Coroners Court, Adelaide.

von Moltke H, u.d, Helmuth von Moltke the Elder, <http://en.wikipedia.org/wiki/Helmuth_von_Moltke_the_Elder>

Acknowledgement

‘Mainstreaming Fire and Emergency Management across Legal and Policy Sectors: Joint Research and Policy Learning’ is the title of a research project being conducted with funding from the Bushfire Cooperative Research Centre (‘Bushfire CRC’). The project is related to other projects looking at urban and regional planning, community engagement and the impact of law and policy on emergency management.

About the authors

Dr Michael Eburn is a barrister and senior fellow, jointly in the ANU College of Law and the Fenner School of Environment and Society. He has written widely on the law and Australia’s emergency services and is a regular speaker at local and international emergency management conferences, and at the Australian Emergency Management Institute. He is the author of Emergency Law (3rd ed, 2009, Federation Press) and Australia’s international disaster response – laws, rules and principles (2010, VDM-Verlag).

Professor Steve Dovers is Director, Fenner School of Environment and Society, ANU. His research interests cross the policy and institutional dimensions of multiple policy sectors and issues including water, urban management, climate adaptation and disasters. His books include Environment and sustainability policy (2005, Federation Press) and The handbook of disaster and emergency policies and institutions (with J Handmer, 2007, Earthscan).

Footnote

1. These interviews were undertaken within the Human Research Ethics protocols of the Australian National University, which place strict conditions on use of the material to protect the anonymity and interests of the interviewees.