What does a well-founded risk decision look like?

By Mark Smith, Mission-Centered Solutions

©Mission-Centered Solutions

This paper is the result of an ongoing dialog around risk I’ve had within the post-Yarnell Honor the Fallen group. One member posed the rhetorical but critical question: “Should we be risking lives for suppression efforts or not? “That prompted my response in The Big Lie essay on the levels of risk I think wildland firefighters operate in routinely, and how we could be more intelligently accepting that risk.

I have been doing a presentation the last few years called “Luck Runs Out.”  Where “The Big Lie” was more a problem statement, “Luck Runs Out” is meant to focus on actionable, practical steps forward for IMTs and Agency Administrators.  What follows started not as an essay but just a handout to go with that presentation, so it was meant to have the context of that larger presentation, although people have told me it reads okay on its own as a stand-alone piece. See what you think!


 

The difficult truth is that wildland firefighting is a high risk-endeavor. Consider the policy that all firefighters on the line carry fire shelters. It is an overt acknowledgment that each time firefighters directly participate in a wildland fire suppression or management activity, their lives are at risk. Moreover, the wildland fire environment is exponentially increasing in complexity, magnifying the risks. What is not keeping pace, however, is our sophistication to plan, operate and support within that complex risk-filled environment. Our tools fall farther behind each fire season.

In an attempt to address this challenge, leaders often make declarations like “No structure is worth a life!” While true, the statement lacks any meaningful guidance. In contrast, it is the job of any risk professional to determine exactly what risk level the structure is worth. And while that assertion may appear straightforward, applying the concept continues to confound senior wildland fire managers.

Where does a well-founded risk decision start? First, it must always start with clearly identified and prioritized values at risk (VaR). It is the “outcome” in “does the outcome justify the risk?” It is the “gain” in “risk vs gain.” It is the “purpose” in the “task, purpose, end state” of leader’s intent.

If the prioritized values at risk are not clearly considered, articulated and displayed for all to see, all subsequent risk decisions will be deeply flawed. They must be specifically discussed in the agency administrator (AA) briefing, and reconsidered during every subsequent objectives, strategy, and tactics meetings.

Second, we must conduct a meaningful assessment of the risks firefighters might encounter. The current risk analysis (ICS Form 215a) on the NWCG website reflects a 1970’s approach to risk. One lists the risks and then mitigations for those risks. It does not quantify the risk in any way, nor is there any discussion of the risk level after mitigation and whether that residual risk is acceptable.

Given what we know about the flaws of that form, its continued use will come to be seen as negligence and opens the agencies and their practitioners to increased liability as time goes on.

In order to see the truth of the risk levels wildland firefighters operate within and evolution required to make the best risk decisions, first consider the two axis, probability/severity model. Most incident management teams (IMTs) now use a modified risk analysis (215a) incorporating this model.

Screen Shot 2018-11-09 at 10.41.25 AM

Typical Probability/Severity Model

The attraction to this model is the simplicity of its Green/Amber/Red “traffic light” appeal. Unfortunately, it is not nuanced enough for the wildland fire environment. When you really do a solid risk assessment, so much falls in places like the “low end of high” or “medium-high”. It also lacks the sophistication of accounting for exposure, such as the number of operational periods, number of fuel cycles, number of people, and so forth.

It also fails to factor in the compounding cumulative effects such as multiple hazards added upon each other. For example, Division A is in high risk because of snags and falling objects, but also because of road conditions and aviation operations. That’s not just high risk, it’s now HighRisk3.

In the exposure curve below, the mathematical reality of just how harshly the odds change as you move up that probability axis is alarming. When you compound risks or factor in exposure, such as being fatigued AND being in steep rocky terrain AND being on Day 12 of an assignment, the percentages increase exponentially.

Screen Shot 2018-11-09 at 10.42.51 AM

Compounding Exposure Curve

 

A key step forward would be adopting a more sophisticated probability/severity matrix that takes these additional factors into account and more accurately depicts the risk spectrum in the wildland fire environment.

Screen Shot 2018-11-09 at 10.43.00 AM

More Sophisticated Probability/Severity Matrix

On the severity axis — You can do a few things on the mitigation side that might influence severity. A jumper’s Kevlar suit, for example, will help when smacking into a tree, but there is good chance the jumper will still break some bones. Similarly, a fire shelter might move the consequences from being a fatality to just being a burn victim, but nonetheless, a significant risk remains.

Given the inherent dangers in the wildland fire environment, severity is always going to be high if things go bad. Given the likely consequences, most of the operations on the fireline are going to be Medium or High risk. It is extremely rare to find a fire where all operations are in the Low risk category.

So most of the decisions concerning risk have to focus on the probability axis. For good risk decisions, a model like this should be part of the AA Briefing and on the wall next to the 215A for discussion. “Should we knowingly risk lives?” –- would more appropriately be re-phrased to “Should we put people where the likelihood of something bad happening is elevated?” You can see on the matrix how quickly risk escalates when you move from Remote to Unlikely to Possible.

One of the inherent challenges in risk management is, as humans, we can’t feel when the odds go from 1 in 1,000,000 to 1 in 1,000. But the universe tracks all that perfectly in real time. This is the discussion that should be happening in front of the 215A. Not just listing the threats and mitigations of rocks, snakes and lightning, but having the dialog and knowing that the real risk is somewhere halfway between the most probable fire and the worst-case fire. Judging how likely it is for an operation to encounter that next elevated risk level and considering the conditions that might precipitate it become essential.

When things move from Unlikely to Possible that’s a big jump, and leaders must reconsider that risk/gain calculus. Unless there’s a cabin in the woods full of babies, puppies and kittens, then the answer is clearly: No, we should not be putting firefighters in a place where the likelihood is moving into the upper end of Possible and the resulting risk Extreme.

Within the wildland fire environment, risk levels are routinely going to be medium or high risk. There probably isn’t anything humans can do to avoid that. We should have very high expectations of our AA and IMT decision makers in terms of critical thinking and their sophistication in making acceptable risk decisions, which means we need tools worthy of the actual risks that firefighters are taking in the current suppression paradigm. Under that current model, we are risking lives and consequently, we have a duty to make sure it’s being done intelligently.

In military special operations, a risk must be determined as necessary during the planning process in order to accomplish an objective. “If we do this, here’s the necessary risk we’re going to have to take.” At that point, the question gets asked “Does the outcome justify that risk?” if so, that becomes the acceptable risk. If not, then you try and mitigate risk down to the acceptable level. If you can’t get it there, it’s not acceptable and military operators look for another way to accomplish that objective with lower risk. In some cases, the objective must be abandoned all together because the risk is too high relative to the outcome.

The difference in special operations is that the small unit (i.e. crew) is heavily involved in the mission planning and the risk decisions. That’s not true in wildland fire –- an echo of why we still have Great Depression/chain-gang era terms like Crew Bosses in wildland fire job descriptions. In wildland fire, risk planning, mitigation and decision-making often occur absent those who will directly encounter the risk. This places a significant responsibility on the AA and IMT staff to discuss strategic and operational risks at the Common Operating Picture (COP) meeting each day, in reviewing the planned end state, and creating (or validating) objectives based on VaR. These strategic and operational risks must be further validated and refined at the strategy and tactics meetings, where the staff flesh out the necessary risks.

Once articulated on the 215A, it is incumbent among every member of the staff to ask the question “Are the residual risks we’re left with — post-mitigation — justifiable?

No tree is worth a life” only tells you what a tree is not worth. But what is it worth? What is the acceptable risk around protecting a tree? A structure? A subdivision? Clearly, the leadership’s responsibility is to make and communicate that decision, but absent a meaningful way to make the acceptable decision, operators are often left to interpret this ambiguous intent on their own.

The shortcoming isn’t a result of the absence of concern, desire or intent, but rather the lack of the necessary tools for wildland firefighters to make any kind of objective decisions about acceptable risk. And that’s because there is nothing that maps the priority of values at risk to the acceptable risks to protect them. We want that to be very clear. Very simple.

Below is an example of what that mapping might look like. As with many examples, things are missing and you may not agree with how the VaR have been prioritized, but that’s intentional. A finished, interagency version should be clear enough so that there is no misunderstanding or disagreement. It is part of the pre-determined playbook. Pending a “Red Book” version, this is an AA and an IMT responsibility to develop and communicate to all:

Screen Shot 2018-11-09 at 10.45.24 AM

EXAMPLE of possible Acceptable Risk Guidelines

This matrix provides clear acceptable risk guidelines for a category of VaR. Incident commanders would be able to make exceptions using the same authority they have now to adjust work/rest and other guidelines, but this removes the “What is a tree worth?” ambiguity.

The challenge remains, however, that without some metrics to assign to an identified risk, the assessment is still subjective. “Hmmm… It will take 2.5 hours to get someone from Division C to the hospital if they get hurt. How do you think that effects the risk level?

Some IMTs are starting to assign some numerical values to each of the hazards and risks identified. Example: “Under 1 hour medevac to a hospital is low risk, 1-1.5 hours + is medium, 1.5 + is high.

As a decision maker during my previous career in the military, I was introduced to a standard risk analysis/risk decision process in the mid 80’s. This example card below was used just for training events, which is why you do not see categories for Enemy Strength, Enemy Cohesion, etc. Imagine that being added to the card as projected incident behavior.

This card is a distillation of the probability/severity matrix, listing the routine variables encountered in training soldiers. This is a “Big Army” tool, so there’s no underwater/night diving type categories, just a plain vanilla tool to help quantify the discussion and get leaders a common operating picture around risk. In Special Forces, we had more sophisticated versions, taking into account such arcane factors as infrared crossover times, moon phase and illumination, solar flare activity, etc. Think 1000-hour fuel moistures and Haines Index.

Once you include the variables of fire weather and fire behavior, and adapt to other common wildland variables you could now have a probability/severity matrix in the AA briefings or tactics meetings and come up with some actual number values to plug into a modified 215A.

Screen Shot 2018-11-09 at 10.46.55 AM

Example Card of Risk Analysis/Risk Decision

This would provide a more quantifiable approach. Inevitably, there will always be situations that despite well-planned mitigations, we’ll still have a residual risk score of high risk. Let’s say a 25 using the above laminated card. This is when we must circle back to the beginning — the values at risk, the “purpose” in leader’s intent. By looking at the acceptable risk table example, you’d see that High risk is just not an acceptable risk to save three chicken coops and a hillside of PJ.

Now what? If we cannot lower the risk through alternate tactics, then we’ll need to back up to the previous C&GS meeting-Strategy. We’ll have to find an alternate strategy to lower that risk and still accomplish the objective. If we can’t create an alternate strategy to lower the residual risk to acceptable, then we need to back up even more and re-look the objective the strategy was meant to accomplish. This process would force decision makers to become way more strategic on suppression actions –- continue the evolution in engagement thinking that moves towards the best ridge versus the next ridge.

The problem with this kind of thoroughness is that it requires time. If an artificial deadline such as IAP production due to copier availability or other factor is driving the quality of our risk analysis, then the tail is wagging the dog.

The question remains, what if we have to accept high risk? Let’s use our earlier example OK, we’ve planned all these mitigations and we still have a residual risk score of 25, high riskbut this time, let’s use a different value at risk and the same acceptable risk guidelines. “The VaR is one of the primary transmission lines of electricity to Phoenix and it’s 114 degrees. High risk is acceptable because if that power shuts off some at-risk people are going to die.” That’s probably a very appropriate risk level.

Finally, in order to have a well-founded risk decision, it is essential to share the risk. Shared risk has been a recent buzzword in wildland fire, but it’s important to truly understand what it means. Shared risk means national leaders create the acceptable risk guidelines based on values at risk, such as the example table. This means they’ve shared the accountability and the risk of putting firefighters to protect the powerline in the example above. It means AA’s and IC’s make the prioritization of the values at risk part of the delegation and dialogue. This increases the quality of fire management interactions with line officers in pre-planning and once fires start.

The net result of this is that everyone involved—national leaders, agency administrators, incident commanders—share in the accountability. It means IMT’s conduct risk assessments with proper tools for the gravity of the job, use more objective criteria, and create leader’s intent with task, purpose and end state. By tying purpose back to a specific VaR, and making the decision on whether the risk is acceptable, now they share in the accountability.

But shared risk means it is also shared down to the operator level. This is way more than having a simple turndown protocol. In the current system, the turndown consideration is completely subjective. “I just don’t feel comfortable.” When operators receive a Division Assignment Sheet (ICS 204), they have no way of knowing what risk level they’ve been asked to accept, so they have no start point to go through the risk management process at their tactical level. A quality 204 would include this: Special Instructions: Risk Level – HIGH – due to increased density of snags in Division A.

If we routinely included the task, purpose, end state on a 204, then each DIVS and crew leader would also understand the VaR they’re being asked to protect. If we included Low, Medium, High, etc. on the 204 then they would know the risk level the IMT decided was acceptable for that VaR. If the acceptable risk guideline table was in the IAP, then they would have all the required ingredients for their own “Does the outcome justify the risk?” assessment at their level and — most importantly — to judge acceptable risks as conditions change throughout the shift. Even in a string of mop up shifts, low risk yesterday could be high risk today because of a wind event overnight. Now crew leaders and sub leaders too, are accountable for risk decisions. That’s what shared risk truly looks like.

Current tools and practices are lagging farther and farther behind the increasing complexity of the wildland fire environment. The growing gap means that more and more, we are relying on luck for success. The worldwide gaming industry’s $90 billion dollars of annual revenue is made possible by one universal truth: “Luck Runs Out.”

The evolution and use of a few simple tools could have a significant impact on the worthy goal of “significantly increasing the odds of everyone going home” at the end of the next fire season. Let’s move wildland fire’s risk management process from the 70’s and 90’s to the 21st Century.

18 thoughts on “What does a well-founded risk decision look like?

  1. This was a great read. Thank you and I would like to see if you would be interested in presenting at our spring IC interagency IC meeting?

  2. Maybe if wildland fire would adopt a similar Risk Assessment Worksheet card as the one depicted and I used in all my Army Aviation operations…hell I carried it when I was an engine slug.

    Wildland fire has borrowed, plagiarized, adopted, stolen, reallocated, cannibalized, etc soooo many SMS and safety item from the aviation industry and military aviation.

    Why not just adopt ONE simple form for ALL fire agencies, not have 429.333 days if Paralysis of the Analysis to produce a card and move forward with this:

    The result of this is that everyone involved—national leaders, agency administrators, incident commanders—share in the accountability. It means IMT’s conduct risk assessments with proper tools for the gravity of the job, use more objective criteria, and create leader’s intent with task, purpose and end state. By tying purpose back to a specific VaR, and making the decision on whether the risk is acceptable, now they share in the accountability.

    I mean…MSC and many of these fire leaders for the last 30 plus years should have had this simplified and adopted…ALREADY

    Tell me why is this such a problem? All the websites and FB postings, FLA’s, etc are great information.

    FIRE like any other Federal Agency has just got so involved in making projects of nearly everything

    “Let’s move wildland fire’s risk management process from the 70’s and 90’s to the 21st Century”

    GREAT Quote? Have we in the fire biz gotten anywhere near the 21 st Century ? I mean we have finally moved to jeeeets as airtankers using 40 yr old and aircraft…cuz the some of the fire agencies are just discovering that portion of the 21st Century since 2006!!

  3. While I do not disagree with your assessment, you are over complicating this process. Remember your audience. Keep is simple.

    Take a look at Gordon Graham’s High Risk, Low Frequency video. He nails it.

    Bottom line:

    Slow Down
    Know your limitations
    Situational awareness
    Discipline
    Communications
    If in doubt, don’t

  4. Why is the Compounding Exposure Curve exponential instead of straight line? The matrices seem to indicate straight line. then, on the card, you add the numbers. That would indicate that all items are of equal weight in the calculation. Is that true?

    • Hi Gordon, a little bit tricky here. Think of the matrices as two-dimensional, and when you compound risks you sort of need something three-dimensional. In other words, We used the “straight line” two-dimensional matrix and found Division A is in high risk because of snags. Then we evaluated for medevac time and again using the two-dimensional matrix we found Division A to be high risk becase it would take 2+ hours to get a FF to the hospital. This is where the 2D/straight line runs out of juice becasue now risk in Div A is not High + High, it is High x High. That’s the exponential part. The latest risk management tools have a way to capture that curve, but as far as I am aware, those tools have not been leveraged yet for risk assessments being done at the AA or IMT level. So the whole deal about using a matrix with more levels of gradiation is a bit of a “cheat” to try and represent that curve using the simpler tools available and on hand now. Not that those advanced tools can’t be used, they are not rocket science, but their adaptation would require a serious and concerted interagency effort and some software development to make them easy to use on an incident. That’s beyond what is in a current practioner’s control, which is what I am focused on. I hope that the conversation does eventually progress to that point in the future. Gordon does that help at all? Thanks. Mark

  5. This statement couldn’t be further from the truth. At least on mine or any good crew.
    The difference in special operations is that the small unit (i.e. crew) is heavily involved in the mission planning and the risk decisions. That’s not true in wildland fire –- an echo of why we still have Great Depression/chain-gang era terms like Crew Bosses in wildland fire job descriptions. In wildland fire, risk planning, mitigation and decision-making often occur absent those who will directly encounter the risk.

    • Hi Johnny! That’s awesome if you have a good crew where you have the input into how you carry out your assignment. I was talking about your crew having direct input and doing the risk assessment or developing tactics before the IAP is published. That’s all done by the C&GS of the IMT and then the Division/Group assignments are done in the tactics meeting by the OSC, SOF, LSC and PSC. Certainly crews have input to their DIVS, who then meet to give their input to Ops before the next operational period, but if your crew checks into a fire on Day 13, you show up to your first shift the next morning and you get an assignment brief. All that was developed without your involvement. I definitely agree there’s lots of good OSCs and DIVS who then encourage crew input into “how” the assignment is to be carried out. But many of those crews are providing that input without ever seeing the risk analysis information from the 215a that effects their assignment. This is something I think will improve as more IMTs are starting to include that information in your 204 division assignment sheet. Does that clarify or are you still seeing things differently? Thanks.

  6. I think Mark nailed it. The federal agencies have been glacial in their adoption of risk management. We have made some progress in articulating risk to responders but have work to do on articulating values at risk and what level of responder risk is appropriate.

  7. Pingback: Saftey – Thinking it Through

  8. Nice article and great comments.
    The Forest Service is releasing an Operational Risk Managment guide and training website. You can find it here:
    https://sites.google.com/firenet.gov/operational-risk-management/operational-risk-management
    We recognized there were multiple RM processes being developed and so we’ve created the FS standard for Operational RM. We’ve tried to make it as interagency friendly as possible and it is deliberately not fire-centric. Importantly we’re also trying to standardize RM terminology which is all over the place, resulting in serious communication difficulties between line and fire.
    I have no doubt some will find this ORM process too complex and others (probably Mark Smith!) will find it not nuanced enough. We’ll continue to improve it over time based on user feedback but I think its a good start at a foundational level. Next step – training site for Strategic Risk Management.
    This ORM guide and website was rolled out to the NWGC Risk Managment committee last month and at 520 last week. We are also teaching to this guide at the Apprentice Academies and the NIMO Risk Managment Workgroup is scheduling to present this at IMT meetings this winter.

  9. I think it’s a great idea to include the VaR and score the risk on the 204. As a single resource boss for a type 2 handcrew I work for a whole spectrum of DIVS/ TFLS from the entire gambit of agencies. Not all agencies have the same definition of VaR. For some timber is the highest VaR and public and structures rank lower or not at all. I think a tool like this would at least get folks on the same page as to what ranks as a VaR and having a quantifiable risk would help in the conversation when assignments are handed down and accepted or not.

  10. Fantastic write up Mark, progressive and to the point. The concepts are easy to recognize, and should not be hard for ANYONE inheriting or operating in the decision making process to come to grips with. My fear is that folks in the upper echelons tend to get “worried” when our partners divulge cut rate information as if the Forest Service was advocating for this or that from people. Simply put, this is education that has been needed for a long time coming, and will continue to be needed from outside the organization/ outside the box thinkers such as Mark and MCS. I applaud all of those that continue to allow information and education such as this to be disseminated, discussed, and acted upon. If … or hopefully “when” ” management” wants to get out of the “managing” game and apply leadership will it be revealed that articles and concepts like this are a veritable gold mine of cultural and operational change. Sometimes its messy when we look in the mirror and realize we have been doing things wrong. The first step to success is then to admit it and open up the vulnerability. Thanks MCS, Mark and the Lessons Learned Center.

  11. Pingback: Safety-1: Antecedent Assessment – Thinking it Through

  12. Sooo…this all folds together in the end, but just hear me out; I recently read an FLA produced due to a loss of life in Oklahoma. The statement was made that the value of the grass being protected was of the highest of values to be protected (my interpretation). I happen to work for a National Grassland and can relate to those statements through the eyes of an agricultural producer.

    I am going to use a grassland fuel model as a specific example in a fire environment where I have witnessed 40,000 acres of growth in a single shift.

    During the initial attack phase of a fire, resources on scene have independent choices to make regarding risk vs. reward. How hard do we “bend the rules” or push the safety envelope in order to secure a fire quickly in an effort to prevent passing that risk on to the multitude (possibly hundreds) of resources who may be called to assist if we can’t get it done? Remember that incoming resources may or may not have been exposed to extreme fire conditions and may or may not be qualified in federal eyes to engage in a fire at all.

    I am the captain on a single type six engine that covers better than 300,000 acres of grasslands and offer mutual aid assistance to adjoining jurisdictions to include federal, state, and private. We have agreements with more than 20 VFD’s that may or may not adhere to NWCG policy and often pave their own way. What I’m getting at is that command structure takes a bit to get dedication to, especially in a rapidly evolving incident. There is always a “risk vs. gain” matrix being used by all involved, but it will most likely be worked through in someone’s head, in a hurry, and the value of the value is very subjective based on the cultural norms associated with the individual responding resources.

    I have a good friend that once told me there are three kinds of suppression fires; There are the one’s that you know you’re going to catch, the one’s that you know that you won’t, and then there are the fires that you can catch if you do your job really well. Doing your job really well often involves “bending the rules”, pushing the safety envelope, and trusting your intuition.

    I am very conscious of the fact that “luck will run out”, but for whom and for what value at risk. What is the value of the value and who get’s to determine that? On what time-table? Who is going to document the risk analysis, pass on the information, and enforce the determination that might well be contradictory to cultural norms and personal values? I can impact the culture of my crew, but attempting to change a culture across a landscape sometimes and sadly involves luck running out.

    My sincerest condolences to the survivors of the Shaw Fire. The lessons learned from you have not been lost.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.