It is possible to “justify” a one hundred fold increase in blowout risk as a result of the Deepwater Horizon accident. This is concerning as the risk has probably actually gone down. In this publication we explain common pitfalls and associated misleading results that can be obtained from risk matrices. Also, an alternative is proposed that is better suited to analysis and decision making for low-frequency but high-consequence events. Finally, we provide a reasoned argument for why we think the risk has reduced.
1.0 Lies, Damn Lies and Statistics
Picture a thirty-year-old man who lives in a lightning-prone area but isn’t afraid of lightning. When a lightning storm threatens, and friends and family confess their terror, he laughs at them. “The chances of getting hit by lightning are one in a million!” he proclaims. “Why worry about a million-to-one shot?”
Then one day it happens – he is hit by lightning. He survives, but only after a painful spell in the hospital and a lengthy recuperation. He now realizes that he was wrong before. The odds of getting hit by lightning are not, he realizes, one in a million – instead, they are one in thirty (because he’s thirty years old).
He resolves to do everything he can to prevent being hit a second time. Leaving his job, his family, and his friends behind, he moves across country to the Northwest, where lightning strikes are uncommon. He festoons his house with lightning rods, and spends a small fortune for the latest high-tech weather forecasting equipment. And whenever a storm portends, he goes into his basement and stays there until it passes.
Was he too cavalier about the risk before he was hit? Or is he overly cautious now?
Well, the answer to both questions is “yes.” According to the National Geographic Society, the actual chance of a person’s being hit by lightning once in their lifetime is about 1 in 3,000. That’s a lot higher than his off-the-cuff “one in a million” prior estimate. Clearly he should have been more careful. But it’s also a lot lower than 1 in 30 – so maybe he is going a bit overboard now.
Risk assessments use the past as a guide to what is likely to happen in the future. Generally, everything else being equal, the more frequently a given incident has occurred in the past, the more frequently we can expect it to occur in the future.
This sound and widely used principle will be used to illustrate potential shortfalls with a well recognized approach to risk assessment used throughout the oil and gas industry.
The number of deepwater wells drilled in the five years since DWH is not significantly different from the number that had been drilled in the same period before. But the major incident database – the history that is used to predict the likelihood of an event occurring – has changed dramatically due to two major blowouts: Montara (North coast of Australia) and Deepwater Horizon. So it’s clear that the risk of blowouts in the Gulf of Mexico (GoM) and elsewhere has increased alarmingly.
Well, we’ll see.
In this blog, RiskCom presents alternative ways to assess blowout risk using a recognized semi-quantitative approach – a risk matrix. We consider both before and after the DWH accident and illustrate how misleading a risk assessment can be.
2.0 Semi-quantitative approach – risk matrix
The image below is of a risk matrix similar to those used by many exploration, production, and engineering companies in the energy and chemical industry. The application of a matrix like this to risk assessment has been termed semi-quantitative analysis.
Use of a matrix to identify the risk of a catastrophic blowout before and after the DWH accident can give results that can be variable, conflicting and confusing.
As I hope to illustrate.
Figure 1: Typical risk matrix
3.0 Blowout risk ranking using a risk matrix
I’ll use the risk matrix presented above to estimate the risk of a blowout in the GoM for the following cases:
- Typical blowout before DWH accident
- Blowout using data from 2007 MMS as a guide
- Extended duration blowout similar to DWH accident
- Blowout risk after DWH accident
3.1 “Typical Gulf of Mexico blowout” risk BEFORE DWH accident
So, let’s start by putting ourselves in a hazard identification (HAZID) workshop before DWH, identifying the risk of a “typical” GoM blowout.
“Typical” being the event agreed by the team in the workshop to be representative of the most likely outcomes from the base event, e.g. blowout. It is very rare for a HAZID workshop to consider several types of blowout and all the associated consequences, so usually just one “typical” event is identified and ranked.
Before DWH, based on what we knew then, we would probably assess the potential worst-case consequence at C or D, and the likelihood at five or six. But choosing D5 (the lower end of both ranges) yields a Medium risk, while opting for C6 (the higher end) results in a Very High risk.
Essentially, that’s the difference between a “tolerable” risk and a show-stopper.
That’s a lot of difference. And the point is that we didn’t really have a firm basis for choosing between C and D, or between five and six.
Figure 2: Typical risk matrix with typical blowout risk identified
3.2 “Typical blowout risk” based on MMS study – BEFORE DWH accident
The reality is that before DWH, blowouts in GoM were not all that infrequent, but the consequences were usually minor.
In a 2007 report to the industry, the Minerals Management Service (MMS) analyzed blowouts in the GoM and found 39 blowouts in the 15-year period between 1992 and 2006. This is equivalent to one blowout in 387 wells.
As to consequences, the MMS concluded: “Only one fatality and two injuries resulted from drilling blowouts during the current period…. As in the previous study, environmental impacts were negligible.”
Plotting this blowout risk data on the matrix, the worst-case safety consequences would not exceed D, while the environmental consequences would be F at worst. The frequency of the safety consequence would be 4, since a blowout with fatality had only happened once; the environmental release likelihood is higher, perhaps as high as 6.
In either case, the risk is ranked as Medium. In this case, using actual data as a guide reduces the uncertainty in the risk estimate.
But are we oversimplifying a more complex situation, with potentially more serious results?
Figure 3: Blowout risk identified based on MMS study data
3.3 Extended duration blowout before DWH accident
Up to this point, we’ve talked about all GoM blowouts, at all water depths, including those blowouts (the vast majority) that are quickly contained with minimal consequences.
Now, let’s look at an extended duration, deepwater blowout, ranked as we would have done before the DWH accident. The potential environmental and/or financial consequence ranking could be as high as A or B, corresponding to a Piper Alpha fatality level or a Montara-sized environmental release. But by definition, the likelihood of such a catastrophic blowout could not be higher than 2, since, before DWH, there had not been such a blowout, in the GoM or anywhere else.
Given the very low likelihood, the resulting risk ranking is most likely B2, a Medium risk. If one elevated the associated consequence to an A, then the risk ranking becomes A2 – High risk, albeit at the low end of the High range. See Figure 4.
Figure 4: Extended duration blowout risk
3.4 Extended duration blowout AFTER DWH accident
After DWH, according to our likelihood definitions, a catastrophic deepwater blowout went from a likelihood of 1 or 2 – being something that was theoretical but almost inconceivable – to something that “Had occurred once or twice in the industry within the last ten years,” a likelihood of 4.
The worst-case consequences are certainly A for environmental damage and financial impact, and probably C for safety.
The corresponding risk is A4, at the upper end of the High range.
Figure 4 shows the “Before DWH” and “After DWH” risk rankings plotted together on the risk matrix.
4.0 Remarks and conclusions
4.1 Putting it all together and producing rubbish!
We’ve looked at several different ways that the risk of a GoM blowout could be ranked.
- A “typical” blowout as it would have been evaluated prior to DWH, using the workshop method
- A “typical” blowout as evaluated prior to DWH, based on MMS data
- A catastrophic blowout (multiple fatalities and major environmental release) as it would have been evaluated both before and after DWH
The final figure shows all of these risk rankings combined on a single matrix. The results are truly “all over the map.”
Therefore, they’re not very helpful.
Figure 5: Plotting blowout risks from all assessments
4.2 Has the risk increased a hundredfold?
Comparing the pre- and post-DWH assessments above, the risk has increased from A2 to A4. Still in the “High” range, but now at the upper end of that range rather than at the lower end. This is a substantial increase.
But does that make sense?
If companies continue to do the same things in basically the same way, how can risk get worse? Was our estimate wrong before? Or have we got it wrong now?
Actually, as with “lightning man,” the answer to both questions is probably “yes.” Once something has happened, we can’t plausibly say that it is virtually inconceivable. So when we said exactly that in our pre-DWH workshop, we were wrong.
But we also don’t want to be “lightning man”, cowering in the basement while life passes us by. His odds of getting hit weren’t really one in thirty; he just got unlucky. Most people never get hit by lightning. And maybe (in reality, almost certainly) DWH just got unlucky too. Extremely unlucky – the sort of extreme bad luck that isn’t really likely to repeat itself just a few years down the road.
What we can say, however, is that the risk has not really increased after DWH – a good thing!
4.3 Why the actual risk has probably been reduced
In the previous section, we said that risk should not change if companies keep doing the same thing in the same way.
That’s not what has happened, of course.
As a result of DWH, companies have made numerous improvements in equipment and procedures for deepwater drilling. One example is the greatly increased availability of capping stacks. If a blowout cannot be stopped by the facility and/or ROV, a capping stack can be deployed “quickly” and the blowout stopped, thereby reducing the likelihood of an extended duration blowout.
Another is the implementation of safety and environmental management system (SEMS).
The combined effect of these two measures almost certainly has reduced the risk of an extended blowout.
4.4 Perils of drawing “vast” conclusions from “half-vast” data
The overarching conclusion is to be very careful when applying a semi-quantitative method such as a risk matrix to an extreme high-consequence, low-frequency event such as a DWH-type blowout.
Risk matrices like the example used here are very dependent on industry experience. But for the most part, industry (thankfully) has limited experience with catastrophic events, so we really can’t say, with any kind of confidence, just how likely such events actually are.
Catastrophic accident events such as DWH don’t result from a single equipment malfunction or operator error, but from a series of malfunctions and mistakes, any one of which might have prevented the accident had it gone properly – the “Swiss cheese” model in action.
Risk matrices, with their off-the-cuff, multiple-order-of-magnitude estimates, can’t really predict the likelihood of such an event.
And, just as important, they can only give limited practical guidance as to how to reduce the risk.
Some companies have a rule that when the risk from a hazard scenario is high enough – or when the consequences are sufficiently severe, regardless of the estimated risk (an implicit admission that we don’t really have good data on the likelihood of catastrophes) – the scenario should be analyzed in more detail with a more quantitative method, such as a layers of protection analysis (LOPA) or a fault tree analysis.
These methods account for industry experience, not with a single catastrophic consequence, but with the reliability of individual devices and decisions, each of which have to go wrong before such a consequence can take place.
This could become standard practice across the oil and gas and chemical industries as these methods would allow a company to take credit for improvements in safety measures (or not!).