Was 9/11/01 a “Black Swan”?
What started the FAIR discussion on TaoSecurity was a quote by Marcus Ranum:
Finally, Marcus slams the idea that one can use an equation to quantify risk. He calls “Risk = Threat X Vulnerability X Asset Value” one wild guess times another wild guess times another wild guess. I agree with this but I would say the concept of separating out those variables helps one understand how Risk changes as one variable changes with the others held constant.
Marcus also offers two approaches to dealing with risk:
- Think of all possible disasters, rank by likelihood, prepare for Top 10. (9/11 showed this doesn’t work.
- Build nimble response teams and command/control structures for fast and effective reaction to threats as they materialize.
There are many problems with Ranum’s statements about, and approach to, risk - but let’s not rehash those. On this anniversary date, let’s take a look at risk and his statement about 9/11:
Essentially, Marcus is calling 9/11 an “unlikely event”. But was it an unlikely event, an extreme outlier, or more than an outlier - A “Black Swan”?
As we consider threat communities, traditional information security professionals like to focus on the technologies they have at hand. The exploits. You can thank the scanner for this mindset - we just love to match threat(sic)/vulnerability pairs. In my mind, this is one of the problems with current unsophisticated information risk analysis (what some would call risk management) - we focus on the plane and the financial value (real estate worth) of the buildings. But threats are not pieces of code (except for self-propagating malware) - they are actors, they have motive and intent. They can be studied. Many times they have past actions.
What past actions do we associate with Islamic terrorists? Well, they like to strap explosives to themselves and blow things up. They also like to hijack planes. In fact, they’ve been hijacking planes since before either Marcus or myself were born. Plenty of prior information there (In fact, some people contend that US Intelligence knew that Osama was considering a hijacking event.).
What about this specific threat, Osama Bin Laden? Well, he did have a past prior for trying to blow up the World Trade Center, didn’t he? I don’t think his past failure would have made it less likely for him to try again, do you?
HINDSIGHT IS 20/20, BUT THERE ARE LESSONS TO BE LEARNED?
Now obviously, we’re dealing in hindsight. But I want to use this example to point out few things:
First, to the Bayesian risk analyst, 9/11 should not have been a “black swan” when considering Osama Bin Laden as a threat. We had priors for both the threat actions (hijacking a plane and the suicidal act of blowing ones self up) and the intent of the threat to impact the assets in question. The right approach would identify these pieces of prior information.
Second, the act of “Think(ing) of all possible disasters, rank by likelihood, prepare for Top 10. (9/11 showed this doesn’t work.) is not good analysis. It suggests the following:
- We ignore impact(!)
- A focus on possibility and not probability
What 9/11 “shows us” is that if we don’t include prior information, or do a crappy job at risk analysis, then Marcus is right - it won’t work.
A SMART IDEA
Finally, I’d like to focus on something very smart that Richard says here:
I would say the concept of separating out those variables helps one understand how Risk changes as one variable changes with the others held constant.
This is hugely important. What Richard is advocating here is a modeling and testing of hypothesis approach. This is of critical importance not only to you, the risk analyst and security professional, but also to the future of our profession. If we don’t adopt a scientific approach with more rigor in analysis, I think the future looks bleak - in much the way Ranum suggests it will be (not in that blog post but another excellent one he wrote somewhere on his website http://www.ranum.com).
Of course, what Richard says we should be doing is susceptible to Garbage-in, Garbage-out - but what, in life, isn’t? As someone just posted on a good statistics weblog, “Any statistical method has assumptions. Maximum likelihood, for example, can be much more unstable than Bayes–that’s why Bayesian inference is sometimes called ‘regularization.’” A good framework makes us account for those assumptions.
Tomas Sep 11
Interesting post!
/Tomas
Ben Sep 11
I think the one thing you’re missing in your “black swan” analysis is this: the tactic of combining plane hijacking with suicide bombing was new (innovative, some might argue). Past actions _were_ considered when looking at hijacking scenarios, and incidentally none of those prior scenarios involved intentionally flying a hijacked plane into a skyscraper (or other structure, for that matter). So, in a sense you’ve perverted your analysis by identifying 2 separate, yet related, attacks and somehow extrapolating that we “should have seen it coming.” Maybe the spooks knew about the attack method, but it must not have been too conclusive because nothing came of those reports (politicize it all you want, if someone had asked most people, regardless of intellect, on 9/10/01 what they thought about 4 groups of terrorists hijacking multiple planes and flying them into buildings like the WTC and Pentagon, I think the response would have been scoffing and/or disbelief).
My point here is this: if you can’t conceive of a specific threat scenario, then how do you estimate the likelihood of it happening? Instead, you really do have to look at the asset value (or, perhaps more correctly, the loss and/or replacement value) and then determine just how much protection you’re willing to throw at something. This line of thinking does, of course, degrade into ranking exercises about probabilities of different threats and vulns, of course. And there we are, back to imperfect numbers again…
Alex Sep 11
Ben,
Good points, and I did consider such a “perversion” (interesting choice of words) when I wrote the article. This is why I’ve stopped well short of statements about ability to predict the act of flying an airplane into a building, and suggesting that we’re way into hindsight land in this discussion. My point wasn’t that could we predict the specific action - it’s that we had non-empirical prior information for use - hijacking actions, suicidal actions, and a preferred target, and a good analysis model for the specific threat actor would force the analyst to identify and use those pieces of prior information. Would the analyst be able to use that prior information to come to the conclusion that Bin Laden had ordered the specific types of attacks he did? I actually kind of hope not (it would mean the analyst had a pretty sick mind).
Back to IT - if you’re thinking that conceiving all specific threat scenarios is difficult and something akin to boiling the ocean, I agree. It’s one of the reasons I came to be displeased with some specific risk analysis methodologies. I personally like FAIR because it doesn’t consider that sort of Threat/Vulnerability pairing - the probable action (misuse, deny, access, disclose, modify) comes in determining probable impact, *not* the frequency of probable loss events. We don’t need that level of precision to be accurate. If we use advanced stochastic methods with the FAIR framework, we can even work with “imperfect numbers” (what a Bayesian might call “noisy data”) in testing our risk hypothesis for probability.
Osama Salah Sep 12
Furthermore to the comments above you shouldn’t really be talking about ‘impact on asset’. Terrorist attacks are supposed to terrorize and that does not necessarily mean that there is a huge direct loss in terms of asset value or human life (and then again maybe citizens are assets of their countries?). For example the anthrax attacks had everyone panicked when receiving mail or spotting any white powder. There weren’t any major asset losses or widespread loss of life but it messed with people’s heads, it terrorized them. Maybe you can go further and try to quantify indirect impact, like losses of the flight industry, loss of productivity etc…where do you stop?
I don’t see how risk analysis could help with terrorism. Even ‘credible’ scenarios are just too complex.
Personally I don’t expect risk calculations in IT to be the same as in Insurance, Engineering, Finance etc. They might have much in common, but the devil lies in the details.
Jack Sep 12
Osama — Excellent point about not focusing on impact to the asset (from the terrorist perspective, especially). In many (most?) loss events, the larger portion of “impact” occurs to the owner of the asset due to diminished value and/or increased liability. After all, assets are “assets” because of some value they provide. As you point out, in the case of terrorists it’s impact to this owner and/or associated stakeholders that matters, and the asset is usually just a means to an end.
I believe you can evaluate and reasonably (but FAR from precisely) quantify indirect impact. Of course there will be a point of diminishing return in how granular or far flung you get in the analysis relative to any improved results. The point is, we don’t have to be precise in order to gain value from the anlysis.
As for risk calculations in IT relative to Insurance, Engineering, and Finance… I agree that the devil lies in the details, but that’s true for those disciplines as well as IT. The insurance actuary has gobs of data but still can’t tell me whether I’m going to live to be 100 or die tomorrow. He/she can only cite odds. And I can’t imagine a rational finance analyst who expects to predict exactly what’s going to happen with the market… even with the massive amounts of data that are available. There are just too many variables and unknowns. And engineering — for all its amazing accomplishments — operates on a level of abstraction in precision that’s limited to the available knowledge and resources. We often seem to forget that capabilities in science and engineering are finite and uncertain too when we get to finer layers of abstraction.
It seems to me that we tend to grant an unwarranted degree of respect to these more mature disciplines when comparing them to our discipline. But there were points in their evolution where they suffered from many of the challenges ours does today.