As Alex discussed a couple of weeks ago, Mike Rothman posted an article discussing concerns he has with risk management models. In his article, Mike reminds us that risk management is not a silver bullet, that we should only do as much risk modeling as is necessary in order to achieve our goals (I assume he means the organization’s goals), and that calculating risk to the Nth degree doesn’t keep attackers at bay. I couldn’t agree more.

That said… there are a number of things Mike has in his article that I would like to offer a different perspective on. Specifically:

In his third paragraph, Mike states that “we (the industry) are all about mitigating risk.” Actually, it might be more accurate to state that we, the industry, should be all about managing risk. The difference is that managing risk (or managing anything) is about achieving a desired result. In our case, it’s about managing how often bad things happen and how bad they are when they do happen, to a degree that’s acceptable to management. Mitigation, by definition, assumes reduction and seems to ignore the notion of the balance point where management is comfortable with their risk position and/or maybe even decides there’s a need to increase risk in order to take advantage of opportunities.

Mike also implies that it takes “considerable resources” to build a risk model. Certainly it CAN take considerable resources, but that doesn’t mean it HAS to. To Mike’s earlier point about not calculating risk to the Nth degree, pragmatic risk analysts seek to find the right level of abstraction/complexity in their analyses to fit the need and available resources.

I’m a bit unclear on Mike’s analogy to building a hotel in a swamp… He is correct that all risk-modeling requires assumptions and estimates. Yup. No question of that. So instead he’s suggesting that we should, ummm, NOT model our risk scenarios and instead just shoot from the hip? That seems to necessarily entail even greater assumptions and fuzzier estimates that are at least as vulnerable to error (and arguably even more vulnerable). This seems even swampier, so to speak, than risk modeling.

Mike is right about the lack of precision in risk quantification. In fact, if your estimates of future events are dead-on, you were lucky. This is true in any analysis of future events. Mike states that actuaries “know” we’ll have 3.7 car accidents in our lifetime. Of course, they don’t really KNOW this because they can’t predict the future, even with their terabytes of empirical data. If we, as individuals, actually experience 3.7 car accidents in the course of our life (assuming we could have a .7 accident), it’s coincidence. Almost everyone will experience more or fewer than 3.7 accidents. What he really means is that, on average, the population will have somewhere in the neighborhood of 3.7 car accidents. There is no means of predicting precisely what the experience of a single individual will be.

The point is, precision in risk analysis is a pipe dream. But it isn’t precision we should be seeking, it’s accuracy. (This was pointed out to me by two senior vice presidents of the actuarial departments in a large insurance company when I first started working on FAIR.) You can be precise and yet terribly inaccurate — e.g., an estimate that John Doe will make $4,325,211 dollars in 2007 would have been a precise figure but inaccurate (he only made roughly $500k). If, on the other hand, we had estimated that John would make between $400k and $550k, our estimate would not have been precise but it would have been accurate. The degree of required precision for any estimate is dependent upon the situation. I don’t know any business people who expect precision in our risk estimates. They expect ballpark accuracy.

I’d also differ with Mike’s assertion that “it’s all about measuring the relative risk” and that “What security managers should strive to do is get a relative idea of the risk to each major business system…” Yes, relative risk is important because you want to focus on the higher risk issues, but this disregards the question of whether the amount of current risk is acceptable. Focusing too strongly on relative risk seems to take us down the old rabbit trail of continually winnowing down the risk issues until they just disappear altogether — which, of course, is also a pipe dream.

Mike’s statement that “any security person worth his or her salt should already know what major systems are most important and what the potential risks to those systems are” is true. That’s not the point though, or at least it isn’t enough. We can know (or think we know) what the important systems are, and what the relevant threats and vulnerabilities are, but what matters is whether the combination of threats, vulnerabilities, and value/liability at risk is acceptable and, if not, how far off it is (i.e., the degree of unacceptability). And at the end of the day, acceptability is not the security professional’s call, it’s the organization’s leaders’ call.

We, as security professionals, are still on the hook to get our jobs done. Mike’s absolutely right about that. This, however, begs the question of what our job is. If we think it’s to “secure things”, then I believe we’re on the wrong track. Securing things is simply a means to a more important end. Loss happens. It will always happen. And because loss happens, we have to recognize that, ultimately, we’re being paid to help ensure that the losses our employers experience are at an acceptable level. This is called risk management. And risk analysis/modeling — done well — provides the means to manage risk more effectively on a more consistent basis.

Posted on December 29