About loss
Happy 4th of July!
Before we get into the meat of this post, we need to establish a common definition for the word “incident”. At least for the purposes of this posting we’ll consider “incident” to mean “loss event” (y’all can use whatever definition you like at any other time – it’s one of the things our profession is best at). In other words, something bad has happened that resulted directly in loss. It does not mean “someone violated policy by not choosing a strong password”.
Poll position…
With definition in hand I’d like to take a silent poll:
-
* How many of you have worked for an organization that suffered a security incident of some kind? (I have, and I suspect most if not all of you have experienced viruses/worms, system or data abuse and/or theft by employees, web defacements, etc.)
* In how many of these incidents was there the potential for significant loss/harm to the organization? (In my experience, many of the incidents have had the potential for significant harm.)
* How many of these incidents actually resulted in worst-case loss? (In my experience, none of them did – they didn’t even come close.)
Okay, I’m going to go out on a limb and say that most if not all of you had the same answers I did – at least if you’re being honest (and my condolences to those who have encountered worst-case losses). In fact, if you plotted incident losses on a graph, you’d see the vast majority of incidents result in low to moderate loss, with almost no incidents resulting in worst-case outcomes. Yet as a profession all we seem to talk about is worst-case loss. “The cost of security is less than the cost of a breach.” Pardon me, but that’s bunk (and I’m sure that’s the point Alex was making). A more accurate and useful statement is “The cost of security is less than the cost of a worst-case breach.” Subtle change in verbiage, but huge change in meaning. Shoot, if we built our cars to withstand worst-case collisions we’d all be driving tanks, and the mortgage payments and fuel costs for an M1 Abraham are, at least from where I sit, cost prohibitive (probably fun to drive though).
Aligning planets
If we really want to understand loss we have to ask ourselves WHY loss magnitude so often falls short of worst-case? What planets align to cut us a break? My recommendation is that IRM organizations ought to include loss analysis in their incident response post mortem process. Evaluate not only what losses were experienced, but also what losses might have been experienced and what the factors were that prevented worst-case outcomes. It can be a real eye-opener. Amongst other things, it can help us identify mitigation controls we hadn’t considered before and can help us do a better job of accurately representing risk to our employers.
shrdlu Jul 4
Preach it, brother!!
I feel like I should be tattooing “Possibility is not probability” on my forehead for work. On the other hand, just because we haven’t had those significant-loss events doesn’t mean I can just pack up my policies and leave.
As you said, the crux of the biscuit is “what the factors were that prevented worst-case outcomes.” That’s an exercise in What Might Have Been, which feels a bit to me like back-assed prognostication, which I don’t like to do. It also could easily cross the line into “retroactive FUD,” which I also don’t want (see first para). But when handled right, a post-mortem like that might turn into a nice back-patting session for me and my team.
(On the other hand, why wait for a mortem?)
rybolov Jul 4
So this is where incident metrics fit in. If you know what happens in your environment, then over 6 months you can start to trend and start to put numbers to costs.
LonerVamp Jul 5
“Shoot, if we built our cars to withstand worst-case collisions we’d all be driving tanks, and the mortgage payments and fuel costs for an M1 Abraham are, at least from where I sit, cost prohibitive (probably fun to drive though).”
I would say file examples like that away and pull them out as often as possible. This directly attacks one of the biggest problems we face in digital security: people want perfection, and that is entirely unrealistic. Then again, that’s what the digital information age has given us: hyper-efficiency in criminal/insecure activity such that our countermeasures have to be hyper-efficient as well.
I think your proposals above work excellently for looking backwards at incidents that have already happened. But worst-case scenarios tend to get used for future planning.
Take the events that led up to the Katrina debacle (I’m not ultimately versed in this, but I think I have the jist correct). If they had planned for worst-case, they would have planned for a Category 4-5 storm. But they took more of a risk management approach by balancing their cost on the proability of that “500-year storm.” Sadly, that didn’t pan out and they’ll always be second-guessed for it. Similarly, W. Bush will get the same treatment for not pre-empting Bin Laden and taking him down when he had reports of risk.
We might be able to value risk properly, but can we as a culture handle that when it goes wrong, even on a microscopic scale of a single small company?
Jack Jul 5
@shrdlu - Good point about “back-assed prognostication”. It could go that direction if one wasn’t careful. My experience with it (so far) hasn’t strayed there. It also generally hasn’t turned into a congratulatory free-for-all either. In many cases, the threat actor simply wasn’t that maliciously minded, was somewhat inept, or was stymied by conditions that we wouldn’t normally think of as “controls”. Regardless of the reasons, it’s always a good learning experience that too often I don’t think we spend enough time leveraging.
@rybolov - Yes, metrics!! Think about it. We fret about the lack of metrics but what I’m finding is that we’re sitting smack dab in the midst of a big (and growing) pool of data. What’s been missing is a framework to tell us what data we need and then gives us an opportunity to turn them into useful metrics. Of course, I’d shamelessly offer that FAIR fills this void, and I fully intend to flesh this out in a future post.
@LonerVamp - You’re right about the disconnect between what we recognize as fallacy (perfect security) and what we seem to gravitate toward. What’s the definition of insanity? Something about doing the same thing over and over while expecting a different outcome. You make a good point too, about how decision makers often are flayed alive for not protecting against worst-case scenarios. Hindsight is always 20-20. It seems to me that, given the reality of never having all of the resources one needs to protect against all eventualities, this is the nature of our beast. I try to comfort myself by thinking that I’m in a better position to deal with those slings and arrows by doing competent risk analysis than if all I did was complete a checklist. Hopefully I’ll never have to find out…
LonerVamp Jul 6
You’re right, it might be futile in extreme events, but we’re still improving things and moving our risks into better levels.
Would you rather have an all-star basketball player on your team for the local 3-on-3 tournament, or a couple random guys from the gym? Will that mean you ultimately will 100% win either way? Nope, but you still improve your odds with the former!