Compliance is critical
Compliance has been getting a bad rap lately, and I’m here to set the record straight… compliance is CRITICAL.
Now, those of you who know me are probably picking your jaws up off the floor and asking whether I’ve suffered a stroke, have started drinking heavily, or have a gun pressed to my temple by a regulator or someone from the PCI lobby. Nope. I still have my full mental faculties (such as they are), and I make the statement without duress — however…
There’s compliance, and then there’s compliance
As usual, our profession tends to not be specific in our use of terms, which sets us up for confusion, inconsistency, and a host of other problems. When I say “compliance is critical”, I don’t mean compliance with some external standard like PCI, ISO, or some hypothetical “best practice”. I mean compliance with an organization’s own policies and standards. Compliance with external standards has its place too (unfortunately), but we’ll pick that up in another post.
Think about it…
In most cases, if an organization was completely, 100% compliant with its own policies and standards, it would almost certainly have a much lower level of risk exposure than most other organizations. In fact, in many cases a 100% compliant organization would be too secure to operate effectively. In other words, the more significant problem isn’t typically a matter of how strong a policy is, it’s the variance from intended/desired state that’s described by policy.
In a perfect world…
The illustration below is intended to represent a “perfect world” condition, where all of the assets/systems/whatever are compliant with an organization’s policies/standards. It also reflects the fact that there is no perfect security, and that the organization has wisely established its policies/standards with an acceptance of some degree of vulnerability (and thus, risk).
The real world tends to be much different
The illustration below represents a more likely condition, where controls applied to a population of assets/etc. tend to vary from what policy calls for. It also reflects the effect that has on vulnerability, which in turn affects risk.
But we knew this already, right?
Yes, it’s true that 99.9% of us already know that variability exists and that it’s bad from a risk perspective — so what’s my point? My point is that variance is one of the most important risk-related metrics we have available to us. Here’s why…
As we see from the illustration above, variance from policy can be a strong indicator of an organization’s risk exposure. At the same time, it’s also a marvelous indicator of an organization’s ability to manage risk (i.e., decision making capabilities and/or the ability to execute against decisions). A little root cause analysis of a highly variant asset population can provide critical insights into what’s not working, which can lead to far more cost-effective risk management measures.
One example of where this could be applied is in the evaluation of a third party’s risk posture. Rather than send a 60 page questionnaire, why not evaluate the organization’s compliance with its own policies across a cross-section of its information risk landscape. I submit that it would provide a more accurate and useful picture of risk exposure and risk management capabilities than the typical questionnaire, at less cost/effort to both parties.
Chris Jul 15
Your example assumes that an organization’s policies are rationally devised so as to yield a desired level of vulnerability.
What if the firm solved another optimization problem, such as “make the auditors go away as quickly as possible” or “spend the least money on controls while doing as much as our main competitor”?
shrdlu Jul 15
I think what Jack is trying to say is that whether you do what you SAY you’re going to do is a key indicator of your ability to be secure overall. No matter what you decide your policy is, if you can’t make your peeps follow it, your risk is higher.
Jack, did I just massively oversimplify?
Jack Jul 15
@ shrdlu — Exactly. Thanks.
@ Chris — Good point, but a different question altogether. In fact, the organization that “adopts” policies based primarily on auditor expectations is begging for trouble downstream. In my experience, management won’t support (through enforcement or resources) policies that don’t reflect their risk tolerance, which leads to inconsistent execution and lots of variance. Therefore, while their policies may be “auditor friendly”, the actual audit results (non-compliance) will not be. They also set themselves up for additional trouble because they “weren’t in compliance with their own policies” if/when something bad happens.
As for the lemming approach to risk management (”What’s everyone else doing?”), it can also lead to significant variance for the same reasons stated above. Bottom line — a truly successful risk management program depends on consistent enforcement and adequate resources, which will only happen if management firmly believes that the policies are theirs.
In my last CISO position, I was fortunate enough to be able to review all of our policies with the CEO. He loosened some of what I proposed, and strengthened others. The result was that he became an outspoken advocate and consistent enforcer, which made life SO MUCH easier than I’d ever experienced before. Now clearly, not everyone is going to have the opportunity to do that with the CEO, but the higher up the food chain you can go (on the business side), the better off you and your program will be.
Rob Newby Jul 16
If you apply some economics to Chris’s statement, what could also be the case is that companies are creating their internal policies based on external, existing policies as a minimum control - another perfectly rational decision, and slightly less cynical than just keeping the auditor at bay.
I’ve never been a CISO (and maybe there’s good reason for that), but it seems to me that this would be a good place to start on internal policies at least. Where do you start as a CISO if you can’t do a full risk analysis for some reason? What if there is no CISO?
Having said that, I can only agree that what you do put in place needs constant review and enforcement, and having someone at the top of the company helping your message can only help that.
JonesJ Jul 17
Rob — You’re right. Many organizations grasp onto something “ready-made” as a starting point rather than create from scratch. I’ve done that myself as a stop-gap. The problem is, I’m not sure how many organizations put any attention on evolving past that.
Aligning policies with management tolerance can take a couple of different forms. Sometimes it happens as a result of an actual management review (preferred, in my mind), and sometimes it happens gradually as issues arise that call attention to the fact that there’s misalignment between policy and management’s risk tolerance. A great example are policy exceptions. If management almost always approves exceptions to policies (and this happens often), that’s a good sign that misalignment may exist. Another good sign (unfortunately) is when management says “How the %#$ did we let that happen?! Does our policy allow that?!”
What’s important is that we pay attention to the fact that policies are intended to reflect management’s risk tolerance (not ours), and that we make the effort to align the two. This is an ongoing process too, for a couple of reasons:
* The risk landscape changes over time, which means policies need to reflect those changes
* Management’s risk tolerance changes over time, either because of changes in the business landscape and/or changes in management
The dialog that takes place with management during the process of aligning policies is also extremely valuable. It helps to establish a relationship where we become trusted advisors rather than “necessary evil”, and it gives us an opportunity to educate. This assumes, of course, that we come to the table to engage in dialog (with lots of emphasis on us being in listening mode) rather than to pontificate/bully/spread FUD.
Organizations that don’t have a CISO should (hopefully) bring in some external expertise to help get them off the ground. If not, then I suspect they’re much more likely to have shelf-ware policies that they don’t come close to complying with.
Jack Freund Jul 20
I would argue that ISO/IEC 27001:2005 compliance is a good external measure of internal compliance with an organization’s polices and standards.
The only things that it “mandates” are the management vehicles to ensure ongoing reevaluation of the threat horizon, risk appetite, scope, etc. There’s even a place for any risk management methodology you enjoy–such as FAIR
Jack Jul 21
Hi Jack - Excellent point. ISO’s less-prescriptive nature does make it far more useful as a source of guidance. Perhaps it was unfair to have included it in my list of “lemming” standards. I guess I was just reacting to some of the “ISO compliant” audit checklists I’ve encountered that are highly specific and prescriptive. Thanks.
Jack Freund Jul 23
In all fairness Jack, you assessment is somewhat accurate. Most people have severe misunderstandings between what 27001 and 27002 actually mean. Case in point, nobody can be 27002 (17799) “compliant.” Without the management system in 27001, 27002(17799) is pointless. 27002 in your original context makes perfect sense.
dunsany Jan 7
Yep. And all of this has ironically led me back to liking a SAS-70 more than I thought I would. Mostly because I get to pick the controls and objectives and then an external auditor rates us on how well we’re doing. As a bonus, I get an outside agency to be the Bad Guy when I need to enforce internal compliance.
And when I look at a third-party, reading their SAS-70 is actually very telling. What controls do they feel can past muster. The choices of what is included and what is not included says a lot about the confidence they have in their own controls.
Vijay Mahajani Aug 24
The stage one of this basic risk assessment guide has 2 steps identify the assets and identify the threats but the guide does not consider asset valuation. May be it is covered under step 9: Probable loss magnitude indirectly. Any comments or suggestions here?
I have seen some of the methodologies consider Asset valuation as one of the key parameters for the calculation of the risk.
Jack Freund Aug 24
Hello Vijay!
Asset valuation is mostly a form of replacement loss. Unless your Threat Community causes a physical loss, it’s not relevant for most InfoSec assessments. I wrote more about this here:
http://riskmanagementinsight.com/riskanalysis/?p=641
I hope this helps.