First, many readers sent us the New York Times/Slashdot “Risk Management” link.  Thank you!

The beginning of a reasoned response was written by Aleks  on Andrew Gelman’s blog (http://www.stat.columbia.edu/~cook/movabletype/archives/2009/01/dont-blame-it-o.html).

Long-time readers of this blog will recall that I believe that Risk Estimation != Risk Management.

We security professionals have a really good idea of why Risk Estimation is not all there is to the management of risk based on our experience.  You, smart  guy or gal that you are, know that having a control in place, or even having a point-in-time positive test of that control’s effectiveness does not make you ’secure’.  It’s some evidence of security, sure.  But there are other variables that we must understand, like the ability of the threat to overcome/bypass the control, the skills and resources of those operating the control, our ability to properly manage those people, etc. These things all contribute to security (and therefore, ‘risk’).

Similarly, if Wall Street folks were only using VaR as a means to understand their portfolio risk, or just plain lazy while the market kept going up, that’s not necessarily a failure of risk estimation or risk analysis as it is of risk management.  But some very high profile critics of Wall Street are focusing on the means of estimation that VaR produces, and not the deficiencies in the concept of risk management.  The article does say one or two things about that:

You face the risk that, in the current state of the world (assuming you can estimate that perfectly), an unlikely event will occur. You also face the risk that the state of the world will change. VaR … can quantify the first risk, not the second.

And understanding the”risk” (what a poor choice of words there, mixing and matching terminology) of that change and our ability to adapt to it is the essence of risk management.  So speaking of the fact that Risk Estimation != Risk Management,  I finally got around to reading:

INTERNATIONAL STANDARD ISO/IEC 27005
First edition 2008-06-15
Information technology — Security techniques — Information security risk management
Technologies de l’information — Techniques de sécurité — Gestion du risque en sécurité de l’information

As you can probably guess, I’ve got opinions.  And since we’re both here (me writing, you reading) why don’t I let you know what those are.

I have a few disagreements:

ISO 27005 IS NOT ABOUT THE MANAGEMENT OF RISK

First, in agreement with our discussion above, is that 27005 has very little to do with the actual management of risk.  It’s really an issue management framework where we create a statement about risk for a very particular scenario, and then tie a Plan-Do-Check-Act cycle to the back of that wagon with the hopes that the issue doesn’t become “red” again.  27005 suffers from the same problems as those who on Wall Street who over-relied on VaR as the end all of Risk Management.  It’s whack-a-mole with a hamster-wheel-of-pain sprinkled on top, and does very little to address the root cause of risk - deficiencies in our capability to manage risk.

ISO 27005 HAS ITS PRIORITIES BACKWARDS

Second, and probably due to this inaccurate view of risk management, is their discussion on the purpose of risk management.  I believe that the purpose of risk management is to align the risk exposure of an organization to that organization’s risk tolerance.  The ISO, not so much.

In section 7.1 (which seems awfully late in the document to start discussing the reason we’re all here today) 27005 states:

It is essential to determine the purpose of the information security risk management as this affects the overall process and the context establishment in particular. This purpose can be:

• Supporting an ISMS
• Legal compliance and evidence of due diligence
• Preparation of a business continuity plan
• Preparation of an incident response plan
• Description of the information security requirements for a product, a service or a mechanism

This is all kind of backwards to me.  An ISMS, BCP, or IRP are not *reasons* for risk management. Rather the management of risk is a reason to have an ISMS, BCP, or IRP (or not have one, depending on the organization’s tolerance for risk).  Similarly, compliance and due diligence are executed because executive management has no tolerance for adding those sorts of losses to the risk they already face.

ISO 27005′S VIEW OF THE ROLE OF ASSETS IS FLAWED

Now with those two disagreements behind us, I actually have very little critique of section 8.2 - risk analysis.  I could launch into a rather wonkish discussion about controls and vulnerability and talk at length as to why FAIR provides better definitions, but I think these differences are less important because the fundamental differences in these differences are less pronounced than other significant deficiencies.  Deficiencies such as the one we’ll find in 8.2.1.2 -

8.2.1.2 Identification of Assets.
An asset owner should be identified for each asset, to provide responsibility and accountability for the asset.

Great, as long as the asset owner is really in the line of business that the asset generates revenue for.

The review boundary is the perimeter of assets of the organization defined to be managed by the information security risk management process.

WHOOOOAAAAAAAAAA! - This “Review Boundary” you speak of is dynamic.  Think Jericho.  Think Vendor Management.  Think 100,000 user networks in which 60,000 of the users are not w-2 employees.  Think of “The Cloud” (won’t someone please think of The Cloud?).

Consider the following - malware infects home PC, home PC used to capture home banking creds.  Those creds are then used to create fraud.  Bank is effected, but the Point of Attack (home PC) is not bank owned.  According to ISO 27005, this isn’t an important source of risk, as you’re not expected to measure it.  I really think there are problems with completely ignoring the fact that assets we don’t directly manage contribute to the amount of risk we have.

DESPITE ALL THIS, 27005 IS FORGIVABLE UNTIL YOU REACH THE ANNEXES

Next, lets talk about the annexes.  To be fair (*cough*) without the annexes the ISO isn’t a bad ’standard’.  It is purposefully vague where it needs to be vague, and purposefully specific where it needs to be specific.

Unfortunately, someone thought that we needed more direction and tacked on several annexes.  The first problem with these annexes is that they’ll be taken for gospel even though they are meant to be “informative”.   And if they were any good, that wouldn’t be so bad.  However the threat and asset valuation ones are disturbingly average for our industry, which is to say, not well thought out at all.  And in some places these annexes are just wrong.  To whit: Annex E page 50 asks you to perform mathematical functions on ordinal values:

The third step is to calculate the measure of risk by multiplying (b × c).

I just multiplied Peanut Butter by Airplane and that equals 723.58!

THE BIG PROBLEM

My final problem with 27005 is that it just wasn’t needed.  It doesn’t really add anything remarkable or special that we don’t already have in place in any number of other documents and standards.  It would seem that its only demonstrative use is for the purposes of auditing to standard compliance.  And I have to think that this is really what this document is all about, something more to serve the ISMS and the cottage industry that surrounds it.   And that’s a shame, because the field of risk management could really use someone like the ISO really putting forth a significant and good effort.

Over the past couple years of blogging, I’ve found that about once or twice a month I’ll write a really long blog post on a subject, only to scrap it before publication.   It might be because my thoughts on the subject are too emotional, because they’re just not terribly interesting, or as I found myself doing last week, it was just a really long-winded way of saying something that could be said more succinctly.   Last week I saw someone in a major magazine refer to something called a “PCI Budget”, and that an informal survey of risk managers suggested that these “PCI Budgets” weren’t going to suffer in 2009.

I proceeded to write a several paragraph post about the role of PCI compliance in Information Risk Management, and how, if you had a significant portion of your budget allocated to “buying more PCI”, you were doing it wrong.  My premise was that you should be investing in those things that lead to maturity in your IRM Program, and that you could count me among the many that believe that ideally PCI compliance is the result of a mature IRM program, not the goal of one (see postscript 1).  But the exercise of writing that article was not without benefit, as I started thinking more about that concept of maturity, and what it means, and how that maturity might be achieved.  I thought I’d share my thoughts on the subject with you today because they seem to be much more interesting than PCI for the sake of PCI.  In fact, I think we can build a couple of axioms or mantras to repeat about IRM programs and achieving maturity.  Think of these as my little contribution to the “New School” movement.

First, maturity does not mean “secure”.

Maturity actually means striving to achieve one of two goals (see postscript 2):

1.)  reducing the probable frequency and impact of loss events i.e., risk, and
2.)  the creation of operational efficiencies in the act of risk reduction.

In order to achieve those goals, we have to first understand how to do two key things, understand the risk tolerance of decision makers given the current business strategy (what we might call “aligning” IRM with business objectives), and figure out the “how’s” and “what’s” of measurement.  We could even say that the first is just a significant piece of the last - we need to figure out what and how to measure the risk tolerance of data owners.  But these two needs leads us deduce our second mantra:

Second, maturity can only be reached by measurement.

This one should be a real “duh” but it’s a lot tougher than it looks.  Press most people about what they are measuring and why, and they’ll have a tough time defending their metrics program.  I’ll give you an example.

Earlier this year, I heard from a well known CISO that he had “metrics under control”, that they were really way ahead of the curve and, basically, seemed to arrogantly imply that they had nothing to learn from the rest of the IRM community at large.  Later in the year, when their metrics person joined one of the projects I worked on, I was very interested to see the quality of what they were going to contribute.  I was keenly disappointed that that this organizations representative contributed to the discussion with much the same condescension as their CISO.  So imagine my surprise when it became knowledge that this wonderful metrics program apparently was built on the foundation of vulnerability scanner results, and not even a useful interpretation or translation of those results, just the straight output.  Needless to say, the arrogance quickly disappeared when others (not myself) confronted them on the usefulness of those metrics.

The moral of the story is - if you’re going to get mature, and if you’re going to measure - you’ll probably need help.  Out of all the projects and mailing lists and so forth that I’m a part of, I can think of about a half-dozen or so people that I would really entrust creating a metrics program for my IRM group.  <shamelessplug>Jack Jones, of course, is the primary person I’d seek to hire </shamelessplug>.

Third, the quality of your measurement will dictate the quality of your maturity (with limits).

Another real obvious point, but doing a good job at measuring enables a quality result.  It doesn’t necessarily mean you’ll execute, no, but it does mean that you’ll be in a position to execute competently.   The funny thing about that organization above that “had metrics all figured out” is that there are now entire groups of people that have an idea about the quality of that IRM program, primarily because the quality of their metrics and metrics interpretation were so faulty.  Whereas they thought they were in the 90th percentile or so of IRM program quality, turns out that they may not be *that* much better than everyone else.

It’s important to note that there are diminishing returns to measurement and how much a metrics program will benefit an organization.  Now I’ve got an idea that once you figure out the “how’s” and “why’s” that there are business and operational theory tools that can help divine the most return for time spent measuring (Lean IRM Programs!), but I’m not aware of *any* organizations that have reached that level of maturity yet.

Fourth, maturity via measurement is not a function of any one set of IRM standards that I’ve seen, but could be achieved using several different available tools.

Well, maybe SABSA is all encompassing but most of the world will never know because that thing is a beast.

This thought is based on my experience(small sample size warning!).  Most people are borrowing from here, from there, and creating an ad-hoc model of how the whole management of risk works, and how/what they need to measure.  As an example, I’m personally involved in a couple of efforts around various ISO standards and FAIR, for example, where FAIR provides the basis for adding rigor into 27k series documents.  Now I’m not out to denounce this experimentation around IRM theory, but it’s worth noting that few people are really satisfied with their results.  This mass creation of one-off models that aren’t shared means that we’re all sort of driving blind.  There are obvious problems with this, which leads to my final thought.

Fifth, visibility and co-operation in creating management models and metrics programs are critical.

If you really want a mature IRM program, if you really want standards compliance to be an after-thought rather than *the* business driver, then I’ll offer that it’s going to be really tough get mature in a vacuum.  If it didn’t work for one of the more well-known CISO’s in the industry, I have to think that the chances are against you doing it all by yourself.  I-4, CIS, Securitymetrics.org are just a few places that you can get involved and get started on the road to maturity.

In conclusion, let me state that mature IRM programs won’t be for everyone.  We are still beholden to the risk tolerance (stated or not) of the business owners.  If their tolerance for Information Risk is great - the resources to get mature might be few and far between.   But measurement and development of mature IRM program practices will at least help you calibrate both your and their expectations around organizational information risk.

====================================================================
Postscripts:
(1) - It always surprises me how my reputation regarding PCI proceeds me.  Let me state for the record that I’m pretty agnostic about PCI - it is simply a piece of the impact landscape CISO’s and IRM managers need to be aware of.  What does get my cackles up is the seriousness others (mainly vendors or those vested in the PCI cottage industry) attribute to it’s ability to make you “secure”.

(2) Recall Jack Jones explains that there are only 3 real value propositions for IRM; reduce risk, create operational efficiencies, and reduce loss.  I’ll rarely reference that last value proposition (reduce loss) because it requires having similar past incidents to compare to and proving that “you didn’t screw up as bad this last time”.   It’s just not a lot of fun, so I normally leave it out in discussion.

Sorry for the slow week. We had two sets of training that went (we thought) really, really well.

One of the things we do is ask learners to bring in scenarios that they want to run through FAIR. FAIR is notoriously applicable, and so we often get some fun analysis. Here’s a sample of what we did:

• Risk to average OS X using SMB owner of an OS X virus (hint: think “low”)
• Sending PII via spreadsheets unencrypted to 3rd party vendors (SS#’s to Payroll services).
• Removing the firewall in front of web apps (How Jericho of us!).
• Losing a laptop (Poop happens).
• 1099s VPN in and try to access core assets they don’t have privileges for.
• Does it make sense for me to buy a whole-house generator for power outages?

Also, I made a graphic for your enjoyment that explains why I believe that certain risk management approaches are really just “issue” management:

Bill Brenner has an article in CSO magazine in which “Fortify Co-Founder and Chief Scientist Brian Chess says:

“2009 will mark the end of pen tests as we know them.”

It’s very vogue, this “X technology is dead” mantra I’m hearing from analysts these days.  To be fair, Brian does say:

“…Death doesn’t mean it goes away, it means it transforms. Pen testing will be reborn in the area of production monitoring and measurement…”

Now he doesn’t tell me what he means by production monitoring and measurement, but I’ll give you my thoughts on the subject.

HEY, HEY - MY, MY!  METASPLOIT WILL NEVER DIE!

Me, I’m very bullish on PenTesting.  PenTests, as a tool, have two purposes:

• A discovery tool (find problems)
• A scientific tool (test hypotheses)

The value proposition of the first purpose is fairly self-evident, we’ve been using PenTests as a tool in this manner for years, and I don’t think I need to take any space here to talk about it.

But the value of “hypothesis testing” may not be so self-evident, so lets talk about that.

YEAH, I’M BIG ON SCIENTIFIC METHOD AS A MEANS TO MANAGE QUALITY OF IT SERVICES, WHAT ARE YOU GONNA DO ABOUT IT?

You and I, the risk people who must translate the value of security into business nomenclature, our job is to make educated hypothesis around the probable frequency and probable impact of a threat event.  To do that in the context of tactical risk identification and reporting, we need at least 4 key pieces of information:

1. The probable frequency with which we can expect threat communities to act against us (Threat Event Frequency)
2. The probable impact ($) the business stands to lose in a successful threat event (Probable Loss Magnitude)
3. The probable force a threat can apply against us (Threat Capability)
4. Our probable ability to resist that force (Resistance Strength).

Until you have useful information for those 4 factors, you can forget understanding your exposure to risk for an asset or group of assets (we could collectively call a ‘business process’).

Penetration Testing involves understanding the balance between 3 & 4 (it is, for the FAIR trained, the understanding of “vulnerability”.   I suppose it could also have some relevance to numbers 1 & 2, but relying on a PenTest to be the significant useful prior for those is folly).  Essentially, the FAIR risk analyst makes calibrated estimates for those factors in 3 & 4, placing those estimates within the context of a population distribution.   PenTesting can be the critical way of gathering evidence that reinforces or refutes those hypotheses.

I can hear some of you saying “OK, Alex, but the value of that isn’t self evident to me.  Why would I, the risk executive, spend the money on PenTesting in order to test some hypothesis?”  Well, maybe you will and maybe you won’t.  But I will offer the following.

After hanging out with Jack and Chris this weekend, looking over some new software features and their implications to getting knowledge & wisdom, what I was seeing was so cool that I could only express it in the following Twitter:

“Risk management, tactically speaking, ultimately becomes variability management”

To which Chris Hoff responded:

“This ultimately becomes VISIBILITY management”

Which, is extremely insightful and absolutely correct.  It begged my response:

“which ultimately becomes about measurement, whose usefulness depends on your model.”

PenTesting validates measurements, which validates Hoffs visibility, which validates variability management, which causes strong risk management.  PenTesting will remain an important tool, but it has to get “lean”.

What might “Lean” PenTesting be?

If I were your enterprise CSO, I’d want my PenTesting functions to move from one big “hack the enterprise” multi-week engagements to several small tests designed to rapidly give me useful information about specific strategic points.   So instead of spending (picking a number out of my elbow here) $50k for one enterprise PenTest, I’d rather have 7 smaller $7k tests that test significant hypothesis (i.e. resistance points that occur the most frequently in our analyses), or areas of significant uncertainty (points where the confidence in our calibrated estimates are very low).  When it comes to reporting, spare me the executive summaries, fault chains designed in OmniGraffle and  the Nessus-devined dashboarding in your clever report formats - just give me the following four paragraphs:

1. The hypothesis we’re testing and why (information provided by my analysts)
2. How you chose the skills and resources to emulate the TCap I wanted
3. How you did against our controls (Resistance Strength)
4. Whether you think we should accept that hypothesis and any recommendations.

Seriously, it shouldn’t be more than a page or two of report and depending on the number of assets involved, it shouldn’t take too long, either.  When I was working with @lbhuston we used to do these sort of small tests all the time, just not in as structured of a manner.

PenTesting isn’t going anywhere anytime soon.  If anything, I think it will actually get “cooler”.  But it, like most tools, will evolve over time to provide us with useful information as our view of the world matures.

======================

PS:  And if I were your consultant and you told me you weren’t mature enough as an organization to utilize this approach - I’d tell you to put your money back in your pocket and reallocate it towards someone would could help you get to that level of maturity.  PenTesting just to re-inforce the fact that you’re screwed-up is silly.

So I should be doing a million other things beside this, but….

I was thinking while I was driving today about PCI (yeah, that might be an indicator that I think about Risk Management too much).  And it occurred to me that PCI DSS might have it all backwards.  Now I’m just thinking out loud here, and throwing this blog post up on a whim (for Twitter conversations that are happening in parallel) so be polite/nice…

1.)  As Jack likes to say, all control efforts are centered around Prevent/Detect/Respond.  An if we can prevent at 100% efficiency, we don’t really need to care about D & R.  Similarly, if we can D/R at 100% efficiency, we don’t really need to care about Prevention.  But the real world is ugly, and you’ll never get 100% at any level.

2.)  The concept of Defense-In-Depth is that we have multiple layers of P/D/R.  They work together such that if the first layer Prevention is compromised (P^1), then D^1/R^1 significantly contributes to a second layer of P for subsequent PDR efforts we could call P^2/D^2/R^2

3.)  Assuming that all the PCI PII is good for is either the fraudulent purchases or obtaining more credit, then….

At a very, very high level, PCI DSS could be thought of to be P^1 in terms of protecting the consumer against the threat actions in #3 above.  The fraud in #3 there is an action that has a separate P/D/R - P^2/D^2/R^2

What we really should care about is that we don’t have to get to R^2, because that means we have to spend money we weren’t planning on spending.

PCI 2.0? (Please don’t hate me for using 2.0)

There are those that believe that all this focus and effort at P^1 is economically wasteful.  So what would our options be?

It would be to give up P^1 (in a very Jericho manner) and focus on D^1/R^1 as a means to prevent the success of threat actions at P^2 and eventually getting to R^2 (#3 above).

So that might be Credit Cards with one time passwords, facial verification steps (my Grandmother had her picture on her credit card), etc… at the PoS.   Alternately, it might just be to eat the cost of ID theft protection/insurance for every consumer that bought something (If I’m CSO of Macy’s I work to build the cost of Debix into the annual CC fees the consumer spends anyways).

This wouldn’t perfectly eliminate fraud, obviously, but it would probably cost the overall economy less than the cottage industry of PCI compliance, allow retailers to focus on being retailers and not financial institutions.

So for PCI 2.0 could we just switch to focusing on hyper effective POS authentication at D^1 and ID Theft at R^1 to do extremely effective P^2 (the actual use of the information)?

So yesterday I asked readers to comment on thoughts I had that came from a question asked on the ISO 27001 Google Group:

“How I can communicate the value of an ISO implementation to non-security management?”

This question came to me after one of the posters on the ISO Google Group asked about KPIs for ISO implementation.  Got great responses in email, blog comments, and on Twitter from current/former CISO folks and consultants and analysts.  Some really great thought and effort, by the way - thank you.  It’s really great to be able to have these sorts of conversations online.

First, I have to point out some resources Brian Honan linked to from Gary Hinson, just because they’re so cool.  Gary has invested gobs of time and effort to become one of the defacto resources on the ISO (you might also want to read or re-read Gary’s web post on the 7 myths of metrics).   Brian links to an implementation guidance document(pdf) and a metrics example(pdf) document.

As full of awesomeness as they are, though, these are simply metrics “mapped” to the ISO (i.e. the ISO isn’t a pre-requisite for generating this information).  They are not KPI’s that express the value of ISO implementation.  Problem is the metrics created here still require some level of “translation” in order to create some value statement that data owners can understand.  As Myrcurial twittered me “27001 is orthoganal to process” meaning (I hope) that metrics have their foundation in events that are generated by processes.  27001 by itself was never meant to create metrics (see above), and so we’re asking a question the ISO can’t answer.  But the desire, the need to measure still exists.  To that extent we can google “ISO compliance” (whatever that means) and if something can be certifiable or deemed “compliant” we can and are “measuring”.  But does that have value? Rybolov (my favorite Guerilla CISO) wrote:

“Whatever you do, don’t start measuring percentage of compliance. Eventually, that’s what all metrics efforts around a framework devolve into.”

I have to agree.  Being ISO “compliant/certified” has little expressive business value prima facia. I find that one KPI that absolutely asserts value when expressed properly is risk - and similarly  Shrdlu wrote:

“I really have no idea. I personally wouldn’t try to justify an ISO implementation by itself. If I could show traceability on how it affected our overall security risk, then that’s what I’d do.”

And that’s a delightful answer.  That “traceability” (geeze-louise Shrdlu - what a word!) is absolutely what I’m after here.  How do I get that?  

If you’re going to do something with corporate budget (time, money - and goodness knows an ISO implementation is time & money) you better be able to communicate the value.  And while the zealotry for ISO implementation differs from person to person, I have yet to come across someone who says that ISO adoption is totally without value.  It’s just not apparent what that value of adoption is and how we can measure (metrics) and express it (KPIs).

Jenean Paschalidis wrote what he thought that value was in a very nice email in which she (Edit- whoops!) puts a qualitative name on the value of adoption:

“Transparency and accountability-this is what all executive/senior management (the company) is on the hook for. ISO provides that. If you want to understand and have confidence in your operations as supported by security (because you will know the who, what, where, when, why and how of a system (human, technical etc.) and you want to be able to trace back why a decision (risk-vetted) had been made - then adoption of this best international practice will assist in providing these answers.”

So working with our above thoughts a little here - if we agree with Shrdlu that the only value of an ISO implementation can only be expressed if we can say how said implementation affected our overall security risk - and we agree with Jenean that the primary benefit is an ability to have confidence in operations as supported by security, then….

The value of the ISO should be expressed as a KPI or set of KPIs that cleary explain how the confidence it generates helps us understand (and then reduce) our risk.

If risk is a probability issue,  ISO adoption helps generate confidence in our predictive analytics.  The dollar value the ISO generates (the ultimate KPI) is part of the cost of being able to make wise risk decisions.

So what is that (making wise risk decisions) worth to you?

SOME CONCLUDING THOUGHTS

First, it occurs to me that this is a real shame.  In a sense, an inability to generate a quantitative value statement for ISO use is simply more witch-doctory (“use it because we, the wise men of the tribe say you should”).  In some future version, the ISO should include some mechanism for measuring and expressing the worth of adoption to the organization (a better reason to use the ISO than “because we said so”).

Second, It should be noted that of Jack Jones’ 3 true value statements from which all metrics/KPIs should point to - we’re only talking about one of those value statements - the ability to reduce risk.  Using the ISO in an organization most certainly could create operational efficiencies (help us do more with less) - but the ISO isn’t a standard that creates operational efficiencies as a primary goal, nor does it give implicit direction on how to create operational efficincies.    The ISO folks do, however, play fast and loose with the idea of “risk” and “risk management” so it’s within this context that I interpreted our conversation.

Finally if you’re going to hire someone to help you with ISO adoption in your organization, the deliverables you ask for in your RFP/SOW/what-have-you should include quantitative (probability) statments about risk reduction and the creation of operational efficiencies.  If the firms answering can’t tell you what value their work will be to your company, then drop me a note and I’ll gladly point you to some friends of RMI’s that know FAIR & all our Risk Management frameworks and also do great ISO work.

On Gary Hinson’s excellent ISO 27001 Google Group, the following question was just posed:

Dear Implementers:
What could be the KPIs by which I, being Management Representative,
can show complete picture in a compiled brief/short report? Your
response would be highly awaited.

Which I think is a great question!  Talk about no-nonsense.  None of this “high-falutin” nonsense about ISO adoption providing ‘piece of mind’ and ‘common language’ or ’strategic currency’.  No this is straight from the hip - tell me right now how I can communicate the value of an ISO implementation to non-security management.

I’m not sure I’ve got a good answer.  Do you?  You guys (loyal, cool, readers) are really bright and many of you CxSO’s in your own organizations.  Leave comments and in our next post  I’ll publish the best and brightest (as well as some of my own thoughts).

Usually I beg off of doing posts that link to other posts (Liquidmatrix does a great job of this on a regular basis), but I was afraid that James & Dave’s usually excellent intern might miss some items of note and so I thought I’d offer up a couple of things today:

1)  Gunnar has put up his speech as the Quality of Protection Keynote:  “The Economics of Finding and Fixing Vulnerabilities in Distributed Systems.” Don’t worry if that title doesn’t turn you on, his post is one of the best this year.  I wanted to make today’s blog post some reflection on what he says there, but I haven’t the time today and we’ll have to table that until next week.  Anyway, it’s excellent.

2)  Aleks Jakulin writes about The Future of Data Analysis.  I spoke with a CSO who is morphing into a CRO role and one of the things he plans on doing is hiring about  a half dozen data analysts.  If you think better use of Security Information is in your future, you’ll want to take a look at that blog.

3)  Brent Huston of the Ohio voting machine fame writes about an incident he just worked on and risk and rational security.

4)  Our friend Mike Rothman and our friends at Business Of Security/Cisco are doing a Pragmatic CSO thing.  Mike is always entertaining and practical (dare I say, pragmatic) so I think this should be a fun webex.  Hope you’ll sign up.

Namaste Risk Geeks!

Hope you all had a great weekend.  I had meant to point you earlier to a FAIR analysis that Chris Hayes did over at his Blog.  But I’ve been a little busy, and before I could mention it, Stuart King put up a kind of angry response on his ComputerWorld blog.  Snark aside, there are a couple of other really troubling aspects of Stuart’s reaction to Chris’ analysis that I thought we could talk about this morning.

The problem is that (Chris’ analysis is) completely impractical. I’ll take a recent, and fairly typical situation as an example. I was taking issue with the manner in which remote access was being provisioned for a third party vendor to connect to a system hosted by one of our European business units. To cut a long story short, it was not only a breach of policy but highly insecure. I wanted the access to be disconnected, the business unit director wanted my risk assessment. And he didn’t want to wait for it.

To quote Chris Hayes, spending time on working out the expected effectiveness of controls, over a given timeframe, as measured against a baseline level of force was not going to pacify an angry Italian fearful that my decision was going to cost him money. He wanted my explanation of the risk and more importantly, what I was going to offer as a solution to keep his business functioning

As Chris is someone who actually does this for a living in a large company, and this is typical of his actual day job, I really find Stuart’s “impractical” comment to be, um, misinformed.

Also, I think Stuart mistakes the purpose of a risk analysis.  The purpose of the risk analysis is not to force someone to be “secure”, but to provide knowledge for decision making.  Using it as a “hammer” to knock in the nail of your personal risk tolerance impairs efficiency and in the long run retards “security” as it creates political resentment.  Seriously, who cares if something might violate policy or not in a pre-implementation discussion?  Policies are not stone tablets handed down from on high, they are state-in-time codification of the data owners risk tolerance.  This risk tolerance changes sometimes, and that’s OK.

To that extent, I appreciate (and I’m sure Chris does, as well) that risk analysis does not create rationality in the data owner.  Someone who sees you as a speedbump on the route to progress they may not be ready to appreciate your point of view even if it is stated in the most rational manner possible.   But it’s worth noting (and Stuart’s example is indicative of this point) that risk analysis does not create rationality in the analyst, either.  If one is being so “security minded” as to ignore the risk tolerance of the business owner - we’re bound to get a reaction similar to that Stuart encountered.  In fact, a practical risk analysis like Chris performed on his blog, done in 30 minutes, should identify the critical point of disagreement between Stuart and the data owner (again, Stuart doesn’t own the data, the agitated Italian does).

But let’s stay rational and open to alternatives to what Chris offers.  Stuart states his approach to risk analysis as:

When I need to document a risk assessment I use a very simple form: list the threats, state the level of vulnerability, list the associated operational costs and potential revenue hits. High, medium, or low risk? Describe the controls and options. Write up who needs to do what, and how much of their time it’s going to take. Job done.

At first glance, I don’t think what Chris has done is any less efficient, and it provides greater insight (using Frequency and Capability instead of just ‘listing the threats’).  But what is key here is that Chris’ approach is consistent and defensible.  Less generous risk geeks and CSO’s I know would have no little difficulty with Stuart’s approach.  But to particularly answer Stuart’s main objection (impracticality) I would offer that with practice, Chris’ work is probably quicker and easier than Stuart’s described process as it eliminates much of the ambiguity an immature risk analysis creates - reducing the need for further discussion and arguments with data owners (regardless of disposition or nationality).

Finally the irony of Stuart’s post is that the reason he had this confrontation may in fact be because he was incapable of bringing a salient model for risk to the table, one that identified the factors that create risk and developed a defensible belief statement concerning risk.   We’ll never know if one would have helped him in this isolated instance, but I can tell you that in organizations like Chris’, good risk models and strong risk anlayses create operational efficiencies, reduce costs, and streamlines intra-departmental communications.

Pre-Script - It should be noted that the outcome of this discussion - in the last paragraph - is one smart way you can approach the “We need to reduce your budget” discussion (if that discussion hasn’t come already).

I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2008, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”