Vulnerability Centrism
We talked yesterday a little about Marcus Ranum’s podcast (which I think he really should have named “Ranum’s Rants” or something more quirky and fun). One of the other things that Marcus (MJR) talks about in his podcast is the current state of computer security, and some of the dumb ideas that are perpetuated by the industry. One of those ideas he believes is dumb is penetrate and patch. And when he mentioned it in the podcast, it got me to thinking about a question I’ve been asking myself for some time now.
ARE WE TOO SCANNER CENTRIC?
Let’s set the WayBack machine to the mid 90’s. NT 3.5 and 4.0, “stateful inspection” and IDS are hot new technologies, and the Security Administrator Tool for Analyzing Networks has just been released. The world is about to change.
The advent of scanners was one of the most significant advances in network security, ever. It (and NT 4.0) leads to “scan and patch” methodologies, and they become the staple of many InfoSec programs.
Fast forward twelve years. These days, scan and patch is an effective control process against automated malware and the bottom 2/3 of the external amateur threat community population distribution. Other threat communities may start with a scan for low hanging fruit, but their mode to intrusion probably won’t be SANS top vulnerability if you’ve got a good scan and patch system going on. What happened? Bad guys have evolved.
But have we?
Take a brief look at the impact of scanners. Two are fairly notable:
- The Most Widely Accepted Risk Assessment Methodologies are Simply Derivatives of “Scan and Patch.” OCTAVE and NIST both use scanning as the “engine” that drives risk assessment. They then add some fairly unsophisticated probability and valuation steps to a scan to create a risk belief statement. Many smart people (MJR and Andrew Jaquith included) now question the validity of risk management partially because of these shortcomings in Risk Assessment.
- The Most Widely Accepted ISMS Certification Processes Are Derivatives of Scan and Patch. ISO certification (and PCI certification, yeah, we better consider it a de facto ISMS now that Texas is considering making it law) are essentially scan and patch with some other stuff thrown in (like audit of management buy-in and awareness programs) to make it look like we know what good risk management means.
Thing is, understanding patch levels is only one aspect of determining current state for control strength. In fact, if you take a look at FAIR as a framework for what makes risk, then control strength is only one of eight factors of somewhat equivalent importance at the point where it is considered.
 I’m thinking that your patch level, the results of your scanner, are simply one piece of data to be used to set the lowest common denominator for Control Strength. But because it was revolutionary back when we were arguing Pentium Pro’s vs. SPARC 20’s for the pinnacle of workstation prowess, we’re going to be stuck with focusing on scan and patch at a time when network scans are becoming about as relevant as firewalls.  They both have their place, to be sure. But are we putting too much emphasis on scans?
 Your comments would be very welcome.
shrdlu May 15
Alex, scanning has become so popular because it’s the only automated, standardized (for some value of “standard”) method of measuring security vulnerability. More people can run Nessus than can do this stuff by hand (as Ranum says, “Lots of pen testers use Nessus because it does a better job than they can.”). If you manage to automate the assessment of your other seven factors, then you’ll probably see them emphasized more.
Alex May 15
shrdlu,
Right. But then is there a case where we’ve got “scanner as a hammer, and everything’s a nail” syndrome?
Dutcher Stiles May 15
I agree scanning has become a vuln assessment lowest common denominator, just as the honeypot collection information has become a threat assessment LCD. Match the results and you can filter out a lot of noise, and get to the meatier risks, i.e., skilled/targeted attackers vs. unknown vulnerabilities in your own environment with all its quirks.
dre May 16
I’m going to have to take the Richard Bejtlich highroad and say that threat-centric beats vulnerability-centric.
Concentrate on removing threats (i.e. adversaries), while also going after as many vulnerabilities from both sides of the scanning coin as possible (both dynamic/static analysis).
Remove threats in any manner possible, but this usually means knowing your enemy, finding your enemy, and (in the case of Internet crime) - putting your enemy behind bars. So the importance of IDS and Honeypots/honeytokens increases when you have the talent capable of producing a result where threats can be removed in a timely and consistent manner.
Ranum is just trying to come up with new methods and ways of getting our community to fight these battles that we are obviously losing. Scanning does not work; it won’t find all the security-related bugs/flaws (and it doesn’t remove threats directly). Threat modeling won’t work by itself either (although the vulnerabilities threat-modeling finds may end up costing a large organization a lot less money than finding a security bug/flaw of this nature after the software is already in production).
Another problem is that many security professionals still concentrate on the systems+networks and fail to even consider web applications. In 2007, reading the book, “XSS Attacks” should be a number one priority for any serious security professional. Anyone reading Cisco or StillSecure product literature as their primary resource for innovation in the security industry has got to go back to school and stay out of the business world for at least a few years.
A possibly bigger issue is less of a social problem and more of an economic one. There just aren’t enough people, particularly talented people, to keep up with the amount of work. Companies do not pay for security education/training because they can’t afford it (their budget is maxed out paying for AV/IPS/FW), and the salary/pay ranges are off by a factor of six to ten.
In order to go from the “sucking chest-wound” category to a more acceptable “rear-guard action” - the security community will need to choose a leader with a solid plan and godlike vision. I nominate Dan Geer.
Andy Steingruebl May 16
I was having a similar discussion in the area of application security the other day.
What we’re looking for as part of our risk assessment methodologies are numbers, facts, probabilities, etc.
Within application security or security threat analysis one of the only ways we an get this certainty is with components where we fully understand the threat model, and where we can make direct calculations of work-factor to break it, break in, etc. Cryptographic systems are those that lend themselves to this type of analysis. In general, apart from class breaks, we can strictly analyze the work required to break a crypto-system and then rely on that as a fixed point to analyze around.
There are precious few other places where we can do analysis in a purely quantitative way. Scanners are one of those places where we can definitively check for a vulnerability, assign it a strict score according to numerous factors, and then do quantitative risk analysis.
Without statistical data on the effectiveness of certain defensive techniques we’re all struggling for a quantitative way to do our jobs, and scanning fits at least part of the bill.
Alex May 16
dre -
Interesting that there is a perceived dichotomy between “threat” and “vulnerability” as the center of the universe. Let me suggest that those are two sides of the “what causes a loss event” equation.
I think you are very correct when you suggest that the concept of “network security” needs, um, updating. BITS AUP is egregious in this area, but that won’t stop dozens if not hundreds of F.I.’s from making a determination of vendor risk that ignores web/app security.
————————————-
Andy,
Oh, you’ve touched my special buttons! You said words like “facts” and “statistical data”. At some point we should sit down and talk…
I’ll buy if you indulge the amateur Bayesian in me.