The trouble with likelihood

It’s common to see charts similar to the one below used to communicate risk.  On one axis we have Impact, and on the other we have Likelihood.  We’ll save a discussion regarding Impact for another post, but in this post I’d like to point out a couple of subtle but important limitations with the term “likelihood”.

Likelihood connotes the probability of an event occurring.  In fact, you may see explicit probability ranges assigned to each qualitative label (e.g., “Very High = 90% to 100% probable”).   And, while this seems to be on the right track, there are two problems with it:

• It often doesn’t include a timeframe reference.  In other words, does the likelihood statement refer to the probability of the event occurring this week, this year, in this lifetime?
• It doesn’t provide the means to differentiate between something that may happen once vs. something that may happen multiple times.  For example, a statement; “The likelihood of a virus infection is Very High” doesn’t differentiate whether the event is likely to happen once or many times.

These two limitations become critical when we’re trying to quantify and/or compare risk issues.

Using frequency, we can account for events that occur many times within the defined timeframe as well as those that occur fewer than once in the timeframe (e.g., .01 times per year, or once in one hundred years).  Of course, this raises the question of how we determine frequency, particularly for infrequent events.  In the interest of keeping this post to a reasonable length, I’ll cover that another time (soon).

Drawing lines

You may have seen charts like the ones below, with lines drawn to differentiate High from Medium, etc.

(NOTE:  Magnitude scales will vary based on the risk capacity/tolerance of the organization)

These can be useful, but a few challenges I’ve encountered with this approach include:

• If the risk point falls barely on one side of the line or the other, do the lines really serve a useful purpose, at least from the perspective of being able to assign a qualitative value?
• Who drew the lines?  At one place I’ve worked, I couldn’t get management to provide guidance on where to draw the lines so I took a stab at drawing them based on what I thought management’s risk tolerance was given their earlier decisions.  This seemed to work okay, as I didn’t experience much push-back from management, but you need to constantly look for evidence that the lines need to be changed.
• Particularly in larger companies with multiple affiliates or subsidiaries, line placement will vary because each part of the enterprise will have its own risk tolerance.  A “critical” loss at the subsidiary level might not equate to a rounding error at the enterprise level.  I’ve dealt with this by plotting results on two charts; one scaled to the enterprise risk tolerance, and another drawn to the subsidiary’s tolerance.

Of course, the fact that the point isn’t really a point at all, but the intersection of two ranges or distributions further affects the utility of lines.

I’ve found two ways of charting risk that seem to be well received by management (below).

(NOTE: These charts were created using Monte Carlo analyses within FAIR-based applications)

My preference is the scatter plot, which does a nice job of visualizing the uncertainty that is a part of any risk analysis.  A couple of things to note:

• No lines have been drawn to label the result “High”, “Medium”, etc.
• I haven’t used a green-to-red background on the charts.

I will use those illustrative tools if requested by management, but I tend not to use them otherwise.  Besides the challenges I noted above regarding lines, my rationale is that lines and colors tend to bias interpretation of the results.  In other words, if someone sees a risk point plotted in a red background or in the “High” section of the chart, they equate those results as “unacceptable”.  The fact is, the acceptability of a risk condition is often dependent on the value proposition of the situation, the cost to mitigate risk, etc.  I’ve found management is intelligent enough to know that the upper-right part of the chart means more risk than the lower-left.

Posted on May 20