The Limitations of Traditional Risk Assessments

3 eggs balanced on frayed ropeDeveloping risk maps, heat maps and risk rankings based on subjective assessments of the severity of impact of potential future events and their likelihood of occurrence is common practice. These approaches provide an overall picture of the risks, seem simple and understandable enough to most people, are often the result of a systematic process and provide a rough profile of the organization’s risks.

Typical attributes of a risk map include: governing objectives drawn from a business strategy or plan that provides a context for the assessment, a common risk language that provides a perspective for understanding risk and predetermined criteria for conducting an assessment. While everyone agrees that an effective risk assessment should never end with just a list of risks, it is not unusual for traditional risk assessments to hit a wall, leaving decision makers with a list and little insight as to what to do next. In addition, there is the common complaint that risk assessments rarely surface an “a-ha!” that alters senior management’s view of the world.

There are a number of reasons why these problems exist. We offer four.

First, the risk assessment process may allow individual biases to affect the assessment, foster “group think” and preempt out-of-the-box thinking. Scales derived from qualitative descriptions of severity and likelihood are often understood and used differently by different people. Assessments by unknowledgeable participants often are “middle of the road” on these scales, skewing the results. Furthermore, intersections on a risk map are mean averages of sometimes widely dispersed views and do not necessarily represent a true consensus of the participating evaluators.

Second, the process is a linear, point-in-time assessment that doesn’t address the unique characteristics of the company’s risks. While using a common analytical framework to evaluate risks with different characteristics and time horizon considerations may make the process easier to execute, it is not robust enough to add value continuously over time, may ignore the interplay among related risks and does not alleviate the fundamental problem of limited information. It also contributes to problems later on when the organization assigns different risks to the appropriate risk owners as the logical next step to solving the problem. Even if these assigned risk owners were involved in the process, they often do not know what to do with the results, leading to frustration over attempts to integrate risk management with core management processes.

Third, subjective assessments are often influenced by past experience. This is a dangerous shortcoming of the process, because one thing we have learned over the years is that the past is not always a reliable indicator of what to expect in the future. For example, the global financial crisis taught us an important lesson: What we don’t know can be more important than what we do know. The integrity of the risk assessment process can be impaired by the overconfidence that can be bred by an overly simplified view of the future. Overconfidence is often driven by the degree of success managers have experienced in the past and the quality and coherence of the story line managers construct regarding the future, rather than by business realities.

Fourth, the process offers little insight as to what to do about exposures to extreme events. The process often leads to a conclusion to de-emphasize the so-called “high impact, low-likelihood” risks because of the low probabilities involved and the false sense of security arising from the lack of historical precedence. These events are often the ones that cause the most damage if and when they occur unexpectedly, particularly if the organization lacks preparedness. We may not know what those events may be, but we can gauge their impact, such as the loss of a strategic supplier, as experienced by companies in the automotive and semi-conductor industries when the Japanese tsunami occurred.  Therefore, the process needs to consider such considerations as the velocity or speed to impact, the persistence of the impact over time and the organization’s response readiness even though the specific cause of the loss may not be known.

There is a place for traditional risk assessment approaches when creating awareness and obtaining a quick overview of risk, particularly when a company is just starting down the path of implementing enterprise risk management (ERM). However, as traditional approaches lose their utility as a source of fresh insight over time, the focus on ERM begins to fade away. Accordingly, more sophisticated assessment mechanisms may be necessary to provide the insights management needs.

In summary, an assessment process that subjects all risks to the same analytical grid has shortcomings that need to be recognized. If very little happens as a result of an organization’s risk assessment process, it is a clear sign that alternative approaches should be considered. We’ll explore these alternatives next month.

————————————————————————–

The following are some suggested questions that executive management and the board of directors may consider, consistent with the entity’s objectives:

  • Do we practice enterprise list management (i.e., do we maintain a list of risks year-to-year with little, if any, follow-up)?
  • Are we satisfied that we are evaluating changes in the business environment to identify the risks inherent in the corporate strategy? Is the board sufficiently involved in the process, particularly when such changes involve acquiring new businesses, entering new markets, introducing new products or altering key assumptions underlying the strategy?
  • Do we have solid agreement on the most critical risks facing the company (i.e., do we agree on why these risks are significant)? Do we understand the organization’s responses to these risks? Are we satisfied that our risk assessment results are sufficiently action-oriented, as evidenced by designated risk owners executing risk responses to reduce the key risks to an acceptable level?

Does management apprise the board in a timely manner of significant emerging risks or significant changes in the enterprise’s risk profile? Is there a process for considering response plans to emerging risks on a timely basis?

About the Author

Jim DeLoach

Jim DeLoach, Protiviti, ERM SpecialistAbout the Author
Jim DeLoach has more than 35 years of experience and is a member of the Protiviti Solutions Leadership Team. His market focus is on helping organizations succeed in responding to government mandates, shareholder demands and a changing business environment in a cost-effective and sustainable manner that reduces risk to an acceptable level. He also assists companies with integrating risk management with strategy setting and performance management. Jim also serves as a member of Protiviti’s Executive Council to the CEO.