In addition to grounding flights and disrupting hospital operations, CrowdStrike’s July outage brings something else into stark relief: Even the most well-regarded vendor can make a costly mistake. Baker Tilly’s Jeff Krull explores how companies can stay vigilant with regard to vendor risk.
From a risk perspective, organizations are right to be concerned about their technology vendor ecosystem. The CrowdStrike outage was a sobering reminder that despite best efforts in vendor selection and management, service disruptions will occur. There is no turning back; organizations will continue to rely on global IT suppliers and software vendors to keep up with the speed of change and the need for specialized skill sets and services to conduct their business.
Organizations should continue to focus on leading practices when vetting and selecting vendors but recognize that even the largest and most well-regarded IT suppliers and services are vulnerable to both intentional cyber attacks and unintentional errors and disruptions.
CrowdStrike’s July outage caused widespread chaos around the world with flights grounded, broadcasts interrupted and 911 lines disrupted. That a regular software update from a leading provider could cause such issues highlights the growing complexity of interconnected and third-party risks.
Before memory of the outage event fades (and until the next one occurs), all businesses should take the time to carefully evaluate their business continuity and recovery plans, particularly with mission-critical third-party vendors. This includes preparation for various outage scenarios including the most extreme, where the system no longer exists.
Organizations will prepare for and react differently to system outages depending on the business type and risk tolerance. Should an outage occur, organizations are advised to have downtime procedures in place and practice them during off-peak times to ensure that they work and that the company is able to get its systems back online.
Here are steps all organizations should be considering to manage vendor-related business interruption risk:
Identify the risks
Performing vendor risk assessments is essential to understanding and managing the risk profiles of your vendors and ensuring that the exposures match your organization’s risk tolerance level. A constantly evolving vendor ecosystem means ongoing due diligence into relevant risks before and after a vendor is selected, onboarded and offboarded, as well as understanding how the systems are interconnected.
Establish and test incident response plans
Conduct these tests with key vendors to ensure you are promptly informed and can properly mitigate any risk you may be exposed to by an outage affecting a third-party vendor. This includes understanding the details in your contracts and service-level agreements involving incidents.
Risk Lessons From CrowdStrike’s Blunder
Organizations continue to grapple with faulty update fallout
Read moreConduct training and awareness efforts
Leading organizations are consistently operating tabletop exercises or other real-time training techniques that often dovetail into vendor risk. Ongoing training and support from the top-down assists in maturing an organization and building a more risk-aware culture.
Practice downtime procedures
Organizations should document their downtime procedures and regularly practice them, including practicing how data will be input/recovered once systems are available. This includes flexible or dynamic procedures that are aligned to your organization’s peak or critical operating periods since you may respond differently (e.g., a retailer during the holidays versus a normal month).
Identify and address the need for redundancies to avoid a service disruption
Organizations with large and mission-critical third-party supplier networks that cannot afford interruptions may need to invest in redundancies that will enable them to avoid business interruption and achieve a rapid, full-scale recovery. This can be a costly way to protect against a system outage. Redundancy efforts could include multiple cloud providers and local data centers that provide “hot sites” that are ready to go in the event of a system outage or failure.
The CrowdStrike outage underscores a critical reality: Despite thorough vetting and selection processes, even the most esteemed technology vendors are not immune to disruptions. As organizations continue to depend on a global network of IT suppliers and software vendors to meet the demands of a rapidly evolving business environment, they must remain vigilant and proactive in their risk management strategies.
Ultimately, successful vendor relationships hinge on mutual understanding, collaboration and a clear comprehension of client needs and expectations. By fostering these relationships and continuously improving risk management practices, organizations can better navigate the complexities of their technology vendor ecosystem and safeguard their operations against future disruptions.
With more than 20 years of experience in process and controls, information technology and internal audit, Jeff Krull is principal and practice leader of Baker Tilly’s cybersecurity practice. His expertise includes cybersecurity, IT controls, system and organization controls (SOC) examinations internal auditing, business process controls and specialized compliance assessments and attestations. Examples of these engagements include cybersecurity, SOC examinations, Sarbanes-Oxley compliance, internal audit, pre- and post-implementation assessments, privacy and HIPAA risk assessments and specialized compliance attestations for clients.
