Examining Systems Output Controls

Technology has become ubiquitous in the processing of millions of business transactions. Computer programs and analytical software have contributed to major increases in productivity, consistency and accuracy of today’s work product. Output controls ensure that computer programs process these transactions accurately and produce the results we expect to see. But what happens when things go wrong or undetected for long periods of time? We place tremendous faith in the accurate operation of these varied output controls. So what happens when system controls themselves are inadvertently changed without the proper authorization, testing, or verification?

System output controls are primarily automated controls but may include manual controls as well. Examples include controls that ensure holiday calendars accurately reflect current or future dates, proxy tabulations are cross-referenced with share counts and customer accounts, data is transmitted without change, and customer confirmations and account statements are complete. Manual controls may include macro uploads from excel spreadsheets into system databases for further processing. Or a manual step required to add a metadata tag to a file to ensure the correct information is posted to a website.

Why should I worry about output controls?

The following are a few examples that should illuminate the importance of focusing on output controls.

In March 2008, the Tokyo Stock Exchange stock trading system failed.

“The fundamental cause was the fact that the TSE did not adequately verify that the setting values that had been appropriate until this point were still currently effective and will continue to be effective in the future. The settings were not appropriate given the environmental change brought about by factors such as the enhanced processing capacity of both the Stock Trading System and trading participants’ systems, as well as participants’ diversified order placement methods.”

In the Washington post, February 7, 2004, a headline read “Shuttle Again Fails to Deploy Satellite as the Guidance System Malfunctions”

A headline from a Associated Press report from August 7, 2009, read “Airspeed Systems Failed on US Jets” . On at least a dozen recent flights by U.S. jetliners, malfunctioning equipment made it impossible for pilots to know how fast they were flying, federal investigators have discovered.

Remember the Y2K scare? Businesses spent millions of dollars to ensure that a simple change of the calendar from 1999 to 2000 would not shut down computer systems the world over! The lessons of Y2K are informative when thinking about the costs and complexity of system output controls. Namely, the expense of Y2K remediation could have been avoided had businesses incorporated a regular output controls assessment protocol .

Was Y2K a waste of money, or a prudent investment to ensure the smooth transition the world experienced? The answer to that question is beyond the analysis of this article. However, to prevent your own mini Y2K scare, now and in the future, it is critical to incorporate an annual periodic review of output controls into your toolkit of internal controls over financial reporting and operations.

Output Controls: Opportunities and challenges

The good news is that performing a review of output controls is not an expensive or labor intensive exercise. One should be able to tackle a few critical output controls each year until you have covered the key controls. The reviews can be conducted in weeks, not months and the intelligence gleamed from these exercises will help to improve your overall control environment.

To start, define the scope of the output controls review. One simple way to decide is to start with the most critical “customer output controls”. (Customer confirmations, monthly financial statements, prospectus delivery, web site controls, etc.) More complex output controls may include system application interfaces with spreadsheets and other upstream applications or change controls for financial databases.

These more complex output controls should be segregated into critical categories such as financial output controls, operations output controls, administrative output controls, or maintenance output controls where one system relies on the accuracy of another system to process data or complete transactional processing. Segregating these controls allows you to prioritize and easily manage the cycle of your review of these controls.

If you conduct annual SysTrust or SAS 70 audits, some of these controls will be tested as part of the independent exams performed by external auditors. External auditors increasingly rely on systems controls for assurance that financial controls are operating effectively. However, one should not rely solely on these examinations to assure themselves that all of their critical output controls are covered.

Once you have identified the scope of your review, assemble a small team of subject matter experts to begin the assessment process. Start with a simple assessment map of the controls: input, activity, output. Your SMEs should be able to determine who is accountable for the control, how the control is suppose to work, how to validate or test its accuracy, and to provide policies and procedures that ensure the controls are operating as designed.

Developing a visual map is critical to ensure all are in agreement that the key controls are presented in entirety and no gaps exist. Visual maps also help illuminate inter-dependencies across departments or other, more formal business line accountabilities. (Think of how many times one department believed another ensured the accuracy of data, only to find otherwise when a serious failure occurred.) Documenting in narrative form the controls and handoffs serves to clarify the controls operation and strengthen gaps that may exist.

The level of detail that one chooses to use is critical here. I would suggest that the reviews and details remain high level enough to ensure that an accurate representation of the controls map show the key controls and accountabilities. The goal of the exercise is to gain assurance that the output controls are accurate and operating effectively. “How do you know” questions typically lead to the desired response. The level of detail is subjective to the SMEs performing the assessment and the level of assurance one obtains from the documentation, reports, or tests performed on the controls.

Don’t repeat your own Y2K

A mini Y2K may exist at many firms in the form of service recoveries, customer rebates for inaccurate calculations, or financial statement restatements. How can these errors be avoided? By implementing a regular health check on your systems output controls you can reduce or eliminate a significant number of these problems.

In today’s fast paced environment where staff changes due to downsizing, business challenges and other demands adversely impact technology we must ensure that changes to our systems are truly understood end to end. If we fail to remember to ensure that one simple change in one application can have serious implications downstream in other applications we begin to build our own Y2K event down the road. There are many examples of a systems change in Year 1 not having an impact until Year 3 due to a change in product features not contemplated when the change occurred. If one is not re-validating assumptions about the operations of output controls IT professionals must be skeptical that business as usual is good enough!

**********

James Bone founded Global Compliance Associates, LLC after more than 20 years in financial services to provide advanced compliance and operational risk approaches to firms seeking to enhance or build a compliance framework beyond the current “COSO” risk model.

James has successfully developed analytical approaches that quantify operational risk, audit, and compliance events to proactively model key controls and processes. Global Compliance Associates, LLC also works with third-party vendors to provide compliance, audit, and risk talent to address targeted or long-term internal control projects.