Internal audit must know how to respond when business process owners want to go faster and document less (such as in Agile environments). Nielsen’s Kevin Alvero and Wade Cassels discuss what IA can do to meet these seemingly contradictory goals.
In the five months between the crashes of the Boeing 737 Max 8 airplanes in Indonesia and Ethiopia that resulted in the deaths of 189 people and 157 people respectively, Boeing received multiple complaints from pilots about the Max 8’s autopilot system, according to an NBC News report.[1]
Several of those complaints mentioned insufficient documentation, with one even referring to the aircraft’s manual as “criminally insufficient.” A CBC report suggested that details about the Max 8’s MCAS computer system, which was the focus of the investigation into both crashes, was at one time included in the Max 8’s manual, but left out of the final draft.[2] Meanwhile, an investigation was also opened into the process by which the Federal Aviation Administration (FAA) had certified the Max’s flight control system.[3]
Regardless of their ultimate outcome, investigations into the circumstances surrounding the two crashes has heightened discussion about the importance of business process documentation. Documentation forms the basis for the quality, reliability, repeatability and legitimacy of all organizational activities, and in some industries – such as transportation and health care – it is a matter of life and death.
Business process documentation is critical to the internal audit process as well. It sets the foundation of the auditor’s understanding of the process and represents the expected state that the process is audited against. Yet increasingly, organizations are trying to become more agile and are challenging the quantity of documentation they are required to produce, particularly if they perceive that it is just extra work that is slowing them down and that it is primarily being produced as an exercise to satisfy auditors. To be clear, Agile methodology does not promote neglecting mandatory or critical documentation for the sake of speed. Rather, one of the tenets of the Agile Manifesto is “working software over comprehensive documentation.”
What, then, should be the response and approach of internal audit when the organization wants to go faster and document less, or differently? First, internal audit should be involved and supportive of efforts to alleviate the burden of excessive documentation. Second, internal audit should reinforce the benefits of documentation to the business and defuse the idea that satisfying internal audit is the primary goal. Third, internal auditors must know what kind of documentation matters and how to determine if it is existent and sufficient in an Agile environment.
If internal audit functions can execute on these three main points, they stand a good chance of striking the right balance between supporting the business in its pursuit to deliver faster and better and providing assurance that appropriate controls are in place.
1. Avoid Pushback
Some organizations have embraced Agile methodology wholeheartedly. However, for other organizations and individual business units, while they may not describe themselves as going Agile, per se, they are looking to somehow lessen the burden of documentation in order to move faster and make the best use of their resources. In either case, when organizations move to adopt a more agile product development process, it is critical that internal audit take the right attitude when it comes to the matter of documentation. Rather than asking, “how is the business going to produce the documentation we need to do our audit?,” internal audit must instead ask, “how are we going to perform our audit based on the documentation that is produced by the process?”
In other words, internal audit should not ask the business to produce documentation for the sole purpose of applying it to a tried-and-true audit process, as this ultimately serves the needs of internal audit, but not of the business. If they do, they risk making themselves irrelevant and not valuable to the organization.[4] (Denning, Forbes) Instead, PwC notes, internal auditors “need to disrupt themselves.”[5]
Internal audit should seek to understand the rationale behind the move to a more agile process and how the business intends to satisfy its requirements as it relates to documentation. That last part is particularly important, because agility and recklessness are not the same. According to Deloitte, two common myths auditors have about Agile development processes is that Agile teams can do whatever they want and Agile projects produce no documentation.[6]
But these are indeed myths, and it is important for internal audit to discern whether the Agile approach is being adopted responsibly and to support efforts to ensure that mandatory documentation is not omitted. This requires communication and involvement in the early stages of the transition, because control activities need to be changed in an Agile environment. Neither process owners nor internal audit should perceive Agile as a means of circumventing controls. Rather, controls must be adapted to account for the use of Agile techniques.
“As the use of Agile becomes pervasive,” says PwC, “all risk, compliance and assurance executives need to embrace how these highly effective methods can coexist with effective controls. With a sufficient understanding of the Agile environment and leading controls development practices, risk professionals can take the right steps to integrate controls that protect against risk and noncompliance without compromising much needed agility.”[7]
2. Reinforce the Value of Documentation
One of the best ways internal audit can ensure that the business produces enough documentation (of the right kind) is to reinforce the notion that it is in the business’s best interest do so. This is particularly important if internal audit perceives that one of the drivers of the effort to lessen the burden of documentation is that process owners don’t fully appreciate its value. Internal audit must be able to understand and communicate the importance of business process documentation beyond the fact that internal auditors need it to be able to do their jobs. Internal auditors should approach requests for documentation as a “help me help you” proposition and be able to articulate benefits such as:
- Process optimization. Documentation review can reveal opportunities to enhance processes, correct inefficiencies or errors and introduce automation.
- New employee training. Documentation helps ensure thorough and consistent training for new employees, to the benefit of the company and the new hires.
- Company knowledge sharing and retention. Documentation preserves specialized knowledge, reducing the risk of relying too heavily on a few individuals. It also helps the organization defend its intellectual property.
- Operational consistency. Documentation forms the basis for ensuring that business processes are performed consistently to management’s specifications.[8]
Documentation imparts value in all these ways and more, but as to what exactly constitutes documentation, auditors and business process owners alike should keep an open mind.
3. Have the Skills to Assess Documentation
According to the Project Management Institute (PMI), one of the key reasons Agile initiatives fail is because of a lack of Agile experience on development teams.[9] Similarly, internal audit teams assigned to Agile projects must have training and experience to recognize how to alter their approach.
When auditing Agile projects, Deloitte states, “IA teams may need to think differently — whether this means recognizing a different set of controls, changing where to look for evidence that controls exist, testing an ongoing control or helping the team gain even more operational efficiencies.”[10]
In order to audit effectively in an environment that is more agile and produces less documentation, auditors need to adapt their skills and understand how to access and interpret documentation in this new environment. As always, documentation must exist, and it must be complete, accurate and timely (i.e., up to date). However, auditors must understand how to assess these attributes in forms of documentation different from traditional reports. Also, they must become accustomed to accessing and assessing these forms of documentation in real time throughout the development process with less reliance on looking at documentation after the fact.
According to PwC, auditors conditioned to reconciling formally documented and approved project charters, business requirements, design documents and user acceptance test reports can instead gain assurance through:
- “Requirements,” typically in the form of Epics, Features and Stories, stored in a tool such as JiRA, Rally or Team Foundation Server (TFS).
- Traceability established to track each story through the tool chain to show approvals, testing results and ultimately its release into production.
- Evidence and approvals retained in the tools to allow for auditability. For example, authorized product owners may have to sign off on user stories using a checkbox approval at the end of a sprint (requires role-based access to be defined and for appropriate business representation to be integrated into the production life cycle).[11]
Conclusion
The importance of business process documentation has not diminished in spite of the rise of Agile methods. What is changing is how internal auditors can expect to access documentation and what it might look like. If internal audit and process owners are willing to engage each other in the early stages of Agile transformation, internal audit can be a valuable partner in ensuring that the business reaches its speed and efficiency goals without putting control in jeopardy.
[1] Blackman, J., Rosenblatt, K. “U.S. pilots complained about Boeing 737 Max 8 months before Ethiopia crash.” NBCNews.com. March 13, 2019.
[2] Gollum, M., Shprintsen, A., Zalac, F. “737 Max flight manual may have left MCAS information on ‘cutting room floor.’” CBC.ca. March 26, 2019.
[3] Gates, D. “Flawed analysis, failed oversight: How Boeing, FAA certified the suspect 737 MAX flight control system.” SeattleTimes.com. March 17, 2019.
[4] Denning, S. “Can Internal Auditing Become Agile? Seven Keys To Thinking The Unthinkable” Forbes.com. March 21, 2017.
[5] PwC. “Agile on the Rise – Integrating Effective Controls.” 2018.
[6] Deloitte. “Auditing Agile projects – Your grandfather’s audit won’t work here. 2018.
[7] PwC. “Agile on the Rise – Integrating Effective Controls.” 2018.
[8] Churazova, A. “The Easy Guide to Business Process Documentation.” Blog.nuclino.com. January 22, 2019.
[9] Miller, G. J. (2013). Agile problems, challenges, & failures. Paper presented at PMI® Global Congress 2013—North America, New Orleans, LA. Newtown Square, PA: Project Management Institute.
[10] Deloitte. “Auditing Agile projects – Your grandfather’s audit won’t work here. 2018.
[11] PwC. “Agile on the Rise – Integrating Effective Controls.” 2018.
Kevin M. Alvero, CISA, CFE, is Senior Vice President, Internal Audit, Compliance and Governance at 
Wade Cassels, CCSA, CISA, is a Senior Operational Auditor at 
