Cherry Bekaert's journey to more efficient audits

An in depth look at how a top accounting firm generates significant return on investment through reliance on data driven techniques.

Who is MindBridge

MindBridge allows you to build and execute a data-driven risk assessment strategy for every engagement. That means you can use our AI auditing software to complement your existing procedures through every phase of an audit.

From planning and fieldwork through to audit completion, you’ll be able to leverage MindBridge to analyze 100% of transactions, and immediately identify errors, potential fraud, and noncompliance issues with a focus on results to uncover valuable insights for your client. You’ll also be able to create comprehensive audit plans and reports with visual graphs and annotations to take discussions with your clients further.

MindBridge commissioned a first-of-a-kind algorithm audit from University College London Consulting (UCLC) who are renowned experts in algorithm audit and safety. The algorithm audit provides independent assurance that MindBridge’s algorithms operate as expected. This audit demonstrates MindBridge’s commitment to transparency in building explainable, credible artificial intelligence.

The MindBridge team knows what it takes to deliver this type of change within your organization. We have developed best practices and have helped a wide range of firms adopt and see value from a data-driven auditing process. We know what it takes from identifying the right resources, planning the project implementation, to managing the communication and training requirements. MindBridge is here to lead, support, and provide guidance every step of the way.

For innovation and methodology leaders

While many have been hesitant to rely on artificial intelligence (AI) and audit data analytics in place of traditional procedures, firms like Cherry Bekaert are confidently innovating and reaping the benefits. Adopting MindBridge firm-wide in 2020, this top accounting firm has focused their analytics strategy on specific innovative technologies having the most impact on their day-to-day audit activities.

Transaction risk over time

Their first achievement: the use of data analytics to reduce sample sizes and increase audit efficiency.

This methodology case study walks through Cherry Bekaert's audit data analytics implementation process and provides a detailed explanation of how they combined revisions to their audit risk model with the MindBridge platform to achieve game-changing reductions in sample sizes.

ROI

For moderate risk of material misstatement in an illustrative client, Cherry Bekaert demonstrated a reduction in sample size from 384 to 252. At $200 per hour and 10 minutes per sample, that's a $4.4k saving on sampling, balanced against the cost of loading data and performing procedures in MindBridge.

The theory of improving audit efficiency

1. Quantified risk of material misstatement (RMM)
2. Reduced RMM by using ADA to identify higher risk items
3. Defined and quantified procedures that made up other components of detection risk
4. Eliminated arbitrary minimums and determined new sample sizes based on the audit risk model

Risk-based sampling in today’s audit

Cherry Bekaert’s AI-enabled approach helps the auditor focus on the higher-risk transactions and tends to reduce the sample sizes necessary to achieve reasonable assurance.

Jessica Everage Helms, CPA, Senior Manager in the Audit Professional Practices Group at Cherry Bekaert explains, “What auditors do now is very manual in terms of understanding the client’s financial state and identifying areas of risk. It takes time to do things manually which means there’s never enough time to evaluate all the different factors that could help determine the highest-risk financial entries. MindBridge uses AI to automate and pinpoint what to look for, turning this random process into something targeted and efficient.”

This approach is based on the idea that risk-based sampling offers a clearer identification of significant items and reduces the residual risk in the remaining population. Such an approach often allows engagement teams to spend less effort in key sections of the audit, significantly reducing the manual, repetitive testing that auditors typically perform to gain assurance on lower risk transactions.

Audit data analytics

Audit data analytics can enhance the auditor’s ability to efficiently and effectively analyze larger volumes of data, and in more depth, than when using manual audit techniques alone. Platforms like MindBridge take this one step further to score and report risk for every financial transaction, based on a set of business rules, statistical methods, and machine learning criteria. These scores are then aggregated for key account balances or business processes.

A focus on the weakest elements of the accounts has also been present in sampling since the 1940s, with auditors often performing 'random testing with special emphasis on vulnerable aspects'. It has long been known that by focusing the sample on where the risk is likely to lie, auditors can most quickly gain assurance that the accounts are free from material misstatement.

Leveraging risk scoring to drive sample selection represents a combination of two long-standing trends in sampling within audit. The introduction of Monetary Unit Sampling was done in part to drive efficiency in the audit process. One of the key advantages of Monetary Unit Sampling is the idea that the auditor can select a smaller number of transactions to gain greater coverage of the 'monetary units' (i.e., the total number of dollars) in the population.

Cherry Bekaert's use of risk-based sampling combines themes from both of these traditional schools of thought. By using risk scoring techniques to stratify the population, the auditor can select transactions which are both high value and contain a high likelihood of material misstatement. Such risk scoring also allows for automated and robust definition of the lower risk transaction sets.

These techniques are also related to the audit risk model itself, and the judgements that the auditors make when selecting their sample size. With the concepts of risk scoring introduced in the AICPA's Audit Evidence Standard (SAS142) and spectrum of risk introduced in risk assessment standards (ISA 315 Revised 2019, SAS145), it is clear the audit industry is continuing its move towards risk based auditing.

This combination of audit data analytics at both the risk assessment and response stage allows audit firms to maximize the efficiency of their auditors, and minimize the potential for over-auditing.

As technologies allowing for interrogation of 100% of the transactions within a population become widespread in use, stretching beyond journal testing, they will clearly have an impact on the cost of audit (less human checking) and on the depth of testing that will be possible.

Some audit firms believe that the use of technology-based tools, in certain instances, provides more persuasive evidence than traditional audit techniques.

Assurance based on selecting samples and testing documents has been the mantra of auditors for decades. Realizing the efficiency gains from the shift to risk-based samples and audit data analytics requires firms to look at their audit methodology, and in particular how they sample and how sampling interacts with the audit risk model.

MindBridge helps lay the foundation for audit data analytics

“One of the challenges with existing auditing standards is there are no clear guidelines around how much we can use audit data analytics or data usage in general,” said Helms. “We first piloted MindBridge to understand how the tool works and determine the best way to apply it to our audits, including a full quality control (QC) review and documentation against the standards to ensure compliance for our specific use cases. In particular, SAS 142 and SAS 145 really opened up what we can do in terms of using analytics within a standards-compliant framework.”

As news of the MindBridge project spread throughout the firm's 25 offices, there was excitement around what the platform could do, mixed with hesitation. As with any major technology shift, Cherry Bekaert adopted a change management strategy that educated and supported everyone, starting with practice development at the national level.

"We had to develop a firm-wide policy for everything we use MindBridge on," said Michael Hoose, CPA, Director at Cherry Bekaert and member of the firm's Audit Professional Practices group. "We defined policies for journal entry testing, a policy for identifying high-risk transactions for revenue testing, and another one for reconciliation of cash received to revenue. We also reperformed every MindBridge control point using test data to ensure it complied with our Firm's quality standards. Using the product's documentation we gained an understanding of the algorithmic settings and how high risk transactions are selected. Overall, we gained a high level of confidence that the risk scoring worked for our Firm's purposes and the knowledge to back it up.

MindBridge is committed to building the confidence and documentation that firms need to place reliance on our techniques. With the upcoming requirements that firms assess vendors present in ISQM1, documentation such as the independent assessment of the MindBridge algorithms completed by University College London Consulting becomes critical to adoption.

At a high level, the QC process within Cherry Bekaert included:

• Explaining MindBridge to national office personnel
• Developing criteria for when use of MindBridge is and is not acceptable
• Identifying senior and manager-level change "champions" who would lead
• Starting implementation efforts and performing quality review of MindBridge outputs

The theory: Six key steps to reduce sample sizes by using risk scoring and analytics

Minimizing sample sizes to maximize audit efficiency requires a detailed understanding of the components of the audit risk model, and how they interact with sample size calculations. There are six key steps in these calculations for firms and engagement teams to consider:

Your Guide Forward

1. Establish the desired Audit Risk
2. Quantify Risk of Material Misstatement by quantifying its components
3. Reduce Risk of Material Misstatement by using data analytics to identify higher risk items and reduce the "residual" inherent risk
4. Test controls, if appropriate
5. Define and quantify procedures that make up the other components of Detection Risk
6. Eliminate arbitrary minimums and determine sample sizes using the highest allowable risk of incorrect acceptance based on the audit risk model

Michael Hoose, CPA Director, Audit Professional Practices group.

Michael is a Director in Cherry Bekaert's National Office working on complex accounting matters. He is a licensed Certified Public Accountant with over fifteen years of experience serving both private and publicly-traded companies.

Michael has also taught numerous internal and external courses including for the North Carolina Association of CPAs (NCACPA) covering a wide range of topics from ASC 606 Revenue from Contracts with Customers to Share-based Compensation, Foreign Currency, Analytical Procedures and Accounting Standards Updates.

Michael serves as a technical resource for Cherry Bekaert's Accounting & Auditing ("A&A") Professional Practices group and is actively involved in quality review and the development of the Firm's A&A technical resources and internal training.

Using a hypothetical engagement, we can demonstrate the impact on sample sizes:

Gross revenue:

• 100M

Tolerable misstatement (TM):

• 500K

Potential sample sizes

The audit risk model

It is worth laying out the audit risk model, and explaining how audit data analytics and risk scoring impact each of these components.

Where,

$$ \mathbf {A R} = \mathbf {I R} \times \mathbf {C R} \times \mathbf {A P} \times \mathbf {O S P} _{\mathrm {R i s k}} $$

Audit risk is the risk that the auditor expresses an inappropriate audit opinion when the financial statements are materially misstated. For practical reasons and because the objective of an audit is to obtain reasonable but not absolute assurance, acceptable audit risk is never zero. Acceptable audit risk is not prescribed by authoritative guidance; however, the consensus is that audit risk should be no higher than 10%.

The lower the risks are (IR × CR × AP × OSP), the higher the allowable risk of incorrect acceptance can be, thus the smaller the sample size.

1. Establish the desired Audit Risk (AR)

$$ \mathbf {A R} = \mathrm {I R} \times \mathrm {C R} \times \mathrm {A P} \times \mathrm {O S P} \mathrm {R i s k} \times \mathrm {S a m p l i n g A l l o w a b l e R i s k} $$

It is worth noting that acceptable audit risk does not have to be the same for all engagements. For example, a firm could have a policy that their baseline audit risk is 7% and decrease the audit risk for higher risk engagements to 5%.

Defining what constitutes higher risk engagements is a matter of professional judgment. One approach might be to require a certain subset of engagements (e.g., public entities) use a lower audit risk or require those engagements where firm policy requires an engagement quality review use a lower audit risk.

2. Quantify Risk of Material Misstatement (RMM) by quantifying its components

$$ A R = I R \times C R \times A P \times O S P R i s k \times S a m p l i n g A l l o w a b l e R i s k $$

3. Reduce the Risk of Material Misstatement by using data analytics to identify higher risk items and reduce the "residual" inherent risk

Traditionally, auditors have used a quantitative threshold for determining higher-risk items to exclude from the sample population and test individually, often called "Individually Significant Items" (ISI). This may include some qualitative criteria such as related party transactions.

Reducing "residual" Inherent Risk decreases the risk of material misstatement, which also reduces the level of assurance needed through Control Risk, Analytical Procedures, Other Substantive Procedures, and sampling. Importantly, the amount of effort required to reduce residual Inherent Risk is generally significantly less than the amount of effort required to achieve the same level of assurance.

Significant Account

Traditional Sampling Approach

Risk-based Sampling Approach

| Total Sample Size | 311 | | Remaining Population (Risk unknown) | | | Value | $97M | | Count | 2K | | Sample | 296 | | Value | $3M | | Count | 15 | | Sample | 15 |

For example, audit data analytics can more effectively and efficiently identify transactions displaying characteristics which are indicative of a risk of misstatement. This could include rules-based techniques and statistical methods such as Benford’s analysis, or identifying transactions that are unusually complex (e.g., transactions that flow into and out of the same account or have many lines).

MindBridge has a number of powerful machine learning indicators, such as Outlier Anomaly and Unusual Amounts.

After identifying the Individually Significant Items, if the population can be distilled into a group of homogeneous transactions with lower risk profiles, the residual inherent risk of the sample population can be assessed lower than the overall Inherent Risk of the account balance. For example, if the starting Inherent Risk is qualitatively assessed as “high”, then the residual Inherent Risk might be “moderate” after identifying the Individually Significant Items using data analytics.

Test controls, if appropriate

There are opportunities to drive further efficiency by using controls-based testing, as control sample sizes are not one-for-one relative to substantive sample sizes to achieve the same assurance.

5. Define and quantify procedures that make up the other components of Detection Risk (DR)

Most firms and third-party audit methodologies use qualitative descriptions of analytical and other substantive procedures risk (e.g., high, moderate, or low). Moreover, few define what constitutes a high, moderate, or low quality analytic or other substantive procedure.

This exercise presents an opportunity to improve what constitutes a high, moderate, and low-quality analytic to avoid the inconsistent application of qualitative descriptions that often result in widely varying sample sizes.

Conclusion

We’d like to thank Cherry Bekaert for working with us to talk about how they are driving their sample sizes using MindBridge. By engaging with experts and taking the right expertise and approach, Cherry Bekaert has been able to forge a new way of thinking about sampling and see benefits because of it. We look forward to seeing how Cherry Bekaert’s use of MindBridge will continue to evolve in the future.

The use of risk scoring to focus the sample where it matters is just one example of benefits that firms can realize from ambitious applications of AI. With standards like ISA 315, SAS 142, and SAS 145 enabling auditors to rely on audit data analytics for evidence, there are a range of other potential avenues for application, including the use of AI as both a risk assessment and response procedure.

Through a strategic and disciplined approach to deploying MindBridge, audit firms are seeing benefits from sampling less, discovering more of their clients. They’re able to deliver a better quality audit at less cost because of it.

As MindBridge continues to apply its risk scoring to a wider variety of data-sets, we are excited by the various use-cases where firms can apply AI, and the increasing degree they can rely on data-driven assurance in place of documents and inquiry.

In the end, these material reductions in sample sizes are achieved through understanding of the audit risk model, quantifying any qualitative terms, and the use of audit data analytics to automate and focus risk identification.

“Through a comprehensive and explainable coverage of risk, MindBridge has given us the tools to identify risky transactions and reduce sample sizes," explains Helms. “We use it on clients of any size, any budget, and our experience proves that data analytics is a viable option towards data-driven assurance and risk-based sampling that firms can start using right now."