Originally published on caats.ca and shared here with permission.
By addressing people, processes, and technology, you can ensure a successful data analytics initiative that is built to last.
Long a staple of reviews, data analytics is no longer a nice-to-have, but a requirement. Evaluators and Internal auditors now have the ability to gain insights from, and test correlations with, a vast array of information on the Internet, which can be as diverse as competitor information, regulatory filings, and conversations on social media. Data analytics provide reviewers with the potential to deliver oversight, insight, and foresight.
Analytics can examine the audit entity from a data-driven perspective (what does the data reveal about the activity?), drive understanding of the risks (what is happening?), and generate insight (why is it happening?). It also provides reviewers with the ability to perform prescriptive analytics to develop recommendations to address issues, as well as predictive tools to look at what will happen and help to prepare for it.
The key to ensuring that data analytics has the best chance of success lies in managing the people, processes, and technology aspects of the initiative. The three are integral to any effort to develop data analytics and must be considered separately and as a whole.
People
When it comes to people, there are several questions to address. Should each review team be responsible for developing its own analytics capabilities or should there be a data analytics function that supports the review teams? Can the department afford to have one or more people dedicated to data analytics, particularly if it’s a small review function? Review functions seeking to develop an analytics capability have a better chance of success if they create a separate analytics function, even if it is one person who has responsibility to support the teams in the analysis requirements. As review functions move along the data analytics maturity curve, the teams should take more responsibility for data analysis, and the analytics function will shift to providing complex analysis and verifying the integrity of the analysis performed by the review teams.
With this approach, the next question will address the level and experience of the person that should be part of the analytics function. A related question is: Should an evaluator or auditor be taught programming (data extraction and analysis) or should a programmer be taught to conduct reviews? Failures in this area have one thing in common — management did not assign the right person or people to the task. Too often, a junior programmer with limited or no audit experience — addressing only the IT aspects of the job — is assigned to develop the analytics function. Given the nature of the task — dealing with business process owners, system programmers, and review team leaders — the analytics function must be staffed at the appropriate level and with the necessary experience. The biggest hurdle is having the business process knowledge to identify the types of analytics to run.
The size of the analytics function will depend on the size of the review function overall, as well as the analyses to be performed and the types of technical expertise and experience that are available in the review organization. If responsibility is assigned to a single person, he or she must be, at a minimum, the equivalent of a team leader level and must have data extraction and analysis and evaluation or audit experience. This will mean hiring someone with the required skills if they do not exist in-house. As the use of data analytics increases, the analytics function can grow, adding junior levels, a career path, and mobility to the function.
The analytics function must be visible and knowledgeable of, and responsive to, the needs of the review teams. At the same time, it must also be proactive in recognizing opportunities for the application of data analysis and in marketing existing and new applications of technology.
Processes
Data analytics needs to be fully integrated into the review process. Ensuring that data analytics are embedded in the process will require support from all levels, starting with senior management. Management will have to reinforce the use of analytics, the data analytics function will have to market its services, team leaders will have to be challenged by management, and team members will have to employ analytics. Management should establish goals for the implementation and use of data analytics and these should be communicated to the entire audit team. It should be clear that data analytics will support the planning processes (examining the controls, risks, and business processes), the executive phase (testing controls, drilling down into the risks and assessing the effectiveness of the business process), and the reporting phase.
Initially, it will be important to highlight success stories and educate managers and team leaders about what is possible. Improving on the traditional review approach of sampling, evaluators and auditors can benefit from the implementation of data analytics to allow for more precise identification of control deficiencies, noncompliance with policies and procedures, and areas of high risk. These same analytics could then be used to ensure appropriate management follow-up has occurred by elevating the identified deficiencies or implementing continuous auditing procedures in areas of higher risk.
While analytics can produce significant benefits, the inappropriate introduction of technology can also have serious negative consequences. In many evaluation and audit organizations, credibility is a valued, but fragile, commodity. Evaluation and internal audit must continually demonstrate the value and utility of their work by producing high-quality, timely reviews in areas of high risk. The incorrect use of technology and data analysis could produce erroneous conclusions and damage the credibility of the review organization with its clients. It also could make any subsequent attempt to use analytics more difficult.
Technology
The most important questions surrounding technology are whether analysis software should be purchased and what the cost will be. To answer these questions, evaluation and internal audit need to understand what analytics are already in place before embarking on efforts to develop its own analysis routines. The existence of data warehouses and business intelligence (BI) tools should be investigated before deciding whether to invest in independent analytics. The organization may already be producing reports that can be adapted for evaluation and audit use. Evaluators and auditors should obtain read-only access to application systems, and the ability to run standard reports and access and use data warehouse and BI tools. If additional analytical capabilities are required, MS Excel and Access can be useful in some circumstances. As the analytics become an integral part of the audit process and more complex, the need for a more robust software package to support data analytics increases. In practice, the use of specialized audit analysis software has distinct advantages — particularly in terms of logging, repeatability of tests and efficient test design, working with large data sets, and dealing with complex data manipulation. The cost of audit software is usually significantly less than the investment in resources and skills required for a successful audit analytics program. Management needs to put this in perspective and be willing to invest in the initiative — on average, a single license for audit software costs approximately US$3,500.
Any analytics initiative must quickly demonstrate a return on investment (ROI). Therefore, management should start with a targeted, ad hoc analytics program that will yield immediate benefits in terms of acceptance, ROI, and the development of the analytics function. At the same time, it should be clear that the initial steps are not sufficientand a strategy will be developed to improve and deploy analytic capabilities across the organization. Senior management should ensure that there is a plan to take action and measure results accurately.
Sustainable Success
Organizations should expect that an individual with strong data analysis skills, armed with software and some training, will be able to drive a successful analytics program on his or her own. Sustainable success in the use of analytics also requires leadership, strategic and tactical goal setting, review process knowledge, team coordination, integration, and good project management. The skills required to remain effective in an increasingly technologically complex world must be developed, nurtured, and supported. In addition, to efficiently and effectively implement and use data analysis by all evaluators and auditors with a variety of computer skills, the organization needs to develop a standard, user-friendly, integrated environment; provide specialized training and IT support; and provide ongoing encouragement. The benefits are significant and well worth the effort.
Effective analytics requires an initial investment of time and a commitment to follow up on results. Early analytics may produce a large volume of results — including false positives — and will need to be honed and evaluated to ensure results are manageable, reliable, and can be followed up on. Because analytics take time to be fully effective, the expectations of senior management, inside and outside of the evaluation and internal audit functions, must be carefully managed.
The question should not be “Should we embark on develop analytic capabilities?”, but “How soon can we start?” Adequately addressing the people, process, and technology aspects of the initiative will increase the likelihood of success.
Responses