Big data, big impact on accounting


Zabihollah Rezaee and Jim Wang look at how predictive analytics is changing the landscape for businesses

Big data and data analytics have been used increasingly in businesses in the past decade. The market for big data technology and services is forecast to grow at a compound annual growth rate of 26.4 percent between 2014 and 2018 to US$41.5 billion, about six times the growth rate of the overall information technology market, according to International Data Corporation. However, big data application in accounting is at an early stage, and the emergence of big data and data analytics creates an opportunity for accountants and auditors to sharpen their skills.

Research and advisory company Gartner defines big data as “High-volume, high-velocity and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision-making and process automation.”

About 2.5 exabytes (an exabyte is 1 billion gigabytes) of data are created each day and the amount doubles every 40 months. For example, Walmart collects more than 2.5 petabytes (a petabyte is 1 million gigabytes) of data every hour from its customer transactions.

The business community and the accounting profession should take advantage of the opportunity provided by big data and data analytics in improving the effectiveness, efficiency, and efficacy of their work. According to EY’s Global Forensic Data Analytics Survey 2016, 79 percent of surveyed companies “use more than 10 million records, which are typically outside the domain of spreadsheets and require more sophisticated tools for analysis.”

The velocity of the big data is evident by the fact that 100 hours of video is uploaded to YouTube every minute, and more than 58 million tweets are sent on average each day. Currently, more than 98 percent of all stored information is electronic, compared with about 25 percent of all stored information being digital in 2000. The data includes both structured (e.g. journal entries) and unstructured data, such as e-mail, voice mail, images, GPS signals, and social media. Big data has the potential to change the landscape of businesses, particularly in the area of customer engagement, automating operation processes, and predictive analytics for decision making. As an example, in Big Data: The Management Revolution the Harvard Business Review reported the case of a major airline uses big data and predictive analytics to cut operating costs.

In the airline industry, flight delays and landing earlier than expected drive up operating costs. As part of the industry’s long-standing practice, pilots provide the estimated time of arrivals (ETAs) during the final approach to the airport. These ETAs are far from ideal because pilots would be paying attention to other things in preparation for landing while making these estimates. PASSUR Aerospace, a provider of decision-support technologies for the aviation industry, found a solution to this issue. In 2001, it began offering its own arrival estimates called RightETA. PASSUR improved estimates by using a variety of sources of data, including publicly available data about weather, flight schedules, and proprietary data of every plane in the local sky. This data was collected from a network of a passive radar station that it had installed near the airports. By 2012, PASSUR had more than 155 radar stations. Each installation collects a wide range of information about every plane in the local sky every 4.6 seconds, yielding a huge and constant flood of digital data. Together, with all the data gathered over more than a decade, it has a huge amount of multidimensional time-series information. The Harvard Business Review article summarized PASSUR as the system asking itself: “What happened in all the previous times a plane approached this airport under these conditions? When did it actually land?” and uses sophisticated analysis and pattern matching to calculate RightETA.

Big data and data analytics will affect accounting in many ways by influencing how business is conducted and how financial statements are prepared and audited.


Big data and financial accounting

There are two noticeable trends of big data in financial accounting. Firstly, different data sources are being integrated into accounting information systems. For example, text, video, and audio data are gradually being linked with traditional data. Accountants need to enhance their data analytic skills to deal with large volumes of available data, including automatically mined data (such as customer purchases, URL click through tracking, and content engagement data). Secondly, one area where big data may affect substantial change is fair value accounting. Data service companies specializing in collecting and evaluating designated data from various sources could emerge, such that big data pertaining to the fair value of assets and liabilities can mitigate subjective assumptions in fair value estimates. For example, Thomson Reuters Valuation Navigator currently provides market data for valuation by collecting scattered financial data into a single, searchable repository, which can automate daily pricing and valuation workflows.

Big data and managerial accounting

Big data poses a challenge for managerial accountants. A 2013 Chartered Global Management Accountant report titled From Insight to Impact Unlocking Opportunities in Big Data, revealed three relevant findings. Firstly, 86 percent of surveyed professionals agree that “their businesses are struggling to get valuable insight from data.” Secondly, professional accountants in businesses must change from decision-making supporters to business partners, to create value for businesses and cultivate an evidence-based decision-making culture rather than one based on management opinions. Finally, management accountants must know more about cyber and information security because of increasing concerns that “commercially sensitive data in the cloud is vulnerable to cyber-attack.”

Big data has the potential to improve performance management systems as well. For example, finance and accounting teams in a manufacturing firm can acquire benchmark metrics from the financial automation services provider and compare whether or not the firm’s performance is below the mean. For instance, to see whether the volume of account reconciliations it rejects is consistent with the experience of other manufacturers. Companies can monitor employee telephone calls, emails, and in-office behaviours such as web use and clickstream. By adopting big data analytics techniques, traditional management can be transformed through implementing comprehensive monitoring and control systems. For example, the use of big data could help to identify new motivational measures and relationships between good management performance and variables not previously considered. Firms may measure employee morale by tone of emails and phone conversations made on company equipment, productivity by the number of emails sent by managers, and customer satisfaction by video-captured body language of customers.


Big data and auditing

Auditors are now facing the challenge of huge amounts of both structured (e.g. general ledger or transaction data) and unstructured data (e.g. email, voice or free-text fields in a database, Wi-Fi sensors, electronics tags, etc.), together with an increasing amount of nontraditional data sources such as third-party watch lists, news media, free-text payment descriptions, email communications, and social media. Auditors use big data because they must be able to follow how their clients manage their own big data. As a consequence of new data analytic tools that have become available, auditors can use big data to reduce the costs of their audits and enhance profitability. For example, database-to-database verification with independent trading partners can be implemented, and automatic audit confirmation has become a viable replacement for manual confirmation. provides an example of automated audit confirmation. The company provides secure audit confirmation services for over 14,000 accounting firms, 100,000 auditors, and 700,000 organizations. Confirmation.describes its service as providing “an all-in-one solution that help to minimize fraud and elevate efficiency for the entire audit confirmation process.” With big data, auditors can analyse both structured and unstructured data to identify potential transactional anomalies (e.g. unauthorized disbursements), patterns of behaviour (e.g. split payments to bypass transaction limit), and trends (e.g. increased fraudulent transactions before a big holiday). As a consequence of using automatic data collection and rule-based analysis techniques to identify errors, auditors may shift responsibilities from detecting errors in data to judging which errors are worthy of further investigation.

In 2015, the American Institute of CPAs (AICPA), CPA Canada and Rutgers Business School partnered to create the Rutgers AICPA Data Analytics Research Initiative, which hopes to help integrate data analytics into the audit process to enhance audit quality. The following four examples explain how the initiative will affect auditing:

  • Population-level tests would be feasible over traditional sampling because of the digitization of transaction data and reduced costs of data analysis.
  • The role of auditors with the emergence of big data will move from statement-level assurance to data-level assurance.
  • Auditors will need to use text analytics to manage unstructured data, such as text in the management discussion and analysis sections of financial reports.
  • Auditors will face a less challenging task in asserting the existence of fixed assets if each asset’s records are complemented with pertinent audio, video, and textual information.

AICPA also set up Audit Data Standard working group to develop a standardized data model that management, internal auditors, and external auditors could utilize for enhanced analytics which would further improve the timeliness and effectiveness of the audit process. The first issuance of the Audit Data Standards include base standard, general ledger standard and accounts receivable sub-ledger standard. The United States regulator Securities and Exchange Commission (SEC) has used big data in the fraud audits since July 2013 when SEC released an analytical Accounting Quality Model, nickname “RoboCop” to detect securities law violations, issuer reporting and disclosure, and audit failures by taking advantage of around one billion records a day from each of the 13 national equity exchanges, time-stamped to the microsecond.


Big data and accounting standards

Big data has the potential to change accounting standards significantly. Many argue that current accounting standards are artifacts of an era subjected to high transmission costs and slow data collection speeds; however, such working conditions have become obsolete. To be relevant, accounting standards must focus on data rather than presentation. The U.S. academic John Peter Krahel and William R. Titera, a retired EY partner, proposed in 2015 that “accounting standards will have to deal with the content of the databases and allowable sets of extractions but not with the particular rules of account disclosure.” Because accounting standards in the age of big data will give users more responsibility for demanding available data, it is important that future accounting standards balance the need for disclosure with the need for the protection of sensitive data. The IT company Information Age predicts that standards-based information/data will be sold and traded on open exchanges by 2024. Accountants need to embrace the concept of recognizing the value of information/data assets. As early as 1990s, Doug Laney at Garnter coined “infonomics” and described information economics and principles of information as an asset that needed to be managed, valued and accounted for in the books of accounts. This is becoming reality.

With big data changing the skills required in the profession accountants should consider the ways they can enhance their knowledge and skills to prepare for the big data challenge through CPD courses and exploring online resources. 


–  Zabihollah Rezaee is Chair Professor of Accountancy at University of Memphis and Jim Wang is an assistant professor in accounting at the School of Business of Tung Wah College