What an AI audit is and how to run one for your business

"AI audit" gets used for two different things. One is a review of an AI system for bias, safety, and compliance, the kind of assurance work covered by frameworks such as the NIST AI Risk Management Framework. The other is a review of a business to find out where AI is worth using at all. This guide is mostly about the second, because it is the question most owners have before they spend money. A later section covers the first, since it matters as soon as AI is talking to your customers.

An audit exists to answer a narrow question: which of our tasks should be handled by plain software, which by AI, and which should stay with people, and in what order? Get it wrong in one direction and you pay for tools nobody needed. Get it wrong in the other and you leave hours, and sometimes revenue, on the table.

Why audit before you buy anything

AI does not improve a business evenly. Research from several directions points the same way.

In a 2017 analysis of more than 2,000 work activities across 800 occupations, the McKinsey Global Institute estimated that 49 percent of the activities people are paid to do could be automated by adapting technology that already existed. Yet fewer than 5 percent of occupations could be fully automated, while about 60 percent had at least 30 percent of their activities that could be. The useful lesson is not the headline percentage. Automation attaches to tasks, not jobs, so an audit has to work at the level of tasks. That study predates today's generative AI tools and measures technical feasibility, not whether automating a task is worth the cost.

A study of 5,179 customer support agents at one company found that access to a generative AI assistant raised the number of issues resolved per hour by 14 percent on average. For novice and low-skilled workers the gain was 34 percent, and for experienced workers the effect was minimal. Who does the task changes how much AI helps.

In a randomized experiment with 453 college-educated professionals doing writing tasks in their own occupations, those given ChatGPT finished about 40 percent faster, and independent graders rated their output about 18 percent higher in quality.

The most useful result for an audit comes from an experiment with 758 consultants at Boston Consulting Group. On 18 tasks inside what the AI could do, consultants using GPT-4 completed 12.2 percent more tasks, finished 25.1 percent faster, and produced work rated more than 40 percent higher in quality. On a task the researchers deliberately chose to fall outside the AI's abilities, consultants using AI were 19 percentage points less likely to produce a correct solution than those working without it. The authors call the boundary between the two a "jagged frontier": tasks that look equally hard can sit on either side of it, and the AI sounds confident in both cases.

Taken together, the gains are real, they vary by task and by person, and on the wrong task AI can make results worse. Sorting your tasks before you find that out the hard way is what an audit is for. Each study covers one setting (a support team, writing assignments, consultants), so use the numbers as direction rather than as a forecast for your business.

What an AI audit covers

Step 1: build a task inventory, not a job list

Jobs are too coarse. "Office manager" contains dozens of tasks and only some suit AI. Ask each person to jot down their recurring tasks for two weeks as they happen, or sit with them for an hour per role. Aim for 20 to 40 tasks across the business, not hundreds. Ask what people dread doing, since those tasks tend to be repetitive and worth a look.

Task:
Who does it:
Times per week:
Minutes each time:
Tool or system used:
Input: always the same format / varies
What a mistake costs:
Notes:

Step 2: sort each task by the kind of work it is

Every task falls into one of three groups.

• Rules. The input has the same shape every time and the answer follows a rule: send a reminder 24 hours before an appointment, create an invoice when a job is marked complete, copy a customer's details from a form into the CRM. Use plain automation.
• Language and pattern. The input is free text, documents, images, or speech, and the answer needs interpretation: reading an inquiry and judging how urgent it is, drafting a reply, summarizing a long email thread, pulling figures out of supplier invoices that all look different. A draft is acceptable. These are candidates for AI with a person reviewing the output.
• Judgment and relationship. The answer depends on context, trust, or accountability: negotiating, handling a complaint, setting a price for unusual work. Keep these with people. AI can prepare material for them but should not decide.

Four questions sort most tasks. Could you write the rule on one page? Does the input arrive in different formats? Does a wrong answer cost money, safety, or a relationship? Would the customer want to know a person handled it? Rules-based tasks are cheaper, more predictable, and easier to test, so when a rule works, use the rule.

Step 3: score and rank

Score every task that is not clearly a human job on three things.

• Value: hours per week the task takes across the business.
• Feasibility, 1 to 5: 5 means the input is consistent and the data can be reached through standard integrations. 1 means the information is on paper or in someone's head.
• Risk, 1 to 5: 1 means a mistake is cheap and caught quickly. 5 means a mistake could harm a customer, break a law, or cost real money.

Then compute priority = value × feasibility ÷ risk. The formula is crude on purpose. It is a way to sort a long list, not a measurement.

An illustrative example for a made-up home-services company. The numbers are invented.

Reminders win by a wide margin. Complaints land at the bottom, which is where they belong. Hours alone undervalue tasks that affect when you get paid or whether a customer says yes, so quote follow-up and invoicing deserve extra weight beyond their hours. Add a notes column for that.

Does the arithmetic pay off?

Add up the realistic hours for the tasks you would automate. Here the three rules-based tasks come to 2.0 + 1.3 + 1.0 = 4.3 hours a week, or about 224 hours a year (4.3 × 52). At an illustrative loaded labour cost of $35 an hour, that is about $7,800 a year of time. It is time freed, not cash saved, unless the hours move to billable work or replace a hire you would otherwise make.

Now subtract running costs. Suppose software and monitoring cost $150 a month. The net monthly benefit is 4.3 × 4.33 weeks × $35 − $150 ≈ $502. For a hypothetical $4,000 build, payback is $4,000 ÷ $502, about eight months. That figure is made up for the arithmetic and is not a quote from us. Eight months is a respectable result, not a spectacular one, and it ignores any effect on faster payment or fewer no-shows, which is why a proper audit prices those separately.

Step 4: check data, tools, and privacy

For every task on the shortlist, ask where the information lives (accounting software, email, a spreadsheet, paper), whether software can reach it through an integration or export, and whether it is accurate enough that a system acting on it would behave sensibly. A system built on messy customer data produces output that looks confident and is wrong.

If a task involves customers' personal information, you are responsible for how it is handled. In Canada, the federal, provincial, and territorial privacy commissioners published nine principles for generative AI on December 7, 2023. They are: legal authority and consent, appropriate purposes, necessity and proportionality, openness, accountability, individual access, limiting collection, use, and disclosure, accuracy, and safeguards. The document is guidance on applying existing privacy law to generative AI, not new law, and obligations differ by sector and province. Quebec's Law 25, for example, includes a requirement to inform people when a decision about them is made exclusively through automated processing. This is a summary, not legal advice. For anything involving sensitive information, check with a privacy professional. The step-by-step checklist for using AI with customer data goes deeper on tiers, vendor questions, and a staff policy.

Questions to answer for each tool before it touches customer data:

• What personal information will the tool receive?
• Does the vendor use your inputs to train its models, and can you opt out?
• Where is the data stored, how long is it kept, and can you delete it?
• Who at the vendor can see it?
• Do customers need to be told that AI is involved?
• Can a person override an automated outcome?

Step 5: design the human checkpoints

The consultant study above is the reason for this step. A task that looks easy may sit outside what the AI does well, and you will not know until you check its output. For each task decide where a person looks at the result, and write it down.

• Review everything customer-facing at first, and anything involving money.
• Once the system has earned some trust, sample rather than review everything, for example one in ten.
• Log every output and every correction, so patterns in the errors are visible.
• Route cases the AI is unsure about to a named person.
• Make the system easy to switch off, and say who takes the task back when it is off.
• Tell staff which outputs are AI-generated, so reviewers know what they are checking.

Step 6: pilot one task

Pick one task from the top of the list, ideally low risk and easy to measure. Before you start, write three things down: the baseline (minutes per task, error rate, customer response), the success criteria (for example, time per task drops by a third without more errors), and the stop criteria (an error rate or a complaint that ends the pilot). Run it for two to four weeks with a reviewer, then compare against the baseline and decide whether to expand, adjust, or stop. The sample will be small, so do not over-read a single week.

Keep a failure log: what happened, what input caused it, how it was caught. The log will tell you more about where the frontier sits in your business than any vendor demo.

The other kind of AI audit: auditing a system

Once AI touches customers or influences decisions about people, the system itself needs checking too: whether it stays accurate over time, whether it treats people fairly, whether it is secure, and who is accountable when it fails. The US National Institute of Standards and Technology published AI RMF 1.0 on January 26, 2023, a voluntary framework organized around four functions: govern, map, measure, and manage.

For a small business those four functions translate into four habits. Name an owner for each AI system. Write down what it does, what data it uses, and where it can fail. Track error rates and complaints. Keep a process for fixing the system or switching it off. You do not need a formal program, but you do need someone responsible for each of those four things.

When AI is not the answer

• The process is unclear or changes every week. Fix the process first.
• The volume is low enough that a checklist costs less than a system.
• The source data is wrong or scattered across places software cannot reach.
• A mistake would be severe and you cannot review the output.
• The real problem sits earlier or later in the process, such as slow replies caused by nobody owning the inbox, and no tool touches it. The revenue leak mapping guide is the better starting point in that case.
• The task is the relationship itself, such as a personal call that customers value because a person makes it.

Questions to ask anyone who offers to audit or build for you

1. Will the audit stand on its own, so I can use it without hiring you to build?
2. Which tasks did you recommend against automating, and why?
3. What data leaves my systems, and where does it go?
4. What happens when the system is wrong? Can you show me a failure you have seen?
5. How do I turn it off, and who owns the workflows and accounts afterward?
6. What are the ongoing costs, including software, monitoring, and your time?
7. How will we know in 30 days whether it worked?

What a good audit gives you

• A task inventory with times and frequencies.
• Each task classified as rules, language, or judgment.
• A scored, ranked list with the assumptions visible, so you can challenge them.
• Risk and privacy notes for every shortlisted task.
• A pilot plan with a baseline, success criteria, and stop criteria.
• A list of tasks to leave alone.
• Honest cost and payback estimates, including the ones that come out unattractive.

The audit we run produces a written report with a prioritized list of opportunities scored by impact and effort, plus a debrief call to go through it. You keep the report whether or not you hire us to build anything.

The first 30 days after an audit

1. Week 1: make the policy and process fixes that need no software, such as naming owners and setting response targets.
2. Week 2: record baselines and choose the pilot task.
3. Weeks 3 and 4: run the pilot with a reviewer and keep the failure log.
4. End of week 4: compare against the baseline and decide whether to expand, adjust, or stop.

Sources

1. McKinsey Global Institute, "A Future That Works: Automation, Employment, and Productivity," January 2017
2. Brynjolfsson, Li, and Raymond, "Generative AI at Work," The Quarterly Journal of Economics 140(2), 2025
3. Noy and Zhang, "Experimental evidence on the productivity effects of generative artificial intelligence," Science, July 2023
4. Dell'Acqua and colleagues, "Navigating the Jagged Technological Frontier," Harvard Business School working paper, 2023
5. Office of the Privacy Commissioner of Canada and provincial and territorial counterparts, "Principles for responsible, trustworthy and privacy-protective generative AI technologies," December 2023
6. National Institute of Standards and Technology, AI Risk Management Framework 1.0, January 2023