Six months into a major AI rollout, our client’s finance team was still copy-pasting data between systems. The “intelligent automation” we’d championed was sitting mostly unused.

The problem wasn’t the technology. The problem was we’d never properly audited what was already there before adding more.

Why AI Audits Matter Now

Most enterprises I work with have between 4-7 different AI tools running simultaneously. Some were official purchases. Others snuck in via individual team subscriptions. A few came bundled with existing software upgrades.

According to Gartner’s latest enterprise software report, the average mid-size organisation has 3.2x more AI-enabled tools than they realise. That’s not a technology problem—it’s a visibility problem.

The Pre-Implementation Audit Checklist

Before your next AI project kicks off, run through these questions:

1. What’s Already Running?

Inventory check:

• List every tool with “AI” or “intelligent” in the marketing
• Include features within existing software (Excel’s Ideas, Outlook’s scheduling suggestions)
• Document unofficial tools staff are using personally

A manufacturing client in Melbourne discovered their operations team had been using Claude for shift scheduling for eight months. Nobody in IT knew. The solution worked brilliantly—they just hadn’t captured the learnings.

2. What’s the Actual Usage?

Licenses tell you what’s available. Logs tell you what’s used.

Pull 90 days of usage data for every AI tool. You’ll typically find:

• 20% of tools get 80% of the use
• At least one “essential” tool nobody’s touched in weeks
• Features marketed as AI that are really just rules-based automation

3. Where Are the Integration Points?

Map how data flows between systems:

• What manual steps exist between AI outputs and the next process?
• Where do people export data just to import it elsewhere?
• Which integrations were “coming soon” six months ago?

4. What’s the Feedback Loop?

For each AI tool, ask:

• How do users report when it gets something wrong?
• Who reviews that feedback?
• When was the last configuration change based on user input?

No feedback loop usually means the tool is either perfect (unlikely) or ignored (more likely).

The Three Questions Every New AI Proposal Must Answer

After the audit, evaluate new proposals against these criteria:

1. What existing capability does this replace or enhance?

If the answer is “nothing—it’s net new,” that’s a warning sign. New capabilities need new processes, new training, and new change management. The cost is always higher than the software license.

2. Who will own this in 18 months?

Pilots have champions. Production systems need owners. If there’s no clear answer, the tool will likely join the zombie software pile within a year.

3. What happens when it’s wrong?

Every AI system will produce incorrect outputs eventually. The interesting question is what your process does when that happens. If the answer involves phrases like “manual review of everything,” you haven’t actually saved time—you’ve added a step.

The Post-Audit Conversation

The audit often surfaces uncomfortable truths:

• The expensive platform purchased last year is mostly unused
• Shadow AI tools are solving real problems the official tools missed
• Some “AI initiatives” are really just automated reports rebranded

These findings typically trigger one of two responses:

Response A: Defensive justification of past decisions, followed by more purchases to “fix” the original ones.

Response B: Honest assessment of what’s working, consolidation where possible, and clearer criteria for future investments.

Response B leads to better outcomes. It’s also harder politically, which is why so few organisations manage it.

Making Audits Routine

Build AI audits into your regular review cycles:

• Quarterly: Usage metrics and feedback review
• Bi-annually: Full inventory reconciliation
• Annually: Strategic alignment assessment

The goal isn’t to slow down AI adoption. It’s to make sure each new tool actually adds value rather than complexity.

What We Changed

After our finance team audit, we made three changes:

1. Retired two overlapping tools nobody used
2. Added proper training for the one tool that was actually valuable
3. Created a simple intake process for new AI requests

Adoption of the remaining tool jumped from 23% to 71% within a quarter. Not because the technology improved—because we finally understood what we were working with.

The unsexy work of auditing what exists often delivers more value than the exciting work of buying what’s new.