Accounting firms are adopting AI to draft client emails, summarise financial documents, prepare reports, and streamline internal workflows. But accounting is not a generic industry. The data involved — payroll records, tax returns, bank statements, management accounts, business valuations — carries professional and legal obligations that most AI adoption guides overlook. The Hermes Agent paired with Google DeepMind's Gemma 4, running locally via Ollama, gives mid-size firms a practical way to use AI without sending a single byte of client data to an external server.

The Question Every Firm Should Be Asking

Most AI tools accountants use today are cloud-based. When a team member types a prompt into a cloud AI product, that prompt is sent over the internet to the provider's servers, processed externally, and the response is sent back. The accountant gets a useful answer. The client's data has left the building.

For general productivity tasks — rewriting an internal email, brainstorming a blog post — this is usually fine. But accounting firms routinely work with information that clients would never expect to leave a controlled environment:

management accounts and board reports,
tax returns and BAS lodgements,
payroll records and employee details,
bank statements and loan applications,
business valuations and sale documents,
audit evidence and workpapers,
confidential client correspondence.

The question is not whether AI can help with these tasks. It clearly can. The question is: where does the client data go when AI processes it?

How Local AI Changes the Data Flow

Local AI means the model runs on hardware the firm controls. Instead of sending prompts to an external provider, the firm hosts the model internally and processes everything on-site.

The simplest way to think about it: cloud AI is like sending your client's documents to an external print shop. Local AI is like having the printer in your own secure office. The output is similar. The difference is where the work happens.

With the Hermes Agent running Gemma 4 through Ollama, a mid-size firm gets an AI assistant that operates entirely within its own infrastructure. Every query, every document processed, every AI-generated response stays on the firm's hardware. There is no external API call and no cloud dependency.

What Accountants Can Actually Do With It

The real value of local AI for accounting firms is not theoretical. Here are the workflows where it delivers immediate, practical benefit.

Draft Client Emails Using Sensitive Context

Feed the Hermes Agent your internal notes on a client's financial position and ask it to draft a clear explanation of margin changes, cash flow movements, or tax adjustments. The notes stay local. The draft is ready for partner review.

Summarise Financial Documents

Board minutes, loan agreements, management reports, audit planning documents — the Hermes Agent can produce concise summaries without those documents ever leaving your network. For firms handling multiple clients with overlapping reporting deadlines, this saves hours of manual work each month.

Create Workpaper Commentary

Accountants spend significant time turning raw analysis into polished commentary for workpapers. A local model can take your notes on revenue, wages, and overhead movements and produce a structured first draft. The accountant reviews, refines, and approves — but the writing time drops substantially.

Build an Internal Policy Assistant

Connect the Hermes Agent to your firm's internal policies and templates through a RAG (Retrieval-Augmented Generation) setup. Staff can then ask natural language questions like "What is our process for onboarding a new payroll client?" or "Which template do I use for a management reporting engagement?" and get precise answers drawn from your approved documents. This is particularly valuable for mid-size firms where institutional knowledge often sits with a few senior staff.

Review Bookkeeping Notes

Before month-end close, ask the Hermes Agent to review transaction notes and flag items that need follow-up — missing descriptions, unreconciled entries, or incomplete coding. The model works from your data without any of it leaving the firm.

Classify Client Documents and Transactions

Supplier invoices, expense descriptions, bank transaction narratives — the Hermes Agent can help categorise and sort these into the right buckets, provided the workflow includes human review. For firms doing volume bookkeeping, this turns repetitive classification into supervised verification.

Why This Matters for Client Trust

Some clients are already asking their accountants: "Are you sending my data to ChatGPT?"

With a local AI deployment, the answer is straightforward:

No. The AI model runs on our internal infrastructure. Your documents are not sent to any external AI provider.

For firms serving privacy-conscious clients, government contractors, or businesses in regulated industries, this is a genuine competitive advantage. It moves AI from a productivity tool into a trust signal.

For Australian firms, this also simplifies the conversation around the Privacy Act 1988. When data does not leave your network, cross-border disclosure concerns under APP 8 do not arise.

Local AI Does Not Mean Automatic Security

Transparency matters, so here is what local AI does not do by itself:

It does not eliminate all risk. A poorly configured local system can still create confidentiality problems if outputs are saved in insecure locations, user access is poorly managed, or the server is not properly maintained.
It does not replace professional judgement. Every AI output requires competent review. AI hallucination is real, and no model should be trusted to produce final client-facing work without human verification.
It does not stay current automatically. A local model only knows what was in its training data. It should not be relied on for current tax rates, recent legislative changes, or updated accounting standards unless connected to current, authoritative source material.

The benefit of local AI is not magic security. The benefit is control. But control only helps if the firm has the right policies, access controls, and review processes in place.

The Future Is Hybrid

For most mid-size accounting firms, the practical path forward is a hybrid approach.

Cloud AI may be appropriate for low-risk general tasks — internal brainstorming, non-sensitive drafting, professional development. Enterprise cloud tools with strong data protection commitments may suit certain workflows where the risk profile is acceptable.

But for work involving sensitive client data — tax records, payroll files, confidential correspondence, board reports, business valuations — local AI offers something cloud tools cannot: the ability to tell your clients exactly where their data was processed and confirm that it never left your environment.

The firms that establish this capability now are building a trust advantage that compounds over time. The Hermes Agent learns from completed tasks and creates reusable skills — meaning your firm's AI gets measurably better at your specific workflows with every passing month.

The AIRGAP LLM Perspective

"Accountants are in a unique position. They handle some of the most sensitive data in any profession, yet AI adoption in accounting is accelerating faster than the governance frameworks around it. Local AI deployment gives firms the ability to move forward with AI confidently — not by avoiding the technology, but by controlling where the data goes."

— Sasa Abe, Co-Founder, AIRGAP LLM

Getting Started

For accounting firms evaluating AI and concerned about where client data ends up, contact AIRGAP LLM for a confidential consultation. Our five step deployment process — Assess, Design, Build, Validate, Support — is designed specifically for firms in regulated industries.

Frequently Asked Questions

Do we need dedicated IT staff to run this?

The Hermes Agent and Ollama are not consumer apps, but they do not require a full IT department either. A firm with basic IT capability — or a deployment partner like AIRGAP LLM — can handle setup and ongoing maintenance. Most mid-size firms find the overhead manageable once the initial deployment is complete.

Can we still use cloud AI for some tasks?

Yes. Many firms use cloud AI for non-sensitive general tasks while keeping local AI for any work involving client data. The two approaches complement each other. AIRGAP LLM can help you define appropriate boundaries for each.

What does this cost compared to cloud AI subscriptions?

Local AI involves a one-time hardware investment and no ongoing per-query or per-user fees. The Hermes Agent is open-source, Gemma 4 is released under the Apache 2.0 license, and Ollama is free. For most mid-size firms, the economics compare favourably to enterprise AI subscriptions within the first year. Contact AIRGAP LLM for a comparison tailored to your firm.

Is the AI good enough for real accounting work?

For drafting, summarisation, document classification, and internal knowledge search — yes, it is genuinely useful today. For complex tax analysis or audit judgement, AI assists but does not replace professional expertise. The key is matching the tool to the task and always applying competent human review.

Local AI for Accountants: Why Your Firm Should Control Where Client Data Goes