Custom LLM vs Self-Hosted LLM: What Australian Businesses Need to Know
The Australian private AI market increasingly splits into two camps: "custom LLM" services that fine-tune cloud-hosted AI for you, and "self-hosted LLM" deployments that put the AI inside your firm. Both call themselves "private AI." Only one keeps your data inside your environment. For Australian organisations with compliance obligations, the distinction matters more than any other factor. This guide explains the difference and when each makes sense.
The Confusing Language Problem
"Custom LLM," "private AI," "self-hosted AI," "on-premises LLM," "sovereign AI" — these terms get used interchangeably in marketing materials. They are not the same thing.
| Term | Common Meaning | Data Location |
|---|---|---|
| Custom LLM | A model customised/fine-tuned for your organisation | Usually vendor's cloud |
| Managed AI | AI service operated by a vendor for your use | Vendor's cloud |
| Private cloud AI | Hosted AI in a "private" cloud tier | Vendor's cloud (sometimes in Australia) |
| Self-hosted LLM | AI running on your own infrastructure | Your hardware |
| On-premises LLM | AI running in your physical premises | Your office or server room |
| Private LLM | Ambiguous — context-dependent | Could be either |
| Sovereign AI | AI entirely within Australian legal/physical control | Should be on-premises, but marketed loosely |
The clearest test: does your data leave your environment to be processed? If yes, it's some flavour of cloud AI (no matter how it's marketed). If no, it's truly private/self-hosted.
Custom LLM Services in Detail
A "custom LLM" service typically works like this:
- You sign a contract with the vendor
- The vendor configures and fine-tunes an LLM on your data (often using your documents, customer queries, or knowledge bases)
- The vendor hosts the resulting model on their cloud infrastructure
- Your staff access the model via an API or web interface
- Each query travels from your network → vendor's cloud → back to your network
Vendor value-add:
- Model selection and fine-tuning
- Infrastructure operation
- Ongoing model improvement
- User interface
- Support and maintenance
What you get: A model that "knows" your business in some sense — has been trained on aspects of your data or configured with your terminology.
What you give up: Direct control over where your data goes during every query. The data flows through the vendor's infrastructure.
Self-Hosted LLM in Detail
A self-hosted LLM works like this:
- You purchase or specify hardware
- A deployment partner (e.g., AIRGAP LLM) installs the model and infrastructure on your hardware
- The deployment partner configures the system for your use cases (RAG over your documents, custom prompts, integrations)
- Your staff access the model via your internal network
- Each query stays entirely within your environment
Vendor value-add (deployment partner):
- Initial deployment and configuration
- Document ingestion and RAG setup
- Integration with your systems
- Ongoing support and updates
- Model upgrades over time
What you get: A model that knows your business through RAG over your documents, with full control over the infrastructure.
What you keep: Complete custody of your data. Nothing flows externally.
The Side-by-Side Comparison
| Dimension | Custom LLM Service | Self-Hosted LLM |
|---|---|---|
| Where the model runs | Vendor's cloud | Your infrastructure |
| Where data is processed | Vendor's cloud | Your hardware |
| Network requirement | Internet connection for every query | None (can be fully offline) |
| Cross-border data transfer | Usually yes (US/EU) | Never |
| Privacy Act APP 8 implications | Material | None |
| APRA CPS 234 third-party obligations | Triggered | Not triggered |
| My Health Records Act exposure | Material | None |
| Vendor lock-in | Strong (proprietary stack, custom fine-tuning) | None (open-source, transferable) |
| Cost structure | Recurring per-user or per-query fees | One-time setup + monthly support |
| Scalability | Vendor's responsibility | Your hardware planning |
| Pricing predictability | Vendor can change pricing | Costs known upfront |
| Compliance auditability | Limited to vendor's records | Full audit trail under your control |
| Foreign government access risk | Yes (US CLOUD Act, etc.) | None |
| Internet outage tolerance | Service unavailable | Continues operating |
| Model upgrades | Vendor decides | You decide |
| Customisation depth | Vendor-mediated | Direct |
| Setup time | 4-12 weeks | 4-8 weeks |
| Ongoing operational overhead | Vendor-managed | Hosted by you (typically minimal) |
When Custom LLM Services Make Sense
Custom LLM services are the right choice for organisations that:
- Have minimal compliance burden (the data isn't sensitive)
- Don't want to manage any infrastructure
- Need a vendor's specific fine-tuning expertise
- Have short-term or pilot needs
- Have variable usage patterns and want elastic scaling
- Lack internal IT capacity even for a managed on-premises deployment
For an early-stage Melbourne marketing agency wanting AI assistance with proposal writing — no sensitive data, no compliance regime — a custom LLM service may be perfectly appropriate.
When Self-Hosted LLM Makes Sense
Self-hosted LLM is the right choice for organisations that:
- Handle privileged, regulated, or sensitive data
- Have compliance obligations under Privacy Act, APRA, MHR Act, etc.
- Need data sovereignty
- Want predictable, predictable costs at scale
- Have 25+ users where per-user pricing becomes punitive
- Want freedom from vendor lock-in
- Need full audit and compliance demonstrability
- Want to operate offline or air-gapped
For a Melbourne law firm with 60 lawyers, a healthcare network, a wealth management firm, or any APRA-regulated entity — self-hosted is almost always the better choice.
The customllm.au Example
A representative custom LLM service in the Australian market is customllm.au. Their offering includes:
- Custom model fine-tuning
- RAG integration
- Multi-modal capabilities
- Tiered monthly pricing ($2,999-$14,999/month)
- Government/sovereign messaging
- Australian data residency claims
For organisations with low compliance burdens that want a polished managed service, this is a credible offering. But on close inspection:
- The infrastructure is the vendor's cloud — your data flows through their systems
- The cost model is monthly subscription — grows with usage and time
- Data residency in Australia helps with APP 8 but doesn't address the underlying third-party processing
- Vendor lock-in: the custom fine-tuning is on the vendor's models, hosted on the vendor's infrastructure
Compared with a self-hosted deployment from AIRGAP LLM:
- Your data stays in your office or server room
- Cost is mostly one-time + modest support
- No third-party processing at all
- No lock-in — the open-source models can be migrated, the infrastructure is yours
Real Decision Examples
Decision 1: Mid-Tier Accounting Firm (45 Staff)
Profile: Tax and advisory firm handling client financial data. Privacy Act applies; client confidentiality is professional obligation.
Custom LLM service option: $5,000/month = $60,000/year. Over 3 years: $180,000. Data flows through vendor's cloud.
Self-hosted option: $50,000 year 1 (hardware + setup) + $24,000/year support = $98,000 over 3 years. Data stays in office.
Decision: Self-hosted. Cheaper, compliance-aligned, no vendor dependency.
Decision 2: Solo Family Law Practitioner
Profile: Single practitioner with paralegal. Privileged communications, but small scale.
Custom LLM service option: $1,500/month = $54,000 over 3 years.
Self-hosted option: $30,000 year 1 + $12,000/year support = $54,000 over 3 years.
Decision: Self-hosted for privilege protection (custom LLM service involves third-party disclosure that may waive privilege — see our analysis). Cost is similar but compliance position is much stronger.
Decision 3: Marketing Agency (15 Staff)
Profile: Creates marketing content for clients. No sensitive data, no compliance regime.
Custom LLM service option: $2,000/month = $72,000 over 3 years.
Self-hosted option: $35,000 year 1 + $18,000/year = $71,000 over 3 years.
Decision: Similar cost; custom LLM service may be operationally easier. Either is reasonable.
Decision 4: Healthcare Network (350 Staff)
Profile: Multi-site clinical network handling patient records. My Health Records Act, Privacy Act, AHPRA all apply.
Custom LLM service option: $12,000/month = $432,000 over 3 years. Patient data flows through vendor's cloud — major compliance issue.
Self-hosted option: $120,000 year 1 + $36,000/year = $192,000 over 3 years. Patient data never leaves the network.
Decision: Self-hosted. Substantially cheaper AND vastly better compliance position. The cloud option is arguably not even legally available given MHR Act s 77.
The Compliance Argument Revisited
For Australian organisations with compliance obligations, the comparison reduces to:
| Question | Custom LLM Service | Self-Hosted LLM |
|---|---|---|
| Will data be disclosed to a third party? | Yes | No |
| Is that disclosure cross-border? | Often yes | Never |
| Do you control the security controls? | No | Yes |
| Can you demonstrate compliance to a regulator? | Indirectly | Directly |
| Is the data subject to foreign legal process? | Often yes | No |
For organisations with material compliance burdens, these answers should drive the decision regardless of cost.
For deeper context, see our private LLM vs public LLM analysis and sovereign AI Australia guide.
The Vendor Lock-In Difference
Custom LLM services typically involve significant lock-in:
- The fine-tuned model lives on the vendor's infrastructure
- Migration requires re-doing the fine-tuning elsewhere
- Integration code targets the vendor's API
- Staff training is on the vendor's interface
- Documentation and prompts are customised for the vendor's model
If the vendor:
- Raises prices significantly
- Changes terms of service unfavourably
- Has a major outage
- Gets acquired by a different company
- Decides to shut down the service
You face significant migration cost and operational risk.
Self-hosted LLM has minimal lock-in:
- Open-source models are portable
- Integration code targets standards (OpenAI-compatible APIs)
- Documents and configurations are yours
- Staff training is on a general AI interface
- Hardware is yours
You can change deployment partners (or even bring it in-house) with limited disruption. The investment in the deployment doesn't evaporate if you change vendors.
The Hybrid Reality
Some organisations adopt a hybrid approach:
- Self-hosted LLM for sensitive data, compliance-bound workflows, and core internal operations
- Custom LLM service or public cloud AI for non-sensitive productivity tasks (calendar assistance, public-facing content drafts)
This minimises compliance overhead while accessing best-of-breed options for low-stakes use cases. The key is rigorous data classification and policy enforcement — staff need to know which tool to use for which task, and tooling should prevent sensitive data from flowing to cloud services.
How to Evaluate Vendors
For organisations considering either custom LLM or self-hosted options, ask vendors:
| Question | What to Listen For |
|---|---|
| Where exactly does our data flow during a query? | Specific answers, named systems and locations |
| What happens if you go out of business? | Concrete migration paths, not "we won't" |
| Can we audit your security controls? | Yes / specific limitations / no |
| What does an APRA examiner see when they ask about your service? | Documentation, certifications, transparent reporting |
| Do your terms allow you to change pricing? | Yes (almost always) — by how much, with how much notice |
| Can the service work offline? | Yes / no — important for sovereign requirements |
| What models do you use? | Open-source named (Llama 3, Gemma 4) vs proprietary |
| Who has access to our data on your side? | Specific roles and controls |
Vendor answers that are vague, defensive, or rely on "trust us" are red flags for compliance-sensitive organisations.
The AIRGAP LLM Perspective
AIRGAP LLM exclusively deploys self-hosted private AI for Australian organisations. We chose this model deliberately:
- For regulated industries (legal, healthcare, financial services, government) the cloud-based "custom LLM" approach struggles to meet compliance obligations
- Open-source models (Llama 3, Gemma 4, Mistral) are now capable enough that the historical argument for proprietary cloud models has largely evaporated
- Australian organisations are increasingly aware that "data sovereignty" requires more than an Australian data centre
- The cost economics favour self-hosting at any team size above ~15-20 users
We are upfront about the trade-offs. Self-hosted requires the organisation to own some infrastructure (typically a single server). It requires a deployment partner relationship for setup and support. It does not eliminate all complexity.
But for the typical Australian organisation with compliance obligations or 30+ users, it is the more durable, more compliant, and ultimately cheaper approach.
For a tailored comparison of custom LLM vs self-hosted options for your specific situation, contact AIRGAP LLM for a free assessment.
Frequently Asked Questions
What is the difference between a custom LLM and a self-hosted LLM?
A "custom LLM" typically means a managed AI service where a vendor configures or fine-tunes a model for your organisation, but the model still runs on the vendor's cloud infrastructure. A "self-hosted LLM" (also called "on-premises LLM" or "private LLM") runs on hardware you control — usually within your own office, server room, or private data centre. The difference is who controls the data and the infrastructure: vendor's cloud (custom) vs your own environment (self-hosted).
Is a custom LLM service the same as a private LLM?
No — these are often confused. A custom LLM service is "private" in the sense that the model has been configured for your organisation, but the data still flows through the vendor's cloud infrastructure. A true private LLM is one where the model AND the data run on infrastructure under your control. For Australian organisations with compliance obligations under the Privacy Act, APRA standards, or the My Health Records Act, the cloud-vs-on-premises distinction matters more than the customisation.
Why do some Australian organisations choose self-hosted LLM over custom LLM services?
Five main reasons: (1) Data sovereignty — data stays in Australia, often in the building; (2) Compliance — easier alignment with Privacy Act APP 8, APRA CPS 234, My Health Records Act; (3) Cost — no per-user or per-query fees scaling with usage; (4) Independence — no foreign vendor lock-in or pricing surprises; (5) Security — full visibility into how data is handled. Custom LLM services address customisation but leave the underlying data flow and compliance issues unchanged.
Can a self-hosted LLM be as good as a custom LLM service?
Yes, for the vast majority of business use cases. Self-hosted LLMs use the same open-source foundation models (Llama 3, Gemma 4, Mistral) that many custom LLM services use under the hood. The differentiator in custom LLM services is fine-tuning and integration work — which can equally be done on a self-hosted deployment. The capability ceiling is the same; the data control is dramatically better with self-hosted.
What's the cost difference between custom LLM services and self-hosted?
Custom LLM services typically charge $2,000-$15,000 per month in subscription/usage fees, depending on tier and consumption. Over 3 years that's $72,000-$540,000. Self-hosted LLM deployment costs $25,000-$110,000 in year one (hardware + setup) and $12,000-$36,000 per year ongoing. By year 2-3 the self-hosted deployment is dramatically cheaper, and the gap grows over time. See our detailed cost guide for breakdowns.