What "private" should mean
Ask any vendor these five questions:
- Where do the weights run? Your VPC, on-prem, or the vendor's cloud?
- Who can see prompts and outputs? Is any of it logged outside your environment?
- Who owns the trained model? Can you keep running it if the contract ends?
- Can the model change without you? Pinned versions and rollback on your authority, or provider-side updates?
- Can you prove it to an auditor? Records of data, training runs and evals.
Three ways to get a private LLM
| Private endpoint from an API provider | Vendor-hosted dedicated model | Owned model in your environment (IRIS8) | |
|---|---|---|---|
| Where it runs | Provider's cloud, region you pick | Vendor's or your cloud | Your VPC, IRIS8-hosted, or a managed on-prem install |
| Who owns weights | Provider | Vendor (usually) | You |
| Model updates | Provider schedule; models get deprecated — e.g. OpenAI is closing fine-tuning, with training possible until 6 Jan 2027 (OpenAI, Supervised fine-tuning guide) | Vendor schedule | You decide |
| Cost model | Per token | Per GPU-hour | Your infrastructure + our managed service |
| Best for | Low volume, broad tasks | Teams wanting a hosted option | Regulated data, repeat workloads, long-term ownership |
How IRIS8 builds it
Six stages, run by our team (iris8.ai homepage and /managed-service.html):
- Procure — choose a licensed open-weights base model and size the hardware.
- Train — prepare your data and fine-tune (LoRA (Hu et al., "LoRA"), preference tuning with DPO (Rafailov et al., "Direct Preference Optimization")). Read how fine-tuning works.
- Benchmark — score it against frontier models on your own cases before it ships.
- Deploy: in your VPC, IRIS8-hosted, or on-prem as a scoped data-centre install in a managed engagement, with an OpenAI-compatible API so existing apps connect.
- Operate — production support, monitoring, rollback.
- Reinforce — retrain on reviewed feedback; new versions ship only if they pass the benchmark.
Most enterprise systems also need current documents, so we pair the model with retrieval (RAG (Lewis et al., "Retrieval-Augmented Generation")). In our work, most production systems end up using both. Details in How to train an LLM on your own data.
When private is cheaper, and when it isn't
Running your own model means paying for GPUs whether or not they're busy. At low volume, a pay-per-token API usually costs less. Owning pays off on repeat, high-volume tasks that a smaller model can do well. In our work, specialised models perform on par with frontier models on narrow, domain-specific tasks. The pilot ends with a cost comparison built from your volumes.
Regulation and data residency
Data residency is set by where you deploy. Your legal and risk teams decide what each regulation requires; we supply the records they ask for. For India, the DPDP Act's main obligations apply from 13 May 2027. We do not yet hold SOC 2 or ISO 27001 certification.
Talk to us about your first workflow
Related: 8-week pilot · On-prem and VPC LLM deployment · Fine-tuning on company data · How to train an LLM on your own data
Frequently asked questions
What is a private LLM?
A language model that runs in infrastructure you control, so prompts, outputs and training data stay inside your perimeter, and ideally whose weights you own.
Is a private LLM the same as on-prem?
Not always. It can run in your own cloud account (VPC), IRIS8-hosted, or on-prem as a scoped data-centre install in a managed engagement. See on-prem LLM deployment.
Do we need to train a model from scratch?
No. We fine-tune an existing open-weights model on your data; that takes far less data and compute.
Is it as good as GPT-class models?
On broad tasks, usually not. On narrow, repeated tasks, fine-tuned smaller models can match or beat them (Zhao et al., LoRA Land). We measure it on your cases.
How do we start?
An 8-week pilot on one workflow.