What you need
- Compute: GPUs sized to the model and traffic. We size it in week 1 of the pilot.
- Network and access: a project/account we can deploy into, with your security team's approvals.
- People: one infra contact. You don't need an ML platform team; we run training, serving and updates.
What we run for you
- Model: a licensed open-weights base model, fine-tuned on your data (how fine-tuning works).
- Serving: OpenAI-compatible API, so existing apps and frameworks (LangGraph, CrewAI, AutoGen, MCP) connect with config changes.
- Gateway: masking of sensitive data, routing, budgets.
- Releases: canary rollouts, pinned versions, rollback.
- Records: logs and audit trail stay in your environment, OCSF-formatted.
- Day two: monitoring, incident response, retraining. See Managed Service.
Start with one workflow in your environment
Book a call · or read about the 8-week pilot
Related: Private LLM for enterprise · Fine-tuning on company data · How to train an LLM on your own data
Frequently asked questions
What is an on-premise LLM?
A large language model that runs on hardware in your own data center (or your own cloud account), not on a vendor's shared service.
Can we deploy an LLM in our own cloud instead of on-prem?
Yes. The model can run in your own cloud account (VPC) instead of your data center. We confirm the cloud, region and GPU options for your setup in week 1.
How big a model do we need?
Usually smaller than you'd expect: for narrow tasks, fine-tuned small models can match larger ones (Zhao et al., LoRA Land). We test this on your data.
Who maintains it?
We do, under the managed service, with your team seeing every run and release.
How does this relate to a private LLM?
On-prem is one way to run a private LLM.