Private LLM on Azure: Azure OpenAI or your own model
What Azure OpenAI does with your data, and how IRIS8 runs a fine-tuned open-weight model inside your own Azure subscription.
Azure is one way to run in your own cloud account. See all three options: Deploy.
The details that matter in Microsoft's terms
| Topic | What Microsoft says (paraphrased) | Why it matters |
|---|---|---|
| Deployment type | Standard deployments process in your chosen geography; Global deployments may process in any geography where the model is deployed; DataZone deployments within the US or EU zone | A "Global" deployment can conflict with data residency rules |
| Abuse monitoring | Flagged prompts and completions may be stored and reviewed by authorized Microsoft staff unless you're approved for modified abuse monitoring | Some regulated data shouldn't sit in any third-party review store |
| Stored data | Files, threads and stored completions sit in your resource's geography, encrypted, with customer-managed key option | Good — but preview features may not support all conditions |
| Fine-tuned models | Available only to you | You still don't hold the weights |
All from Data, privacy and security for Foundry Models sold by Azure.
Azure OpenAI or your own model?
| Question | Azure OpenAI | Your own model on Azure GPUs |
|---|---|---|
| Who holds the weights | Microsoft | You |
| Network | Private endpoints available | Inside your VNet only |
| Model changes | Model versions are retired on Microsoft's schedule (model retirements) | Changes only when you release one |
| Best for | Open-ended, general work | High-volume narrow tasks, strict residency |
In our work, most production systems end up using both. Measured in our pilots: roughly 60-80% lower cost than frontier API models on specialised tasks, depending on the workload. Published research points the same way: a production study of a narrow feature found that self-hosted small open models gave competitive output quality at 5 to 29 times lower cost than a frontier API, while noting that at low volume the API can be cheaper (Irugalbandara et al., arXiv:2312.14972).
See how this would work on your data.
Book a callHow IRIS8 deploys on Azure
Subscription
a resource group in your tenant; access through Entra ID roles you grant and can revoke.
Serving
AKS or VMs on Azure GPU instances; model files in your storage account, encrypted with keys in your Key Vault.
Network
your VNet, private endpoints, no public IP.
Records
Azure Monitor and your own prompt/output logs.
What we'd do first
- 1Pick a narrow, high-volume task that's either costly on Azure OpenAI or blocked by residency or logging rules, and run it as an 8-week pilot in your subscription.
- 2Keep Azure OpenAI for the rest.
Frequently asked questions
Does Azure OpenAI train on our data?
Microsoft says prompts, completions and training data aren't used to train foundation models without your permission (Microsoft).
Where is our data processed?
In your geography for standard deployments; possibly elsewhere for Global and within the zone for DataZone (Microsoft).
Can we turn off human review of prompts?
Eligible customers can apply for modified abuse monitoring (Microsoft).
Why run our own model?
You hold the weights, it only changes when you choose, and it can later move on-prem.
Who owns the model?
You do.