IRIS8 Talk to us
Cloud

Private LLM on Azure: Azure OpenAI or your own model

What Azure OpenAI does with your data, and how IRIS8 runs a fine-tuned open-weight model inside your own Azure subscription.

Azure is one way to run in your own cloud account. See all three options: Deploy.

Your environmentYour cloud account (VPC), or on-prem as a scoped data-centre install in a managed engagement
You own the modelWeights, prompts and logs stay with you
8-week pilotOne workflow, a pass mark agreed in week 1
RegionsGlobal

Last reviewed

01

The details that matter in Microsoft's terms

Topic What Microsoft says (paraphrased) Why it matters
Deployment type Standard deployments process in your chosen geography; Global deployments may process in any geography where the model is deployed; DataZone deployments within the US or EU zone A "Global" deployment can conflict with data residency rules
Abuse monitoring Flagged prompts and completions may be stored and reviewed by authorized Microsoft staff unless you're approved for modified abuse monitoring Some regulated data shouldn't sit in any third-party review store
Stored data Files, threads and stored completions sit in your resource's geography, encrypted, with customer-managed key option Good — but preview features may not support all conditions
Fine-tuned models Available only to you You still don't hold the weights

All from Data, privacy and security for Foundry Models sold by Azure.

02

Azure OpenAI or your own model?

Question Azure OpenAI Your own model on Azure GPUs
Who holds the weights Microsoft You
Network Private endpoints available Inside your VNet only
Model changes Model versions are retired on Microsoft's schedule (model retirements) Changes only when you release one
Best for Open-ended, general work High-volume narrow tasks, strict residency

In our work, most production systems end up using both. Measured in our pilots: roughly 60-80% lower cost than frontier API models on specialised tasks, depending on the workload. Published research points the same way: a production study of a narrow feature found that self-hosted small open models gave competitive output quality at 5 to 29 times lower cost than a frontier API, while noting that at low volume the API can be cheaper (Irugalbandara et al., arXiv:2312.14972).

See how this would work on your data.

Book a call
03

How IRIS8 deploys on Azure

Subscription

a resource group in your tenant; access through Entra ID roles you grant and can revoke.

Serving

AKS or VMs on Azure GPU instances; model files in your storage account, encrypted with keys in your Key Vault.

Network

your VNet, private endpoints, no public IP.

Records

Azure Monitor and your own prompt/output logs.

Data rules

pin the region for GDPR or DPDP needs.

04

What we'd do first

  1. 1
    Pick a narrow, high-volume task that's either costly on Azure OpenAI or blocked by residency or logging rules, and run it as an 8-week pilot in your subscription.
  2. 2
    Keep Azure OpenAI for the rest.
FAQ

Frequently asked questions

Does Azure OpenAI train on our data?

Microsoft says prompts, completions and training data aren't used to train foundation models without your permission (Microsoft).

Where is our data processed?

In your geography for standard deployments; possibly elsewhere for Global and within the zone for DataZone (Microsoft).

Can we turn off human review of prompts?

Eligible customers can apply for modified abuse monitoring (Microsoft).

Why run our own model?

You hold the weights, it only changes when you choose, and it can later move on-prem.

Who owns the model?

You do.

Talk to us

A 30-minute call with the IRIS8 team about your data, your environment and a first pilot.

Talk to us