IRIS8 Talk to us
Cloud

Private LLM on AWS: your model in your own AWS account

How Bedrock, SageMaker and your own EC2 GPUs differ in where your data and model live, and how IRIS8 runs a fine-tuned model inside your account.

AWS is one way to run in your own cloud account. See all three options: Deploy.

Your environmentYour cloud account (VPC), or on-prem as a scoped data-centre install in a managed engagement
You own the modelWeights, prompts and logs stay with you
8-week pilotOne workflow, a pass mark agreed in week 1
RegionsGlobal

Last reviewed

01

Three ways to run an LLM on AWS

Option Where the model runs What you control Fits when
Amazon Bedrock (hosted models) In AWS-operated Model Deployment Accounts; model providers have no access to them or to your prompts (AWS docs) Your IAM, network path and logging config; not the base weights You want a frontier model and accept a managed service
SageMaker AI endpoints Managed instances, can be attached to your VPC (AWS docs) Your model artifact, instance type, VPC You want managed hosting of your own model
EC2 / EKS on GPU instances Your instances in your VPC Everything, including the serving stack You need full control, or the same stack on-prem later

AWS works on a shared responsibility model: AWS secures the infrastructure; you're responsible for your data, access and configuration — whichever option you pick.

02

How IRIS8 deploys on AWS

Account

a dedicated account or VPC in your AWS Organization; we work through roles you grant and can revoke.

Serving

SageMaker or EKS on GPU instances, depending on your team's preference; both keep the model artifact in your S3 bucket, encrypted with your KMS key.

Network

private subnets; applications reach the model over AWS PrivateLink or internal load balancers, not the public internet.

Records

CloudTrail for API activity; prompt and output logs in your account.

Regulated data

if you handle PHI, use services on AWS's HIPAA eligible services list under your AWS BAA. See Private LLM for HIPAA workloads.

See how this would work on your data.

Book a call
03

Bedrock, or your own model?

In our work, most production systems end up using both: a frontier model through Bedrock for open-ended work, and a smaller model trained on your data for the high-volume, narrow tasks where format, cost and consistency matter. Measured in our pilots: roughly 60-80% lower cost than frontier API models on specialised tasks, depending on the workload. Published research points the same way: a production study of a narrow feature found that self-hosted small open models gave competitive output quality at 5 to 29 times lower cost than a frontier API, while noting that at low volume the API can be cheaper (Irugalbandara et al., arXiv:2312.14972). Owning the fine-tuned model also means it doesn't change when a provider updates or retires a model version.

04

What we'd do first

  1. 1
    Pick one narrow task, agree the pass mark, set up the account and network with your cloud team in week 1, and reach a canary release in 8 weeks.
  2. 2
FAQ

Frequently asked questions

Does Bedrock use our prompts to train models?

AWS says model providers have no access to Bedrock's deployment accounts, logs or customer prompts (AWS). Check the Bedrock data privacy FAQ for current terms.

Can the model run with no internet access?

Yes — private subnets with VPC endpoints for the AWS services it needs.

Which region?

Any region where your GPU instance type is available and your data rules allow.

Can we move it on-prem later?

Yes, if served on EKS/containers; the same model and stack can run in your data center.

Who owns the model?

You do — it sits in your S3 bucket.

Talk to us

A 30-minute call with the IRIS8 team about your data, your environment and a first pilot.

Talk to us