Private LLM on AWS: your model in your own AWS account
How Bedrock, SageMaker and your own EC2 GPUs differ in where your data and model live, and how IRIS8 runs a fine-tuned model inside your account.
AWS is one way to run in your own cloud account. See all three options: Deploy.
Three ways to run an LLM on AWS
| Option | Where the model runs | What you control | Fits when |
|---|---|---|---|
| Amazon Bedrock (hosted models) | In AWS-operated Model Deployment Accounts; model providers have no access to them or to your prompts (AWS docs) | Your IAM, network path and logging config; not the base weights | You want a frontier model and accept a managed service |
| SageMaker AI endpoints | Managed instances, can be attached to your VPC (AWS docs) | Your model artifact, instance type, VPC | You want managed hosting of your own model |
| EC2 / EKS on GPU instances | Your instances in your VPC | Everything, including the serving stack | You need full control, or the same stack on-prem later |
AWS works on a shared responsibility model: AWS secures the infrastructure; you're responsible for your data, access and configuration — whichever option you pick.
How IRIS8 deploys on AWS
Account
a dedicated account or VPC in your AWS Organization; we work through roles you grant and can revoke.
Serving
SageMaker or EKS on GPU instances, depending on your team's preference; both keep the model artifact in your S3 bucket, encrypted with your KMS key.
Network
private subnets; applications reach the model over AWS PrivateLink or internal load balancers, not the public internet.
Records
CloudTrail for API activity; prompt and output logs in your account.
Regulated data
if you handle PHI, use services on AWS's HIPAA eligible services list under your AWS BAA. See Private LLM for HIPAA workloads.
See how this would work on your data.
Book a callBedrock, or your own model?
In our work, most production systems end up using both: a frontier model through Bedrock for open-ended work, and a smaller model trained on your data for the high-volume, narrow tasks where format, cost and consistency matter. Measured in our pilots: roughly 60-80% lower cost than frontier API models on specialised tasks, depending on the workload. Published research points the same way: a production study of a narrow feature found that self-hosted small open models gave competitive output quality at 5 to 29 times lower cost than a frontier API, while noting that at low volume the API can be cheaper (Irugalbandara et al., arXiv:2312.14972). Owning the fine-tuned model also means it doesn't change when a provider updates or retires a model version.
What we'd do first
- 1Pick one narrow task, agree the pass mark, set up the account and network with your cloud team in week 1, and reach a canary release in 8 weeks.
- 2See the 8-week pilot and on-prem and VPC deployment.
Frequently asked questions
Does Bedrock use our prompts to train models?
AWS says model providers have no access to Bedrock's deployment accounts, logs or customer prompts (AWS). Check the Bedrock data privacy FAQ for current terms.
Can the model run with no internet access?
Yes — private subnets with VPC endpoints for the AWS services it needs.
Which region?
Any region where your GPU instance type is available and your data rules allow.
Can we move it on-prem later?
Yes, if served on EKS/containers; the same model and stack can run in your data center.
Who owns the model?
You do — it sits in your S3 bucket.