IRIS8 Talk to us

Optimize

Send each request to the right-sized model

Caching, budgets and routing at the gateway, set by rules you control, with no change to your apps.

Caching

Repeat requests come from cache

Repeat and near-duplicate requests are answered without a new model call.

Gateway · cacheSample data
  • Request Amodel
  • Request A, repeatedcache

Tunable match thresholdOpt out per route

Budgets

Team budgetSample dataWithin budget

Spend this month against the cap

Budgets that stop at your cap

Set a monthly budget per team and choose between an alert and a hard stop.

Routing

Routine work goes to smaller models

Each request goes to the smallest model that clears the quality bar you set for that task.

IRIS8 console · loan-file-qa / gatewaySample data

Gateway · live requests

  • Summarise loan fileyour model
  • Extract KYC fieldsyour model
  • Repeat questioncache
  • Draft credit memofrontier model

Account number masked before sendEvery request logged

Your model takes routine work. A frontier model you choose takes the rest.

Attribution

One provider invoice
  • Lendingshowback
  • Contact centreshowback
  • Engineeringchargeback

Spend shown by team

Usage is attributed to teams and products, so finance sees who spent what.

Rules

Your rules at the gateway

Compress long prompts, cap output length and A/B route between models, all in configuration.

Gateway configurationSample data
Compress verbose prompts
On
Maximum output tokens
Set per route
A/B route between models
On
Fallback when a provider fails
On

Library

Specialist model library

A library of small models for classification, extraction, summaries and triage is in development.

In development

Classification

In development

Extraction

In development

Summaries

In development

Triage

Cost

Check the cost on your own volume

The calculator shows both monthly totals and the break-even point.

Questions

Questions

Talk to us
Q1Do our apps need to change?

No. Routing, caching and budgets are set in the gateway configuration.

Q2Who decides the quality bar?

You do, per task. A request only moves to a smaller model that clears it.

Q3Does caching risk stale answers?

Match thresholds and expiry are set per route, and any route can opt out. Routes that need a fresh answer stay live.

Next step

Try it on one workload