Private LLM for banks: trained on your data, inside your perimeter
A model fine-tuned on your documents and run in your VPC or data centre, and what SR 26-2, RBI guidance and the EU AI Act ask of it.
Which rules apply, and what they mean for an LLM
| Rule | Jurisdiction | What it says (paraphrased) | Covers GenAI? | Source |
|---|---|---|---|---|
| SR 26-2 / OCC Bulletin 2026-13 (Revised Guidance on Model Risk Management, 17 Apr 2026) | US | Replaces SR 11-7 and SR 21-8; risk-based model risk management; most relevant to banks over $30B in assets; not enforceable standards | No | OCC Bulletin 2026-13 checked 2026-11-04 |
| RBI FREE-AI Committee report (Aug 2025) | India | 7 principles ("Sutras"), 6 pillars, 26 recommendations, including a board-approved AI policy and lifecycle governance for regulated entities | Yes | Reserve Bank of India checked 2026-11-04 |
| RBI, Storage of Payment System Data (Apr 2018) | India | Payment-system data must be stored only in India | Unclear | Reserve Bank of India checked 2026-11-04 |
| Digital Personal Data Protection Act, 2023 | India | Consent and purpose limits for processing personal data | Unclear | Digital Personal Data Protection Act checked 2026-11-04 |
| EU AI Act (Regulation (EU) 2024/1689), Annex III 5(b) | EU | AI systems used to evaluate creditworthiness of individuals are high-risk (fraud detection excepted) | Yes | Regulation checked 2026-11-04 |
Two things are easy to get wrong. First, SR 26-2 does not cover generative AI: the agencies say GenAI and agentic AI are "not within the scope of this guidance" (OCC Bulletin 2026-13, Model Risk Management: Revised Guidance). That doesn't mean anything goes; it means your own model-risk and third-party policies have to carry the weight, and examiners will still ask how you tested and monitor the system. A model you own, with versioned training data, eval results and pinned releases, is easier to document than an API that changes underneath you. Second, data location rules apply to the data, not the AI label: if a workflow touches payment data in India, inference has to stay where the data must stay (Reserve Bank of India, Storage of Payment System Data, RBI/2017-18/153). In the EU, a use case — not a model — becomes high-risk: the same model used for complaint triage and for credit scoring sits under different obligations (Regulation (EU) 2024/1689 (AI Act), Annex III).
Use cases that suit a private model
| Use case | Input data | Output | Fine-tune or RAG? | Human review |
|---|---|---|---|---|
| KYC document extraction | ID and address proofs, forms | Structured fields | Fine-tune (fixed format) | Exceptions only |
| Complaint triage | Customer emails, call notes | Category, priority, route | Fine-tune (your categories) | Sample QA |
| Credit memo first draft | Financials, analyst notes | Draft memo in house format | Both (fine-tune format, retrieve current policy) | Always — analyst signs |
| AML alert narrative draft | Alert data, transaction history | Draft investigation narrative | Fine-tune | Always — investigator decides |
| Policy and procedure Q&A | Policy manuals, circulars | Answer with citation | RAG (facts change) | Spot checks |
Data you would use, and how it is handled
| Data type | Sensitivity | Handling |
|---|---|---|
| Customer PII (names, IDs, account numbers) | Personal / regulated | Masked before training unless the task needs it; stays in perimeter |
| Payment transaction data | Regulated (India: localisation ([Reserve Bank of India, Storage of Payment System Data, RBI/2017-18/153](https://www.rbi.org.in/Scripts/NotificationUser.aspx?Id=11244))) | In-country deployment only |
| Internal policies, credit manuals | Confidential | Used via retrieval; versioned |
| Analyst-written memos and decisions | Confidential | Training examples after SME review |
See how this would work on your data.
Book a callA use case end to end
Example: complaint triage. Your team already labels complaints by product and urgency. We take a few thousand past complaints with the labels your team gave them, mask account numbers and names, and fine-tune an open-weights model in your VPC to produce the same labels in the same format. Before anything ships, we score it on a held-out set of recent complaints against your current process and a frontier model, with the pass mark agreed in week 1. It then runs alongside your team, version-pinned, with a sample of outputs reviewed weekly. On customer-VPC deployments, data and model stay in your environment.
Where it runs
For India, deploy in an Indian region or on-prem where payment data is involved (Reserve Bank of India, Storage of Payment System Data, RBI/2017-18/153). Details: on-prem and VPC deployment.
What we'd do first
- 1Pick one high-volume, rule-bound workflow — complaint triage or KYC extraction are usually good first candidates because outcomes are easy to score.
- 2Agree the pass mark, build the benchmark from your own cases, and take it to a canary release in 8 weeks.
- 3Credit decisions come later, once the documentation and review process has been proven on a lower-risk task.
- 4See the 8-week pilot.
Frequently asked questions
Does SR 26-2 apply to our LLM?
Not directly. The revised guidance says generative and agentic AI are outside its scope (OCC Bulletin 2026-13, Model Risk Management: Revised Guidance). Your bank's own model-risk and third-party risk policies still apply, so document testing and monitoring anyway.
Can a private LLM keep Indian payment data in India?
Yes, if it is deployed in an Indian cloud region or on-prem. RBI requires payment-system data to be stored only in India (Reserve Bank of India, Storage of Payment System Data, RBI/2017-18/153).
Is an LLM used for credit scoring high-risk under the EU AI Act?
Yes. Systems that evaluate individuals' creditworthiness are listed as high-risk in Annex III (Regulation (EU) 2024/1689 (AI Act), Annex III).
What does the RBI FREE-AI report expect of banks?
Among its 26 recommendations: a board-approved AI policy and governance across the AI lifecycle (Reserve Bank of India, FREE-AI Committee Report).
Do we need to train a model from scratch?
No. We fine-tune an existing open-weights model on your data. See fine-tuning.
How do we start?
One workflow, 8 weeks: the pilot.