Access Control for Private AI: RBAC, SSO & MFA Done Right
Owning the box is only half the job. The other half is deciding who on your team can do what with it — ask a question, change a prompt, swap a model, or read the logs. Done well, access control is the difference between a server only your people can reach and one where a single shared login can read every document on it. Here is how to set it up properly: the roles that matter, the single sign-on and multi-factor login that hold it together, and the audit trail that proves who did what.
The control people skip — and the risk
When a team stands up its own AI server, the rush is to get it answering questions. Access control gets a shrug: one login, shared around, and everyone is effectively an administrator. It is the single most common — and most costly — mistake we see on self-hosted deployments.
The risk is concentration. If every account can do everything, then one stolen password, one careless click, or one disgruntled login can read every document the model can see, change the model itself, or quietly delete the logs that would have shown what happened. Access control is not bureaucracy; it is the brake that keeps a small mistake from becoming a full breach. It belongs in the foundation of any private AI infrastructure, not bolted on later.
The four roles every private AI server needs
Role-based access control (RBAC) grants permissions by job, not by person. Most private deployments map cleanly onto four roles. Start here and split further only if you genuinely need to.
| Role | Can do | Cannot do | Who it's for |
|---|---|---|---|
| Inference consumer | Ask questions; use the chat or app | Change prompts, models, or settings | Most staff — the everyday users |
| Prompt engineer | Build and tune prompts, tools, and RAG sources | Swap models or alter system config | The person shaping how the AI answers |
| Model admin | Manage models, configuration, and integrations | Necessarily see raw audit logs unsupervised | IT lead or the technical owner |
| Auditor (read-only) | Review access records and logs | Change anything — no write access at all | Compliance, security, or an outside reviewer |
Separating the auditor from the admin matters: the person who can change the system should not be the only person who can read the record of those changes.
SSO & federated identity — one login, central control
Single sign-on (SSO) lets the AI app trust the identity provider your company already uses, instead of inventing its own usernames and passwords. People log in once with their normal work account, and the AI server simply asks that provider "is this really them, and what are they allowed to do?" The two standards that carry this are OIDC (OpenID Connect, built on OAuth 2.0) and SAML — both are mature, widely supported ways for an application to delegate login to a central directory.
The payoff is control in one place. There is no separate AI password to scatter, reuse, or forget to revoke. When someone leaves, disabling their company account closes their AI access at the same moment — no orphaned login lingering on the box. Federated identity also lets you map directory groups straight onto the four roles above, so access follows the org chart instead of a spreadsheet.
MFA on inference endpoints — closing the open door
The most common hole on a self-hosted AI server is an inference endpoint that answers anyone who can reach it. The model is up, the API responds, and nobody put a real login in front of it. On a flat network, that means a stolen laptop or a single shared credential is a free pass.
Multi-factor authentication (MFA) closes that door by requiring a second proof of identity — an authenticator code or a hardware key — on top of the password. Run through your SSO provider, MFA covers the AI app the same way it covers email, so there is one consistent rule rather than a special case for the AI box. Keeping the endpoint LAN-only reduces exposure, but it does not replace authentication — the two work together, and an unauthenticated endpoint is a risk even inside your own walls.
Least privilege & token hygiene
Least privilege is one idea: give every account and every integration the minimum it needs, and nothing more. Run this checklist on any private AI deployment.
Start everyone as a consumer
Default new accounts to the inference-consumer role and grant up only when a real need appears — not the other way around.
One identity per person
No shared logins. SSO ties every session to a named human, which is the whole point of an audit trail.
Scope API tokens narrowly
A token for an app or script gets only the permissions that app needs, not a full admin key copied around.
Expire and rotate tokens
Give tokens a lifetime, rotate them on a schedule, and revoke any that leak — long-lived secrets are the ones that hurt.
Revoke on offboarding
Removing the company account should remove AI access in the same step; check no local accounts survive it.
Review access regularly
Walk the role list every quarter and pull permissions nobody is using — privilege creeps quietly.
Logging access for audit
Roles decide who can do what; logs prove who actually did. Every login and request on a private AI server should be recorded on hardware you own — who accessed it, when, from where, and what they touched — in tamper-evident records with retention you set rather than a vendor sets for you. That is what turns "trust us" into evidence you can hand to a reviewer.
This is also where access control meets compliance. Encryption, access control, and audit logging are exactly the infrastructure controls that HIPAA, PCI, SOC 2 and CMMC programs expect to see. To be clear about the boundary: TIS builds and configures these controls; we are not auditors and do not certify your organization — that sign-off stays with your QSA, CPA, C3PAO, or counsel. Access logging also pairs with encryption at rest and in transit and with keeping data where you can point at it, covered under business data privacy.
We configure the roles and logins here in Texas
You do not have to design the role map alone. We set up RBAC, wire the server into your existing SSO, turn on MFA, and confirm the audit logging works — then install it on-site across Houston, Katy, Fulshear and the Fort Bend area and stay on call. Want it scoped first? Start with an AI readiness audit, then see our Texas service areas.
Access control questions
Why is "everyone is admin" a problem on a private AI server?+
Because a single compromised or careless account then has the run of the whole box — it can read every document, change the model, or wipe the logs that would have shown what happened. Role-based access control limits each person to what their job needs, so a mistake or a stolen login does far less damage.
What roles does a private AI server actually need?+
Four cover most deployments: an inference consumer who only asks questions, a prompt engineer who builds and tunes prompts and tools, a model admin who manages models and configuration, and a read-only auditor who can review access and logs without changing anything.
How does single sign-on work with a self-hosted LLM?+
The AI app trusts your existing identity provider over a standard protocol — OIDC or SAML — so people log in once with the company account they already have. There is no separate password to scatter or reuse, and removing someone in one place removes their AI access too.
Do I need MFA if the server never touches the internet?+
It is still worth it. An on-prem box reduces outside exposure but does not stop a stolen laptop, a shared password, or an insider on the LAN. Requiring a second factor on the login closes the unauthenticated-endpoint hole that self-hosted deployments most often leave open.
How is access to the AI logged for an audit?+
Every login and request is recorded on hardware you own — who accessed what, when, from where — with retention you set. Those tamper-evident records are the evidence that supports a HIPAA, PCI, or SOC 2 program; we configure the controls, your auditor does the certification.
Access control is one layer of the bigger picture — see the full local LLM security guide, or how the controls map to Texas AI compliance.
Make "who can do what" a decision, not an accident
Tell us who needs to use your AI and how — we'll map the roles, wire in your SSO and MFA, and turn on the logging, on a server you own outright.