Encrypting a Private AI Server: At Rest, In Transit, and Where Keys Live
Encryption on a private AI server comes down to three plain questions: is the data scrambled on disk, is it protected as it moves across the wire, and who holds the keys? Get those three right and a stolen drive is unreadable, a tapped network shows nothing, and the keys stay in your hands. Here is how we set up each one on a TIS build — what it protects, what it honestly does not, and the spots people forget like vector stores and backups.
The three encryption questions that matter
People ask "is it encrypted?" as if it were one switch. It is really three. At rest is about data sitting on disk — protected if the drive is pulled or the box is stolen while powered off. In transit is about data moving across the network — protected from anything that can read the wire between the user and the server. Key custody is the part that actually makes the other two mean something: an unbreakable algorithm is worthless if the key is sitting in the open or held by someone else.
Answer all three honestly and you have real protection. Answer one and skip the rest and you have a false sense of it. The rest of this page walks each question in plain English, then shows the spots that get missed and how the keys are handled.
At rest — AES-256, full-disk vs. volume, and what it doesn’t protect
Encryption at rest scrambles the data stored on the server's drives so that a drive pulled out of the machine reads as noise. On Linux AI servers we do this with LUKS (the standard Linux disk-encryption layer), using the AES-256 cipher. That covers the model weights, your document store, the vector store, caches, and logs — everything written to disk.
There are two ways to scope it. Full-disk encryption protects the entire drive including the operating system, unlocked once at boot. Volume (or partition) encryption protects specific data volumes — useful when you want the data store encrypted separately from the OS. Both rely on AES-256 under LUKS; the difference is how much of the disk is in scope and when it unlocks.
What encryption at rest does not protect
This is the honest part most pages skip. Encryption at rest protects a powered-off or stolen drive. It does not protect a server that is running with the volume already unlocked — once the system is booted, the data is plain to the operating system and to anyone who can reach it through a logged-in session, a network service, or stolen credentials. So disk encryption is a layer, not the whole defense. It pairs with access control (RBAC, SSO & MFA) and network isolation, which guard the live, unlocked system.
In transit — TLS even on the LAN, because “internal” still needs it
Every prompt a user sends and every answer the model returns travels across your network. Encryption in transit protects that traffic with TLS — the same protocol that secures web traffic — between the client and the inference endpoint. Without it, prompts and responses move in the clear, readable by anything else sitting on the same network segment.
The common mistake is assuming "it never leaves the building, so it's safe." It isn't. A LAN is not a private channel — another device on the same segment, a compromised machine, or a misconfigured switch can see unencrypted traffic. So we use TLS on the inference endpoint even when the model is LAN-only. Internal traffic gets the same protection as anything crossing the public internet, because the threat of someone reading it does not disappear at your front door.
The spots people forget
Encrypting the obvious database is easy. These are the places your data quietly lands that an at-rest setup has to cover too — we check each one on a build.
Vector store
A RAG system stores embeddings of your documents in a vector database — that is your data, and it sits on disk like anything else. It must fall inside the encrypted volume, not outside it.
Caches
Inference caches and KV caches can write fragments of prompts and context to disk. If a cache lands on an unencrypted scratch volume, it is a leak; we keep it on the encrypted store.
Swap space
When memory fills, the system pages to swap on disk — which can include in-flight prompt or model data. Encrypted swap (or no swap) keeps that from becoming a plaintext copy.
Backups
A backup of an encrypted server is useless if the backup itself is plaintext. Backups get their own AES-256 encryption, whether they stay on-site or go offsite-but-still-yours.
Logs
Audit and application logs can capture prompt content or identifiers. They live on the encrypted volume and under the same retention you set.
Temporary files
Document uploads and conversion steps often write temp files. Those scratch paths point at the encrypted volume so nothing slips out the side door.
Keeping all of this in-house is the heart of business data privacy — and the resilience side, including encrypted backups, is covered in AI backup & disaster recovery.
Key management — where the keys live, rotation, and who holds them
AES-256 is only as good as the key behind it. This is the part that actually keeps encryption meaningful, and it comes down to three questions. Where do the keys live? On a TIS build, on hardware you own in your building — not handed to a vendor cloud where someone else controls access. Who holds them? A named, limited set of people you decide on, not "whoever set it up." How are they rotated? On a schedule, so a key that may have been exposed can be retired without re-encrypting from scratch.
The recovery side matters just as much. A disk encrypted with a key nobody can find is a self-inflicted disaster, so we document the unlock keys and recovery path up front — encryption you can't recover from is just data loss with extra steps. Where your organization answers to a framework, this maps directly to access-control and audit expectations; see AI data privacy & compliance in Texas for how the controls line up. For the hardware those keys protect, see our custom AI servers.
Encryption vs. the compliance frameworks
How encryption at rest and in transit maps to the frameworks a Texas business actually faces. These controls support your compliance program; they do not certify it. Your QSA, CPA, or counsel handles the assessment and sign-off, not TIS.
| Framework | At rest (AES-256 / LUKS) | In transit (TLS) | How the control helps |
|---|---|---|---|
| HIPAA Security Rule | Supports | Supports | Safeguards ePHI on disk and across the network; pairs with access control and audit logs you control. |
| PCI-DSS 4.0 | Supports | Supports | Helps protect stored and transmitted account data; the SAQ/QSA assessment is still done by your assessor. |
| SOC 2 | Supports | Supports | Feeds the Security/Confidentiality criteria as evidence; a licensed CPA performs the examination. |
| CMMC 2.0 / NIST 800-171 | Supports | Supports | Maps to the protection-of-CUI control families; the C3PAO or self-assessment handles certification. |
"Supports" means the control contributes to that framework's safeguards — it is not a claim of certification. Re-confirm current requirements with your assessor before relying on them; rules change.
We set up the encryption and hold the keys in your building
We configure LUKS at rest, TLS in transit, and document where the keys live and who holds them — then hand the unlock keys to you. We build and install across Houston, Sugar Land, Missouri City and the Fort Bend area, on hardware you own outright. See our Texas service areas.
Encryption questions
Is my data encrypted on a private AI server?+
Yes — we encrypt the disk at rest with LUKS using AES-256, and we use TLS for data in transit, including traffic that never leaves your LAN. That covers the model weights, your document store, the vector store, logs, and backups.
Does encryption at rest protect a running AI server?+
No. Encryption at rest protects a powered-off or stolen drive. Once the server is booted and the volume is unlocked, the data is readable to the running system and to anyone who gets access to it. That is why access control, network isolation, and logging matter alongside encryption — disk encryption is one layer, not the whole defense.
Do you still need TLS on a local network?+
Yes. "Internal" traffic can still be read by anything else on the same network segment, so we use TLS between clients and the inference endpoint even on the LAN. It closes the gap where prompts and responses would otherwise travel in the clear inside your building.
Where do the encryption keys live on a TIS build?+
On hardware you own, in your building — not in a vendor cloud. We set up key storage, document who holds the unlock keys, and plan rotation so a single lost key is recoverable. The custody question matters as much as the algorithm: AES-256 only protects you if the key is controlled.
Related: business data privacy, AI access control, and AI compliance in Texas · or read more on private AI infrastructure on the main site.
Want the data on your AI server actually protected?
We’ll set up AES-256 at rest with LUKS, TLS in transit, and key custody you control — built and installed on-site across Houston and Fort Bend County. No monthly-fee pitch.