Every organization using AI faces the same question it cannot defer: where does the data go, who can see it, and what record exists of what the AI did. For a business, that question is the practical content of compliance for private AI, and it is settled by configuration rather than by a vendor's assurance.
Short answer: Self-hosted AI moves the orchestration runtime, task queue, documents, and conversation history onto infrastructure you own or rent, which changes where the compliance questions land. It does not produce compliance by itself: you still have to set agent access boundaries, decide how the model inference boundary handles regulated data, and write down your data flows. The seven steps below are that work, in the order that makes each one checkable; for a broader foundation, see private AI for business.
Compliance concerns at a glance
Six questions decide most compliance reviews. The architecture answers five of them by design; the sixth is a decision only you can make.
| Compliance concern | What the architecture provides |
|---|---|
| Where is our data stored? | Single-tenant: the runtime, storage, and conversation history run on infrastructure you own or rent. |
| Will the AI vendor use our data for training? | No. Data in local storage is not used to train third-party public models. |
| Are credentials exposed in the AI context? | No. API keys and passwords stay in the secret store and never appear in model prompts. |
| Is there an audit trail of AI actions? | Every agent action is written to an append-only event log. |
| Who approves irreversible actions? | Human-in-the-loop gates require explicit confirmation before any external message, deletion, or privileged command. |
| What about calls to external AI model services? | Prompt context sent to an external gateway crosses that provider's network. This boundary requires deliberate planning. |
The trade is explicit: the data boundary moves onto your infrastructure, and the operational work of running it moves with it.
What you need before you start
Compliance configuration begins before installation, not after it. Arrive with these five things decided, because the software cannot infer any of them.
- Identify which regulations apply to your business and your data: GDPR, HIPAA, and PCI DSS are the ones organizations usually arrive with.
- List the data categories your agents will process: personal data, health records, financial data, internal documents.
- Confirm a Linux server or VPS meeting the minimum profile of 2 vCPU, 4 GB RAM, and 20 GB SSD for a node that uses remote model gateways.
- Name the person or integrator responsible for the host. Operating system updates, monitoring, backups, and recovery are yours, not the platform's.
- Draft an access control policy and a data retention policy you can map onto agent roles.
You leave this stage with a written scope: the regulations in play, the data categories, the server, the owner, and the two policies the rest of the configuration will be checked against.
Steps: configure for compliance
- Scope which data categories your agents will handle, and which regulation covers each one. List the categories first (personal data, health records, payment data, internal documents), then attach a regulation to each. You finish with a scope document and a short "agents may not touch" list for anything you could not classify.
- Deploy on infrastructure you own or rent. The orchestration runtime, the task queue, documents, conversation history, and the configuration database run exclusively on hardware you own or rent, on a Linux server or VPS running Ubuntu or Debian. That single-tenant property lets you answer "where is the data" with a location instead of a policy statement. You finish with a named home for every category from Step 1.
- Grant each agent the minimum set of tools its role needs. Agents run as separate operating system users in isolated processes, under declared scopes and least-privilege tool grants. An agent's reach is the union of the tools granted to it, so an agent that drafts customer replies holds no tool that reads payroll or deletes records. You finish with one tool list per role, where every entry is justifiable in a sentence.
- Decide which agents may call external model gateways, and which may not. The platform uses a hybrid inference model: runtime and storage stay on your server, but complex reasoning frequently routes prompts to external frontier LLM gateways such as Anthropic, Google Gemini, or OpenAI unless you provision local CPU or GPU inference. When a request goes to an external gateway, the context payload of that request crosses the network to that provider. Make the decision per role for every agent that touches regulated data, and write it down. You finish with a permitted list: which agents may use external gateways, and which data categories each may include in a request.
- Turn on human-in-the-loop gates for every irreversible or externally visible action. Sending external email, publishing content, deleting records, executing privileged shell commands, and spending money all stop and wait for explicit human confirmation. This is the mechanism that keeps an automated action from becoming a compliance event with no record of who authorized it. You finish with a system where every action that leaves your server has a named human behind it.
- Move every credential into the secret store. Third-party API keys, mailbox passwords, and database tokens belong in the local secret store, where agents resolve them through authenticated local sockets under a strict allowlist. Those values are never injected into model prompts, so a hostile instruction embedded in a document has nothing to read: the credential was never in the context to begin with. You finish with agent conversations and documents that contain no credential values.
- Confirm the append-only event log covers every agent that handles regulated data. All system events, state mutations, and agent actions are written to an append-only event log, ordered by time and not editable after the fact. This is the layer an auditor or an incident review asks for. You finish able to answer "what did the AI do on 3 March, and in what order" from a record rather than a reconstruction.
How to check it worked
Four tests. Run them before go-live, then repeat them on the interval your retention policy sets.
- Open an agent conversation and search it for a credential value. Nothing should match.
- Pick three actions an agent took today and find each one in the event log with its timestamp.
- Trigger a test that would send an external email. The system should present a confirmation request and send nothing until a person approves.
- Read each agent's tool list against its role description. Every tool listed should be one the role actually needs.
If a test fails, treat it as a configuration gap rather than a software defect: each of the four maps to a setting you control.
What goes wrong and how to tell
Three failure modes undo a compliance configuration, and each has an indicator you can check without special tooling.
Regulated data reaches an external model gateway without review. The indicator: an agent that handles regulated data is configured to use an external model gateway, and nobody has audited what that agent can read.
Warning: Before go-live, list every agent configured to use an external model gateway and audit exactly what data those agents can access. In a hybrid deployment, this boundary is the primary compliance risk.
The installation is treated as compliance complete. The indicator: once the node is live, there is no documented data flow map, no access policy matched to agent roles, and no written retention rule.
Warning: If you cannot describe in one document which agent accesses which data and under which policy, the compliance configuration is not complete, whatever the software provides.
Agent tool grants are broader than the role requires. The indicator: an agent's tool list contains tools its declared role never needs.
Warning: A tool grant that exceeds the role is a standing permission. The agent keeps it for every later task, so review tool lists when a role changes, not only at installation.
Frequently asked questions
Does installing this platform make us GDPR compliant?
No. Installing software does not create regulatory compliance, and no self-hosted platform can hand it to you. The platform supplies the technical mechanisms: single-tenant storage, process isolation, credential isolation, human approval gates, and an append-only event log. Your compliance position depends on the infrastructure configuration, access policies, retention practices, and administrative procedures built around those mechanisms.
Where does our data go when an agent uses an AI model?
Data at rest stays on your server. Documents, conversation history, configuration, and the Memory used for retrieval are stored locally, and none of it is used to train third-party public models. When a task calls an external model gateway, the context payload of that request does travel over the network to that provider and is governed by that provider's commercial API data policies. You decide which agents and which data categories are permitted to cross that boundary.
Can we run private AI with no data going to external AI services at all?
Yes, and it changes the hardware bill. Full local inference runs on host CPU or GPU with local model runners, sized to the model weights you choose, which is well beyond the 2 vCPU and 4 GB RAM minimum that covers deployments using remote gateways. Nothing leaves the server only when local weights are the sole inference path and outbound network access is closed; a local-model deployment with an open outbound network is a different statement.
What does an append-only event log give us for compliance?
A time-ordered record of what each agent did and in what sequence, written so that earlier entries are not rewritten later. Every system event, state mutation, and agent action goes into it, which is the evidence layer for audit review, incident investigation, and a regulator's question about AI behavior on a specific date.
Who is responsible for keeping the server secure and patched?
You are, or the integrator you contract. Host security, operating system updates, monitoring, backups, and recovery procedures sit with the organization running the node; the platform does not manage the underlying host. Budget for that work before go-live, because it is the part a self-hosted deployment does not remove.
Next step
When you have mapped your data flows and chosen your deployment model, set up your node to begin the configuration.