The server where an AI agent runs determines what data leaves your infrastructure, what the monthly bill looks like, and what technical obligations fall on your team.
Short answer: AI agent hosting comes in three configurations: a vendor's shared cloud, a virtual private server running your own runtime, or dedicated hardware with private model inference. Each one draws a different data boundary and carries different cost and operational requirements. This guide explains how each configuration works, walks through a concrete deployment, and gives you the steps to match one to your situation.
How AI agent hosting works
| Aspect | Vendor cloud | Self-hosted runtime, external model API | Self-hosted runtime, local inference |
|---|---|---|---|
| Where the runtime lives | The provider's shared infrastructure | Your server or VPS | Your server or dedicated hardware |
| Where stored data lives | The provider's storage | Your server: local databases, document repository, memory store | Your server: local databases, document repository, memory store |
| Where inference runs | The provider's infrastructure | An external model gateway such as Anthropic, Google Gemini, or OpenAI | A local model runner on your CPU or GPU |
| Who manages operations | The vendor | You, or a managed service provider you appoint | You, or a managed service provider you appoint |
| Cost structure | Subscription, commonly priced per seat | Infrastructure hosting plus per-token gateway fees | Infrastructure hosting plus hardware sized to the model weights |
| Minimum compute | None on your side | 2 vCPU, 4 GB RAM, 20 GB SSD | Dedicated CPU or GPU sized to the chosen model weights |
Read that table by row, not by column. Where the runtime lives tells you who patches the operating system. Where stored data lives tells you who holds the backups. Where inference runs tells you what leaves the building.
A self-hosted runtime with an external model gateway keeps your documents, task queue, conversation history, and configuration on a machine you own or rent, and still sends the context payload of each reasoning request to Anthropic, Google Gemini, or OpenAI. That egress is explicit, metered, and controlled by system configuration rather than incidental. Only the third configuration changes the boundary: with local model weights on your hardware, no external gateway configured, and no outbound network adapter active, reasoning context stays on the server. In both self-hosted configurations, data held in local databases, document repositories, and the memory store is never used to train third-party public models. The architecture behind configurations two and three is covered in more depth on the platform overview at /self-hosted-ai-agents.
What a self-hosted deployment looks like in practice
A six-person operations team rents a VPS at the minimum specification: 2 vCPU, 4 GB RAM, 20 GB SSD, running Ubuntu. They deploy a single node on it and route complex reasoning to an external frontier model gateway.
Documents, task history, conversation threads, and credentials all live on that VPS. Credentials sit in a local secret store, and agents resolve secret references through a strict allowlist at the moment of use. The API key for the model gateway and the password for the mail account are never inserted into a model prompt.
The team connects its staff to the same node. Headcount does not change the licensing cost, because there is no per-seat fee.
One agent is assigned to monitor a shared inbox and draft replies. The agent runs a reasoning step, and the context payload for that request goes out to the external model gateway under that provider's commercial zero-retention policy. The draft comes back. Before anything is sent, the agent stops at a human-in-the-loop gate: sending an external email requires explicit confirmation. The same gate applies to publishing content, deleting records, executing privileged shell commands, and any action that spends money.
Two details make that promise checkable rather than aspirational. Each agent runs under its own isolated Linux system account, so the agent watching the inbox cannot read the files or secrets of an agent handling payroll unless that access was granted. Every action, state change, and system event is written to an append-only event log, so the person reviewing a draft can see what the agent did before asking for approval. Agents coordinate through an internal Message Bus and a shared task list, and multi-step work runs as a deterministic Workflow with defined states.
Steps to choose and configure your hosting
- Define your inference boundary requirement. Decide whether it is acceptable for the context payload of a reasoning request to transit to an external model gateway under that provider's commercial zero-retention policy, or whether all reasoning must stay on hardware you control. You finish this step with a yes or no answer, not a preference.
- Size the compute. If external gateway routing is acceptable, a standard VPS at 2 vCPU, 4 GB RAM, and 20 GB SSD runs the runtime. If all reasoning must stay local, provision dedicated CPU or GPU hardware sized to the model weights you intend to run. You finish this step with a machine specification.
- Assess your operational capacity. Self-hosting on Ubuntu or Debian means you or someone you appoint handles host security, OS updates, monitoring, backups, and recovery. You finish this step knowing whether that capacity exists in house or needs a managed service provider.
- Map the answers to a configuration. External gateway acceptable plus standard operational capacity points to a VPS with a remote model API. Local inference required, or gateway transit unacceptable, points to dedicated hardware with a local model runner. If neither is viable, evaluate a vendor cloud product instead. You finish this step with one configuration chosen and the reasons recorded.
- Install the runtime. Follow the prerequisites and installation runbook at /docs/install. You finish this step with a node running and a first agent to configure.
Common mistakes
Warning: A self-hosted runtime with an external model API is not an air-gapped system. When an agent performs a reasoning step, the context payload for that request traverses the network to the external model provider. This is not a defect; it is how frontier model access works. The question to answer is whether that provider's zero-retention commercial policy meets your requirements, not whether the traffic exists.
Most compliance misjudgments in this area come from treating "self-hosted" as one thing. It is two things with different data boundaries, and the difference is inference, not installation.
Before you commit to a configuration, verify:
- All model providers in your configuration publish and enforce a zero-retention API policy, checked before you store sensitive content.
- You are not treating software installation alone as regulatory compliance. GDPR, HIPAA, and SOC 2 depend on the full infrastructure configuration, access control policies, data retention practices, and administrative procedures.
- Your budget covers infrastructure hosting and per-token model gateway fees. Per-seat subscription fees are eliminated, but infrastructure and inference costs are not.
- Your team has Linux server administration capability, or an integrator relationship, before you commit to a self-hosted configuration.
Frequently asked questions
Does self-hosting mean my data never leaves my server?
No, not by itself. It depends on where inference runs. With an external model gateway, stored data stays on your server while the context payload of each reasoning request goes to the provider. With local model weights, no external gateway configured, and no outbound network adapter active, reasoning context stays on your hardware.
What does it actually cost to host AI agents yourself?
Infrastructure hosting plus inference. On a VPS with an external gateway, that means the monthly server cost and per-token gateway fees. On dedicated hardware with local inference, that means the server or GPU sized to the model weights you run. Per-seat subscription fees disappear, but no configuration reaches zero operational cost.
Can I run an AI agent on a cheap VPS?
Yes, for the runtime, if reasoning routes to an external model gateway. The base node needs 2 vCPU, 4 GB RAM, and 20 GB SSD. The same VPS will not run high-accuracy local models; fully private inference needs dedicated CPU or GPU compute sized to the model.
What technical skills do I need to self-host AI agents?
Linux server administration on Ubuntu or Debian, or a relationship with a managed service provider. The hosting organization is responsible for host security, OS updates, monitoring, backups, and recovery procedures. Zero technical expertise is not a supported configuration.
Is self-hosted AI automatically GDPR or HIPAA compliant?
No. Installing the software does not make you compliant. Compliance depends on your overall infrastructure configuration, access policies, data retention practices, and administrative procedures, and on how you have configured the inference boundary described above.
What happens to my data if I use an external model like Claude or GPT?
The specific context payload sent in that request traverses the network to that provider, and is governed by the provider's commercial zero-retention API policy. Confirm that policy directly with each provider before sending sensitive content. Credentials are a separate case: they stay in the local secret store, are resolved by the tool execution layer through an allowlist, and are never exposed in prompt context.
How many people can use one self-hosted AI agent node?
A single node serves an entire team, with no per-user licensing. The practical limit is compute and configuration, not headcount: what the agents are asked to do, which models they route to, and how much concurrent work they handle.
Next step
The next step is to review the technical requirements for the configuration you chose and deploy a node.