If you are looking for the best self-hosted AI platform, you will find comparisons of features. This page compares something else: the eight architectural properties that decide whether a deployment keeps the promises the word "self-hosted" implies. They are ranked by how often the real implementation falls short of the vendor claim, so the first items are the ones worth checking hardest.
Short answer: The three most commonly misrepresented properties are data boundary clarity, credential isolation, and human approval gates on irreversible actions. A platform can keep its storage and runtime entirely on your server while routing every reasoning request to an external cloud model, and those are separate architectural properties that a vendor has to state explicitly. How they interact is covered in the pillar article on self-hosted AI agents.
| Aspect | Hybrid deployment | Fully local deployment |
|---|---|---|
| Runtime, documents and event log | On your server | On your server |
| Where inference runs | External frontier model gateway | Local model weights on your CPU or GPU |
| Minimum hardware | 2 vCPU, 4 GB RAM, 20 GB SSD | Compute sized to the model weights you choose |
| What leaves the server | The context payload of each external request | Nothing, provided no outbound adapters exist |
| Recurring costs | Hosting plus per-token gateway charges | Hosting plus hardware |
Warning: "Self-hosted" describes where the runtime and the data live, not where the reasoning happens. A hybrid node can hold every document, conversation and event on your server while sending the context of each reasoning request to an external provider. If a vendor uses the two terms interchangeably, ask which one they mean.
The eight properties, most useful first
- Data boundary clarity. Storage and runtime can sit entirely on your server while every reasoning request routes to an external cloud model. Those are separate properties, and a vendor describing only one has told you half the architecture. In a hybrid deployment the specific context payload of each reasoning request travels to the provider's API under that provider's commercial zero-retention terms, and fully local inference removes that egress but requires dedicated CPU or GPU hardware above the base specification. What to ask: "Does the context payload leave my server when an agent reasons, and if so, to which endpoint, under which data policy?"
- Credential isolation from model context. API keys, mailbox passwords and database tokens should never appear in the context sent to a model, whether that model runs on your host or at a provider. On a correctly built node, credentials stay in a local secret store, agents resolve them through local tool wrappers over authenticated local channels under strict per-agent allowlists, and the model receives only the result of the tool call. If a provider logs or retains a context payload, no raw credential is in it. The test to apply: can an agent complete an external API call without the credential appearing anywhere in the reasoning context?
- Human approval gates on irreversible actions. Sending external email, deleting records, publishing content, executing privileged shell commands and spending money should pause for explicit human confirmation, enforced by the platform rather than requested in a prompt. Agents run under bounded autonomy: they can prepare an irreversible action and propose it, but they cannot execute it alone. This is the category list to check a platform against, because a gate that covers email but not deletion is not a boundary.
- Headcount-independent licensing. Self-hosting is meant to remove per-seat fees, so one node should serve a team of five and a team of fifty at the same licence cost. What remains is infrastructure: server hosting, operating system maintenance, and per-token charges when a task calls an external model gateway. Those are infrastructure-sized costs that scale with usage rather than with how many people log in. Ask directly: "Does the price scale with users, or with server usage and API volume?"
- Which of the two deployment kinds you are buying. Hybrid mode keeps runtime and storage local while inference runs remotely, and it fits modest hardware: 2 vCPU, 4 GB RAM and 20 GB SSD. Fully local mode keeps everything on the server and removes external payload egress, but needs dedicated CPU or GPU compute sized to your chosen model weights. On a single node, different agents can be pointed at different models, so routine triage uses a small model while reasoning-heavy roles use a larger one. Ask which mode a vendor defaults to, because the hardware bill and the data boundary both follow from the answer.
- Per-agent process isolation with bounded tool grants. Each agent should run in its own isolated process with its own explicit list of permitted tools, granted through a capability allowlist and executed through Model Context Protocol servers. An agent that shares a runtime with others, or that can call any tool simply by asking, cannot be given least privilege in any meaningful sense: there is nothing to bound. Ask: "Does each agent have its own process and its own explicit tool list, or do agents share a runtime and call tools on request?"
- Multi-agent task coordination. A chat assistant handles one conversation at a time. A multi-agent platform runs parallel workstreams, with inter-agent messaging over a Message Bus, work assigned and tracked as tasks, and deterministic Workflows for processes that must run the same way every time. Without those three, "multi-agent" usually means several chat windows. The distinction matters when your operations involve handoffs, dependencies and audit requirements, and it does not matter at all if you only need answers to questions.
- Channel coverage. Agents only help on the channels where the work arrives, so list your own first: browser web chat, internal messaging between agents, SMTP/IMAP email, Telegram bots, and voice through the Gemini Live API. Channel adapters sanitize inputs and map conversations to persistent threads, which means an agent answering an email works on the same task and the same document as one answering a web chat message. Compare the platform's coverage against your live channels, not against a generic feature list.
What to do first if you only do one thing
Tip: Ask the vendor to trace one request. From the moment an agent starts reasoning to the moment the result returns, where does the context payload travel, to which endpoint, and under which data policy? The hybrid inference boundary is the property most often left vague. A vendor who cannot answer that question has not given you data boundary clarity, whatever the sales page says.
Frequently asked questions
Is self-hosted AI actually private if it uses OpenAI or Anthropic?
Partly, and the split is architectural rather than promotional. Runtime, storage, documents, the event log and credentials stay on your server. When an agent is configured to use an external frontier model, the context payload of that request goes to that provider's API under their commercial zero-retention terms. The remaining question is how clearly the platform declares that boundary, and whether you provision local inference instead.
Do I need a GPU to run self-hosted AI?
Not in hybrid mode. A node that uses remote model gateways for inference runs on a minimum of 2 vCPU, 4 GB RAM and 20 GB SSD. Dedicated GPU hardware, or CPU compute sized to the model, is required only when you want fully local inference with no external payload egress.
How much does self-hosted AI cost compared to a SaaS subscription?
Self-hosting eliminates per-seat subscription fees, not costs. What remains is server hosting, operating system maintenance and backups, and per-token charges for calls to external model gateways. The comparison turns in favor of self-hosting once per-seat fees for your team exceed the combined infrastructure and gateway spend, which usually happens as headcount grows.
Can self-hosted AI run with no internet connection?
Only in a configuration with exclusively local model weights and no outbound network adapters. Hybrid deployments need network access for the model gateways, and any platform using email, Telegram or cloud voice needs it for those channel adapters. Air-gapped operation is a specific configuration you build and verify, not a default you receive.
What technical skills does self-hosted AI require to run?
At minimum, someone who can administer a Linux server on Ubuntu or Debian: operating system updates, security patches, backups, monitoring and recovery. The platform runs agents and workflows; it does not maintain the host for you. In-house Linux administration or a managed service provider is a practical operational requirement, not an optional extra.
Set up your node
If these eight properties match what your deployment needs, the next steps are hardware prerequisites and the installation runbook. Get started to provision a node, or read the installation prerequisites before you choose hardware. The security architecture documents the boundaries described above in detail.