Running an AI chatbot on your own server means your conversation history and configuration never touch a SaaS vendor's database, but the boundary is more precise than that phrase suggests.
Short answer: A self-hosted AI chatbot stores its conversation threads, documents, and credentials on your own Linux server and serves the chat interface from that same machine. The reasoning step, where a language model reads the conversation and produces a reply, still sends a prompt to an external model gateway unless you run a local open-weight model on dedicated hardware. That boundary determines your actual data exposure, and it is the one most guides skip.
How a self-hosted AI chatbot actually works
A chatbot node is not one system. It is four separable layers, and each has its own data boundary. Product pages compress all four into a single word, private, which is why the claim is hard to check. Separating them makes it answerable.
| Layer | What runs there | Where the data sits |
|---|---|---|
| Chat interface | The browser panel, served from your own machine | Served from your own server; conversation data is not written to any third-party host |
| Conversation storage | A local database on the same server | Threads, documents, and full history stay on your disk |
| Agent process | Isolated processes under their own system accounts | Reads and writes local data; credentials stay in the local secret store |
| Inference | A call to a model, either external or on the host | The prompt leaves the server through a gateway unless the model runs locally |
The first three layers are local by construction. The chat panel is delivered from your server, and the messages typed into it are stored on that machine. Threads, the documents the chatbot draws on, and every action the software takes are written to a database on your machine, and that database is not replicated to a vendor. The agent runs as an isolated process with its own system account and an explicit set of permitted tools, so one agent cannot reach another agent's mailbox or files. Credentials such as mailbox passwords, API keys, and database tokens sit in a local secret store and are resolved at the moment of use through an allowlist. They are never written into a model prompt.
The fourth layer is where deployments differ. To produce a reply, the agent assembles a prompt: the conversation so far, its own instructions, and any passages it retrieved from local Memory. If the model behind it is an external gateway such as Anthropic, Google Gemini, or OpenAI, that prompt leaves your server, is processed under the provider's commercial API terms, and returns as text. If the model runs locally on your own CPU or GPU, it does not leave. Nothing else changes: same storage, same tools, same event log, same gates. Egress to a gateway is explicit and metered by system configuration, so you can check it on your own node rather than trusting a description.
This page covers the configuration most people start with: gateway-backed inference on a modest Linux VPS. Self-hosted agents as a wider category are covered at self-hosted AI agents.
A worked example: one support question, end to end
A staff member opens the web chat panel on the company server and types: what is our refund window for annual plans?
The agent process on that server picks up the message, searches local Memory for the refund policy Document, and assembles a prompt from the question and the relevant policy excerpts. The prompt travels over the network to the configured model gateway, which processes it under the provider's commercial API terms and returns a generated reply. The agent writes the full exchange to the local event log, keeps it in the conversation thread, and sends the answer back to the chat panel.
Nothing in that sequence writes conversation data to a third-party database. The only external transit is the prompt on the way out and the generated text on the way back. The question and the retrieved policy excerpts form the payload; the thread, the log entry, and the credential that authorised the call stay on the server.
Run the same conversation on a node configured with a local model runner and enough CPU or GPU capacity, and only that middle step changes. The prompt is processed on the machine that stored it, and no payload leaves at all.
Steps to set up your own chatbot node
Six decisions, in this order. Each one stays configurable afterwards, but doing them in sequence avoids rework.
- Provision the server. Ubuntu or Debian, with 2 vCPU, 4 GB RAM, and 20 GB of SSD storage as the minimum for gateway-backed inference. That is a floor, not a ceiling: running the model locally needs CPU or GPU capacity sized to the weights you choose.
- Install the platform. Work through the prerequisites and installation runbook at /docs/install. The installer sets up the runtime, the local databases, and the process isolation that keeps each agent in its own system account.
- Define the chatbot agent. Give it a Role (what it is for and how it behaves) and attach a web chat Channel so the browser panel can reach it.
- Add the model gateway key to the secret store. The agent resolves it at runtime through an explicit allowlist. The key is never placed in a prompt and never written into a file the agent's process reads directly.
- Grant the tools the chatbot needs, and no others. Capability grants are declared per agent. An agent without a grant cannot call the tool, whatever the conversation asks it to do.
- Enable the human-in-the-loop gates. For a customer-facing chatbot, that means outbound email and any operation that changes records outside the agent's declared scope. Send a test message, then confirm the exchange appears in the event log before real traffic arrives.
Common mistakes
Mistake 1: treating self-hosted as air-gapped
Self-hosted means the storage, the runtime, and the configuration sit on your server. It does not mean the server never talks to the internet. In a gateway-backed deployment the inference step sends conversation context to a model provider, and that provider's data handling terms apply to those prompts. The egress is explicit and controlled by configuration rather than hidden, but if your data carries strict retention requirements, verify the provider's zero-retention API policy before you go live, or plan for local model inference from the start.
Warning: Self-hosting does not by itself satisfy HIPAA, GDPR, or SOC 2. Compliance depends on the whole infrastructure: configuration, access policies, retention practices, and the terms of any gateway you call. Do not describe a deployment as compliant until someone has reviewed those factors against your obligations.
Mistake 2: expecting self-hosting to remove the bill
One node serves the whole team with no per-seat licensing, and that is where the saving comes from. It does not remove the rest of the cost: hosting for the server, the hours spent administering it, and per-token charges from whatever model gateway you call. For a team currently paying per-seat prices, self-hosting usually wins at moderate volume, but the cost structure is different, not zero.
Warning: Deploying and administering a Linux server takes technical proficiency or support from an integrator. The platform runs on standard Linux and does not abstract server operations away.
Frequently asked questions
Does my chat data stay private when I self-host?
Conversation threads, documents, and credentials stay on your server. The prompt sent to an external model gateway during inference is the exception: that payload travels to the provider and is governed by the provider's commercial API terms. Everything before and after that step stays on your machine.
Can I run this on a cheap VPS?
Yes, when inference goes to an external gateway. 2 vCPU, 4 GB RAM, and 20 GB SSD is the stated minimum for that configuration. Running the model itself on the node needs hardware sized to the weights and is not practical on entry-level VPS instances.
Do I need a GPU to self-host an AI chatbot?
Not for the standard configuration. A GPU is needed only when you want the inference step to run on your own hardware instead of routing to an external gateway. Without one, the node still stores, retrieves, and logs everything locally; only the reasoning step travels.
Is a self-hosted AI chatbot GDPR compliant out of the box?
No. Self-hosting gives you control over where data is stored and who can reach it, but compliance depends on your full configuration, access controls, retention practices, and the terms of any external model gateway you use. Installing the software does not confer compliance.
How many users can one node serve?
There is no per-seat license, so the whole team can use one node. A maximum number of concurrent chat sessions is not published; practical capacity comes from the hardware you provision and the rate limits of the gateway or local runner behind it. The minimum specification above is a floor for gateway-backed inference, not a sizing estimate for a busy support queue.
What happens if the chatbot tries to send an email or delete a record?
Those actions stop at a human-in-the-loop gate. External messages, deletions, and privileged system operations require explicit confirmation before they execute, and the agent cannot route around that gate.
Set up your node
Deploy a node of your own and watch the boundary on your hardware: Set up your node. The installation prerequisites and the full runbook are at /docs/install.