Most guides on how to use AI for small business begin with the tool. The stall happens earlier. Owners who get stuck usually have no single task named, no honest account of what the setup costs and requires, and no way to tell whether the thing is working.
Short answer: Pick one repetitive internal task with defined inputs and outputs, configure a private agent for that task alone, and run it under human review before extending its responsibilities. A single node serves the whole team with no per-seat licensing fees, so cost scales with the server you rent and the model calls you make, not with headcount. Expect two to three weeks of supervised operation before output is consistent enough to reduce manual checking.
What you need before you start
Three things have to exist before installation, plus a decision about where inference runs. Missing any one of them stalls the project rather than failing it gradually.
- Name one task that already exists in your operation: one input type, one output type, currently done by a person. "Use AI for the business" is not a task.
- Rent or provision a Linux server running Ubuntu or Debian with at least 2 vCPU, 4 GB RAM and 20 GB SSD storage. That specification is enough while prompts route to an external model gateway.
- Obtain at least one external model gateway API key (Anthropic, Google Gemini or OpenAI) if inference will route off the server.
- Size dedicated CPU or GPU hardware to the model weights you intend to run if it will not. The minimum server cannot run local inference.
- Assign a person, in-house or a contracted integrator, who can install, update and monitor a Linux server. Self-hosting puts OS updates, host security, backups and recovery on that person.
Tip: The first task should run at least several times a week and take a person more than a few minutes each time. A monthly task will not produce enough runs to judge whether the agent is doing it correctly.
Steps
Order matters here. Skipping or reordering is the most common cause of an early failure, and that failure usually looks like a bad model rather than a bad sequence.
- Name the task precisely. A good first task is repetitive, runs several times a week, takes a person more than a few minutes each time and produces a document or a message as its output. Candidates that fit the channel and tool architecture include drafting replies to inbound email inquiries, summarizing supplier or client documents before a meeting, and routing support messages to the right person. You finish with one sentence that names the input and the output. For a wider map of what a private node can handle across an operation, see what a private node can handle.
- Provision the server. Install Ubuntu or Debian at the specification above. You finish with a node that starts and an administration interface you can reach from a browser. Confirm both before you touch any agent configuration.
- Create one agent with a named role, limited to the tools that task actually requires. Bounded tool grants are what make an agent's behavior predictable, not an obstacle to work around: an agent with fewer tools produces more consistent output. You finish with capability grants that match the task and nothing else.
- Connect the agent to the channel the task arrives on: web chat, email over SMTP and IMAP, or another supported interface. Send a message yourself and confirm it lands in the agent's thread. You finish with a verified inbound path.
- Turn on human-in-the-loop confirmation for every externally visible action. Sending external email, deleting records, publishing content, running privileged shell commands and spending money all pause and wait for explicit approval. You finish with a confirmation queue and a named person who clears it. Operate this way for the first two to three weeks.
- Decide whether to widen the role, and only after the agent has completed the task at least twenty times without a configuration error and your review of its output shows consistent quality. The decision is to reduce supervision on low-risk outputs or to add a second task. You finish with a deliberate choice instead of a drift.
How to check it worked
Four signals, all observable in the first week. None of them is a judgment about quality.
- The task completes end to end at least once: an input arrives on the configured channel and the agent produces an output you can review.
- Externally visible actions are waiting in the confirmation queue, not dispatched on their own. If nothing is queued after a task run, check the gate configuration before you assume the agent is being careful.
- The event log lists every action the agent took, in sequence, with timestamps. An empty log after a run means the task did not complete, whatever the chat window suggests.
- No API key, mailbox password or database credential appears in the agent's output or in the prompt context you can see. Credentials resolve through the secret store and are never placed in model-visible text.
Run this check on day one, day three and day seven.
What goes wrong and how to tell
The task is too wide
An agent asked to "handle customer communication" has no single definition of correct output, so neither do you. The visible symptom is a review queue full of answers to questions you never scoped.
Warning: If outputs in the queue do not match the task definition, the scope is the problem, not the model. Rewrite the role to name one input type and one output type, and remove every tool that specific task does not use.
Nobody owns the server
Self-hosting means the operator owns OS updates, host security, monitoring, backups and recovery. With no named owner, the node keeps running while it drifts out of date, which is the harder state to notice.
Warning: If the server has not received security patches in more than two weeks, it has no owner in practice. Assign a person or contract an integrator before going live, not after something breaks.
The team expects no human involvement
Human-in-the-loop gates pause externally visible actions until a person confirms them. This is a design constraint, not a defect, and a team that was not told about it reports that the agent is stuck.
Warning: If people say actions are stuck, those actions are sitting in the confirmation queue. Decide before launch who receives the queue and how quickly they are expected to clear it.
The team expects fully local inference on a minimum server
A 2 vCPU, 4 GB instance routes complex reasoning to an external model gateway. The context payload for those requests travels to that provider, under that provider's commercial API terms.
Warning: If prompts are reaching an external endpoint while the team believed all processing stayed on the server, there are two honest options: accept the hybrid model and its data path, or provision dedicated GPU hardware and configure local model inference. The minimum specification does not offer a third.
Frequently asked questions
Do I need a developer to set this up?
You need someone with the technical proficiency to administer a Linux server, and the platform does not remove that requirement. If nobody in the business has it, a managed service provider or an integrator can handle installation, updates and monitoring, but the responsibility is still yours to place.
Does my data go to OpenAI or Anthropic when I use this?
The runtime, the documents, the conversation history and the configuration database stay on infrastructure you own or rent. When a task routes a prompt to an external frontier model gateway for inference, that specific request payload crosses the network to that provider and is governed by the provider's commercial zero-retention API policies. Inference with no external calls at all requires dedicated local hardware sized to the model.
What does it cost to run for a team of ten people?
There are no per-seat licensing fees, so ten people on one node cost the same in licensing as one: nothing. What remains is the infrastructure you rent (a VPS at the minimum specification, or a GPU host if you run local models) and per-token fees on the external model gateway for each inference call. Server maintenance time is a cost as well, whether you pay someone or spend your own hours on it.
Can one node run more than one agent at the same time?
Yes. Multiple agents run on a single node, each with its own role, its own tool grants and its own channel configuration. Every agent runs as a separate isolated system process with a strict allowlist of the tools and secrets it may reach, so one agent cannot use another agent's credentials or read its mail.
Is this compliant with GDPR or HIPAA out of the box?
No. Installing the software does not produce compliance. Compliance depends on the infrastructure configuration, the access policies, the data retention practices and the administrative procedures around the system, which is why the data-boundary documentation is worth reading before you make a claim to a client or a regulator.
Start with one node
Start with one task on one agent and keep human review switched on for the first two to three weeks. Set up your node.
If you would rather verify the prerequisites first, the installation runbook lists what the host must provide, and the security architecture describes where data stays and where it leaves.