BYOK LLM configuration¶

Hivemind is bring-your-own-key: it never proxies LLM calls through Seglamater. You point it at your own provider — Anthropic, OpenAI, Ollama (local), or any OpenAI-compatible endpoint (LM Studio, vLLM, LocalAI, Azure OpenAI, LiteLLM, …). Hivemind boots and runs without a provider configured; LLM-dependent surfaces (public chat, agent runtime tool-call loop, content studio drafting) stay degraded until you set one. The boot log makes the degraded state loud:

WARN  LLM startup ping FAILED — LLM-dependent features degraded; fix
      HIVEMIND_LLM_* env vars and restart  provider=anthropic
      endpoint=https://api.anthropic.com model=claude-haiku-4-5-20251001
      error="missing config: HIVEMIND_LLM_API_KEY (anthropic)" hint=""

The envelope¶

Every provider uses the same five environment variables; the values differ. Set them in your install dir's environment file and then sudo docker compose up -d server (the server picks them up on restart):

Variable	What	Default if unset
`HIVEMIND_LLM_PROVIDER`	`anthropic` \| `openai` \| `ollama` \| `openai-compat`	`anthropic`
`HIVEMIND_LLM_ENDPOINT`	Base URL (no trailing `/`)	provider-specific: `anthropic` → `https://api.anthropic.com`, `openai` → `https://api.openai.com`, `ollama` → `http://localhost:11434` (unreachable from inside the container — see "Common pitfalls" below; set `http://host.docker.internal:11434` explicitly), `openai-compat` → no default, required
`HIVEMIND_LLM_API_KEY`	Provider API token	unset → ping fails
`HIVEMIND_LLM_MODEL`	Provider-specific model id	provider-specific (see below)
`HIVEMIND_LLM_TIMEOUT_SECS`	Per-request timeout in seconds	`60`
`HIVEMIND_LLM_PACE_MS`	Inter-call gap (rate-limit politeness)	`400`
`HIVEMIND_LLM_RETRY_MAX`	Total attempts on transient upstream failure (`1` = no retry)	`3`
`HIVEMIND_LLM_RETRY_BASE_MS`	First backoff before jitter	`500`
`HIVEMIND_LLM_RETRY_CEIL_MS`	Backoff ceiling	`8000`

HIVEMIND_LLM_PACE_MS is a Tidal-style respectful inter-call gap, not a rate limit. It enforces a minimum of N milliseconds between successive calls to the upstream so a burst of agent tool-calls doesn't hammer the provider. The default 400 ms is conservative; tighten for local providers.

Transient-overload retry¶

Hosted providers (Anthropic especially) routinely return transient 429 rate_limited and 529 overloaded responses under load, plus the occasional gateway 5xx. Hivemind retries these automatically with bounded, full-jittered exponential backoff — by default 3 attempts, doubling from 500 ms up to an 8 s ceiling, honoring the provider's Retry-After header (clamped to the ceiling). A transient blip becomes a slightly slower but successful answer instead of a hard bot failure. Auth errors (401/403) and ordinary 4xx are not retried — those are configuration faults, not blips.

If every retry still fails, the customer-facing bot returns a clean "try again in a moment" (HTTP 503), never a misleading 500. Set HIVEMIND_LLM_RETRY_MAX=1 to restore the old single-shot behavior (no retry). Retries that fail carry no token usage, so cost-cap accounting is unaffected.

If your instance predates this release, setting the three RETRY variables above has no effect, and that is not something you can fix from .env. They were documented here before the server service declared them, so on an older instance they never reach the process and the built-in defaults (3 / 500 ms / 8 s) stay in force. This is a real gap we shipped, not a configuration mistake on your side — and it is worth being precise about why, because the obvious fix does not work:

docker-compose.yml is written once, by the invite bootstrap, at install time. Re-running the installer does not rewrite it, and a managed update rewrites only the pinned image digest. So a variable added to the shipped compose file reaches new installs and not existing ones. Putting the name in .env does not help either: Compose's .env feeds variable substitution inside the compose file, so a variable the service does not declare is never passed to the container.

Two ways to tell, and one way to fix:

Tell: docker inspect hivemind-server and look for the names in Config.Env. If they are absent, they are not reaching the process. From this release the managed updater also reports them as undelivered after an apply, rather than reporting a plain success.
Fix: re-run your invite bootstrap, which rewrites docker-compose.yml. Note that it overwrites that file, so keep a copy of any local edits first.

The retry behavior itself is on by default and needs no configuration — this affects only changing it.

Provider snippets¶

Pick one. Copy the four lines into the environment file under <install-dir>/, then restart the server.

Anthropic (Claude)¶

HIVEMIND_LLM_PROVIDER=anthropic
HIVEMIND_LLM_ENDPOINT=https://api.anthropic.com
HIVEMIND_LLM_API_KEY=sk-ant-…              # console.anthropic.com → Settings → API Keys
HIVEMIND_LLM_MODEL=claude-haiku-4-5-20251001

Default model is claude-haiku-4-5-20251001 if you leave _MODEL unset. For agent-runtime / content-studio work the haiku tier is the right default; bump to a sonnet tier (claude-sonnet-4-5-…) if you want richer reasoning at higher per-call cost.

OpenAI¶

HIVEMIND_LLM_PROVIDER=openai
HIVEMIND_LLM_ENDPOINT=https://api.openai.com
HIVEMIND_LLM_API_KEY=sk-…                  # platform.openai.com → API keys
HIVEMIND_LLM_MODEL=gpt-4o-mini

gpt-4o-mini is a reasonable default; pick whichever chat model your account has access to.

Ollama (local, no key)¶

HIVEMIND_LLM_PROVIDER=ollama
HIVEMIND_LLM_ENDPOINT=http://host.docker.internal:11434
HIVEMIND_LLM_MODEL=llama3.2:3b

No API key. Note the host.docker.internal hostname: the hivemind-server container needs to reach Ollama on the host. The shipped docker-compose.yml adds extra_hosts: - "host.docker.internal:host-gateway" for the server service so this resolves correctly on Linux Docker.

Pull the model on the host first:

ollama pull llama3.2:3b

For larger models (8B+) tighten HIVEMIND_LLM_PACE_MS if you find the agent loop is artificially slow:

HIVEMIND_LLM_PACE_MS=50

OpenAI-compatible (LM Studio / vLLM / LocalAI / Azure / LiteLLM)¶

HIVEMIND_LLM_PROVIDER=openai-compat
HIVEMIND_LLM_ENDPOINT=http://host.docker.internal:1234     # LM Studio default
HIVEMIND_LLM_API_KEY=                                       # leave blank for unauth local
HIVEMIND_LLM_MODEL=local-model                              # whatever your server reports

This path speaks the OpenAI Chat Completions API shape; it works against anything that does too. Common targets:

LM Studio — default http://host.docker.internal:1234, no key.
vLLM — http://<host>:8000/v1, no key by default.
LocalAI — http://<host>:8080/v1, key optional.
Azure OpenAI — https://<resource>.openai.azure.com/openai/deployments/<dep>, key required. Set HIVEMIND_LLM_MODEL to the deployment id.
LiteLLM proxy — http://<host>:4000, key per your LiteLLM config.

Verify¶

After you set the four vars and restart the server, look for these INFO lines in the boot log (no WARN LLM startup ping FAILED):

sudo docker compose logs --tail=30 hivemind-server | grep -E "LLM|listening"
# INFO  hivemind_server: BYOK LLM provider initialised provider="anthropic"
#       endpoint="https://api.anthropic.com" model="claude-haiku-4-5-20251001"
# INFO  hivemind_server: listening addr=0.0.0.0:3000

If you see the WARN LLM startup ping FAILED line instead, the error message names what's wrong (most often a missing key or a wrong endpoint). Re-edit the environment file and sudo docker compose up -d server — provider init runs at every server start, so you'll see the new attempt on the next restart.

Switching providers¶

Change the four variables in the environment file and restart the server:

sudo docker compose up -d server

There is no on-disk LLM-provider state to migrate. Agent profiles + the chat history stay; the model that backs them just changes. Re-run any agent-quality checks you have after the switch — output quality differs across providers and especially across model tiers.

Common pitfalls¶

Wrong endpoint path. HIVEMIND_LLM_ENDPOINT is the BASE URL — no trailing /v1, no path. Hivemind appends the right path per provider shape. Setting https://api.openai.com/v1 makes the ping fail.
Model name doesn't exist on the account. The startup ping is a real call against the model; if your account doesn't have access, the ping errors. Use a model id you've confirmed in the provider's console.
Local provider unreachable from the container. Ollama / LM Studio on the host requires host.docker.internal (Linux Docker honors this via the host-gateway extra_hosts entry the shipped compose includes). localhost from inside the container points at the container itself, not the host.
Aggressive HIVEMIND_LLM_PACE_MS=0. You can set it to zero — but most cloud providers will rate-limit you. The default 400 ms is the polite floor; lower it deliberately and watch for 429s.
Timeout too tight for local large models. Local 70B+ inference on modest hardware easily exceeds 60 s for the first ping. Bump HIVEMIND_LLM_TIMEOUT_SECS=180 if you're seeing timeout errors on a local provider that you know is working.

If the ping keeps failing after the obvious fixes, troubleshooting.md has the "missing HIVEMIND_LLM_API_KEY" entry with the exact log line + fix pattern.