Operator notes

A short reference for operators running a Hivemind instance. This page will grow as we hit more real-world ops scenarios — if you trip over something not covered here, email support@seglamater.com so we can add it.

Updating

Hivemind self-updates against the channel set in /srv/hivemind/.env (HIVEMIND_UPDATER_CHANNEL=canary is the default during alpha). When a new release lands, the server polls updates.seglamater.app/hivemind/channels/<channel>/latest.json, verifies the signed manifest, pulls the pinned image digest, and performs a rolling restart.

To force a check immediately:

docker compose exec hivemind-server hivemind-cli update --check

To pin to a different channel:

# In /srv/hivemind/.env
HIVEMIND_UPDATER_CHANNEL=stable    # once stable is populated

Then docker compose up -d to apply.

During alpha, only canary is populated

beta and stable channels exist but currently have no releases. Pin to canary until that changes.

Backups

Hivemind stores all durable state in PostgreSQL. Back up:

The Postgres data volume (hivemind-db-data by default).
/srv/hivemind/.env — contains DB password and signing keys.

A typical nightly backup:

docker compose exec -T hivemind-db \
    pg_dump -U hivemind hivemind | gzip > /var/backup/hivemind-$(date +%F).sql.gz

Restore is symmetric — see the email/ticket from support@seglamater.com if you need the full DR runbook.

Logs and health

docker compose ps
docker compose logs --tail=200 hivemind-server
docker compose logs --tail=200 hivemind-db

The server exposes a health endpoint at /api/v1/health (returns JSON). Reverse-proxy health checks should hit this path.

Where things live

Path	Contents
`/srv/hivemind/.env`	Runtime config + secrets. Back this up.
`/var/lib/hivemind/`	Compose stack working dir + volumes.
`/var/log/hivemind/`	Docker log overflow if size limits hit.

Bots, knowledge, and integrations

/bots is the surface where you mint, edit, and connect your bots — each bot is one agent profile plus optional per-bot knowledge plus an optional channel integration. Day-to-day operator work lives here.

Creating bots

Use + New bot for both flows:

From a template (recommended starting point) — pick from the bundled catalog (Customer support, Sales SDR, Internal IT FAQ, HR FAQ, Product docs). The bot's prompt, tier, and starter knowledge come from the template; you can override the prompt before submitting or edit it later.
Blank slate — leave the template dropdown on "(Blank — write your own prompt)" and fill in name, slug, security tier, and system prompt yourself. Useful when none of the templates fit your business shape.

Slugs are URL-safe (lowercase letters, digits, hyphens) and become part of the API path + the chatalot bot username if you connect that integration. They can't be renamed after creation today — pick deliberately.

Editing bots

Click any bot on the /bots list to open its detail page, which has four tabs:

Tab	What you do there
Overview	See the bot's profile, model, cost cap, rate limits
Prompt	Edit the system prompt; save persists immediately
Knowledge	Add, view, delete per-bot knowledge snippets (RAG corpus)
Integrations	Connect / test / disconnect channel integrations (chatalot)

The Knowledge tab's snippets are searched at chat time via Postgres full-text search and the top relevance-ranked hits are injected into the system prompt. The bot's knowledge_search_enabled flag flips on automatically when you add the first snippet.

Treat knowledge as published content. It goes to the LLM verbatim and is stored in your database — do not paste API keys, customer PII, or internal credentials.

Integrations

The Integrations tab on a bot's detail page lets you connect that bot to an external channel. Today the only supported integration is chatalot; more channels are on the roadmap. The connect flow takes a bot:provision-scoped token for your chatalot instance (one-time, not persisted — not an admin JWT), provisions a bot user + scoped token, encrypts the token at rest, and starts the live message listener.

Make the bot discoverable — add it to a community. Chatalot scopes user search to community peers, so a bot in no community is invisible to everyone but the instance owner. In the Connect modal, Load communities and pick a default community: Hivemind adds the bot to it on connect (auto-joining its public channels) so users can find and message it. If you skip it, the bot connects but stays unsearchable until you add it to a community (Connect modal on a re-connect, or your chatalot admin panel). A "the bot connected but nobody can find it" report means it has no community membership.

Where the bot replies — channel_reply_mode. Each profile chooses its reply surface:

dm_only (default) — 1:1 direct messages only.
channel_mention — channel messages, only when the bot is @-mentioned by username; DMs ignored.
both — DMs and channel @-mentions.

There is no self-service API/UI for this yet (tracked in HIVE-106); set it directly in the database:

UPDATE agent_profiles SET channel_reply_mode = 'both' WHERE slug = '<slug>';

In a channel the bot answers only when directly @-mentioned — it never auto-responds to general chatter, and @everyone/@here/ @channel do not trigger it. A "the bot doesn't respond when I @-mention it" report on a dm_only profile is expected; switch the profile to channel_mention or both.

See Chatalot integration for the full operator + integrator surface (connect / test / disconnect / send tools / audit log).

Rotating your admin API key

The admin API key is what hm CLI calls, daemons, and MCP integrations use to authenticate against /api/v1/*. The key is created during install and stored:

On the server, in the users.api_key column for the bootstrap admin user (also surfaced as ADMIN_API_KEY in /srv/hivemind/.env if your installer wrote it there).
On any workstation that calls the API, typically in ~/.config/hivemind/env as HIVEMIND_API_KEY=....

You'll need to rotate it when:

You redeploy onto a fresh database (drops the old user row).
You migrate Hivemind to a different host and the workstation env still points at the previous instance.
You suspect the key has been disclosed.

The rotation procedure (until a hm admin-key rotate subcommand lands — tracked):

Generate a new random key on the server:
```
NEW_KEY=$(openssl rand -hex 24)
```

Update the bootstrap admin's row:

docker compose exec -T hivemind-db psql -U hivemind hivemind -c \
    "UPDATE users SET api_key = '$NEW_KEY' WHERE role = 'admin' AND email = 'you@example.com';"

Update /srv/hivemind/.env if ADMIN_API_KEY lives there, then docker compose up -d (a restart is only needed if the server reads ADMIN_API_KEY from env at boot — usually no).

Update ~/.config/hivemind/env on every workstation that calls the API:

sed -i "s/^HIVEMIND_API_KEY=.*/HIVEMIND_API_KEY=$NEW_KEY/" ~/.config/hivemind/env

Verify:

curl -fsS -H "X-API-Key: $HIVEMIND_API_KEY" $BASE/api/v1/me
# returns your user JSON if the key is good

Symptoms of a stale key:

Response	Meaning
`401 {"error":"missing X-API-Key header"}`	Header isn't reaching the server (proxy stripped?)
`401 {"error":"invalid or inactive API key"}`	Header arrived, key doesn't match a user row

If you see "missing X-API-Key header" via your reverse proxy but the header arrives correctly when you call the container directly, something in your proxy chain is dropping it — check Caddy / Traefik / nginx config for header-passthrough rules.

Tailscale + Hivemind hostname resolution

If you run your install behind Tailscale and use a non-public hostname (e.g. hivemind.qlab resolved by an internal pi-hole), note that Tailscale's MagicDNS overrides /etc/resolv.conf on client devices. Workstations on the tailnet will get NXDOMAIN for your internal hostname unless one of these is true:

The Tailscale admin console has Split DNS configured for your hostname suffix, pointing at your internal DNS server (cleanest).
Your internal DNS server is set as a Global nameserver in the Tailscale admin console (routes ALL DNS through it — only do this if your internal DNS forwards everything else upstream).
The workstation has a static /etc/hosts entry — quick local workaround, doesn't scale across devices.

Symptom: dig hivemind.example.com returns nothing on the workstation but works fine on a non-tailnet device on the same network. /etc/resolv.conf shows nameserver 100.100.100.100 (MagicDNS).

Reaching out

Bug reports / ops help: support@seglamater.com
Roadmap / sales: sales@seglamater.com
Status page: TBD — until we publish one, watch the canary channel pointer for release timing: https://updates.seglamater.app/hivemind/channels/canary/latest.json.

Bootstrap your first admin

The Hivemind installer doesn't create an admin account for you — that's intentional, since the right path depends on whether you're running local-auth or SSO. Two options:

Path A: local-auth (default)

POST /auth/register works exactly once. The first registered user gets the admin role. Subsequent calls return HTTP 409.

curl -X POST http://localhost:8585/auth/register \
     -H 'Content-Type: application/json' \
     -d '{"username":"you","email":"you@example.com","password":"twelvechars+"}'

Password must be ≥ 12 characters. Response is JSON with the new user's ID + role. Then sign in at /auth/login (returns an hm_session cookie):

curl -X POST http://localhost:8585/auth/login \
     -H 'Content-Type: application/json' \
     -c /tmp/cookies \
     -d '{"email":"you@example.com","password":"twelvechars+"}'

If you ever lock yourself out, you can use ADMIN_API_KEY from your .env as a bearer token for service-to-service API access — that key bypasses the user table entirely.

Path B: SSO via forward-auth (recommended for production)

If you've already wired up Authentik (or any forward-auth IdP per the section below), there's no manual registration step. The first user who signs in via SSO is auto-provisioned. Their role comes from group membership: members of the HIVEMIND_GROUP_ADMIN group (default hivemind-admin) get the admin role; everyone else gets user.

This is the cleaner long-term path because it avoids ever needing /auth/register or password rotation in Hivemind itself — your IdP owns identity and lifecycle.

Hivemind v0.1.x ships API-only registration to keep the security surface minimal during alpha. A UI flow ("Create the first admin" button) is on the v0.2.x roadmap. Until then, paste the curl command or enable SSO.

Reverse proxy + TLS

Hivemind binds to 127.0.0.1:8585 by default, so a reverse proxy on the same host is required for any external access. Below is a minimal Caddy snippet that handles TLS termination and forwards to Hivemind. Drop this into /etc/caddy/Caddyfile (or wherever your Caddy config lives) and replace hivemind.example.com with your real domain.

hivemind.example.com {
    encode zstd gzip

    # Hivemind's own UI + API.
    reverse_proxy 127.0.0.1:8585 {
        # Pass through real client IP so the audit log and any
        # rate-limiting logic see the actual user, not 127.0.0.1.
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-For {remote_host}
        header_up X-Forwarded-Proto {scheme}
    }
}

Caddy auto-provisions a Let's Encrypt cert as long as the DNS A/AAAA records point at the host. For nginx, the equivalent is a standard proxy_pass http://127.0.0.1:8585 block plus your TLS config.

Authentik / forward-auth SSO

Hivemind has built-in support for forward-auth (Authentik, Pomerium, etc.) — the SSO middleware lives in the server binary and is enabled via environment variables in your .env. The flow is: a reverse proxy in front of Hivemind authenticates the user against your IdP, forwards a few trusted headers, and Hivemind auto-provisions / updates the user's account based on group membership.

When you'd use this

Your team already runs Authentik (or another OIDC IdP) and you want Hivemind logins gated by it.
You want group-based RBAC: members of hivemind-admin get admin, members of hivemind-user get the normal role, everyone else gets the default role.

Enabling forward-auth

Add the following to your .env (next to DB_PASSWORD, ADMIN_API_KEY, etc.):

# Toggle. Local password login is disabled when this is true.
HIVEMIND_FORWARD_AUTH=true

# Trusted upstream CIDRs — load-bearing security boundary. Hivemind
# only trusts forward-auth headers from peer IPs in this list. Set
# this to whatever Caddy / your reverse proxy presents as its source
# IP (typically the docker bridge subnet on the same host).
HIVEMIND_FORWARD_AUTH_TRUSTED_NETS=172.17.0.0/16,127.0.0.1/32

# Header names — defaults match Authentik's ProxyProvider outpost.
HIVEMIND_FORWARD_AUTH_USER_HEADER=X-Authentik-Username
HIVEMIND_FORWARD_AUTH_EMAIL_HEADER=X-Authentik-Email
HIVEMIND_FORWARD_AUTH_GROUPS_HEADER=X-Authentik-Groups
HIVEMIND_FORWARD_AUTH_GROUPS_DELIM=|

# Group → role mapping. Authentik group named on the left maps to
# the Hivemind role on the right.
HIVEMIND_GROUP_ADMIN=hivemind-admin
HIVEMIND_GROUP_USER=hivemind-user