Upgrade — pre-managed-update install → managed-update pipeline

Runbook for taking an existing chatalot deployment whose chatalot-updater sidecar is not actively running and migrating it onto the managed-update pipeline. Covers two starting points: (a) a pre-managed-update install with no updater services in the compose at all, and (b) a previously-staged install where the updater services were defined in the compose but the profile was never activated.

This is a distinct path from a fresh install (docs/self-hosting/production.md). Don't run scripts/install.sh against an existing instance — it will refuse to overwrite secrets and the result will be inconsistent.

Audience for this doc: an AI assistant OR a human operator with sudo on the chatalot host. Steps are deliberately conservative — every change is preceded by a backup, and the rollback path is one command.

What "managed updates" means in this context

After the upgrade:

• chatalot-server pulls its image from registry.seglamater.app (or your private registry) by digest, instead of building from local source.
• A chatalot-updater sidecar runs alongside the server. The admin "Updates" tab in the web panel shows current vs latest, and applying an update is one click.
• The updater takes a pre-apply Postgres snapshot, runs pre_flight.sh + migrate.sh against the new image in throwaway containers, and rolls back automatically if the new version's health check fails.
• A docker-socket-proxy with a strict ACL fronts the docker daemon for the updater — the updater can pull, create, start, stop, and remove containers, but cannot exec into anything or read host volumes.

Inputs you need before starting

Input	Required?	Notes
sudo on the chatalot host	Yes	Editing compose files, restarting containers, generating secrets.
Current chatalot dir on the host	Yes	Often /srv/chatalot, /opt/chatalot, or ~/services/chatalot. Find via docker inspect chatalot --format '{{.Mounts}}' or docker compose config --format json from inside the dir.
Read access to https://registry.seglamater.app	Yes	Either anonymous (if the chatalot package was made public) or via a per-instance read-only token. Test: docker pull registry.seglamater.app/seglamater/chatalot:0.24.6 from the host.
5-10 minutes of brief downtime acceptance	Yes	The container swap is ~30s; if migrations need to apply on first start, add ~60s. Schedule outside business hours for a real client.

Pre-checks

Run these on the chatalot host before touching anything. Stop and ask the operator if any answer is unexpected.

First, set the path to the compose dir. The runbook references $COMPOSE_DIR throughout; the operator points it at wherever their stack lives:

# Common locations: /srv/chatalot, /opt/chatalot, ~/services/chatalot.
# Find via `find / -name docker-compose.yml -path '*chatalot*' 2>/dev/null`
# if you're unsure.
COMPOSE_DIR=/srv/chatalot

# Auto-detect the chatalot service container name. The container_name
# varies between deployments — `chatalot` on some compose files,
# `chatalot-server` on others. We look it up by compose service name
# (which is consistently `chatalot`) and resolve to whatever the
# operator's compose declared.
CHATALOT_CONTAINER=$(sudo docker compose -f "$COMPOSE_DIR/docker-compose.yml" \
    ps --format '{{.Name}}' chatalot 2>/dev/null | head -1)
echo "chatalot service container: ${CHATALOT_CONTAINER:-NOT FOUND}"
# If empty, the COMPOSE_DIR is wrong or the chatalot service is stopped —
# fix that first before running anything below.

Then the actual checks:

# 1. What version is running, and is the API healthy?
#    Run via `docker exec` so we don't depend on host-port binding —
#    many production setups bind chatalot only to an internal network
#    and front it with a reverse proxy, so port 8080 isn't on host
#    loopback.
sudo docker exec "$CHATALOT_CONTAINER" curl -sf http://127.0.0.1:8080/api/health | jq .
# Expected: {"status":"ok","version":"0.X.Y", ...}
# (If the deployment exposes port 8080 to host loopback, plain
# `curl -sf http://127.0.0.1:8080/api/health` from the host works too.)

# 2. Inventory secrets/
ls -la "$COMPOSE_DIR/secrets/"
# Expected baseline (pre-0.24): jwt_private.pem, jwt_public.pem, totp_encryption_key, db_password
# Already-on-0.24.x baseline: also updater_token, cosign_pub
# After this upgrade: also registry_creds
# Watch for 0-byte placeholders in cosign_pub / updater_token —
# those are signs of a prior session that staged the upgrade
# but never finished. They need to be populated, not skipped.

# 3. Compose layout
grep -E '^\s*(services|build|image|profiles):' "$COMPOSE_DIR/docker-compose.yml" | head -30
# Look for: chatalot service uses 'build:' or 'image:'? Are there any 'profiles:' lines?

# 4. Running container count
sudo docker compose -f "$COMPOSE_DIR/docker-compose.yml" ps --status running
# Pre-upgrade typical: chatalot, postgres (sometimes named chatalot-db), and maybe coturn / cloudflared.
# Post-upgrade target: those + chatalot-updater + chatalot-socket-proxy.

# 5. Is the updater profile already DEFINED in the compose file?
#    (A prior session may have staged the services without activating them.
#    This is independent of the running version.)
# Match exactly TOP-LEVEL service definitions (2-space indent, name, colon,
# end-of-line). A looser pattern would also match a `depends_on:` reference
# nested inside another service.
grep -cE '^  (chatalot-updater|chatalot-socket-proxy):$' "$COMPOSE_DIR/docker-compose.yml"
# Expected: 0 (full upgrade path — no updater services in compose at all)
#       or  2 (variant — both services are defined; pick the staged variant below)

The path you take depends on what's already in the compose file, not the running chatalot version:

• Updater services NOT defined in the compose (typical pre-0.24 install): follow the full path below.
• Updater services already defined but not running (compose was pre-staged in a prior session — chat.seglamater.app's situation as of 2026-04-24): jump to Variant: updater profile staged below. Skip the compose-shape surgery entirely; just populate secrets + env vars + activate.

Backup

Do not skip this. A failed migration without a backup means the next steps are "restore from your VPS provider's snapshot" which may or may not exist.

TS=$(date +%Y%m%d-%H%M%S)
BACKUP_DIR=/var/backups/chatalot-pre-managed-update-${TS}
sudo mkdir -p "$BACKUP_DIR"

# 1. Database dump.
#    Note the `| sudo tee … > /dev/null` instead of `sudo … > path`:
#    the redirect operator is processed by the user's shell, NOT by
#    sudo, so a plain `sudo cmd > /var/backups/...` fails with
#    "Permission denied" because the user can't write to root-owned
#    /var/backups/ regardless of what's running on the left of the
#    redirect. `sudo tee` is the canonical fix.
sudo docker exec -i $(sudo docker ps --format '{{.Names}}' | grep -E 'postgres|chatalot-db' | head -1) \
    pg_dump -U chatalot -Fc chatalot | sudo tee "$BACKUP_DIR/chatalot-${TS}.dump" > /dev/null

# 2. Secrets dir (encrypted contents — keep safe)
sudo cp -a "$COMPOSE_DIR/secrets" "$BACKUP_DIR/secrets"

# 3. Current compose file
sudo cp -a "$COMPOSE_DIR/docker-compose.yml" "$BACKUP_DIR/docker-compose.yml.before"
sudo cp -a "$COMPOSE_DIR/.env" "$BACKUP_DIR/env.before" 2>/dev/null || true

# 4. List of running containers + their image refs (for rollback verification).
#    Same `| sudo tee` reason as the pg_dump above.
sudo docker compose -f "$COMPOSE_DIR/docker-compose.yml" ps --format json | \
    sudo tee "$BACKUP_DIR/containers.json" > /dev/null

ls -la "$BACKUP_DIR"
echo "Backup at: $BACKUP_DIR — keep this until the upgrade is verified working for at least 24h"

# Sanity-check the DB dump landed and isn't empty (see "thinking you have
# a backup but you don't" failure mode of the redirect-permission gotcha).
sudo test -s "$BACKUP_DIR/chatalot-${TS}.dump" && echo "DB dump OK" || echo "DB dump MISSING — STOP"

Confirm to the operator the backup landed before continuing. The DB dump alone should be in the 10s of MB to GB range depending on usage.

Variant: updater profile staged but inactive

The compose file already defines chatalot-updater + chatalot-socket-proxy services and the updater_token / cosign_pub / registry_creds secrets in the top-level secrets: block, but the profile has never been started. Steady state: the chatalot service runs from local source (build: .), no managed-update sidecar.

This is the smaller migration. You don't add services or secret slots; you just populate the placeholder secret files, add the chatalot-server-side admin-UI env vars, switch the chatalot service to a registry-pulled image, and bring up the profile.

Inventory before changing anything:

# What's actually in the secrets dir?
ls -la "$COMPOSE_DIR/secrets/"
# Look for: cosign_pub, updater_token, registry_creds. Some may be 0-byte
# placeholders (touched by a prior session); some may be missing entirely.
# Also note ownership — files commonly owned by the user that did the rsync;
# the in-container chatalot user is UID 999, so file ownership matters.

# What admin-UI env vars does the chatalot service already have?
grep -nE 'UPDATER_API_URL|UPDATER_API_TOKEN|UPDATE_CHANNEL|UPDATE_MANIFEST_HOST' \
    "$COMPOSE_DIR/docker-compose.yml" | head -10
# UPDATER_API_TOKEN_FILE is usually already there (added with the staged
# services); UPDATER_API_URL / UPDATE_CHANNEL / UPDATE_MANIFEST_HOST
# typically still need to be added (those landed with v0.24.5).

Then:

Step A — Populate the staged secret files

cd "$COMPOSE_DIR"

# updater_token (HMAC shared between chatalot-server and chatalot-updater).
# If the file exists empty, overwrite it. If it has content already, KEEP IT —
# the server may already be using that value; rotating without coordination
# would lock the server out of the sidecar.
if ! [ -s secrets/updater_token ]; then
    sudo bash -c "openssl rand -hex 32 > secrets/updater_token"
fi
sudo chmod 0644 secrets/updater_token  # mode 0644 — see "Why 0644" below.

# cosign_pub: the trust anchor for managed-update signature verification.
sudo curl -sfo secrets/cosign_pub https://updates.seglamater.app/.well-known/keys/chatalot.pub
echo "f1ff3f7ff3a1a7fe53620bbd8db0e362e55dcf622780e271740bc27d1850d082  secrets/cosign_pub" | \
    sudo sha256sum -c -
sudo chmod 0644 secrets/cosign_pub

# registry_creds: per-instance read-only token for authenticated pulls.
# Format: {"username":"...","password":"...","serveraddress":"registry.seglamater.app"}
# For anonymous pulls, an empty file works:
sudo bash -c "echo -n '' > secrets/registry_creds"
sudo chmod 0644 secrets/registry_creds

# Make sure db_password is also 0644 (the updater sidecar reads it via UID 1001
# even though the chatalot server reads it via UID 999 — they need different
# UIDs to read the same file, so 0644 is required regardless of host owner).
sudo chmod 0644 secrets/db_password

# Lock the parent dir so non-privileged users on the host can't enumerate.
# This is the actual access gate; per-file modes don't matter once the
# dir is 0700.
sudo chmod 0700 secrets/

Why 0644 for files the sidecar reads (db_password, updater_token, cosign_pub, registry_creds): docker compose v2.40 and earlier mount file-driven secrets preserving the host file's mode and owner. The chatalot server runs as UID 999 inside the container, the updater sidecar as UID 1001 — they don't share a UID, so neither can read the other's secret if the file is 0600 owner-restricted. Mode 0644 lets both UIDs read; the parent secrets/ directory at 0700 enforces actual host-level access control. Compose v5.1+ defaults secret modes to 0444 inside the container regardless of host, which sidesteps the issue — but the conservative move is to set 0644 on the host and not assume the compose version.

The chatalot-server-only secrets (jwt_private.pem, jwt_public.pem, totp_encryption_key) can stay 0600 because only chatalot reads them and it's the host file owner — but if your host file owner is something other than UID 999 (the chatalot in-container user), even those need 0644 on older compose versions. Safest default: chmod 0644 across the board.

Step B — Add admin-UI env vars to the chatalot service

In docker-compose.yml, find the chatalot service's environment: block. The four keys below must all be present — depending on how the service was previously staged, some may already exist. Add only the keys that are missing; do not duplicate existing keys.

    environment:
      # ...existing keys...
      UPDATER_API_URL: "http://chatalot-updater:8081"
      UPDATER_API_TOKEN_FILE: /run/secrets/updater_token
      UPDATE_CHANNEL: stable
      UPDATE_MANIFEST_HOST: https://updates.seglamater.app

Quick check for what's already there before editing:

sudo grep -nE 'UPDATER_API_URL|UPDATER_API_TOKEN_FILE|UPDATE_CHANNEL|UPDATE_MANIFEST_HOST' \
  "$COMPOSE_DIR/docker-compose.yml"

Each key should appear exactly once after editing.

Ensure updater_token is in the chatalot service's secrets: list (so chatalot-server's request-signing can read it). If it's already present from prior staging, leave it as-is:

    secrets:
      # ...existing entries...
      - updater_token

Step C — Pin BOTH images (one-time switch off build:)

Two services may still be on build: from prior staging — the chatalot server itself and the chatalot-updater sidecar. Both must be pinned to a registry image at the same release version. If only one is pinned, the running updater binary can lag behind the server's protocol and admin-UI calls will fail.

Use the latest stable release from the manifest host:

curl -sf https://updates.seglamater.app/chatalot/channels/stable/latest.json | jq '.version, .image, .image_digest'

Pin the chatalot service:

  chatalot:
    # Was: build: .
    image: registry.seglamater.app/seglamater/chatalot:0.24.6

Pin the chatalot-updater service. Replace any build: { context: ., dockerfile: Dockerfile.updater } block:

  chatalot-updater:
    # Was: build: { context: ., dockerfile: Dockerfile.updater }
    image: registry.seglamater.app/seglamater/chatalot-updater:0.24.6

For bit-exact reproducibility, pin by digest instead of tag:

    image: registry.seglamater.app/seglamater/chatalot@sha256:3aac2ee6e6fb...
    image: registry.seglamater.app/seglamater/chatalot-updater@sha256:43151eab0dc7...

After this step, the host no longer needs the chatalot source tree — every future upgrade is image-only.

Step D — Pull the new images (one-time docker login if registry is private)

sudo docker login registry.seglamater.app -u <vendor-username>
# Paste the per-instance read-only token at the password prompt.

sudo docker pull registry.seglamater.app/seglamater/chatalot:0.24.6
sudo docker pull registry.seglamater.app/seglamater/chatalot-updater:0.24.6
sudo docker pull registry.seglamater.app/seglamater/docker-socket-proxy:0.3.0

The pull happens once on the host; from here on, the sidecar will handle future image pulls during applies (using secrets/registry_creds for its own auth).

Step E — Validate the compose

sudo docker compose -f "$COMPOSE_DIR/docker-compose.yml" --profile updater config > /dev/null
echo "Exit code: $?  (0 means valid)"

Skip to Step 7 (sidecar bring-up) below — the rest of the full-upgrade path applies verbatim.

Full upgrade (from pre-0.24)

Step 1 — Pin the chatalot image

Replace the build: . directive with a pinned image: ref. Edit docker-compose.yml:

services:
  chatalot:
    # Was: build: .  (or build: { context: ., dockerfile: Dockerfile })
    image: registry.seglamater.app/seglamater/chatalot:0.24.6
    # everything else stays the same

Use the latest stable digest-pinned version. Check with:

curl -sf https://updates.seglamater.app/chatalot/channels/stable/latest.json | jq '.version, .image_digest'

Pin to <image>@<image_digest> instead of :<version> if you want bit-exact reproducibility (recommended for production):

    image: registry.seglamater.app/seglamater/chatalot@sha256:3aac2ee6e6fb...

Step 2 — Add the updater services

Append to the end of the services: section. These two services and their networks/secrets are unchanged across deployments — copy verbatim:

  chatalot-socket-proxy:
    image: registry.seglamater.app/seglamater/docker-socket-proxy:0.3.0
    container_name: chatalot-socket-proxy
    restart: unless-stopped
    environment:
      CONTAINERS: "1"
      IMAGES: "1"
      NETWORKS: "1"
      POST: "1"
      EXEC: "0"
      VOLUMES: "0"
      SYSTEM: "0"
      BUILD: "0"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - updater_internal
    read_only: true
    tmpfs:
      - /tmp
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges:true
    profiles:
      - updater

  chatalot-updater:
    image: registry.seglamater.app/seglamater/chatalot-updater:0.24.6
    container_name: chatalot-updater
    restart: unless-stopped
    depends_on:
      chatalot-socket-proxy:
        condition: service_started
      postgres:
        condition: service_healthy
    environment:
      DOCKER_HOST: tcp://chatalot-socket-proxy:2375
      CHATALOT_UPDATER_SQLITE_PATH: /var/lib/chatalot-updater/state.db
      CHATALOT_UPDATER_SERVER_CONTAINER: chatalot
      DATABASE_URL: postgres://chatalot:__DB_PASSWORD__@postgres:5432/chatalot
      CHATALOT_SNAPSHOT_DIR: /var/backups/chatalot
      CHATALOT_UPDATER_API_TOKEN_FILE: /run/secrets/updater_token
      CHATALOT_UPDATER_COSIGN_PUBKEY_PATH: /run/secrets/cosign_pub
      CHATALOT_UPDATER_REGISTRY_CREDS_FILE: /run/secrets/registry_creds
      CHATALOT_UPDATER_LISTEN_ADDR: "0.0.0.0:8081"
    volumes:
      - updater_data:/var/lib/chatalot-updater
      - chatalot_backups:/var/backups/chatalot
    secrets:
      - updater_token
      - cosign_pub
      - db_password
      - source: registry_creds
        target: registry_creds
        mode: 0444
    networks:
      - updater_internal
      - internal
      - external
    read_only: true
    tmpfs:
      - /tmp
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:8081/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 10s
    profiles:
      - updater

Add the network if it doesn't exist:

networks:
  updater_internal:
    internal: true

(Keep the existing internal: and external: networks if your compose has them.)

Add the new secret definitions to the top-level secrets: block:

secrets:
  # ...existing entries...
  updater_token:
    file: ./secrets/updater_token
  cosign_pub:
    file: ./secrets/cosign_pub
  registry_creds:
    file: ./secrets/registry_creds

Add the new volumes:

volumes:
  # ...existing entries...
  updater_data:
  chatalot_backups:

Step 3 — Add admin-UI env vars to the chatalot service

In the chatalot service's environment: block:

    environment:
      # ...existing entries...
      UPDATER_API_URL: http://chatalot-updater:8081
      UPDATER_API_TOKEN_FILE: /run/secrets/updater_token
      UPDATE_CHANNEL: stable
      UPDATE_MANIFEST_HOST: https://updates.seglamater.app

Add updater_token to the chatalot service's secrets: list (so chatalot-server can sign requests to the sidecar):

    secrets:
      # ...existing entries (jwt_private_key, jwt_public_key, totp_encryption_key, db_password)...
      - updater_token

Step 4 — Generate the new secret files

cd "$COMPOSE_DIR"

# Shared HMAC token between chatalot-server and chatalot-updater.
# 64 hex chars = 32 bytes. Both sides read this; never check into git.
sudo bash -c "openssl rand -hex 32 > secrets/updater_token"
sudo chmod 0644 secrets/updater_token

# Cosign public key — the updater sidecar's verifier reads this to validate
# manifest signatures. The current published key has SHA-256
# f1ff3f7ff3a1a7fe53620bbd8db0e362e55dcf622780e271740bc27d1850d082.
sudo curl -sfo secrets/cosign_pub https://updates.seglamater.app/.well-known/keys/chatalot.pub
echo "f1ff3f7ff3a1a7fe53620bbd8db0e362e55dcf622780e271740bc27d1850d082  secrets/cosign_pub" | \
    sudo sha256sum -c -
# Expected: secrets/cosign_pub: OK

# Registry credentials. For anonymous pulls (public registry), an empty
# file is fine — the sidecar treats whitespace-only as "no creds":
sudo bash -c "echo -n '' > secrets/registry_creds"
sudo chmod 0644 secrets/registry_creds

# For authenticated pulls (private registry, the default for
# Seglamater-hosted clients), populate it with a per-instance
# read-only token. Format:
# {"username":"client-name","password":"read-only-token","serveraddress":"registry.seglamater.app"}
# The operator should provide these; do not hard-code.

Step 5 — Pull the pinned image (one-time)

If the registry requires authentication (the default for registry.seglamater.app):

sudo docker login registry.seglamater.app -u <username>
# Paste the token at the password prompt.

Then pull the pinned images into the local docker cache:

sudo docker pull registry.seglamater.app/seglamater/chatalot:0.24.6
sudo docker pull registry.seglamater.app/seglamater/chatalot-updater:0.24.6
sudo docker pull registry.seglamater.app/seglamater/docker-socket-proxy:0.3.0

If pull fails with 401, secrets/registry_creds won't help here — that file is for the SIDECAR's pulls during applies, not for the host's initial pull. The host operator must docker login once.

Step 6 — Validate the compose file before applying

sudo docker compose -f "$COMPOSE_DIR/docker-compose.yml" config > /dev/null
echo "Exit code: $?  (0 means valid)"

If non-zero, the YAML has a syntax issue or a missing reference. Don't proceed.

Step 7 — Bring up the new sidecar (without disrupting the running server yet)

cd "$COMPOSE_DIR"
sudo docker compose --profile updater up -d chatalot-socket-proxy chatalot-updater
sudo docker compose ps

The chatalot service is still running on the OLD image (since you only edited the compose file but haven't run up -d chatalot yet). The new sidecar comes up and begins healthchecking on port 8081.

Verify:

sleep 15
sudo docker compose ps chatalot-updater chatalot-socket-proxy
# Both should be "running (healthy)" or "running"
sudo docker logs chatalot-updater 2>&1 | tail -20
# Look for: "http listener bound" and "registry credentials loaded" (only if creds were set)

If the sidecar crashloops, check that: - secrets/updater_token is non-empty - secrets/cosign_pub is the actual PEM public key (not the example placeholder) - The internal network exists (chatalot-updater needs DB access)

Step 8 — Recreate the chatalot-server with the new image

This is the brief-downtime moment. The chatalot service gets recreated from the pinned image: ref and picks up the new env vars. Existing volumes (file_storage, postgres data) are unaffected.

sudo docker compose up -d chatalot
# Wait for health to come back
for i in $(seq 1 30); do
    if curl -sf http://127.0.0.1:8080/api/health 2>/dev/null | grep -q '"status":"ok"'; then
        echo "Healthy after ${i} attempts"
        curl -sf http://127.0.0.1:8080/api/health | jq .
        break
    fi
    sleep 2
done

Expected version in the health JSON: matches the version in your pinned image ref (e.g., 0.24.6).

Verification

# 1. Server health
curl -sf http://127.0.0.1:8080/api/health | jq .

# 2. Admin Updates endpoint exists (returns 401 without auth — that's correct)
curl -sS -o /dev/null -w 'HTTP=%{http_code}\n' http://127.0.0.1:8080/api/admin/updates/status
# Expected: HTTP=401

# 3. Sidecar reachable from chatalot-server
sudo docker exec chatalot curl -sf http://chatalot-updater:8081/health
# Expected: {"status":"ok"}

# 4. All expected containers running
sudo docker compose ps
# Expected at minimum: chatalot, postgres, chatalot-updater, chatalot-socket-proxy

Open the admin panel in a browser, log in as an admin, click the new Updates tab. Expected display:

• Running now: <your version>
• Latest available: 0.24.6 (or whatever stable is)
• "Up to date" if you're on the latest, otherwise an Apply button

If the Updates tab shows a 401 error, the chatalot service is missing the UPDATER_API_TOKEN_FILE env or the updater_token secret mount. Re-check Step 3.

Rollback

If the new version is unhealthy or behaves unexpectedly:

cd "$COMPOSE_DIR"

# 1. Take down the new sidecar (no-op if it's already failed)
sudo docker compose --profile updater down chatalot-updater chatalot-socket-proxy

# 2. Restore the old compose file
sudo cp "$BACKUP_DIR/docker-compose.yml.before" docker-compose.yml

# 3. Recreate chatalot from the old (build-from-source) compose
sudo docker compose up -d --build chatalot

# 4. Verify health on the old version
curl -sf http://127.0.0.1:8080/api/health | jq .

Database state is preserved across the rollback (the old version reads the same postgres volume). If the new version applied a forward-only schema migration, the old version may fail to start — in that case restore the DB from the backup:

# Stop chatalot first to avoid concurrent access
sudo docker compose stop chatalot

# Restore
sudo docker exec -i $(sudo docker ps --format '{{.Names}}' | grep -E 'postgres|chatalot-db' | head -1) \
    pg_restore -U chatalot -d chatalot --clean --if-exists < "$BACKUP_DIR/chatalot-${TS}.dump"

sudo docker compose up -d chatalot

After the upgrade

Tell the operator (and document for the client):

• Future updates: click the Updates tab → review release notes → click Apply. ~60s of brief downtime per apply, fully automatic rollback on failure.
• Backup secrets/ offline. The new updater_token is added to that dir; losing it breaks the admin Updates UI's auth to the sidecar.
• The cosign_pub fingerprint (f1ff3f7ff3a1a7fe53620bbd8db0e362e55dcf622780e271740bc27d1850d082) is your trust anchor for signed releases. If chatalot's release-signing key is ever rotated, you'll see a release-notes notice and need to re-fetch this file. Don't accept a key change you didn't see announced.
• Channel choice. UPDATE_CHANNEL=stable is the right default for production. canary is for early adopters who want to help find regressions; beta sits between.

What this runbook does NOT cover

• DNS / domain / TLS: still your reverse proxy's job. The upgrade doesn't touch any of that.
• Multi-tenant: same one-instance-per-VPS model.
• Cross-major-version jumps with breaking schema changes: this runbook assumes contiguous version updates. For a jump from 0.18 → 0.24 across multiple migration generations, validate intermediate versions in a test environment first.

References

• production.md — fresh-install version of this runbook.
• ai-deployment.md — fresh-install runbook tuned for AI agents.
• ../admin-guide/updater.md — full description of the managed-update pipeline + HMAC auth model.
• backup-and-restore.md — ongoing backup strategy beyond the one-shot pre-upgrade snapshot.