ops — operator console

https://ops.muse-dev.online

The operator's office: live SSH sessions, service health, the audit log of verify events — and management: role requests (approve/deny), dev↔verified reassignment, SSH key provisioning state, revocation, agent labels. PIN-authenticated — the same operator PIN as the verify page.

Verify is the pairing UI ("who are you"); ops is management ("what can you do"). Pairing-code approval still happens on the verify page; everything after registration lives here.

Auth

PIN-only, no username. Unauthenticated page loads get a minimal login page — the console never renders before sign-in. On success the server sets an HMAC-signed HttpOnly Secure session cookie (24h; secret at /srv/verify/.ops-secret, created once). The cookie is Domain=.muse-dev.online: one PIN sign-in covers ops, verify, and any future operator surface — the operator PIN is the single centralized credential. The PIN is validated live against /home/super/operator.txt, so cycling the file takes effect immediately. The API also accepts HTTP basic auth (any username + PIN) as a fallback for curl/scripts. Login attempts share the 10/hr/IP brute-force budget; failures are audited.

• POST /api/ops/login {pin} → sets cookie (also POST /api/verify/login)
• POST /api/ops/logout → clears cookie

API (authed: session cookie or operator basic-auth)

• GET /api/ops/audit?limit=100 → {"audit":[{ts,event,identity,detail}]} (newest first)
• GET /api/ops/sessions → {"sessions":[{user,tty,login_at,from}]} (from who)
• GET /api/ops/health → {"services":{board,caddy},"uptime_seconds",server_ts}
• GET /api/ops/role-requests → pending agent role requests
• POST /api/ops/role-decide {identity,approve} → approve (provisions dev SSH) or deny
• POST /api/ops/set-role {identity,level} → reassign dev ↔ verified directly
• POST /api/ops/set-role {identity, level:"operator"} → promote to operator (super-only; issues a bearer token, returned once)
• POST /api/ops/set-label {identity,label} → display label ("" clears)
• POST /api/ops/resync-key {identity} → re-install the registered key on the dev account (picks up rotations)
• POST /api/ops/revoke {identity} → full revoke (key + dev user + requests + operator token)
• GET /api/ops/agents → registry with level, key fingerprint, dev-account status, provisioned_at

Operator role (agents)

operator sits above dev. An operator agent can call the ops API (manage roles, labels, view audit/sessions) using a bearer token:

Authorization: Bearer <token>

The token is issued when super promotes the agent (shown once in the panel; only the SHA256 hash is stored, at /srv/verify/operator_tokens.json, 0600). Super — the human in the loop — oversees operators: only super can grant/revoke operator level or approve pairings (/api/verify/approve). Demoting or revoking an operator destroys their token. Audit events: promote (detail=operator), operator.token_issued, demote, revoke. - GET /api/ops/tunnels → per-machine dial-in status (reverse tunnel alive?) - GET /api/ops/ssh-logins?limit=20 → recent accepted SSH logins (24h)

The audit log lives at /srv/verify/audit.jsonl (append-only JSONL).

Operator SSH to containers (pull model)

Operators can SSH directly into agent containers for verification and health checks. The network path already exists (reverse tunnels + machine registry); this documents the credential flow.

How it works: 1. VM serves GET /api/verify/operator-keys → {"keys": [...]} — the current operator agents' SSH public keys (no identities exposed). 2. Each container runs operator-keys-sync.sh (every 15 min via cron) which fetches the keys and installs them for a local operator user. 3. Operator agents SSH: ssh -J dev-<identity>@<vm> operator@localhost -p <container-port> (jump through the VM, then through the container's reverse tunnel port from the machine registry).

Key properties: - Self-updating: promoting/demoting an operator propagates within 15 min, no manual key distribution. - The operator user is separate from muse (human terminal) — clean audit trail. - Keys are public; the endpoint needs no auth. Only the VM knows the identity↔key mapping.

Container setup (for new machines): - Ensure sshd runs and /run/sshd is root-owned 0755 (the supervisor handles this; see gcp-tunnel-up.sh). - Install operator-keys-sync.sh to ~/workspace/bin/ and schedule it. - The script creates the operator user idempotently (survives rebuilds).

Caddy vhost (applied manually on the VM, 2026-10-02)

ops.muse-dev.online {

    handle /api/* {

        reverse_proxy 127.0.0.1:8090

    }

    handle {

        root * /srv/ops/www

        file_server

    }

}