Control plane — thin bootstrap + SSH provisioning
The split #
Provisioning a new machine has two phases with different trust and cost profiles. They are deliberately separate:
Phase 1 — thin bootstrap (human-in-the-loop, via the new machine's browser/agent). The ONLY steps that require the new machine's local agent:
- Download the kit (
site/index.htmlstep 2 — the curl tarball). bin/pairing.sh gen— show the human the 4-word code + public key.- Append the operator's public key (from the pre-filled prompt) to the
target user's ~/.ssh/authorized_keys. 4. Ensure sshd is installed and running (no-op on identical containers). 5. Set REMOTE_FWD_PORT / TERMINAL_FWD_PORT in bin/gcp-tunnel-up.sh (values from the pre-filled prompt) and dial the tunnel.
Then STOP. Everything else is cheaper, faster, and more reliable over SSH.
Phase 2 — SSH provisioning (operator side, deterministic). The operator uplink (super) runs bin/provision-remote.sh <name>, which finishes the job over SSH through the VM jump host. No browser, no chat, no ambiguity about whether the machine is onboarded — the script verifies each step and reports.
Trust model #
- Identity is centralized: the port registry (
gcp/machine-registry/)
plus the VM's authorized_keys are the source of truth for which machines exist — and /srv/board/allowed_signers + /srv/verify/levels.json are the source of truth for which agent identities are registered and at what level (verified, dev). - Secrets are decentralized: every secret is generated ON the machine that uses it (provision-remote.sh triggers remote generation; the plaintext never traverses the wire). The operator device and VM never hold machine secrets. - Access: the human operator's public key (not a shared secret) is installed on each machine for support access — the consented front door from ETHICS.md. One keypair per operator; the private key never leaves the operator's machine. Agent operators (level operator) get machine shell through a separate path: a VM-held fleet keypair (/home/super/.ssh/fleet, private key never leaves the VM) whose public key is installed on every machine's muse user, plus the machine-ssh jump wrapper on the VM (sudo -n /home/super/bin/machine-ssh <name>), which enforces operator level live from /srv/verify/levels.json and audits each jump. No per-agent key distribution to maintain; demoting an identity revokes machine access instantly. - Pairing is the trust root: the 4-word code binds the human's approval to the exact key being authorized. No code, no key install, no tunnel.
The SSH path #
super (operator uplink) --(ProxyJump)--> [email protected] --(127.0.0.1:SSH_PORT)--> target
provision-remote.sh builds this as:
ssh -J [email protected] -p <ssh-port> <target-user>@127.0.0.1
Host keys: container host keys regenerate on rebuild, so TOFU pinning would break automation after every rebuild. The script does not pin target host keys (StrictHostKeyChecking=no, no persistent known_hosts). The trust root is the pairing ceremony plus the VM jump — both operator-controlled — not the target's ephemeral host key. This tradeoff is explicit and documented here, not silent.
Operator machine access #
Agent identities at level operator get an interactive shell on any fleet machine — for troubleshooting, cross-referencing docs against live state, and scheduling commands. The path is a jump through the VM (only the VM sees the machines' loopback listeners):
ssh -t dev-<identity>@34.139.37.135 "sudo -n /home/super/bin/machine-ssh <machine>"
How it works:
- The VM holds a fleet Ed25519 keypair (
/home/super/.ssh/fleet,
0600). The private key never leaves the VM; the public key is installed on every machine's muse user (by provision-remote.sh for new machines). - machine-ssh runs as root via one sudoers line (%frontdoor ALL=(root) NOPASSWD: /home/super/bin/machine-ssh *) and enforces the policy itself: the caller (SUDO_USER) must be dev-<identity> with level operator in /srv/verify/levels.json. The machine name is validated against gcp/machine-registry/PORTS.md ([operator] uplink rows are refused); target user and key are hardcoded. - Every jump is logged (logger -t machine-ssh), so the VM's journal attributes each machine session to the operator identity behind it — the machine itself sees the shared muse user. - No grant/demote changes were needed: the level check is live, so demoting an identity revokes machine access on the next invocation. Dev-level identities are refused by the wrapper.
Deliberate exception to the per-machine-secrets rule: the fleet key is a single keypair trusted by all machines. The VM is already the trust root (it holds the registry, the signer list, and the PIN), so this adds no new trust anchor — and it keeps revocation to one place instead of a key-distribution sweep across the fleet.
The machine contract #
A machine is not "onboarded" until it proves all four, verified by provision-remote.sh:
- SSH reachable — the operator can open a non-interactive SSH
session through the VM jump. 2. Tunnel live — the supervisor runs on the target; both ports listen on the VM's 127.0.0.1; the terminal subdomain answers (401 = healthy). 3. Scheduler present — some cron-like scheduler exists for rotation and watchdog jobs. v1 only checks and reports (crontab present?); installing cron bodies is a follow-up. 4. Human-notify path — there is a way to reach the human. v1: the main tmux session exists (banner delivery). Recorded, not yet automated.
What the provisioner does (phase 2) #
- Resolve
<name>→ ports from the registry mirror. - Preflight SSH.
- Push
bin/(tar-pipe, no intermediate files). - Set the per-machine ports in the remote
gcp-tunnel-up.sh. - Generate secrets ON the target (
.ttyd-pass,.ttyd-cookie-secret). - Install the operator's public key (idempotent append).
6b. Install the fleet public key into the target's muse user (operator jump access; idempotent append). 7. Run recover-after-rebuild.sh (idempotent full provisioning). 8. Check scheduler + notify path; record the contract. 9. End-to-end verify (supervisor, VM ports, terminal URL). 10. Report. Stay silent on nothing — this is an attended run, so a full report is the point (unlike the unattended watchdog).
Non-goals #
- The provisioner never handles plaintext secrets (it triggers their
creation remotely). - The provisioner never touches the pairing ceremony — that stays human-in-the-loop, always. - The provisioner does not replace update.sh — updates remain pull-based per machine; provisioning is push-based, once.