Runbook — Rotate the Tailscale auth key

Prerequisites

• gcloud auth login + gcloud auth application-default login on the Mac.
• Tailscale admin console access.
• Accept the side effects of VM replacement: data/ token DBs and .env are lost — claude.ai re-registers, connector OAuth re-authorized, new broker admin key.

Steps

1. Mint the new key — [Tailscale admin → Settings → Keys]: reusable off, expiry short (7 d), pre-authorized on, ephemeral off (ephemeral nodes are deleted when offline — wrong for a server).
2. Add as a new secret version (note printf '%s' — echo would embed a newline in the key and break tailscale up): printf '%s' 'tskey-auth-…' | gcloud secrets versions add tailscale-auth-key --data-file=- --project=bob-mcp-project-2026
3. Destroy the old version: gcloud secrets versions list tailscale-auth-key → gcloud secrets versions destroy <N> --secret=tailscale-auth-key
4. Revoke the old key — [admin → Settings → Keys → ⋯ → Revoke]. This is the step that actually neutralizes any leaked copies (state files, old instance metadata).
5. Delete the old machine — [admin → Machines → gcp-mcp-broker → Remove] — before the new VM registers, or it will join as gcp-mcp-broker-1 and break the Ansible inventory (rename via ⋯ → Edit machine name if that happens).
6. Replace the VM: terraform plan -out=tfplan (expect 1 destroy / 1 add if the startup script changed; otherwise taint first: terraform apply -replace=google_compute_instance.mcp_broker) → terraform apply tfplan.
7. Redeploy per gcp-mcp-standalone/runbooks/deploy step 3.

Verification

• New node active in tailscale status; SSH over the mesh works.
• grep -c tskey terraform.tfstate → 0.
• Serial console (gcloud compute instances get-serial-port-output gcp-mcp-broker) shows the startup script completed — this is also the only diagnostic path if the bootstrap fails, since there is no SSH until Tailscale is up.
• Broker healthy per the deploy runbook's checks.

Rollback

None needed for the key itself (old one is dead by design). If the new VM fails to bootstrap, fix main.tf, re-plan, re-apply — the box holds no state worth saving until Ansible has run.