# Secure setup ceremony

Before the vault will sign or accept policy changes, two things must happen: IT verifies cross-cluster trust (Phase 1), and the Setup Admins enroll their passkeys (Phase 2). This page covers what to do in each phase and why each check matters.

![Setup ceremony sequence: IT verification phases followed by Setup Admin passkey enrollment](/img/diagrams/setup-ceremony.svg)

```mmd
sequenceDiagram
    participant IT as IT Admin
    participant K8s as Kubernetes (all 3 clusters)
    participant ORC as Orchestrator
    participant POL as Policy Servers (3x)
    participant Admin as Setup Admins

    Note over IT,POL: Phase 1: Cluster Trust Verification

    IT->>K8s: kubectl logs key-store-mpc-server-0 (x3 clusters)
    IT->>K8s: kubectl logs policy-server-0 (x3 clusters)
    K8s-->>IT: 6 log files with own_pubkey, cluster_pubkeys, cluster_pubkeys_fingerprint

    IT->>IT: Check 1: own_pubkey appears in cluster_pubkeys (all 6 logs)
    IT->>IT: Check 2: cluster_pubkeys_fingerprint identical across all 6 logs

    Note over IT: Out-of-band human check

    alt Any check fails
        IT->>IT: Stop, contact Sodot support with all 6 log files
    else All checks pass
        Note over IT,Admin: Vault operational
        Note over IT,Admin: Phase 2: Passkey enrollment
        IT->>Admin: Share vault URL
        loop Each Setup Admin
            Admin->>ORC: Sign in with Google SSO
            Admin->>ORC: Enroll passkey (WebAuthn)
            ORC->>POL: Replicate passkey enrollment (all 3 nodes)
        end
    end
```

*Phase 1 is IT-only cluster trust verification; Phase 2 is Setup Admin passkey enrollment. No Sodot representative participates.*

## Why the ceremony exists

When the Orchestrator starts in Stage 2, it collects each node's identity public key and distributes the complete ordered set back to every node. Each node uses this set as the basis for cross-cluster trust. The ceremony gives IT a mechanism to verify this propagation was correct and was not tampered with in transit. No Sodot representative participates in or observes the ceremony.

Until cluster trust propagation completes, the Orchestrator rejects signing and policy operations. Once it completes, the vault is operational and the Setup Admins sign in to enroll their passkeys.

## Phase 1: Cluster trust verification

### What to verify

After Stage 2 startup, pull the logs from all six cluster pods: the Key Store and Policy Server reader on each of the three clusters. Each cluster's Deployment name carries its shard index - `cluster-0` = spoke-0 (shard 0), `cluster-1` = hub (shard 1), `cluster-2` = spoke-1 (shard 2):

```bash
# cluster-0 = spoke-0 (shard 0)
kubectl --kubeconfig out/cluster-0.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-mpc-key-store-0 > log-ks-spoke0.txt
kubectl --kubeconfig out/cluster-0.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-policy-server-0-reader > log-pol-spoke0.txt

# cluster-1 = hub (shard 1)
kubectl --kubeconfig out/cluster-1.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-mpc-key-store-1 > log-ks-hub.txt
kubectl --kubeconfig out/cluster-1.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-policy-server-1-reader > log-pol-hub.txt

# cluster-2 = spoke-1 (shard 2)
kubectl --kubeconfig out/cluster-2.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-mpc-key-store-2 > log-ks-spoke1.txt
kubectl --kubeconfig out/cluster-2.kubeconfig -n crypto-vault \
    logs deploy/crypto-vault-policy-server-2-reader > log-pol-spoke1.txt
```

Each node logs its cluster-identity fields at startup under the `cluster_identity` target:

```
cluster_identity: own_pubkey=<hex string>
cluster_identity: cluster_pubkeys=[<pk_1>, ..., <pk_6>]
cluster_identity: cluster_pubkeys_fingerprint=<hex hash>
```

### Two checks per log (all six logs)

**Check 1: self-inclusion.** The pod's `own_pubkey` must appear in its own `cluster_pubkeys` list. If it does not, the Orchestrator distributed a trust set that excludes the node itself, which is a propagation error.

**Check 2: fingerprint agreement.** The `cluster_pubkeys_fingerprint` must be identical across all six logs. The fingerprint is a hash of the ordered six-pubkey set. If any two logs show different hashes, the Orchestrator distributed different trust sets to different nodes, which is a hard failure.

A helper script to surface the values quickly:

```bash
#!/usr/bin/env bash
# verify-cluster-trust.sh
for f in log-ks-spoke0.txt log-ks-hub.txt log-ks-spoke1.txt \
          log-pol-spoke0.txt log-pol-hub.txt log-pol-spoke1.txt; do
    echo "=== $f ==="
    grep -E "own_pubkey|cluster_pubkeys_fingerprint" "$f" | tail -2
done
# Every cluster_pubkeys_fingerprint must be identical across the six files.
# Each own_pubkey must appear in that file's cluster_pubkeys list.
```

### Cross-human check (recommended)

If multiple IT admins are involved, each should independently pull logs from their own cluster and compare fingerprints out-of-band over a channel of their choice (internal Slack, email, or voice). Do not use a Sodot-provided surface for this comparison.

### Failure path

If any fingerprint disagrees or any pod's own public key is absent from its cluster pubkey list: stop, do not proceed, and contact Sodot support. Attach all six log files. The system remains locked until the issue is resolved.

## Phase 2: Setup Admin passkey enrollment

Once Phase 1 passes, the vault is operational and each Setup Admin enrolls a passkey. Share the vault's internal URL with every admin listed in `policyServer.initialAdmins` (and `googleAuth.adminEmails`) and point them to the [First login](/guide/getting-started/first-login) guide. For each admin:

1. Navigate to the vault URL.
2. Sign in with Google.
3. The UI immediately presents the passkey enrollment modal. There is no way to skip it.
4. The admin enrolls a passkey. Enrollment replicates across all three Policy Server nodes.

A passkey is required before an admin can perform any state-changing action, so every Setup Admin must complete this step. Once they have, the Setup Admins can configure the Administrative Policy, create user groups, and onboard the first keys.

## Security properties of the ceremony

* **Identity keypairs are generated locally** on each cluster node at first startup. Sodot never provides or observes private key material.
* **The Orchestrator is a pubkey propagator only.** It collects and distributes the six identity public keys; it never touches private key material.
* **The fingerprint comparison** is the customer's mechanism to verify the trust set was not tampered with in transit.
* **First passkey enrollment** is limited to the Setup Admins configured at deployment, each signing in through Google SSO.
* The per-shard backup encryptor public keys (`mpcKeyStores[].backupEncryptorPublicKey`) must be configured before any key creation or import is permitted. This is a hard system gate.