# Backup & disaster recovery

The vault protects key material through two complementary mechanisms: storage encryption on each cluster node, and a backup encryptor that makes key shares recoverable if a cluster is permanently lost. Both must be in place before the first key is created or imported.

## The backup public key requirement

Each Key Store shard is configured with a backup encryptor public key (`mpcKeyStores[].backupEncryptorPublicKey`). The vault uses this key to produce an encrypted recovery kit for each MPC key share. Key creation and import are blocked until every shard has a backup public key configured.

Generate a backup key pair before installation and store the private key in a secure offline location, separate from your cluster infrastructure. The private key is only needed during an actual recovery.

:::note\[When it must be set]
The public key must be present before the first key is created or imported. This is a hard system gate: key creation and import fail until every shard has one. You can set it at install time (recommended) or add it later by updating the Helm value. Because the Key Store reads it from config at startup, adding or changing it takes effect only after a pod restart (`kubectl rollout restart`).
:::

## Generating the backup key pair

The backup encryptor uses **HPKE** ([RFC 9180](https://www.rfc-editor.org/rfc/rfc9180)) with the **X25519** KEM, HKDF-SHA256, and ChaCha20-Poly1305. Each shard's `mpcKeyStores[].backupEncryptorPublicKey` is the **raw 32-byte X25519 public key, lowercase-hex-encoded (64 characters)**, with no PEM wrapper, no base64, and no `0x` prefix. Sodot does not ship a tool for this, so generate the key pair yourself with standard tooling.

:::warning\[Generate offline, one unique pair per shard]
Generate each key pair on a host **isolated from the cluster**, ideally an offline or air-gapped machine or an HSM. Give **every Key Store shard its own unique key pair**, and never reuse one key across shards, or a single node could decrypt the others' share backups. The **private key never enters the cluster**: keep it offline, geographically separated, under the same controls as your most sensitive key material. It is only needed during recovery.
:::

Only the **public key** goes into Helm values. Because it is public, it is safe to commit to Git or GitOps. Unlike the sensitive values you source from Secrets Manager, it needs no `existingSecret` indirection.

### Using OpenSSL

```bash
# 1. Generate the X25519 private key. Keep this file OFFLINE (it is the recovery key).
openssl genpkey -algorithm X25519 -out shard-0-backup.key

# 2. Derive the raw 32-byte public key as hex. This is the Helm value.
openssl pkey -in shard-0-backup.key -pubout -outform DER | tail -c 32 | xxd -p -c 64
# -> e.g. 3f1c9a...  (64 hex characters)
```

### Using Python

```python
from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey
from cryptography.hazmat.primitives import serialization

priv = X25519PrivateKey.generate()

# Public key -> the 64-char hex string for Helm values.
pub_hex = priv.public_key().public_bytes(
    serialization.Encoding.Raw, serialization.PublicFormat.Raw
).hex()

# Private key -> store OFFLINE (recovery key).
priv_hex = priv.private_bytes(
    serialization.Encoding.Raw,
    serialization.PrivateFormat.Raw,
    serialization.NoEncryption(),
).hex()

print("public  (Helm value):   ", pub_hex)
print("private (keep offline): ", priv_hex)
```

Repeat once per shard and set each shard's value independently:

```yaml
mpcKeyStores:
  - name: mpc-key-store-0
    mpcShardIndex: 0
    backupEncryptorPublicKey: "<shard-0 64-hex public key>"
  # one unique key per shard, do not reuse
```

This key is independent of `mpcKeyStores[].importedKeyEncryptorPem`, which is a separate **RSA** key used only for unwrapping externally imported keys. Different algorithm, different purpose, configured separately.

## How backup works

When a key is generated or imported, each Key Store shard produces an encrypted backup of its share. The backup encryptor public key wraps the share material so that recovery is possible with the corresponding private key, without requiring the shard's operational credentials.

Backup data is written to the Key Store database and can be exported from there for storage in an immutable location such as an encrypted S3 bucket with versioning enabled and object lock applied.

## Operational guidance

**Verifying backup configuration at install time.** We recommend confirming, after installation, that each shard's backup encryptor public key is present and matches the key you intended. Key creation fails until one is configured, so it is worth checking early.

**Rotating backup keys.** Rotating a backup encryptor public key requires re-encrypting all existing key share backups with the new key, so we recommend coordinating it during a maintenance window.

**Storing the backup private key.** The backup private key has no role in normal vault operation. We recommend keeping it in offline, geographically separate storage, with the same physical and procedural controls you apply to your most sensitive key material.