# Provision on GCP with Terraform

The GCP example Terraform module provisions three regional GKE clusters in one region, Cloud SQL,
Cloud KMS, Workload Identity, private networking, and the hub front doors. It writes per-cluster Helm
handoff files to `./out/`, but never installs the chart.

:::warning\[GKE Confidential Nodes security model]
The Key Store runs as a pod on a GKE Confidential node pool. Confidential Nodes encrypt the node VM's
memory against the host and hypervisor. Cloud KMS access is controlled by the Key Store's Workload
Identity, without per-workload attestation. This differs from AWS Nitro Enclaves, where a KMS policy
can require a measured enclave identity.
:::

:::warning\[Example module disclaimer]
The combined three-cluster example is an integration and proof-of-concept harness. Even when you set
one GCP project per shard, one Terraform state and one caller credential span all three parties. For
production, each party should administer its own project, Terraform root and backend, credentials,
GKE cluster, Cloud SQL databases, Workload Identities, and HSM key. Use the
[`sodot/crypto-vault/gcp`](/deploy/terraform/module-gcp) building block separately for each party and
exchange only the PSC and cluster-identity handoff data required to connect the topology.
:::

## 1. Get a JFrog access token

Open the **Set Me Up** panel on the `sodot-terraform-modules` and `sodot-helm-charts` repositories in
the JFrog UI. Export the resulting credentials:

```bash
export ARTIFACTORY_USERNAME="your-username"
export ARTIFACTORY_TOKEN="your-access-token"
export TF_TOKEN_repo_sodot_dev="$ARTIFACTORY_TOKEN"
```

## 2. Download the GCP example

Browse the versions in
[`sodot-terraform-modules/sodot/crypto-vault-example/gcp`](https://repo.sodot.dev/ui/repos/tree/General/sodot-terraform-modules/sodot/crypto-vault-example/gcp/)
and select the latest `vX.Y.Z.zip`, or download it with curl:

```bash
curl -fSL -u "$ARTIFACTORY_USERNAME:$ARTIFACTORY_TOKEN" \
  -o crypto-vault-example-gcp.zip \
  https://repo.sodot.dev/artifactory/sodot-terraform-modules/sodot/crypto-vault-example/gcp/vX.Y.Z.zip

unzip crypto-vault-example-gcp.zip -d crypto-vault-example-gcp
cd crypto-vault-example-gcp
```

The example references the version-matched
[`sodot/crypto-vault/gcp`](/deploy/terraform/module-gcp) building-block module and its connectivity
add-ons from the private Terraform registry.

:::warning\[Terraform 1.14+ required]
The example and building-block modules require Terraform `>= 1.14`. Check with `terraform version`
before continuing.
:::

## 3. Prepare GCP projects and authentication

For production, use three projects, one per shard. This keeps each Key Store and Policy Server inside
an independent IAM boundary. A single-project deployment still creates three separate VPCs and GKE
clusters, but it does not preserve that account-level separation.

Authenticate Terraform with Application Default Credentials and install the GKE authentication plugin:

```bash
gcloud auth application-default login
gcloud components install gke-gcloud-auth-plugin
gke-gcloud-auth-plugin --version
```

Enable the required APIs in every project. Replace the three project IDs below:

```bash
for project_id in cv-spoke-0 cv-hub cv-spoke-1; do
  gcloud services enable \
    compute.googleapis.com \
    container.googleapis.com \
    sqladmin.googleapis.com \
    servicenetworking.googleapis.com \
    cloudkms.googleapis.com \
    secretmanager.googleapis.com \
    dns.googleapis.com \
    --project "$project_id"
done
```

Production requires regional GKE, an N2D machine type with AMD SEV-SNP, and Cloud HSM, so the region
must be one that offers SEV-SNP on N2D: `asia-southeast1`, `europe-west3`, `europe-west4`, or
`us-central1`. See [Supported production regions](/deploy/prerequisites#supported-production-regions).
Verify N2D, GKE, Cloud SQL, and Cloud HSM quota in your chosen region before applying. Note that N2D
availability alone is not sufficient: `europe-west1` offers N2D and does not support SEV-SNP.

:::warning\[Trust boundary]
If two or more clusters use the same GCP project, a compromise of that project can expose multiple MPC
shards. Use one project per shard in production.
:::

## 4. Configure `terraform.tfvars`

Copy the template and edit it:

```bash
cp terraform.tfvars.example terraform.tfvars
$EDITOR terraform.tfvars
```

Set these variables:

| Variable | Required | Description |
|---|---|---|
| `gcp_project` | yes | Base project inherited by all clusters unless a per-cluster override is set. |
| `gcp_project_hub` / `gcp_project_spoke_0` / `gcp_project_spoke_1` | production | Set three distinct project IDs to preserve the one-shard-per-project trust boundary. |
| `gcp_region` | no, default `us-central1` | Region shared by all three regional GKE clusters and their PSC endpoints. Must offer SEV-SNP on N2D plus Cloud HSM. See [Supported production regions](/deploy/prerequisites#supported-production-regions). |
| `artifactory_username` / `artifactory_password` | yes, sensitive | JFrog credentials used by the clusters to pull Sodot images. |
| `relay_api_key` | yes, sensitive | Shared relay API key. Generate a non-empty random value. |
| `mpc_key_stores` | yes, sensitive | Exactly three shard entries with `name` and `backupEncryptorPublicKey`. |
| `policy_servers` | yes, sensitive | Exactly three entries with `writerName` and `readerName`. |
| `ui_host` | recommended | UI hostname. Setting it enables the GKE Ingress and drives OAuth and WebAuthn origins. |
| `orchestrator_host` | no | Separate hostname for the Orchestrator API. |
| `dns_managed_zone` / `dns_project` | no | Cloud DNS managed-zone resource name and its project. Leave empty to manage DNS elsewhere. |
| `google_oauth_client_id` / `ui_google_oauth_client_secret` / `google_admin_emails` | required for UI login | Google OAuth credentials and Setup Admin emails. |
| `gke_endpoint_public_access` | no, example default `true` | Expose each GKE control plane's public endpoint for an apply-from-a-laptop workflow. Set `false` when Terraform runs in the VPC, through peering, or over VPN. The building-block module defaults to private-only. |
| `master_authorized_networks` | strongly recommended | CIDRs allowed to reach public GKE control-plane endpoints. Replace the example's `0.0.0.0/0` first-run default before any non-test use. |
| `confidential_node_machine_type` | no, default `n2d-standard-4` | N2D machine type for the enforced SEV-SNP Confidential Key Store pool. |
| `db_availability_type` | no, default `REGIONAL` | Cloud SQL availability. Keep `REGIONAL` for synchronous standby and failover; use `ZONAL` only for a cheaper test deployment. |
| `db_deletion_protection` / `gke_deletion_protection` | no, default `true` | Keep both guards enabled for long-lived deployments. Set them to `false` and apply before destroying a disposable environment. |
| `db_final_backup_enabled` / `db_final_backup_retention_days` / `db_retain_backups_on_delete` | no, production-safe defaults | Control the final Cloud SQL backup and whether backups survive instance deletion. The defaults create a final backup, retain it for 30 days, and retain existing backups. |
| `kms_key_protection_level` | no, default `HSM` | Store Key Store KMS material in Cloud HSM. `SOFTWARE` is for throwaway test environments. Changing this on a live deployment replaces the key and makes shares wrapped by the old key unrecoverable. |
| `kms_deletion_protection` | no, example default `false` | The combined example is teardown-friendly. Set `true` before the first apply for a long-lived deployment. The building-block module defaults to `true`. |
| `chart_ha` | no, default `true` | Set `false` only for a low-cost test footprint. |
| `deployment_name` / `name_nonce` / `labels` | no | Resource-name prefix, optional suffix for Cloud SQL and KMS names that remain reserved after deletion, and GCP labels. Leave `name_nonce` empty for a stable long-lived deployment. |
| `gke_version` / `release_channel` / `private_cluster` | no | GKE version policy and private-node setting. Leave `gke_version` empty to follow the selected release channel, or pair an explicit version with `release_channel=UNSPECIFIED` when you need a pin. Keep private nodes enabled. |
| `gke_node_pool_delete_timeout` | no, default `2h` | Timeout for each GKE node-pool deletion. Keep the default unless a bounded CI teardown needs a shorter failure window. |
| `confidential_node_max_count` / `general_node_machine_type` / `general_node_max_count` | no | Autoscaling ceilings and the standard-node machine type. |
| `db_tier` / `db_use_iam_auth` / `db_app_username` | no | Cloud SQL sizing and application authentication. IAM authentication is enabled by default, with a distinct database identity and in-pod Cloud SQL Auth Proxy for each workload. |
| `managed_certificate` | no, default `true` | Attach Google-managed TLS certificates to enabled front doors. |
| `ui_client_secret_ref` | no | Use an existing Kubernetes Secret instead of passing `ui_google_oauth_client_secret`. |
| `mpc_key_store_image_repository` / `mpc_key_store_image_tag` | no, release-pinned defaults | Override the GCP Confidential Node MPC Key Store image. Keep both version-matched defaults for released deployments. |
| `image_tags` / `app_image_registry` / `app_image_tag` / `app_image_pull_policy` | no | Advanced application-image overrides. Keep the chart-pinned defaults for released deployments. |
| `observability` | no | OTLP export settings for the Orchestrator and Policy Servers. Empty endpoint keeps export off. |
| `chart_oci_registry` / `chart_version` | no | Chart reference shown in `helm_handoff`. Published example archives carry version-matched defaults. |

The [GCP example reference](/deploy/terraform/example-gcp) lists every input.

:::note\[Backup key gate]
Every `mpc_key_stores` entry needs a unique `backupEncryptorPublicKey` before the vault can create or
import keys. Generate the key pairs offline and keep each private key outside GCP. See
[Backup and disaster recovery](/deploy/operations/backup-dr).
:::

:::warning\[Terraform state security]
Terraform state and the generated `./out/` files contain sensitive values. Use an encrypted,
access-controlled GCS backend with versioning, and never commit the handoff files.
:::

## 5. Configure TLS and DNS

Set `ui_host` to enable the UI's GKE Ingress. The example reserves a global static IP, configures the
chart to use that IP, and attaches a Google-managed certificate. If you also set `dns_managed_zone`,
Terraform creates the A record. Otherwise, point your DNS provider at `ui_front_door_ip`.

```hcl
ui_host         = "vault.example.com"
orchestrator_host = "vault-api.example.com"

managed_certificate = true
dns_managed_zone     = "example-com"
# dns_project        = "shared-dns-project"
```

Register `https://vault.example.com/auth/callback` as an authorized redirect URI on the Google OAuth
client. The managed certificate becomes active only after the hostname resolves to the reserved IP and
the Helm install creates the Ingress.

## 6. Apply

```bash
terraform init -upgrade
terraform apply
```

Terraform creates the three VPCs and Dataplane V2 GKE clusters, Confidential node pools, Cloud SQL
instances, HSM-backed Cloud KMS keys, per-workload Workload Identity bindings, GKE-managed internal
load balancers with reserved VIPs, Private Service Connect endpoints, a dedicated NATS StorageClass,
and the optional front-door resources. It then writes the Helm handoff files.

## 7. Verify the handoff

Inspect the summary and confirm access to all three clusters:

```bash
terraform output helm_handoff
terraform output ui_front_door_ip

kubectl --kubeconfig out/cluster-0.kubeconfig get nodes
kubectl --kubeconfig out/cluster-1.kubeconfig get nodes
kubectl --kubeconfig out/cluster-2.kubeconfig get nodes
```

The generated files use the same shard mapping as the AWS example:

| File | Cluster |
|---|---|
| `out/cluster-0.values.yaml` | Spoke, shard 0 |
| `out/cluster-1.values.yaml` | Hub, shard 1 |
| `out/cluster-2.values.yaml` | Spoke, shard 2 |
| `out/cluster-{0,1,2}.kubeconfig` | Per-cluster kubeconfig using `gke-gcloud-auth-plugin` |
| `out/extra.values.yaml` | Present only when `chart_ha = false` |

The values already include `cloudProvider: gcp`, the canonical `gcp.kmsKey` rendered as
`[storage_encryptor.gcp_kms].kms_key`, Confidential node
placement, distinct Workload Identities and in-pod Cloud SQL Auth Proxy settings for each database
workload, role-specific PSC NetworkPolicy CIDRs, private service addresses, and the GKE ingress
settings. The same files are used for both Helm stages. No post-install address patching is required.

## Production: operate one root per party

The combined example shows the complete topology, but its shared state and credentials do not enforce
the production trust boundary. For production, create three independently administered Terraform
roots. Each root calls the [`sodot/crypto-vault/gcp`](/deploy/terraform/module-gcp) building block once,
uses a party-controlled GCS backend, and authenticates only to that party's project and GKE cluster.

| Party | Building-block role | Producer LoadBalancer Services |
|---|---|---|
| Spoke, shard 0 | `is_hub = false`, `shard_index = 0`, and only shard 0's `mpc_key_store` and `policy_server` inputs | Enable `ks_internal_lb` and `pol_internal_lb`. |
| Hub, shard 1 | `is_hub = true`, `shard_index = 1`, and only shard 1's `mpc_key_store` and `policy_server` inputs | Enable `nats_internal_lb` and `relay_internal_lb`. |
| Spoke, shard 2 | `is_hub = false`, `shard_index = 2`, and only shard 2's `mpc_key_store` and `policy_server` inputs | Enable `ks_internal_lb` and `pol_internal_lb`. |

Use the same `helm_release_name`, namespace, region, UI origin, Google OAuth client ID, Setup Admin
emails, and relay API key in the roots where those values apply. Give every party non-overlapping node,
Pod, Service, control-plane, Private Service Access, and PSC NAT CIDRs. Keep the OAuth client secret in
the hub root only.

Every `psc-gcp` call must map both `google.producer` and `google.consumer`, even in producer-only or
consumer-only mode. Map the unused alias to that party's configured Google provider.

Wire the roots in this order:

1. Agree on the CIDR plan and exchange the three project IDs. PSC producer accept lists use the
   consumer project IDs. Each producer's `internal_lb_source_cidrs` uses its dedicated PSC NAT CIDR.
2. Apply the hub root with empty `remote_mpc_key_stores` and `remote_policy_servers` inputs to create
   its local infrastructure and the NATS and relay LoadBalancer Services. Do not install Helm yet. Use
   one [`psc-gcp`](/deploy/terraform/psc-gcp) module instance per producer role: one for
   `nats_ilb_forwarding_rule` and one for `relay_ilb_forwarding_rule`. Share the two
   `service_attachment_self_link` outputs with both spoke parties.
3. In each spoke root, create consumer PSC endpoints for the hub attachments. Feed the resulting
   `consumer_endpoint_ip` or `consumer_endpoint_dns_name` into `nats_hub_url` as
   `nats://<endpoint>:4222` and `relay_address` as `http://<endpoint>:80`. Create one producer-side
   `psc-gcp` module instance for each of `ks_ilb_forwarding_rule`,
   `pol_reader_ilb_forwarding_rule`, and `pol_writer_ilb_forwarding_rule`, allowing only the hub
   project.
4. Each spoke shares those three service-attachment links and the non-secret `ks_shard_name`,
   `shard_index`, `pol_component_name`, and `policy_server_port` outputs with the hub operator.
5. The hub creates one consumer endpoint for each spoke attachment. Build `remote_mpc_key_stores`
   from each spoke's Key Store name, shard index, and hub-local `http://<endpoint>:8080` URL. Build
   `remote_policy_servers` from each spoke's Policy Server component name and the hub-local
   `http://<endpoint>:<policy_server_port>` reader and writer URLs.
6. Apply all three roots again with the final endpoint inputs, then export one final values file from
   each root. Do not install Helm from an intermediate values document.

The Key Store and Policy Server identity keypairs are not part of this handoff. Each service generates
its identity on first boot, and the Orchestrator propagates the cluster identity set during Stage 2.
Do not exchange identity private keys, database credentials, KMS material, values files, or
kubeconfigs between parties.

Have each root re-export its own sensitive Helm output:

```hcl
output "helm_values_yaml" {
  value     = module.cluster.helm_values_yaml
  sensitive = true
}

output "helm_release_name" {
  value = module.cluster.helm_release_name
}

output "namespace" {
  value = module.cluster.namespace
}
```

Each party writes its result locally, using its assigned shard index:

```bash
SHARD_INDEX=0 # Use 1 for the hub or 2 for the other spoke.
install -d -m 0700 out
terraform output -raw helm_values_yaml > "out/cluster-${SHARD_INDEX}.values.yaml"
chmod 0600 "out/cluster-${SHARD_INDEX}.values.yaml"
```

Each party then runs its own [two-stage Helm command](/deploy/helm/install) with that values file and a
party-controlled kubeconfig. The two spokes and hub perform Stage 1 independently. After all three
parties report Stage 1 readiness, the hub operator performs Stage 2. The parties coordinate readiness,
but do not centralize their Terraform state, deployment credentials, values files, or kubeconfigs.

## Cross-cluster connectivity

The example creates private service paths before Helm runs:

* The hub exposes NATS and relay through internal load balancers and Private Service Connect service
  attachments consumed by both spokes.
* Each spoke exposes its Key Store and Policy Server through Private Service Connect endpoints consumed
  by the hub.
* VPC Network Peering provides the supporting private routing between each hub and spoke VPC.

Every PSC endpoint must be in the same region as its target service attachment. Global access lets
clients in another region reach that regional endpoint; it does not allow a cross-region
endpoint-to-attachment pairing. The combined example therefore places all three clusters and PSC
endpoints in `gcp_region`.

If you use your own network design, preserve the ports listed in
[Architecture and topology](/deploy/architecture#required-network-paths), and never expose Key Store or
Policy Server services publicly.

Once all three clusters are reachable, continue to [Install with Helm](/deploy/helm/install). Before
production handoff, Sodot validates the regional GKE and SEV-SNP pool, private control-plane posture,
per-workload identities, PSC contracts, Cloud SQL safeguards, and HSM-backed KMS key for each party.