# Terraform example: GCP three-cluster mesh

Generated reference for the GCP example that provisions the three-cluster mesh. For the guided setup, see [Provision on GCP with Terraform](/deploy/provision-gcp).

### Requirements

| Name | Version |
| ---- | ------- |
| terraform | >= 1.14 |
| google | ~> 7.42.0 |
| google-beta | ~> 7.42.0 |
| helm | ~> 3.0 |
| kubernetes | ~> 3.0 |
| local | ~> 2.0 |
| tls | ~> 4.0 |

### Providers

| Name | Version |
| ---- | ------- |
| google.hub | 7.42.0 |
| local | 2.9.0 |
| tls | 4.3.0 |

### Modules

| Name | Source | Version |
| ---- | ------ | ------- |
| cluster\_hub | ../ | n/a |
| cluster\_spoke\_0 | ../ | n/a |
| cluster\_spoke\_1 | ../ | n/a |
| ks\_psc\_spoke\_0 | ../psc-gcp | n/a |
| ks\_psc\_spoke\_1 | ../psc-gcp | n/a |
| nats\_psc\_spoke\_0 | ../psc-gcp | n/a |
| nats\_psc\_spoke\_1 | ../psc-gcp | n/a |
| peering\_hub\_spoke\_0 | ../peering-gcp | n/a |
| peering\_hub\_spoke\_1 | ../peering-gcp | n/a |
| pol\_reader\_psc\_spoke\_0 | ../psc-gcp | n/a |
| pol\_reader\_psc\_spoke\_1 | ../psc-gcp | n/a |
| pol\_writer\_psc\_spoke\_0 | ../psc-gcp | n/a |
| pol\_writer\_psc\_spoke\_1 | ../psc-gcp | n/a |
| relay\_psc\_spoke\_0 | ../psc-gcp | n/a |
| relay\_psc\_spoke\_1 | ../psc-gcp | n/a |

### Resources

| Name | Type |
| ---- | ---- |
| [google\_compute\_global\_address.front\_door](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_global_address) | resource |
| [google\_dns\_record\_set.front\_door](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dns_record_set) | resource |
| [local\_sensitive\_file.chart\_values](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/sensitive_file) | resource |
| [local\_sensitive\_file.extra\_values](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/sensitive_file) | resource |
| [local\_sensitive\_file.kubeconfig](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/sensitive_file) | resource |
| [tls\_private\_key.external\_rule\_signing](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) | resource |
| [tls\_private\_key.imported\_key\_encryptor](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) | resource |

### Inputs

| Name | Description | Type | Default | Required |
| ---- | ----------- | ---- | ------- | :------: |
| artifactory\_password | JFrog Artifactory password / access token. | `string` | n/a | yes |
| artifactory\_username | JFrog Artifactory username (used by every cluster to pull chart images). | `string` | n/a | yes |
| gcp\_project | Base GCP project ID used by all three clusters unless overridden per-cluster below. For a single-project PoC, set just this. | `string` | n/a | yes |
| mpc\_key\_stores | MPC shard identities, one per cluster. Hard-pinned to 3-of-3, so exactly 3 entries. Required (no default) — set in the gitignored terraform.tfvars. | <pre>list(object(\{<br />    name                     = string<br />    backupEncryptorPublicKey = string<br />    importedKeyEncryptorPem  = optional(string, "")<br />  }))</pre> | n/a | yes |
| policy\_servers | Policy-server identities, one per cluster (3 in the 3-of-3 deployment). Required (no default). The reader/writer keypairs self-generate on first boot; the reader's external-rule signing key is generated by the example's tls\_private\_key. | <pre>list(object(\{<br />    writerName = string<br />    readerName = string<br />  }))</pre> | n/a | yes |
| relay\_api\_key | Relay API key (all clusters use the same key). | `string` | n/a | yes |
| app\_image\_pull\_policy | imagePullPolicy for the orchestrator/policy-server/vault-ui containers. null keeps the chart default (IfNotPresent); set "Always" for a mutable tag like latest so restarts re-pull. | `string` | `null` | no |
| app\_image\_registry | Registry host+namespace for the orchestrator/policy-server/vault-ui images. null (default) keeps the sodot-docker-oci production registry. | `string` | `null` | no |
| app\_image\_tag | Tag for the orchestrator/policy-server/vault-ui images when app\_image\_registry is set (e.g. "latest"). Ignored when app\_image\_registry is null (the tag then follows image\_tags). | `string` | `"latest"` | no |
| chart\_ha | When true (default) the whole stack runs HA: orchestrator/policy-server/UI at 2 replicas (with HPAs), the MPC keystore at 2 replicas, clustered (3-node RAFT) NATS, relay HPA min 2, PodDisruptionBudgets on, and the pool-wide confidential-node floor sized to 2. When false, main.tf writes extra\_helm\_values collapsing every workload to a single replica, disabling HPAs, using non-clustered NATS, PDBs off, and dropping the confidential floor to 1 — the cheapest footprint (a single orchestrator is a hard signing ceiling — the accepted cheap-mode tradeoff). | `bool` | `true` | no |
| chart\_oci\_registry | OCI registry (incl. oci:// scheme, no chart name) the operator should install the crypto-vault chart from, e.g. "oci://repo.sodot.dev/sodot-helm-charts". Drives the helm\_handoff output hint only. null (default) points at the local chart dir. | `string` | `null` | no |
| chart\_version | crypto-vault chart version the operator should install when chart\_oci\_registry is set. Drives the helm\_handoff output hint only. | `string` | `null` | no |
| confidential\_node\_machine\_type | N2D machine type for the enforced SEV-SNP Confidential keystore node pool, applied identically to all three shards. Defaults to n2d-standard-4. | `string` | `"n2d-standard-4"` | no |
| confidential\_node\_max\_count | Pool-wide autoscaling ceiling for the Confidential keystore node pool. The total floor tracks the keystore replica count (chart\_ha: 2 else 1) via main.tf. | `number` | `3` | no |
| db\_app\_username | Generated-password login and database owner used only when db\_use\_iam\_auth=false, and a known ownership-migration source on the IAM path. Forwarded to every cluster. Renaming it on an existing stack requires recording the old value, applying the new value in a maintenance window, and manually migrating the recorded role before workloads resume; Terraform cannot infer the former value after this input changes. | `string` | `"crypto_vault_app"` | no |
| db\_availability\_type | Cloud SQL availability: REGIONAL (default — synchronous standby failover, matching AWS db\_multi\_az=true) or ZONAL (single zone, cheaper PoC). | `string` | `"REGIONAL"` | no |
| db\_deletion\_protection | Cloud SQL deletion protection. Defaults to true (production-safe). Set false in terraform.tfvars for a throwaway PoC where painless `terraform destroy` matters. | `bool` | `true` | no |
| db\_final\_backup\_enabled | Create a final Cloud SQL backup on instance deletion. Defaults to true, matching the AWS example's production-safe final-snapshot default. Set false for throwaway/CI stacks. | `bool` | `true` | no |
| db\_final\_backup\_retention\_days | Days to retain each final Cloud SQL backup (1-365). | `number` | `30` | no |
| db\_retain\_backups\_on\_delete | Retain Cloud SQL automated/on-demand backups after instance deletion. Defaults to true for recovery; set false for throwaway/CI stacks. | `bool` | `true` | no |
| db\_tier | Cloud SQL machine tier for each cluster's PostgreSQL (stands in for the AWS db.t3.small). Defaults to db-custom-1-3840 (1 vCPU / 3.75 GB). Raise it (e.g. db-custom-2-7680) for more headroom on the orchestrator's SELECT ... FOR UPDATE row-lock path under higher concurrency. | `string` | `"db-custom-1-3840"` | no |
| db\_use\_iam\_auth | Cloud SQL IAM database authentication path (default true): each DB-using pod connects through its colocated Cloud SQL Auth Proxy sidecar as that workload's distinct GSA-derived IAM user with an EMPTY password. Set false to fall back to one generated-password google\_sql\_user (mirrors the RDS master-secret pattern). Terraform automatically migrates database ownership in either auth-mode direction when db\_app\_username is unchanged. | `bool` | `true` | no |
| deployment\_name | Prefix used to name resources in each project (e.g. "crypto-vault"). Must be RFC1035-safe (lowercase, hyphens, leading letter) and \<= 13 characters — it seeds each cluster's app\_name (`<name>-spoke-0`), whose generated GSA account IDs must fit GCP's 30-character limit. The child module also requires `length(app_name) + length(project_id) <= 49` so exact Cloud SQL IAM usernames fit PostgreSQL's 63-byte identifier limit; shorten this prefix when using long project IDs. | `string` | `"crypto-vault"` | no |
| dns\_managed\_zone | Name (the Cloud DNS *resource* name, not the dns\_name) of a managed zone in which to create the UI + orchestrator host A records, pointing at the reserved global static IP(s). Leave empty to skip DNS entirely — point your own DNS provider at the ui\_front\_door\_ip / orchestrator\_front\_door\_ip outputs instead. | `string` | `""` | no |
| dns\_project | Project that owns dns\_managed\_zone, when the Cloud DNS zone lives outside the hub project. null = the hub project. | `string` | `null` | no |
| gcp\_project\_hub | Override the GCP project for the hub cluster (shard 1, runs ORC/NATS/UI). null = use gcp\_project. | `string` | `null` | no |
| gcp\_project\_spoke\_0 | Override the GCP project for spoke-0 (shard 0). null = use gcp\_project. | `string` | `null` | no |
| gcp\_project\_spoke\_1 | Override the GCP project for spoke-1 (shard 2). null = use gcp\_project. Give this its own project for the one-shard-per-project trust boundary. | `string` | `null` | no |
| gcp\_region | GCP region shared by all three regional GKE clusters and their PSC endpoints. Launch certification is limited to us-central1. | `string` | `"us-central1"` | no |
| general\_node\_machine\_type | Machine type for the general node pool carrying the stateless workloads (orchestrator/policy-server/UI/NATS/relay). Defaults to e2-standard-4. | `string` | `"e2-standard-4"` | no |
| general\_node\_max\_count | Per-zone autoscaling ceiling for the general node pool. | `number` | `4` | no |
| gke\_deletion\_protection | GKE cluster deletion protection, forwarded to every cluster. Defaults to true (production-safe: `terraform destroy` refuses to delete the cluster and everything on it). Set false in terraform.tfvars for a throwaway PoC where painless teardown matters — the Cloud SQL counterpart is db\_deletion\_protection, and either one left on stops a destroy. | `bool` | `true` | no |
| gke\_endpoint\_public\_access | Expose each GKE control plane's public endpoint. The building-block module defaults this to false (private-only); this apply-from-a-laptop example explicitly opts in by default. Set false when Terraform runs in-VPC, over peering, or through VPN. | `bool` | `true` | no |
| gke\_node\_pool\_delete\_timeout | Timeout supplied to each GKE node-pool deletion phase, forwarded to every cluster. The 2h default preserves the pinned Google provider's behavior; short-lived CI may lower it to fit a bounded retry into its teardown budget. | `string` | `"2h"` | no |
| gke\_version | GKE version target. Empty (default) tracks release\_channel and receives auto-upgrades. For optional fixed-version mode, pair a full version (for example "1.35.7-gke.100") with release\_channel=UNSPECIFIED. | `string` | `""` | no |
| google\_admin\_emails | Setup admin emails for the bootstrap flow. Forwarded to every cluster. | `list(string)` | `[]` | no |
| google\_oauth\_client\_id | Google OAuth client ID for the vault UI. Forwarded to EVERY cluster (each policy-server-reader verifies the Google JWT). | `string` | `""` | no |
| image\_tags | Per-component image tag overrides (orchestrator/policy\_server/ui). null keeps the chart's pinned-release default. | <pre>object(\{<br />    policy\_server = optional(string, null)<br />    orchestrator  = optional(string, null)<br />    ui            = optional(string, null)<br />  })</pre> | `{}` | no |
| kms\_deletion\_protection | Protect each keystore KMS key from Terraform deletion. The building-block module defaults to true; this explicitly throwaway-friendly example defaults to false so unattended destroy/reaper flows can complete. Set true before the first apply for a long-lived stack. | `bool` | `false` | no |
| kms\_key\_protection\_level | Where each cluster's keystore crypto-key material lives, forwarded to all three clusters: "HSM" (default — FIPS 140-2 Level 3 Cloud HSM, matching the module default and the AWS sibling's posture) or "SOFTWARE". The keystore's runtime contract is identical either way — same key resource, same cryptoKeyEncrypterDecrypter binding, same Encrypt/Decrypt calls — only the material's residency and the price differ. Set "SOFTWARE" for a throwaway PoC or CI stack. ⚠️ Choose before the FIRST apply: changing it on a live deployment replaces the crypto key and makes shares wrapped under the old key unrecoverable. | `string` | `"HSM"` | no |
| labels | Labels merged onto every labelable resource via the providers' default\_labels. GCP labels are lowercase/hyphen only. | `map(string)` | <pre>\{<br />  "project": "crypto-vault"<br />}</pre> | no |
| managed\_certificate | Attach a Google-managed TLS certificate to the UI/orchestrator Ingress (the GCLB terminates TLS). true (default) is the zero-cert-management path; set false only if you terminate TLS yourself via a pre-created certificate referenced in the chart's ingress annotations. | `bool` | `true` | no |
| master\_authorized\_networks | CIDRs allowed to reach each cluster's control-plane endpoint when gke\_endpoint\_public\_access is true. This example defaults to UNRESTRICTED (\["0.0.0.0/0"]) for first-run convenience; the endpoint is still IAM/cert-gated, but TIGHTEN this before any non-test use. | `list(string)` | <pre>\[<br />  "0.0.0.0/0"<br />]</pre> | no |
| mpc\_key\_store\_image\_repository | Repository for the non-AWS MPC keystore image that runs on the GCP Confidential node pool. Rendered into `mpcKeyStoreImage.nonAws.repository`. | `string` | `"repo.sodot.dev/sodot-docker-images/mpc-key-store"` | no |
| mpc\_key\_store\_image\_tag | Tag for the non-AWS MPC keystore image used by GCP. Rendered into `mpcKeyStoreImage.nonAws.tag`. | `string` | `"0.4.0"` | no |
| name\_nonce | Forwarded to each cluster module's name\_nonce: a short suffix on ONLY the Cloud SQL instance and the Cloud KMS key ring + key — the two GCP name families that stay reserved after deletion. Empty (default) changes nothing. The E2E sets it so repeated create/destroy cycles under one stable deployment\_name never collide with their own tombstones. | `string` | `""` | no |
| observability | OTLP export for the orchestrator + policy-server. Leave otlp\_endpoint empty to keep export off (stdout only). Mirrors the module's observability variable field set. | <pre>object(\{<br />    otlp\_endpoint            = optional(string, "")<br />    otlp\_protocol            = optional(string, "grpc") # "grpc" or "http/protobuf"<br />    otlp\_console             = optional(bool, true)<br />    otlp\_timeout             = optional(string, "")<br />    otlp\_resource\_attributes = optional(string, "")<br />    otlp\_log\_filter          = optional(string, "")<br />    otlp\_headers\_secret\_name = optional(string, "")<br />    otlp\_headers\_secret\_key  = optional(string, "")<br />  })</pre> | `{}` | no |
| orchestrator\_host | Orchestrator REST API hostname on YOUR domain (e.g. vault-api.example.com), distinct from ui\_host. Empty = no external API front door. Uses the SAME reserved-IP + managed-certificate + Cloud DNS mechanism as the UI. | `string` | `""` | no |
| private\_cluster | Private nodes (no public node IPs; egress via Cloud NAT). true (default) is production-safe. This is independent of control-plane endpoint exposure. | `bool` | `true` | no |
| release\_channel | GKE release channel: RAPID, REGULAR (default), STABLE, EXTENDED, or UNSPECIFIED. Use UNSPECIFIED with gke\_version only for optional fixed-version mode; it disables node auto-upgrade. | `string` | `"REGULAR"` | no |
| ui\_client\_secret\_ref | BYO mode: name of an existing Kubernetes Secret in the chart namespace holding the UI's Google OAuth client secret under key CLIENT\_SECRET. Forwarded to the hub module. Empty = use the literal-var path (ui\_google\_oauth\_client\_secret). | `string` | `""` | no |
| ui\_google\_oauth\_client\_secret | Convenience mode: literal Google OAuth client secret. Forwarded to the hub module, which creates a Kubernetes Secret and wires ui.clientSecretRef to it. Set in terraform.tfvars (gitignored) or via TF\_VAR\_ui\_google\_oauth\_client\_secret. | `string` | `""` | no |
| ui\_host | UI hostname on YOUR domain (e.g. vault.example.com). Drives the UI Ingress host, the Google-managed certificate domain, and the UI's OAuth REDIRECT\_URI + WebAuthn origin (register https://\<ui\_host>/auth/callback on the Google OAuth client). Empty = no UI front door (the UI Service stays cluster-internal). | `string` | `""` | no |

### Outputs

| Name | Description |
| ---- | ----------- |
| deployment\_preflight | Per-shard expected identity and PSC-ingress contracts used for internal deployment-handoff verification. |
| helm\_handoff | Where the per-cluster Helm handoff files were written. Follow README.md "Installing the chart" for the Stage 1 / Stage 2 helm commands. |
| hub\_cluster\_name | Hub GKE cluster name. |
| kubectl\_hub | Command to fetch a kubeconfig entry for the hub. Regional cluster, so --region (not --zone). |
| kubectl\_spoke\_0 | Command to fetch a kubeconfig entry for spoke-0. |
| kubectl\_spoke\_1 | Command to fetch a kubeconfig entry for spoke-1 (shard 2). |
| orchestrator\_front\_door\_ip | Reserved GLOBAL static IP for the orchestrator API Ingress. Point orchestrator\_host at this if you manage DNS yourself. null when no orchestrator host is set. |
| orchestrator\_url | The orchestrator REST API URL. Same reachability preconditions as ui\_url. null when no orchestrator host is set. |
| psc\_endpoint\_addresses | Consumer-side PSC endpoint IPs the rendered chart values dial. spoke-\* nats/relay are the IPs each spoke uses for the hub's NATS/relay; the \*\_ks / *\_pol\_* IPs are what the HUB uses to reach each spoke's KS/POL. |
| psc\_service\_attachments | Producer-side PSC service-attachment IDs, by logical service. Useful for `gcloud compute service-attachments describe` when a consumer endpoint is stuck PENDING (check the accept list). |
| spoke\_0\_cluster\_name | Spoke-0 GKE cluster name. |
| spoke\_1\_cluster\_name | Spoke-1 (shard 2) GKE cluster name. |
| ui\_front\_door\_ip | Reserved GLOBAL static IP for the UI Ingress. With dns\_managed\_zone set, the A record is created for you; otherwise point your own DNS (ui\_host -> this IP). null when no UI host is set. |
| ui\_url | The vault UI URL (TLS terminates at the Google-managed cert on the GKE Ingress). Reachable once (a) the chart is installed so the Ingress exists, (b) DNS for ui\_host resolves to ui\_front\_door\_ip, and (c) the managed certificate finishes provisioning. null when no UI host is set. |