gke-backup-dr

GKE Backup & Disaster Recovery

Protects stateful GKE workloads using Backup for GKE. Backup for GKE can capture both Kubernetes resource metadata (manifests, configurations, and secrets) and the underlying persistent volume (PV) data — but volume data and secrets are only captured when the backup plan explicitly enables them (see the flags below).

CLI Reference

# Enable the BackupRestore addon (Slow cluster-level update)
gcloud

[!WARNING] --include-volume-data and --include-secrets BOTH DEFAULT TO FALSE. If you omit them, the backup plan silently produces config-only backups with no persistent volume snapshots and no Secrets. Always pass both flags explicitly when the goal is full workload protection.

Notes:

• The backup-restore command group requires the gcloud beta component (gcloud components install beta).
• --cluster requires the full resource path projects/{project_id}/locations/{location}/clusters/{cluster_name} (or projects/{project_id}/zones/{zone}/clusters/{cluster_name} for zonal clusters), not a bare cluster name.
• Restore plans require exactly one namespaced-resource scope flag: --all-namespaces, --selected-namespaces={ns1},{ns2}, --excluded-namespaces=..., --selected-applications=..., or --no-namespaces.

Restore Safety (CRITICAL)

A restore writes into a live cluster and, depending on the conflict policy, can overwrite or delete existing resources:

• --cluster-resource-conflict-policy=use-existing-version keeps existing cluster-scoped resources (safe default); use-backup-version deletes the existing version first — deleting a CRD deletes all of its CRs.
• --namespaced-resource-restore-mode=fail-on-conflict aborts on any conflict (safe default); merge-skip-on-conflict skips conflicting resources; merge-replace-on-conflict and merge-replace-volume-on-conflict overwrite existing resources or volumes; delete-and-restore deletes entire conflicting namespaces (and all resources in them) before restoring.

Rules:

1. Validate the restore in a non-production target cluster first.
2. Prefer the safe defaults (use-existing-version + fail-on-conflict) unless the user explicitly needs to revert live resources.
3. Always obtain explicit user confirmation before executing a restore into a production cluster, and state which conflict policy is in effect and what it may overwrite or delete.

Best Practices

1. CMEK Encryption: Encrypt backup plans using Customer-Managed Encryption Keys: --encryption-key=projects/{project_id}/locations/{location}/keyRings/{ring}/cryptoKeys/{key}.
2. Scope: Prefer backing up specific namespaces rather than the entire cluster: --selected-namespaces={ns1},{ns2} (instead of --all-namespaces).
3. Application Consistency: Recommend quiescing the database or pausing application writes (e.g. using pre-backup hooks or database-specific tools) prior to backups to ensure data integrity.
4. CSI Volume Snapshots: Ensure that stateful backups utilize GKE's CSI (Container Storage Interface) driver for volume snapshots to capture persistent volume data.
5. Service Terminology: Always explicitly refer to the service as Backup for GKE in your response. This distinguishes it from the broader (but complementary) Google Cloud Backup and Disaster Recovery (DR) Service, ## Golden Path Backup Defaults

The recommended production golden path configuration for Backup for GKE:

• Addon: BackupRestore addon enabled (--update-addons=BackupRestore=ENABLED).
• Volume Inclusion: --include-volume-data explicitly passed (enabled, since the service default is false).
• Secret Inclusion: --include-secrets explicitly passed (enabled, since the service default is false).
• Retention: Defined retention period (e.g. 30 days via --backup-retain-days=30).
• Encryption: CMEK enabled (--encryption-key=...).

Recent Changes

• Cross-project backup and restore (GA): Backup plans can store backups in a different project than the source cluster, and restore plans can target clusters in a third project. Enables centralized backup projects (with immutability/retention managed by a platform team) and cross-project environment seeding without granting access to the source project.
• Pricing change (effective 2026-03-02): The backup management fee moved from pod-based to NAMESPACE-based pricing — charged per non-system namespace in the most recent successful backup of each plan (system namespaces like kube-system are excluded). Existing committed use discount (CUD) holders keep pod-based management pricing until their commitment ends; everyone else moves to the new model. See https://cloud.google.com/products/backup-for-gke/pricing-changes.
• Smart Scheduling: RPO-driven backup scheduling as an alternative to fixed cron schedules — pass --target-rpo-minutes={minutes} instead of --cron-schedule when creating the backup plan (optionally with RPO exclusion windows via --exclusion-windows-file).
• Hyperdisk support: Backup and restore of Hyperdisk ML and Hyperdisk Balanced High Availability volumes is supported on GKE clusters running 1.33.1-gke.1959000 and later (Hyperdisk throughput, extreme, and balanced types are also supported).

Troubleshooting & Common Pitfalls (CRITICAL)

[!IMPORTANT] Slow Operations: Enabling the BackupRestore addon (--update-addons=BackupRestore=ENABLED) triggers a slow Google Cloud control plane cluster update that takes several minutes. * Rule: Do not run a terminal loop waiting for the GKE Backup addon to become active. * Action: Provide the command to enable the addon, explain that the operation will proceed in the background, and immediately proceed to write the backup plan configs. Do not block.