Rebuilding the kluster

When rebuilding the kluster from scratch, the CNPG containers will be considered as new and will be set up according to their initdb config.

Since most of the clusters here are formally defined as a fresh clusters, the following will happen:

  • in the relevant PVC the pgdata folder will be renamed to pgdata-old
  • a fresh pgdata folder will be created
  • a database with RBAC as defined in the initdb config will be created

This is problematic since the PVC content is the actual state of the database in the present setup. In order to get back to a functional state, some manual intervention is therefore required.

  1. Bootstrap the kubernetes cluster
  2. Wait for the CNPG containers to be up and running - they will be setup fresh at this point
  3. follow the procedure from https://cloudnative-pg.io/documentation/1.20/declarative_hibernation/:
    • hibernate the postgresql cluster
    • wait for the pod to be shut down
    • copy the pgdata-old content to the pgdata folder
    • de-hibernate the postgresql cluster
  4. The database should now be in a functional state

Also see https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Controlling-Resource-Modification/#preserving-changes-made-to-an-applications-annotations-and-labels