Rotate OpenStack Credentials

The following document describes the necessary actions that must be taken LCM-wise after an OpenStack credential rotation.

1. Change the OpenStack credentials (how to do that is out of scope).

2. Trigger rotation of managed components

   ch-k8s-lbaas disabledch-k8s-lbaas enabled

   Immediately afterwards renew the OpenStack connection of the Kubernetes cluster. This will update the kube-system/cloud-config secret and restart the cloud-controller-manager, csi-cinder-controllerplugin and csi-cinder-nodeplugin in the kube-system namespace.

   $ bash managed-k8s/actions/apply-k8s-supplements.sh connect-k8s-to-openstack.yaml
   

3. Verify that everything is able to come up after it has been restarted.

4. Check which Pods besides the above mentioned have mounted the kube-system/cloud-config secret:

   kubectl get pods --all-namespaces -o json | jq --raw-output '.items[]
       | select(.spec | has("volumes"))
       | select(.spec.volumes[].secret.secretName=="cloud-config")
       | "\(.metadata.namespace)/\(.metadata.name)"'
   

5. Check which Pods are referencing the kube-system/cloud-config secret in their env:

   kubectl get pods--all-namespaces -o json | jq --raw-output '.items[]
       | select(.spec.containers[].env[]?.valueFrom.secretKeyRef.name=="cloud-config")
       | "\(.metadata.namespace)/\(.metadata.name)\n"'
   

6. Figure out how these Pods are controlled and (rollout) restart them.

7. Update Thanos bucket configuration

   Thanos enabledThanos disabled

   Thanos is enabled if yk8s.k8s-service-layer.prometheus.use_thanos is set to true. If the custom bucket management setting yk8s.k8s-service-layer.prometheus.manage_thanos_bucket is unset or set to true, apply the required changes by running the following update script:

   $ bash managed-k8s/actions/apply-k8s-supplements.sh install-monitoring.yaml
   

   This ensures that the Kubernetes secret thanos-bucket-config for Thanos is updated.