Each Prometheus instance needs a persistent volume (PV) to store the recorded metrics.
To set up such persistent volumes, modify the Backyards control plane custom resource to allow for acquiring such volumes. Complete the following steps.
Find which storage classes does your Kubernetes cluster support. Log in to your cluster, then run the following command:
$ kubectl get storageclass (kubernetes-admin@turip-host/backyards-system) NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 7h17m
Select the appropriate storage class from the output list.
The previous example shows the default AWS settings for EKS or PKE clusters, in this case it is safe to use gp2 volumes.
Set up the Persistent Volume.
To enable Persistent Volumes for Prometheus, first customize this YAML patch file:
Save this patch file as
Enter the Storage Class name obtained in the previous step into the
storageClassNameproperty, and set the storage space required by updating the value of
Backyards is controlled by a
ControlPlanecustom resource found in the Backyard’s namespace (default:
backyards-system) named backyards. To update the Custom Resource with the patch, run the following command.
$ kubectl patch controlplane --type=merge --patch "$(cat persistent-storage.patch.yaml)" backyards controlplane.backyards.banzaicloud.io/backyards patched
If you are using Backyards in operator mode, skip this step.
Otherwise, execute a reconciliation so Backyards updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
$ backyards operator reconcile
Verify that Persistent Volumes exist. To verify that Prometheus has the physical volumes attached, run the following command:
$ kubectl get pv -n backyards-system NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-79cc1a86-bcfa-46a9-a0e7-24e38d001d29 50Gi RWO Delete Bound backyards-system/prometheus-backyards-prometheus-db-prometheus-backyards-prometheus-0 gp2 7m10s
The list shows that a Physical Volume (named
backyards-system/prometheus-backyards-prometheus-db-prometheus-backyards-prometheus-0in the example) has been created and attached (
Next steps 🔗︎
If you haven’t done yet, Set up High Availability for the Monitoring Stack.