SLO-based alerting requires you to configure Backyards to use an Alertmanager deployment. This procedure describes how to configure Backyards to use an existing Alertmanager deployment.
Note: If you don’t have and existing Alertmanager deployment, or you want to use a separate deployment, see Deploy a new Alertmanager for Backyards.
Prometheus Alertmanager installed and configured.
Note: We recommend configuring alert grouping in the following way:
route: receiver: 'slack-notifications' group_by: [ service, severity ]
That way you can route notifications based on severity, using the
severitylabel of Backyards-generated alerts.
To configure Backyards to use an existing Alertmanager deployment, complete the following steps.
Download the following configuration snippet as
your-alert-manager-X-hostpart with your Alertmanager’s fully qualified domain name, and the
your-alert-manager-X-portwith the port Alertmanager is listening on.
Backyards is controlled by a
ControlPlanecustom resource found in the Backyard’s namespace (default:
backyards-system) named backyards.
The following command changes the
spec.backyards.prometheus.alertmanagervalue to connect to the existing Alertmanagers. Run the following command:
$ kubectl patch controlplane --type=merge --patch "$(cat enable-alert-manager.yaml)" backyards
If you are using Backyards in operator mode, skip this step.
Otherwise, execute a reconciliation so Backyards updates your Kubernetes cluster to the desired state described by the ControlPlane Custom Resource. Run the following command:
$ backyards operator reconcile
- In case your Alertmanagers are not part of your service mesh setup, create destination rules in the
backyards-systemnamespace to allow communication with your Alertmanagers.