So, you’re officially up and running with Harness and you have some pretty awesome pipelines and workflows that your teams depend on to get their applications reliably deployed into production. You even have multiple instances of your delegates running so you’ll never have an issue deploying your applications even if one of them goes down. Amazing! But as we all know computers (and people) are never as reliable as we hope they are and sometimes things goes wrong. Someone can push bad configuration, networks can go down, etc. These kinds of failure scenarios can also affect your delegates and prevent you from shipping code. So how do we get alerted when things like this happen?
1. Configure your incident management system
Most incident management systems allow you to trigger an incident by sending an email to a specific address. Follow the instructions below for your specific incident management system for configuring the email integration:
Now that we have this new email address, we can configure Harness to send it certain notifications.
2. Setup a new user group
A user group in Harness allows us to not only setup fine-grained permissions, but also gives us the ability to control how users in that group receive notifications.
Let’s create a new User Group called Ops Alerts. Go to Security -> Access Management -> User Groups -> +Add User Group.
At the bottom of the screen edit the Notification Settings and add the email address that you created in step 1. We don’t actually need to add any users to this group. You can leave all of the other settings blank.
3. Setup a notification rule
We can setup granular notification rules for the various system events that happen in Harness and send them to specific user groups. Here we’ll add a rule that sends a notification to our new Ops Group whenever a delegate goes down.
Go to Setup -> Alert Notification Rules -> +Add New Rule. Here we’ll choose Setup as our Category and Matching as our Alert Filter. Then from the Alert Type dropdown we’ll choose Delegates Down. Under the User Group(s) section we’ll add our newly created Ops Alerts group that we created in step 2.
That’s it. Now any time a delegate goes down it will send an email to your incident management system.
Further reading
Here are some links with more documentation about how notifications and user groups work in Harness: