Alerting

This Helm chart is designed to help configure alerting system, AlertManager for now.

It consists of several sections.

SectionsDescription
routeAlertmanager route definition which will be added to the generated Alertmanager configuration as a first-level route.
receiversReceiver defines one or more notification integrations
inhibitRulesInhibit rules used to silent relevant alerts (#FIXME: not ready yet)
prometheusRuleSimplifies creating PrometheusRules resources

TL;DR, How to use this chart?

Simple example

route:
  routes:
    - receiver: chat
      matchers:
        - name: namespace
          value: prod-.+

receivers:
  chat:
    webhookConfigs:
      - url: https://chat.example.com/hooks/XC6ptt3jceSeu4NJC/7KWt5iKYYegxPHchTfX6uyzmJ8wKnGkWEp8TWAeiN3DmAKqa
        sendResolved: true

Route

Alertmanager route definition which will be added to the generated Alertmanager configuration as a first-level route.

ParameterTypeDescriptionDefault
receiverstring(Required) Name of the receiver for this route. If not empty, it should be listed in the receivers section.--
continuebooleanBoolean indicating whether an alert should continue matching subsequent sibling nodesfalse
groupBylist of stringsList of labels to group by
groupIntervalstringHow long to wait before sending an updated notification. Must match [0-9]+(ms\s\m\h)
groupWaitstringHow long to wait before sending the initial notification. Must match [0-9]+(ms\s\m\h)
matcherslistList of matchers that the alert’s labels should match.
matchers[].namestringLabel to match
matchers[].valuestringLabel value to match
matchers[].regexbooleanWhether to match on equality (false) or regular-expression (true)false
matchLabelsmappingMapping of label
to add to matchers, It will be treated as regex matcher if have characters other than A-Za-z0-9_-
repeatIntervalstringHow long to wait before repeating the last notification. Must match the regular expression [0-9]+(ms\s\m\h)
routeslist of routeChild routes

Receivers

Receiver defines one or more notification integrations. it is a list if items with following properties.

ParameterTypeDescriptionDefault
namestring(Required) Receiver name
emailConfigslist of emailConfigsList of Email configurations
slackConfigslist of slackConfigsList of Slack configurations
webhookConfigslist of webhookConfigsList of webhook configurations

emailConfigs

Receiver Email configurations

#FIXME: complete the table

ParameterTypeDescriptionDefault
authIdentitystring
authPasswordobject
authSecretobject
authUsernamestring
fromstring
headerslist
hellostring
htmlstring
requireTLSstring
sendResolvedstring
smarthoststring
textstring
tlsConfigobject
tostring

slackConfigs

Receiver Slack configurations

#FIXME: complete the table

ParameterTypeDescriptionDefault
actions
apiURL
callbackId
channel
color
fallback
fields
footer
httpConfig
iconEmoji
iconURL
imageURL
linkNames
mrkdwnIn
pretext
sendResolved
shortFields
text
thumbURL
title
titleLink
username

webhookConfigs

Receiver Webhook configurations

#FIXME: complete the table

ParameterTypeDescriptionDefault
httpConfig
maxAlerts
sendResolved
url
urlSecret

inhibitRules

List of inhibition rules. (The rules will only apply to alerts matching the resource’s namespace.)?

ParameterTypeDescriptionDefault
equalstringLabels that must have an equal value in the source and target alert for the inhibition to take effect.
sourceMatchlistMatchers for which one or more alerts have to exist for the inhibition to take effect
sourceMatch[].namestringLabel to match
sourceMatch[].valuestringLabel value to match
sourceMatch[].regexbooleanWhether to match on equality (false) or regular-expression (true)false
sourceMatchLabelsmappingMapping of label
to add to sourceMatch, It will be treated as regex matcher if have characters other than A-Za-z0-9_-
targetMatchlistMatchers that have to be fulfilled in the alerts to be muted.
targetMatch[].namestringLabel to match
targetMatch[].valuestringLabel value to match
targetMatch[].regexbooleanWhether to match on equality (false) or regular-expression (true)false
targetMatchLabelsmappingMapping of label
to add to targetMatch, It will be treated as regex matcher if have characters other than A-Za-z0-9_-

PrometheusRule

Section: prometheusRule

#FIXME: incomplete contents

ParameterTypeDescriptionDefault
enabledbooleanEnables this prometheusruletrue
tplstringTreat rules as helm templatefalse
intervalpositive integerInterval between each time prometheusRule.rules are run--
grafanaDomainstringgrafana domain for connection from prometheus to grafana--
ruleslistPrometheus Rules--
rules.exprinteger or string(Required) The expression to be run #FIXME--
rules.recordstringRecord name to save expr result--
rules.forstringHow long to wait before raising the alert. Must match the regular expression [0-9]+(ms\s\m\h)--
rules.labelsmappingsLabels for this rule--
rules.annotationsmappingsAnnotations for this rule--

last update: 2022-01-24