Prometheus Alertmanager Tutorial

1. Alertmanager Architecture Overview

Prometheus handles metric evaluation and fires alerts based on PromQL rules. Alertmanager acts as the central notification router: handling deduplication, grouping, inhibition, and dispatch to Slack, PagerDuty, or Opsgenie.

2. Configuring Alert Routing (`alertmanager.yml`)

Below is a battle-tested Alertmanager YAML configuration file provided in VibeInfra's monitoring stack:

global:
  resolve_timeout: 5m
  slack_api_url: 'https://hooks.slack.com/services/T00/B00/X00'

route:
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 12h
  receiver: 'slack-notifications'

receivers:
- name: 'slack-notifications'
  slack_configs:
  - channel: '#alerts-production'
    send_resolved: true
    title: '[{{ .Status | toUpper }}] {{ .CommonLabels.alertname }}'
    text: '{{ range .Alerts }}{{ .Annotations.summary }}\n{{ end }}'

3. High Availability (HA) & Deduplication

To prevent duplicate notifications when running multiple Alertmanager instances, connect them via gossip mesh networking: