Setting Up Alerting with Grafana

Alert Types in Grafana

• Threshold alerts — trigger when a value crosses a limit
• No data alerts — trigger when metrics stop arriving
• Multi-condition — combine multiple conditions

Create an Alert Rule

1. Go to Alerting → Alert Rules → New Alert Rule
2. Define the query (e.g., CPU usage from Prometheus)
3. Set conditions: WHEN avg() OF query IS ABOVE 80
4. Set evaluation interval (how often to check)
5. Set pending period (how long before firing)
6. Choose notification channel

Common Alert Examples

High CPU Usage

Query: 100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)
Condition: IS ABOVE 85
For: 5 minutes

High Memory Usage

Query: (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100
Condition: IS ABOVE 90
For: 5 minutes

Disk Space Low

Query: (1 - node_filesystem_avail_bytes / node_filesystem_size_bytes) * 100
Condition: IS ABOVE 85
For: 10 minutes

Notification Channels

Configure in Alerting → Contact Points:

• Email (SMTP)
• Slack webhook
• Discord webhook
• Telegram bot
• PagerDuty
• Generic webhook

Silence and Mute

Use Alerting → Silences to temporarily suppress alerts during maintenance windows.