Log Monitoring with Loki and Promtail

What Is Loki?

Loki is a log aggregation system designed by Grafana Labs. Unlike Elasticsearch, Loki indexes only metadata (labels) rather than the full log content, making it much more resource-efficient.

Architecture

• Promtail — agent that collects and ships logs
• Loki — stores and indexes logs
• Grafana — queries and visualizes logs

Install with Docker Compose

services:
  loki:
    image: grafana/loki:2.9.3
    ports:
      - "3100:3100"
    volumes:
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml

  promtail:
    image: grafana/promtail:2.9.3
    volumes:
      - /var/log:/var/log:ro
      - ./promtail-config.yml:/etc/promtail/config.yml
    command: -config.file=/etc/promtail/config.yml

volumes:
  loki-data:

Promtail Configuration

Create promtail-config.yml:

server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: system
    static_configs:
      - targets: [localhost]
        labels:
          job: varlogs
          host: my-server
          __path__: /var/log/*.log

  - job_name: nginx
    static_configs:
      - targets: [localhost]
        labels:
          job: nginx
          host: my-server
          __path__: /var/log/nginx/*.log

Query Logs in Grafana

1. Add Loki as a data source: http://loki:3100
2. Go to Explore
3. Use LogQL to query logs:

# All nginx error logs
{job="nginx"} |= "error"

# SSH login failures
{job="varlogs"} |~ "Failed password"

# Rate of errors per minute
rate({job="nginx"} |= "error" [1m])

Why Loki over ELK?