Centralized Log Management with Loki and Grafana

What is Loki?

Loki is a log aggregation system by Grafana Labs. Unlike Elasticsearch, Loki only indexes metadata (labels), making it lightweight and cost-effective while integrating seamlessly with Grafana.

Architecture

• Promtail — agent that ships logs to Loki (like Filebeat for Elasticsearch)
• Loki — stores and indexes logs
• Grafana — query and visualize logs

Install Loki

wget https://github.com/grafana/loki/releases/download/v2.9.4/loki-linux-amd64.zip
unzip loki-linux-amd64.zip
sudo mv loki-linux-amd64 /usr/local/bin/loki

Create /etc/loki/config.yml:

auth_enabled: false
server:
  http_listen_port: 3100
common:
  path_prefix: /var/lib/loki
  storage:
    filesystem:
      chunks_directory: /var/lib/loki/chunks
      rules_directory: /var/lib/loki/rules
schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

Install Promtail

wget https://github.com/grafana/loki/releases/download/v2.9.4/promtail-linux-amd64.zip
unzip promtail-linux-amd64.zip
sudo mv promtail-linux-amd64 /usr/local/bin/promtail

Configure Grafana

Add Loki as a data source in Grafana with URL http://localhost:3100. Then use the Explore panel to query logs with LogQL:

{job="syslog"} |= "error"
{filename="/var/log/nginx/access.log"} | json | status >= 400