AI-Powered Log Analysis

AI for Log Analysis

Traditional log analysis with grep and regex misses complex patterns. AI-powered analysis can detect anomalies, correlate events across services, and generate plain-language summaries of issues.

Using Ollama for Log Analysis

import subprocess, ollama

def analyze_logs(log_file, lines=100):
    # Get recent logs
    result = subprocess.run(
        ["tail", f"-{lines}", log_file],
        capture_output=True, text=True
    )

    response = ollama.chat(model="llama3", messages=[{
        "role": "system",
        "content": "You are a Linux system administrator. Analyze these server logs and report any errors, warnings, anomalies, or security concerns. Be specific and actionable."
    }, {
        "role": "user",
        "content": f"Analyze these logs:\n\n{result.stdout}"
    }])

    return response["message"]["content"]

# Analyze various logs
for log in ["/var/log/nginx/error.log", "/var/log/auth.log", "/var/log/syslog"]:
    print(f"\n=== {log} ===")
    print(analyze_logs(log))

Anomaly Detection Script

#!/usr/bin/env python3
# /usr/local/bin/log-anomaly-check.py
import ollama, subprocess, smtplib

LOGS_TO_CHECK = {
    "nginx_error": "/var/log/nginx/error.log",
    "auth": "/var/log/auth.log",
    "mysql": "/var/log/mysql/error.log",
}

alerts = []
for name, path in LOGS_TO_CHECK.items():
    logs = subprocess.run(["tail", "-50", path], capture_output=True, text=True).stdout
    if not logs.strip():
        continue

    analysis = ollama.chat(model="llama3", messages=[{
        "role": "system",
        "content": "Analyze logs. Respond with ALERT if critical issues found, OK if normal. Brief explanation."
    }, {
        "role": "user",
        "content": logs
    }])

    result = analysis["message"]["content"]
    if "ALERT" in result.upper():
        alerts.append(f"{name}: {result}")

if alerts:
    # Send notification
    print("ALERTS FOUND:", alerts)

Use Cases

• Automated daily log review with AI summaries
• Real-time anomaly detection in security logs
• Correlating errors across multiple services
• Generating incident reports from log data
• Identifying performance degradation patterns