Symptoms
- Services failing to start or crashing
- "No space left on device" errors
- Cannot create new files or write to databases
- SSH may still work (it is already loaded in memory)
Emergency Cleanup
# Check disk usage
df -h
# Find largest directories
du -h --max-depth=1 / | sort -rh | head -15
Clear Log Files
# Truncate large log files (keeps the file descriptor open)
sudo truncate -s 0 /var/log/syslog
sudo truncate -s 0 /var/log/nginx/access.log
# Remove old journal logs (keep last 2 days)
sudo journalctl --vacuum-time=2d
Clear Package Cache
# Debian/Ubuntu
sudo apt clean
sudo apt autoremove -y
# RHEL/Rocky
sudo dnf clean all
Remove Old Kernels
# Ubuntu
sudo apt autoremove --purge
Docker Cleanup
docker system prune -a --volumes
Find and Remove Large Files
# Find files over 100 MB
find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null
# Find files modified in the last 24 hours (possible log explosion)
find /var/log -type f -mtime -1 -size +50M
After Cleanup
# Restart affected services
sudo systemctl restart nginx mysql php8.2-fpm
# Verify services are running
systemctl --failed
Prevention
- Set up disk space monitoring with alerts at 80% usage
- Configure logrotate for all application logs
- Schedule regular cleanup of temporary files
- Use separate partitions for
/var/log to prevent log files from filling the root partition