homelab/compose/monitoring/logging/DOCKER-LOGS-DASHBOARD.md

Docker Logs Dashboard - Grafana

A comprehensive dashboard for viewing all Docker container logs via Loki.

Features

📊 Panels Included

1. Docker Container Logs (Main Panel)

   • Real-time log streaming from all containers
   • Filter by container, image, or search term
   • Expandable log details
   • Sortable (ascending/descending)

2. Log Volume by Container

   • Stacked bar chart showing log activity over time
   • Helps identify chatty containers
   • Per-container breakdown

3. Error Logs by Container

   • Time series of ERROR/EXCEPTION/FATAL/PANIC logs
   • Automatically detects error patterns
   • Useful for monitoring application health

4. Total Logs by Container

   • Bar gauge showing total log lines per container
   • Color-coded thresholds (green → yellow → red)
   • Based on selected time range

5. Statistics Panels

   • Active Containers: Count of containers currently logging
   • Total Log Lines: Sum of all logs in time range
   • Total Errors: Count of error-level logs
   • Log Rate: Logs per second (current rate)

Access the Dashboard

1. Open Grafana: https://logs.fig.systems
2. Navigate to: Dashboards → Loki folder → Docker Logs - All Containers

Or use direct link:

https://logs.fig.systems/d/docker-logs-all

Using the Filters

Container Filter

• Select specific containers to view
• Multi-select supported
• Default: "All" (shows all containers)

Example: Select traefik, loki, grafana to view only those

Image Filter

• Filter by Docker image name
• Multi-select supported
• Useful for viewing all containers of same image

Example: Filter by grafana/loki:* to see all Loki containers

Search Filter

• Free-text search with regex support
• Searches within log message content
• Case-insensitive by default

Examples:

• error - Find logs containing "error"
• (?i)started - Case-insensitive "started"
• HTTP [45][0-9]{2} - HTTP 4xx/5xx errors
• user.*login.*failed - Failed login attempts

Time Range Selection

Use Grafana's time picker (top right) to select:

• Last 5 minutes
• Last 15 minutes
• Last 1 hour (default)
• Last 24 hours
• Custom range

Auto-Refresh

Dashboard auto-refreshes every 10 seconds by default.

Change refresh rate in top-right dropdown:

• 5s (very fast)
• 10s (default)
• 30s
• 1m
• 5m
• Off

LogQL Query Examples

The dashboard uses these queries. You can modify panels or create new ones:

All logs from a container

{job="docker_all", container="traefik"}

Errors only

{job="docker_all"} |~ "(?i)(error|exception|fatal|panic)"

HTTP status codes

{job="docker_all", container="traefik"} | json | line_format "{{.status}} {{.method}} {{.path}}"

Rate of logs

rate({job="docker_all"}[5m])

Count errors per container

sum by (container) (count_over_time({job="docker_all"} |~ "(?i)error" [1h]))

Tips & Tricks

1. Find Noisy Containers

• Use "Log Volume by Container" panel
• Look for tall bars = lots of logs
• Consider adjusting log levels for those containers

2. Debug Application Issues

1. Set time range to when issue occurred
2. Filter to specific container
3. Search for error keywords
4. Expand log details for full context

3. Monitor in Real-Time

1. Set time range to "Last 5 minutes"
2. Enable auto-refresh (5s or 10s)
3. Open "Docker Container Logs" panel
4. Watch logs stream live

4. Export Logs

• Click on any log line
• Click "Copy" icon to copy log text
• Or use Loki API directly for bulk export

5. Create Alerts

In Grafana, you can create alerts based on log patterns:

• Alert if errors exceed threshold
• Alert if specific pattern detected
• Alert if container stops logging (might be down)

Troubleshooting

No logs showing

1. Check Promtail is running: docker ps | grep promtail
2. Verify Loki datasource in Grafana is configured
3. Check time range (logs might be older/newer)
4. Verify containers are actually logging

Slow dashboard

• Narrow time range (use last 15m instead of 24h)
• Use container filter to reduce data
• Increase refresh interval to 30s or 1m

Missing containers

Your current Promtail config captures ALL Docker containers automatically. If a container is missing, check:

1. Container is running: docker ps
2. Container has logs: docker logs <container>
3. Promtail can access Docker socket

Advanced Customization

Add a New Panel

1. Click "Add Panel" in dashboard
2. Select "Logs" visualization
3. Use query:

   {job="docker_all", container="your-container"}
   

4. Configure options (time display, wrapping, etc.)
5. Save dashboard

Modify Existing Panels

1. Click panel title → Edit
2. Modify LogQL query
3. Adjust visualization options
4. Save changes

Export Dashboard

1. Dashboard settings (gear icon)
2. JSON Model
3. Copy JSON
4. Save to file for backup

Integration with Other Tools

View in Explore

• Click "Explore" on any panel
• Opens Loki Explore interface
• More advanced querying options
• Better for ad-hoc investigation

Share Dashboard

1. Click share icon (next to title)
2. Get shareable link
3. Or export snapshot

Embed in Other Apps

Use Grafana's embedding features to show logs in:

• Homarr dashboard
• Custom web apps
• Monitoring tools

Related Resources

• LogQL Documentation
• Grafana Dashboards Guide
• Loki Best Practices

Support

For issues with:

• Dashboard: Edit and customize as needed
• Loki: Check /home/eduardo_figueroa/homelab/compose/monitoring/logging/
• Missing logs: Verify Promtail configuration

Dashboard file location:

/home/eduardo_figueroa/homelab/compose/monitoring/logging/grafana-provisioning/dashboards/docker-logs.json