Observed System State
Raw signals, no interpretation
ScopeThese metrics aggregate nginx access logs across all sites on my host stack. There are currently two. Data is analyzed as a single surface to observe traffic patterns and probing behavior.
Log volume (last 24h)
Response volume by status (last 24h)
Temporal Comets (last 24h)
Animated request bursts over the last 24 hours. Larger comets indicate higher volume.
Top 404 Endpoints (24h)
Loading…
Top 404 Source IPs (24h)
Loading…
Observer Script (Retired)
Goodbye, Perl.
You did your job here, and you did it well. Since day one this site ran on a simple observer script that read nginx logs, recorded exactly what happened into MariaDB, and stayed out of the way.
Though my traffic levels are still the same, I wanted realtime updates. The Perl script ran on a 1 minute scheduled job. Datadog Vector + OpenSearch gives me near realtime access.
A full writeup of how I am using Vector and OpenSearch for this is here: