Watch It! Monitoring Lessons from Two Decades in Computing - Markus Krüger - NDC Oslo 2026
Summary
This talk provides a comprehensive overview of modern observability practices, emphasizing the shift from reactive debugging to proactive monitoring. The core message is that robust monitoring requires collecting three types of telemetry—metrics, logs, and traces—and implementing sophisticated systems to detect issues before end-users are impacted. Key recommendations include adopting OpenTelemetry for standardization, using tools like Prometheus and Grafana for visualization, and leveraging AI for anomaly detection and automated incident response.
Key takeaways
-
Focus on Observability Pillars
21:20
Effective monitoring relies on collecting three distinct signals: Metrics (numerical measures over time, e.g., CPU load), Logs (arbitrary text providing detail), and Traces (tracking request execution across distributed components).
-
Adopt OpenTelemetry for Standardization
23:50
OpenTelemetry is a vendor-agnostic framework for generating, exporting, and collecting telemetry, allowing organizations to switch backends without rewriting application code.
-
Alert on Symptoms, Not Causes
35:00
Alerting should focus on the end result (the symptom, e.g., 'server not responding') rather than the underlying cause, which can be numerous and unpredictable. Alerts must be actionable and concise.
-
Implement Proactive Testing
40:00
To detect failures before customers do, implement synthetic testing (exercising functionality in production) and Real User Monitoring (RUM) to capture client-side errors.
-
Leverage AI for Incident Management
50:00
AI can assist in anomaly detection (identifying deviations from expected behavior) and, in advanced setups, can propose or even execute fixes for incidents, though caution regarding access control is advised.
Technical details
-
Telemetry Types
1280s
Metrics are efficient for overall state tracking but are coarse-grained. Logs provide deep, arbitrary text detail. Traces are essential for distributed systems, allowing tracking of a request's flow across multiple services and components.
-
Metric Types and Best Practices
1550s
Metrics include Counters (monotonically increasing values, e.g., request count), Gauges (values that can increase or decrease, e.g., CPU load), and Histograms (counting by buckets, necessary for calculating percentiles like the 95th percentile).
-
Logging Systems
1750s
Logs should be aggregated and indexed (e.g., using the ELK/EFK stack: Elasticsearch, Kibana, Fluent). Structured logging (e.g., JSON format) is highly recommended for easier querying and alerting.
-
Visualization and Dashboarding
2000s
Visualization tools like Grafana are recommended because they unify data from multiple sources (Prometheus, Elasticsearch, Loki) without storing data themselves. Dashboards should be simple and focused.
-
Advanced Monitoring Techniques
2400s
Techniques include Synthetic Testing (automated checks of functionality) and Real User Monitoring (RUM) to capture client-side experience, which is critical as backend metrics may miss front-end issues.
Mentioned resources
- OpenTelemetry
- Prometheus
- Grafana
- ELK/EFK Stack
- Jaeger
- Site Reliability Engineering
- Accelerate
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.