Topic
All digests tagged Reliability Engineering
Suggested route
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.
Fuad Ali details the self-improvement loop for AI agents: trace failures, use Signal to detect issues, create living regression datasets, and run experiments to prove fixes. This automates reliability.
Read article
Why are static benchmarks failing? Felipe Blanes details the 'benchmark illusion' in AI agent development and introduces the 'eval flywheel'—a customer-driven process for building truly reliable products.
Read article
Evals are 'fuzzy unit tests' for non-deterministic AI systems. They provide ahead-of-time reliability, complementing runtime harnesses. Learn how to build an Eval suite, mitigate judge biases, and integrate it into a CI gate for production-grade safety.
Read article
A browser agent that's 99% accurate per step succeeds only about 36% of the time over 100 steps. Here's how to beat that math.
Read article
Marc Brooker argues that the hardest part of software is specification, not code. He details how agentic AI is shifting the focus to rigorous testing, agentic policy, and learning from historical postmortems.
Read article
AI agents have transitioned from model problems to distributed systems problems. Reliability requires incorporating concepts like idempotency keys, circuit breakers, and scoped credentials.
Read article
Building an LLM Gateway requires balancing availability, latency, guardrails, and cost. The speaker advises using per-request fallbacks over retries and tracking P99 latency per model/route to prevent silent outages.
Read article
Perception agents close the architectural gap by enabling them to perceive rendered UIs, understand shared context, and verify their own output against design specs, moving beyond simple clicking.
Read article