When traffic is measured in millions per second, everything ordinary gets hard — a deploy, a config change, losing a region. My job is making them ordinary again.
Five million ad requests a second have to land somewhere, get answered in milliseconds, and be visible while it happens. Here is roughly where they go.
Two pieces of this are mine. The monitoring stack — Prometheus collecting, Thanos for global query and long-term storage, Alertmanager and Grafana Cloud carrying it through to on-call — built to keep the signal alive when a region goes. And the ingress: running the HAProxy controller without a cloud load balancer, which takes a standing bill off the platform at no cost to latency.
Three opinions that show up in everything above.
The monitoring stacks I build — Prometheus collecting, Thanos for global query and long-term storage — get designed alongside the thing they watch, not bolted on after the first incident. Conditional paging, so people are woken for reasons and not for noise.
Developers should get a container deployed, a database, a pub/sub topic or a bucket — monitoring already wired — without opening a ticket. My users are the engineers on the other side of the API.
Three cloud migrations under live traffic: OVH to AWS, AWS to GCP, OVH / LeaseWeb to GCP. The interesting part is never the target platform — it is the cutover nobody notices.
Reliability, scale, cost, or a cloud migration you would rather not do twice. Tell me what is breaking.