Chapter 26: Traffic

Traffic is the lifeblood of a distributed system. Understanding traffic patterns — when requests arrive, where they come from, and what they ask for — is essential for capacity planning, performance optimization, and anomaly detection.

Traffic patterns exhibit several common characteristics. Diurnal patterns follow the daily rhythm of human activity, with peaks during waking hours and valleys overnight. Weekly patterns distinguish weekdays from weekends. Seasonal patterns reflect events like holidays, product launches, and marketing campaigns. Understanding these patterns allows operators to pre-provision capacity and schedule maintenance during low-traffic periods.

Traffic management techniques include rate limiting (capping the number of requests a client can make per unit of time), throttling (slowing down requests rather than rejecting them), and admission control (rejecting requests at the edge before they consume backend resources). These techniques protect the system from overload, whether from organic growth, flash crowds, or malicious attacks.

The monitoring service plays a crucial role in traffic management by providing real-time visibility into request rates, error rates, and latency distributions. Sudden changes in these metrics may indicate a traffic surge, a dependency failure, or a bug in a newly deployed version.