Meta engineers developed Intelligent Delivery, a system that optimizes compute resource allocation for ads ranking by predicting the marginal utility of inference tasks. The system dynamically shifts resources across models, hardware fleets, and ranking stages to maintain global efficiency despite increasing model complexity and strict latency constraints. This approach has yielded a 10% performance gain in ranking effectiveness since 2017.
Key points
Compute resource allocation must be dynamic and request-aware to account for heterogeneous returns on investment.
Ad ranking models at LLM scale require reimagining traditional CPU-based scheduling levers for GPU-specific memory and throughput constraints.
Adaptive optimization layers using agentic AI help systems converge on global optima despite shifting traffic patterns.
System-wide efficiency is maximized by directing compute to the stages with the highest marginal utility per unit.