Datadog Engineering·Oct 1, 2026·16 read·seniorHow we extended Apache DataFusion to execute one query across many machines
Datadog developed Distributed DataFusion, an open-source Rust framework, to enable cross-node query execution for Apache DataFusion. By extending DataFusion's capabilities, this framework allows large-scale interactive queries to maintain low latency across distributed compute resources. The project provides a composable foundation for building distributed query engines rather than acting as a fixed, proprietary service.
AI summary