Datadog developed FETCh, a runtime monitoring tool that validates production system behavior against TLA+ formal specifications. This approach addresses the problem of specification drift, where abstraction layers in formal models diverge from evolving codebases over time. By continuously checking real-world execution traces against the model, the team identified hidden inconsistencies that traditional testing and manual review failed to detect.
Key points
Formal specifications suffer from drift as production systems evolve over time.
Runtime monitoring allows for continuous validation of production states against formal models.
Abstraction in formal methods is necessary for tractability but requires automated validation to remain accurate.
FETCh enables detection of subtle correctness bugs that static analysis or manual testing may miss.