Datadog developed a service criticality framework by using LLMs to analyze unstructured postmortem data, combining it with architectural signals to prioritize reliability efforts. This multi-signal approach allows engineering teams to distinguish between critical and non-critical services to optimize deployment speeds and SLO management. The framework provides a weighted classification system that replaces uniform service treatment with data-driven infrastructure decision-making.
Key points
Use LLMs to convert unstructured legacy postmortem data into structured telemetry for service classification.
Criticality should be determined by a multi-signal approach, combining incident history, system dependencies, and traffic patterns.
Implement confidence scores in automated decision systems to allow tuning between risk tolerance and false-positive sensitivity.
Categorize services based on customer impact to avoid over-investing in non-critical service reliability and deployment speed.