Explainer
What is system drift?
A plain-language guide to system drift, how it appears in production telemetry, and how deterministic drift detection can help teams find issues before they become incidents.
Key takeaways
- System drift is a meaningful movement away from normal production behavior.
- Drift can appear in error rate, latency, traffic volume, logs, traces, or change events.
- Deterministic baselines are a useful first step before advanced prediction.
System drift is behavior moving away from baseline
System drift happens when a production service starts behaving differently from its recent or expected baseline. The change may be subtle at first: error rate ticks upward, p95 latency widens, traffic volume shifts, or a new log pattern starts appearing after a deployment.
Not every movement is an incident. Drift detection is about identifying the changes that are large enough, persistent enough, or correlated enough with production changes to deserve attention.
Common drift signals
Error rate drift suggests that a service is failing more often than expected. Latency drift suggests that work is taking longer than normal. Traffic volume drift can reveal a routing issue, client behavior change, or dependency slowdown.
Deployment and configuration drift matter because many incidents are change-induced. When telemetry moves shortly after a deploy or config update, responders need that context in the same investigation path.
Why deterministic rules come first
Advanced AI can be useful later, but production teams need explainable signals first. A baseline comparison can say what changed, what the expected value was, what the observed value is, and how severe the movement appears.
Driftdog's MVP drift engine is intentionally deterministic. It compares current metric windows against recent baselines, classifies severity, and links the finding back to logs, metrics, incidents, and change events.