Anomaly Detection: Methods, Evaluation, and Deployment

Anomaly detection assigns an observation, sequence, or group an unusualness score or label relative to a stated reference. Unusual is not the same as fraudulent, unsafe, defective, or causally important: a rare event can be legitimate, while a harmful event can look ordinary.

A point anomaly is unusual by itself; a contextual anomaly is unusual given time, location, or operating mode; a collective anomaly is a sequence or group whose joint pattern is unusual. These are descriptions, not guarantees that one algorithm will find the event.

Statistical process monitoring

Walter A. Shewhart developed control-chart methods in the 1920s. A common convention places limits three standard deviations from an in-control process center. For a normal distribution with known parameters, about 99.73% of probability lies within ±3 standard deviations. This does not mean an observed point has a 0.27% “chance of being normal,” and it does not apply unchanged to skewed, heavy-tailed, autocorrelated, seasonal, count, or nonstationary data.

Choose the chart for the process and sampling plan. Estimate limits from a stable reference period, investigate special causes, and validate false-alert behavior. For time-varying signals, see time-series analysis techniques.

Outlier detection versus novelty detection

Scikit-learn distinguishes outlier detection, where training data may contain anomalies, from novelty detection, where a clean reference sample is used to identify new observations. The distinction affects fitting, thresholding, and interpretation. “Unsupervised” does not mean assumption-free; contamination, distance, density, representation, and preprocessing still encode judgments.

Common methods and their assumptions

Method	Useful signal	Important limits
Robust rules or control charts	Known process structure and interpretable deviations	Distribution, stationarity, dependence, repeated alerts
Isolation Forest	Points isolated by short random partition paths	Feature representation, sample size, threshold; score is not probability
Local Outlier Factor	Local-density difference	Neighborhood and metric sensitivity; novelty mode differs
DBSCAN	Dense regions and noise under a chosen metric	eps, min_samples, scale, varying density, dimensionality
One-Class SVM	Boundary around a reference distribution	Kernel, scale, hyperparameters, computational cost
Autoencoder	Reconstruction error from learned representation	Can reconstruct anomalies; architecture, loss, training contamination, threshold

K-means distance can be a heuristic, but spherical-cluster assumptions and outlier-sensitive centroids matter. DBSCAN noise points are not automatically business anomalies. Isolation Forest’s contamination helps define a threshold; it does not reveal true incident prevalence.

Time series, images, and sequences

Time-series detectors must model trend, seasonality, autocorrelation, regime changes, missingness, and event duration. Split evaluation by time and avoid fitting preprocessing on the future. Sequence or deep models can help when their representation captures relevant structure, but added complexity does not guarantee better detection. Visual defect systems require representative imaging conditions, ordinary variation, defect coverage, domain-shift tests, and an inspection workflow.

Evaluate the alerting workflow

1. Define the event and action: observation window, recipient, investigation, and cost of misses and false alarms.
2. Create leakage-safe splits: split by time and, when needed, by user, asset, or site; fit transformations only on training data.
3. Preserve prevalence: balanced test sets distort precision and alert volume.
4. Select thresholds on validation data: use explicit recall, capacity, or cost rules; keep final test data untouched.
5. Report operational measures: event precision/recall, false alerts per time, detection delay, queue volume, and performance by context.
6. Run shadow mode: compare with current controls and sample non-alerted events.

ROC area can appear strong under severe class imbalance while precision remains poor. Precision-recall curves are often more informative, but neither captures alert deduplication, investigation cost, or detection delay.

Deploy and monitor

Track input quality, score distribution, alert rate, label delay, feedback quality, threshold and version changes, and downstream outcomes. Analyst-reviewed alerts are a selected sample, not automatic ground truth. Preserve independent audits and sample non-alerted events. An anomaly can be a precursor associated with later failure, but a detector does not “see the future.”

Connect detector telemetry to pipeline monitoring and maintain versions, approvals, rollback, and incident ownership using AI model management.

Originally published July 12, 2025; technically reviewed and substantially updated September 4, 2026.