Long short-term memory (LSTM) networks are recurrent neural networks that can model nonlinear sequential relationships. They are one forecasting option, not a default winner. Many time series are better served by seasonal-naive, exponential-smoothing, autoregressive, tree-based, or domain-specific models.

Evaluate an LSTM against strong baselines under the same forecast origins, horizons, information set, metrics, and operational constraints. A lower error on one split does not establish future or business performance.

How an LSTM works

An LSTM maintains a cell state and hidden state. Learned input, forget, and output gates control candidate updates and exposed hidden state at each step. Gates do not consciously identify important events or filter noise; they are differentiable functions learned from the objective and data.

The architecture was designed to address decaying error flow, but modern LSTMs can still experience vanishing or exploding gradients and do not guarantee useful memory over arbitrarily long sequences.

Define the forecast before building tensors

  1. Define the target, forecast origin, horizon, frequency, unit, aggregation, and decision.
  2. State which lagged targets and covariates are available at each historical origin.
  3. Choose metrics from the decision cost and report them by horizon and relevant segment.
  4. Predefine seasonal-naive and other credible baselines.

Future promotions may be known if scheduled; observed weather, supply disruptions, or revised economic values are unavailable unless separately forecast. Reconstruct the information set that would have existed at every origin.

Split and transform without leakage

Sort by event time and inspect duplicates, gaps, revisions, time zones, daylight-saving transitions, and irregular intervals. Do not delete outliers automatically: determine whether they are errors, events, or target behavior.

Split chronologically before fitting scalers, imputers, feature selection, or encoders. Fit learned transformations on each training fold only. Min-max scaling is not inherently better or worse than standardization, and standardization does not require normally distributed data.

import numpy as np
from sklearn.preprocessing import StandardScaler

train_end = int(len(y) * 0.70)
valid_end = int(len(y) * 0.85)
scaler = StandardScaler().fit(y[:train_end].reshape(-1, 1))
y_scaled = scaler.transform(y.reshape(-1, 1)).ravel()

def make_windows(values, lookback, horizon, start, stop):
    X, target = [], []
    for target_start in range(max(start, lookback), stop - horizon + 1):
        X.append(values[target_start-lookback:target_start, None])
        target.append(values[target_start:target_start+horizon])
    return np.asarray(X), np.asarray(target)

lookback, horizon = 28, 7
X_train, y_train = make_windows(y_scaled, lookback, horizon, 0, train_end)
X_valid, y_valid = make_windows(y_scaled, lookback, horizon, train_end, valid_end)
X_test, y_test = make_windows(y_scaled, lookback, horizon, valid_end, len(y_scaled))

This illustrates one series. For panels, partition by entity and time without future entity-period leakage. In rolling-origin tuning, refit the scaler in every training fold.

Choose architecture and optimization empirically

There is no universal number of units, layers, dropout rate, batch size, learning rate, or epochs. Predefine a bounded search and compare validation performance, stability across seeds, compute, and latency. Dropout may regularize selected activations but does not prevent overfitting.

A bidirectional LSTM is valid only when the complete input window is available at inference, such as classifying a finished sequence. It must not read observations after the forecast origin. Multi-step forecasts may be direct, recursive, multi-output, or encoder-decoder; none is inherently most accurate.

early_stop = keras.callbacks.EarlyStopping(
    monitor="val_loss", mode="min", patience=10,
    min_delta=1e-4, restore_best_weights=True,
)

Early stopping can reduce unnecessary training but cannot rescue a leaking validation design. Record optimizer, schedule, clipping, seeds, precision, software, hardware, dataset snapshot, code commit, and every tuning trial. The bias-variance trade-off remains relevant, but time dependence changes validation design.

Use rolling-origin evaluation and uncertainty

Evaluate across forecast origins representing seasons and regimes, then confirm once on a protected final period. Forecast errors across overlapping origins and horizons are dependent, so an ordinary paired t-test is often invalid. Report per-horizon loss and uncertainty; use a comparison method whose assumptions match the backtest.

LSTMs do not automatically produce valid prediction intervals. Evaluate coverage and width by horizon and regime using a justified probabilistic, ensemble, conformal, or other method. See time-series analysis techniques for alternative models and validation concepts.

Monitor and retrain deliberately

Monitor input availability and quality, residuals when outcomes arrive, error by horizon and segment, interval coverage, latency, failures, overrides, and downstream decisions. Input drift does not prove performance loss.

A schedule or alert should start diagnosis, not automatic retraining. Any update must repeat data validation, rolling evaluation, baseline comparison, approval, staged release, and rollback preparation. Historical relationships can break during regime changes; use scenario plans and fallback baselines. For deeper architecture context, see how LSTM became a forecasting workhorse.

Originally published June 9, 2025; technically reviewed and substantially updated September 4, 2026.