Feature engineering represents information in forms a model can use. A feature may encode domain structure, improve data efficiency, or aid interpretation. It may also introduce leakage, unstable proxies, unfair effects, or maintenance cost. No technique guarantees higher accuracy.

Start with prediction-time availability

For every feature, record its source, event time, availability time, transformation, owner, and expected behavior. Ask whether the exact value would exist when a real prediction is made.

  • Calculate tenure from signup time to the historical prediction cutoffβ€”not today.
  • If price is the target, price per square foot contains the answer and cannot be a feature.
  • Build rolling aggregates with events strictly before the cutoff.
  • Fit imputers, encoders, selectors, and scalers on training data only.

Keep learned preprocessing inside validation

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

preprocess = ColumnTransformer([
    ("num", Pipeline([
        ("impute", SimpleImputer(strategy="median", add_indicator=True)),
        ("scale", StandardScaler()),
    ]), ["tenure_days", "orders_90d"]),
    ("cat", Pipeline([
        ("impute", SimpleImputer(strategy="most_frequent")),
        ("encode", OneHotEncoder(handle_unknown="ignore")),
    ]), ["region", "plan"]),
])
model = Pipeline([("preprocess", preprocess),
                  ("classifier", LogisticRegression(max_iter=1000))])

Scaling often matters for distance-based models and regularized linear optimization; tree models generally do not require it. The choice is estimator-specific.

Engineer time and categories carefully

For a prediction at time t, a 30-day count should normally include eligible events in [t βˆ’ 30 days, t). Specify time zone, cutoff, late-arrival rules, and entity key. Raw month numbers incorrectly imply December and January are far apart; consider categorical or cyclical representations.

One-hot encoding is a transparent baseline for moderate cardinality. For target encoding, compute training values out of fold, use smoothing, fit the final mapping on training data, and define unseen-category behavior. Never include a row's own label in its encoding.

Treat transformations as hypotheses

Log transforms require defined handling for zero and negative values. Ratios require zero-denominator and missing-value rules. Interactions can expose structure to constrained models but amplify measurement error. Binning discards information and creates boundary effects. Polynomial features grow dimensionality quickly and can extrapolate implausibly; validate them with regularization.

Select and interpret features cautiously

Low marginal correlation does not prove irrelevance. Fit selectors inside each fold. Impurity importance can favor variables with many split points; permutation importance measures a fitted model's dependence on a feature but can mislead when features are correlated. Neither establishes causality.

Match validation to deployment

Random cross-validation is unsuitable when future records, repeated entities, sites, or households can cross splits. Use time-based, grouped, or nested designs as required. Keep a final test set untouched until the pipeline and decision rule are fixed. Evaluate calibration, subgroup behavior, latency, cost, missingness, and stability as well as a primary metric.

Operational feature checklist

  • Document definition, source, owner, freshness, lag, type, unit, and range.
  • Version code, reference data, and training snapshots.
  • Test offline/online parity and point-in-time correctness.
  • Monitor missingness, unknown categories, drift, latency, and performance.
  • Define access controls, fallbacks, and retirement criteria.

Start with data cleaning in Python, evaluate complexity through the bias-variance trade-off, and manage production features through AI model management.

Originally published August 10, 2025; technically reviewed and substantially updated September 4, 2026.