The report wasn’t wrong. The finance team asked me twice, and both times the number held: weeks of inventory on hand had come down hard, harder than anyone expects a forecasting project to move it. That was the only result the room actually cared about, not any accuracy number, and it is worth holding onto before the model talk starts, because the model talk very nearly seduced me into solving the wrong problem.
This was a marketplace I led, forecasting demand across a catalog in the hundreds of thousands of SKUs, feeding replenishment and warehouse allocation. The brief sounded simple: predict next-week demand per SKU per region, well enough that we could hold less stock without running dry. Hold too much and you are financing dead inventory in a warehouse. Hold too little and you stock out, lose the sale, and, worse over time, corrupt your own training data, because a stockout looks exactly like low demand to a model that only sees units sold.
Why no single model was enough
I tried to make one model carry it. I tried hard, because one model is so much easier to serve, retrain, and explain to a skeptical ops director. None of them was good enough alone, and they failed in different places. That difference is what the ensemble would later run on.
The classical statistical model, a seasonal ARIMA per series, was excellent on the high-volume SKUs with clean, regular history. Stable demand, strong weekly seasonality, it nailed those. It fell apart on anything sparse or spiky, and it had no way to see a promotion coming because it only knew its own past.
The sequence model, an LSTM trained across many series at once, learned shared patterns the per-series models never could. It borrowed strength from neighbors, which is exactly what you want for a SKU with three months of history. It was also a moody overfitter that would invent confident demand for items that had simply gone quiet.
Gradient boosting (XGBoost, fed with lags, calendar features, price, and promo flags) was the workhorse. It ate the messy external signals the other two could not touch. On its own it was jumpy week to week and weak on long seasonal cycles.
The ensemble did not win because three models are smarter than one. It won because the models were wrong about different SKUs, and averaging away uncorrelated errors is close to a free lunch in this game. The lift over the best single model was real and it was boring: never the best on any one series, rarely the worst, and across the whole catalog that consistency is what let us drop the safety stock.
The blend, written out by hand
I did not want a black-box stacker I could not reason about at 2am during a stockout, so the blend is a per-segment weighted average, with weights learned on a recent validation window, not the full history. Recency matters because the right weighting drifts: promos got more aggressive over the year, and the boosting model’s weight crept up as it learned them.
import numpy as np
from scipy.optimize import minimize
def fit_blend_weights(preds, actual):
# preds: dict of model_name -> array of predictions over the
# validation window. actual: the true demand over that window.
# We learn one set of weights PER SEGMENT (fast/slow/new movers),
# because a weighting that's right for a steady SKU is wrong for
# a three-week-old one. Global weights were the first thing I tried
# and the first thing the slow movers punished me for.
names = sorted(preds)
P = np.vstack([preds[n] for n in names]).T # rows = weeks, cols = models
def weighted_mae(w):
w = np.clip(w, 0, None)
w = w / w.sum() # keep it a real average
blend = P @ w
# MAE, not MSE. Squared error lets a couple of promo spikes
# bully the weights toward whichever model chases spikes,
# and that model is usually wrong the other fifty weeks.
return np.mean(np.abs(blend - actual))
w0 = np.ones(len(names)) / len(names)
res = minimize(weighted_mae, w0, method="Nelder-Mead")
w = np.clip(res.x, 0, None)
return dict(zip(names, w / w.sum()))That is the whole trick, and it is not much of a trick. Non-negative weights that sum to one, optimized for absolute error on a rolling window, fit separately for fast movers, slow movers, and new SKUs. We retrained the weights weekly. We retrained the base models much less often, because the base models were stable and the right blend was not.
Serving three models is its own headache
One model is one artifact, one latency profile, one thing to monitor. An ensemble is three model lifecycles that have to agree on a schedule, plus a blending layer, plus the truth that your forecast is only as fresh as your slowest model. The LSTM needed a GPU box and a batch window. ARIMA was cheap but had to be fit per series, which at this catalog size is a lot of tiny fits. XGBoost was fast to score and slow to feature-engineer.
We ran them as separate batch jobs writing predictions to a shared table, keyed by SKU, region, and forecast week, with the model name as a column. The blender read that table, applied the per-segment weights, and wrote the final number. One rule saved us here: if any base model was stale or missing for a SKU, we refused to just blend the survivors. We fell back to a named, logged default and flagged it, because a quietly degraded forecast feeding replenishment is worse than an obviously broken one. (We learned this the week the LSTM job failed and nobody noticed for two cycles. The blend kept producing perfectly plausible, perfectly worse numbers.)
The boring problems that actually decided it
Now the real work, the unglamorous data fixes that moved the inventory number more than any model swap.
Stockouts were the first offender. A day with zero sales because you were sold out is not zero demand, but your sales table cannot tell the difference. Train on raw units sold and you teach the model to under-forecast exactly the items that already hurt you, a feedback loop that starves your bestsellers. We reconstructed censored demand: where availability dropped below a threshold, we treated the observation as a lower bound and imputed from the pre-stockout run rate. It was crude, and it mattered more than the LSTM.
Promotions were the next. A promo spike is real demand, but it is demand the model should attribute to the promo, not to next Tuesday. Without a clean promo calendar as a feature, every model learns that random Tuesdays sometimes triple, and then hedges upward forever. Getting the marketing team’s promo plan into the feature pipeline, reliably and ahead of time, did more for accuracy than anything I trained.
Holidays rounded it out, obvious in theory but painful in practice across regions with different calendars and a lunar new year that walks around the solar one. A holiday feature that is off by a few days is worse than no holiday feature, because the model anchors to the wrong week.
def clean_demand(units_sold, in_stock_ratio, run_rate, floor=0.2):
# If we were mostly out of stock, units_sold is a lower bound on
# demand, not demand. Impute from the pre-stockout run rate so the
# model doesn't learn to starve the exact SKUs that stocked out.
# This one function changed the inventory number more than swapping
# any base model did. Sit with that.
if in_stock_ratio < floor:
return max(units_sold, run_rate)
return units_soldIf I could hand a team starting this exactly one thing, it would not be a model but a clean, trustworthy record of when each SKU was actually available to be bought, paired with a promo calendar that lands in the feature store before the promo runs, not after. Get those and a dull three-model average will cut your inventory. Skip them and the fanciest architecture on the leaderboard will forecast, with total confidence, the past you accidentally taught it.
Which model to reach for was never the real question. I still think about those two cycles the blend spent running on a dead LSTM, when the forecasts came back clean, confident, and completely wrong, and nobody caught it. A censored-demand problem or a late promo calendar does the same thing, only without a failed job to find in the log, and the warehouse fills up anyway.