When predicting zero beats every model you have

I recently worked on forecasting monthly demand for around a hundred key pharmacy accounts, at customer by product by month. It looks like a classic tabular problem. Then you run the baselines, and a model that always predicts zero beats every moving average you throw at it.

That isn't a bug. When only about a quarter of the cells are non-zero, putting real demand in the wrong month costs more than predicting nothing at all. The rule of thumb in use before this did worse than zero too, with a large positive bias.

Lock the rules before you train

Before training anything, I wrote down the selection criteria and hashed the file with SHA-256. After a bake-off of 24 contenders across 6 families and 6 rolling origins, those criteria disqualified every one of them, including the zero baseline that set the bar. It was a humbling result, and it was only credible because I couldn't move the goalposts afterwards.

Where the value actually was

• Whether an account buys at all is learnable: a classifier clearly beat the best baseline on that question.
• The same forecast that looks hopeless per cell gets far better per customer-month, and better again nationally. Value lives in totals.
• On dense pairs, which hold most of the demand, a scoped per-cell forecast is genuinely useful.

The most useful thing a baseline can do is embarrass you early.

The takeaway isn't that ML failed. It's that the question needed a better shape, and you only find that out if your evaluation is allowed to say no.