Better classifications, similar ordering outcomes
An ordering system needs more than a correct reading of the situation. It also needs a useful way to turn that reading into an action. I tested whether models that interpreted seasonal and disruption signals more accurately would produce better orders when their labels were passed to a fixed formula.
How the labels affected the supply chain
I tested gpt-4.1-mini, o4-mini, phi4:14b, and nemotron-super 120B. Each selected one of five direction labels, from strong decrease to strong increase. Software mapped the label to a safety-stock multiplier between 0.80 and 1.30, then an Order-Up-To formula calculated the order. The model interpreted the information; code performed the calculation.
The fictional three-tier chain used 36 months of synthetic demand shaped around Indian automotive seasons. Simulated pandemic, geopolitical-conflict, and port-strike events affected the environment, while lead times and the proportion of orders fulfilled varied randomly. Headlines supplied to the models were written for the simulation, not taken from live news.
Each model received numbers alone, calendar and seasonal context, or context plus event text. The two frontier models ran twenty repetitions per condition and the local models ran ten. The baseline policies ran one hundred times because supply conditions varied between runs.
I measured directional classification accuracy, label compliance, stockouts, and chain OVAR. The last measure averages each tier's ratio of order variance to demand variance; values above one indicate amplification. Measuring classification and ordering separately allowed me to see whether recognising the situation actually improved the resulting orders.
Accuracy improved more than ordering performance
Without context, directional accuracy ranged from 0.41 to 0.48. Calendar context raised it to approximately 0.72–0.80, and event text brought some conditions as high as 0.84. The order-variance results changed much less.
Across all twelve model conditions, mean OVAR ranged from 1.726 to 1.780. That spread of 0.054 was smaller than the within-condition standard deviations of 0.088–0.133. The study called this clustering the “Equaliser Effect.” It was a description of the observed results, not a statistical demonstration that the models were equivalent.
Phi4:14b with context had the lowest model mean, 1.726. The formula-only Order-Up-To baseline recorded 1.767, while exponential smoothing reached 1.185. The model results therefore stayed near the formula-only result rather than approaching smoothing.
Smoothing had a cost on the other measure: it averaged 89.62 stockout tier-periods per run, compared with 86.95 for Order-Up-To and 84.7–87.0 for the models. A stockout tier-period counts a shortage at a particular supply-chain tier in a particular period. Lower OVAR did not mean fewer shortages.
Checking what context and the labels were doing
For every model, adding event text produced slightly higher OVAR than calendar context alone. The hypotheses that an AI condition would beat smoothing, that event text would reduce OVAR, and that it would improve disruption-period directional accuracy by at least ten percentage points were not supported. The models did meet the required label-compliance threshold.
The simulated events themselves also changed the ratio. Removing them from otherwise matched phi4 conditions increased OVAR to about 2.08–2.12, supporting the hypothesis that the events materially affected OVAR. This is a reminder that a variance ratio depends on the demand environment as well as the ordering policy.
A separate comparison added an instruction to default to neutral. Across six matched model-and-input comparisons, mean OVAR shifted by −0.015 to +0.005. Neutral still selected the multiplier 1.0 and executed the formula; it did not mean placing no order. This check left open how much better labels could improve the downstream calculation.
Conclusion
More accurate direction labels did not produce comparable reductions in order-variance amplification in this experiment. Model results remained close to the formula-only baseline. Smoothing achieved lower OVAR but more stockouts, so it did not lead on every outcome.
The pattern suggested that the lookup and formula might limit the benefit of better classification, but clustering alone could not establish that limit. The next experiment tested it directly by supplying perfect simulation-defined labels and deterministic rules. The finding here remains specific to this synthetic demand profile, stochastic supply environment, and label-to-multiplier mapping.