Correctly formatted labels, wider order swings
A language model can return a valid answer and still make a poor ordering decision. This experiment tested that distinction by giving gpt-4.1-mini five labels to choose from and letting software calculate the resulting order.
From a label to an order
The model's choices ranged from strong decrease through neutral to strong increase. At each month and supply-chain tier, a fixed lookup converted its chosen label to a safety-stock multiplier. An Order-Up-To formula then calculated the quantity to order. The model supplied neither that quantity nor the numerical multiplier.
The fictional chain contained a vehicle manufacturer, an ancillary supplier, and a component maker. It served one product over 25 months of synthetic demand representing Indian automotive seasons, with fixed one-month lead times and no injected disruptions or demand shocks.
I compared three ways of supplying information: numbers alone, numbers with calendar context, and a stateful version with recent history. The final comparison used twenty runs per condition. Exponential smoothing and a fixed-buffer version of the ordering formula provided deterministic reference policies.
Two measurements kept format and performance separate. Label compliance recorded whether the answer used one of the allowed labels. Chain OVAR measured each tier's ratio of order variance to demand variance, averaged across the chain: below one means damping and above one means amplification. I also counted stockouts so that a lower ratio would not be mistaken for a complete improvement if it came with more shortages.
Every label was usable, but the orders still lagged the baselines
| Condition | Chain OVAR | Mean stockouts |
|---|---|---|
| Exponential smoothing | 0.545 | 5 |
| Fixed-buffer control | 1.710 | 14 |
| Numbers only | 3.843 ± 0.129 | 12.05 |
| With context | 3.268 ± 0.062 | 15.70 |
| With history | 3.762 ± 0.084 | 12.05 |
Label compliance was 1.000 throughout, and no fallback responses were needed. The model therefore used the interface successfully in these runs. Its choices produced roughly six to seven times the OVAR of exponential smoothing, however, and smoothing also recorded fewer stockouts.
Calendar context lowered OVAR relative to numbers alone, but all three model conditions remained above the fixed-buffer control as well as smoothing. Software could read every response, yet the resulting orders still performed poorly on the variance measure.
The experiment began with ten repetitions per condition and was expanded to twenty. The headline means changed by less than one standard deviation. The table reports the expanded set; the extra repetitions checked repeatability on the same demand series, not performance across different demand profiles.
Conclusion
In this test, reliable formatting was not enough for reliable control. The model always returned an allowed label, yet every model condition produced more order-variance amplification than either reference policy. Smoothing also produced fewer stockouts.
The result is limited to one model and this particular mapping from labels to multipliers. Those multiplier values were chosen from domain logic, not shown to be optimal. The following experiment tested the label approach with disruptions and random variation in supply conditions. Its results describe that different environment and should not be combined with the numbers here.