Language models against simple ordering rules
The first ordering experiment compared language models with one another. This one added a more useful reference: simple rules that could place orders without a model. I wanted to know whether direct AI ordering offered anything over those rules in either order stability or shortages.
What the models had to do
A fictional vehicle manufacturer, lighting supplier, and component supplier formed a three-tier chain serving one product. The simulation used 25 months of synthetic demand covering two Indian festive cycles, with fixed one-month lead times. Every run started from the same initial conditions.
I tested gpt-4.1-mini and o4-mini through Azure, and phi4:14b and GPT OSS 120B locally through Ollama. Each model chose order quantities from numerical inventory information, either alone or with a tier-specific role and calendar context. That gave eight model-and-information combinations, each run twenty times. The agents had no memory of earlier decisions.
Three rules supplied the comparisons: exponential smoothing with α = 0.30, passing demand through directly, and ordering up to a stock target with a fixed safety buffer. Each ran once because the rule and environment were deterministic. Repeating that calculation would not test how the rule behaved under different demand; this study used only one series.
I measured chain OVAR alongside stockouts. OVAR divides order variance by demand variance at each tier and averages the ratios across the chain. Above one means the orders amplify demand fluctuations; below one means they dampen them. The study required an OVAR difference of at least 0.5 to count as practically meaningful, and lower variance was not enough if shortages became worse.
The gap to simple rules
| Policy | Chain OVAR | Stockouts |
|---|---|---|
| Exponential smoothing | 0.545 | 5 |
| Demand passthrough | 1.00 | 3 |
| Order-up-to | 1.71 | 14 |
| phi4:14b, numbers only | 4.33 | 41.0 mean |
Even the model condition with the lowest mean OVAR was far from the rules. Model averages ranged from 4.33 to 6.35, and every rule recorded both lower OVAR and fewer stockouts than every model configuration. The extra order variability was not buying better service.
Context did not consistently narrow the gap. For gpt-4.1-mini, it reduced OVAR from 4.70 to 4.47, less than the study's practical-importance threshold. For phi4:14b, it increased OVAR from 4.33 to 6.35. The latter context condition also had a standard deviation of 2.53, showing substantial variation between runs. The reasoning models did not deliver the expected operational advantage either. None of the study's seven hypotheses was supported; their definitions and individual results are in the repository.
What this comparison can explain
These models were responsible for the order quantities themselves. The results therefore concern direct replenishment decisions under these prompts, not other possible uses such as reading supplier documents or helping a planner interpret an unusual event.
The agents' lack of memory may have contributed, but the experiment did not vary memory independently. It cannot establish why the models performed poorly, or predict how an agent with a different design would behave. What it does establish is the observed gap to the rules in this fixed simulation.
Conclusion
None of the tested language-model configurations improved on the simple ordering policies. Every rule did better on both measured outcomes, so selecting the best model among this group would still have left a substantial gap.
That result led to a different question: could a model contribute useful information while a formula retained responsibility for calculating the order? The following experiments narrowed the model's role to test that possibility.