Does context help AI place supply chain orders?
An inventory balance tells an ordering agent how much stock is available. A business description adds a different kind of information: what the company makes, whom it supplies, and where the current month falls in its calendar. I tested whether that extra context helped two language models place steadier orders.
The ordering task
The experiment used a fictional vehicle manufacturer, lighting supplier, and component supplier. Each tier started with 43,000 units of inventory and received deliveries after a fixed one-month lead time. All runs followed the same 13-month synthetic demand series, from December 2024 to December 2025, with twelve active ordering periods.
I tested gpt-4.1-mini and o1. In the “blind” condition, they received numerical inventory information. In the context condition, they received the same numbers with company, product, market, and calendar information. The models chose order quantities without remembering their earlier decisions.
The outcome of interest was the bullwhip effect: orders fluctuating more than the demand they respond to. I measured it using chain OVAR, the ratio of order variance to the variance of demand received at each tier, averaged across the chain. A value above one indicates amplification; below one indicates damping. I also measured stockouts and excess inventory, because fewer shortages can come at the cost of holding more stock.
Context changed the two models differently
The original plan called for five runs per configuration. One stored run in each gpt-4.1-mini condition identified a different model, so I excluded those runs. The table therefore uses four runs per condition for gpt-4.1-mini and five for o1. Some older aggregate files still include the excluded runs.
| Model | Runs per condition | Blind OVAR | Context OVAR |
|---|---|---|---|
| gpt-4.1-mini | 4 | 3.149 | 2.914 |
| o1 | 5 | 3.835 | 4.412 |
Adding context reduced gpt-4.1-mini's mean chain OVAR by about 7.5%. For o1, it increased the ratio by 0.577. All four averages remained above one: the extra information helped one model relative to its own numbers-only result, but neither model avoided amplification. The o1 results also varied substantially between runs, limiting how much confidence to place in either average.
The o1 context condition had fewer stockouts but substantially more excess inventory. Its mean chain stockout count was 12.8, and its excess-inventory measure was 654,728 units. A judgment based only on shortages would therefore miss an important cost of its ordering behaviour.
Repeating the gpt-4.1-mini comparison
In July 2026, I repeated the two lightweight conditions with five runs each, using the pinned model version gpt-4.1-mini-2025-04-14. Chain OVAR was 3.088 ± 0.091 without context and 2.794 ± 0.080 with it. The ± values describe standard deviation across runs, not confidence intervals. The 9.5% reduction was in the same direction as the original result.
This replication covered only gpt-4.1-mini. It did not rerun o1, replace the original comparison, or provide a simultaneous test of both models. Both batches also used the same simplified setting: one product, one demand series, fixed lead times, and limits on the order quantities the models could return. The hypotheses were not pre-registered.
Conclusion
Business context helped gpt-4.1-mini on the order-variance measure and hurt o1 in this experiment. That difference matters: supplying more information was not a dependable way to improve both models. The later lightweight replication strengthened the evidence for that model's response, while leaving the o1 result unchanged.
The larger question remained unanswered. There was no simple ordering-rule baseline, so the study could not show whether either model was preferable to an ordinary formula. The next experiment added those baselines and more repetitions.