Comparing sarvam-30b with GPT OSS 120B
This experiment examined a specific reason to try sarvam-30b: whether a model developed for an Indian-language and cultural context would make different ordering decisions on demand shaped around Indian automotive seasons. The completed runs showed little difference from the earlier GPT OSS 120B reference. Getting those runs to finish was a separate problem.
The model and the seasonal task
I ran the Q4_K_M GGUF version of sarvam-30b locally through llama-server. It is a mixture-of-experts model with 30 billion total parameters and approximately 2.4 billion active parameters. GPT OSS 120B was not rerun; its results came from the earlier ordering experiment.
The fictional three-tier automotive chain served one product, with fixed one-month lead times. Its 25-month synthetic demand series included festive peaks and monsoon troughs based on Indian passenger-vehicle seasonal patterns. It was not proprietary sales data. Agents received their numerical inventory state, a supply-chain role, and the current month, but no year or explicit seasonal-event labels. They placed orders without remembering previous decisions.
I obtained ten completed runs with the native reasoning flag disabled and ten with it enabled. Both configurations used temperature 1.0 and top_p 1.0, following the local GGUF documentation. A numbers-only condition was dropped after calibration failed to produce dependable completion. The comparison therefore covers only prompts containing context.
The main measures were stockouts and chain OVAR: order variance divided by demand variance at each tier, averaged across the chain. Below one means damping; above one means amplification. The study required a difference of at least 0.5 OVAR to count as practically important. A separate pattern score combined seasonal keyword use with whether orders increased at the expected seasonal times.
Similar ordering results
| Condition | Chain OVAR | Mean stockouts |
|---|---|---|
| sarvam-30b, reasoning disabled | 4.504 ± 0.044 | 39.9 |
| sarvam-30b, reasoning enabled | 4.501 ± 0.093 | 40.5 |
| GPT OSS 120B, earlier reference | 4.52 ± 0.05 | 39.6 |
The differences were much smaller than the 0.5 threshold. Seasonal pattern scores were close too: 0.219 and 0.232 for sarvam-30b, compared with 0.21 for GPT OSS 120B. These prompts did not reveal the expected seasonal advantage. They do not test everything either model might know about Indian seasons or business conditions.
Nor do close averages demonstrate that the models are equivalent. The GPT OSS reference had twenty runs rather than ten, and came from a different execution setup. Both sarvam configurations also remained far from the exponential-smoothing reference, which achieved about 0.54 OVAR with five stockouts.
Why completed runs are only part of the result
With reasoning disabled, 15 of 25 attempts needed replacement to obtain ten completed runs. With reasoning enabled, none of ten attempts needed replacement. Reporting only the completed-run table would hide that difference.
Its cause is not established. The findings summary says a conflicting “Think silently” instruction affected the main disabled-flag run, while the code and calibration notes record removing that instruction. The available run records do not resolve which prompt was used. The difference in completion therefore cannot be attributed to the reasoning flag alone.
These observations concern the local GGUF and llama-server configurations tested here. A cloud deployment could behave differently. A useful follow-up would first record the exact prompt used in each configuration, then separate completion reliability from the quality of completed orders.
Conclusion
Sarvam-30b did not show a practically meaningful ordering advantage over the historical GPT OSS 120B reference in this context-only test. Neither sarvam configuration approached the simple smoothing policy. That answers the narrow comparison without claiming equivalence or drawing a general conclusion about cultural knowledge.
The completion failures are equally important, but their explanation remains uncertain. This study shows the need to assess both the orders a model produces and whether the serving setup can reliably produce them.