Muse Glimmer vs Microsoft-Decision-1: Local AI Within Four Answers
How close can a model running on my personal workstation come to a cloud-hosted decision model on industrial engineering questions? I compared Meta’s Muse Glimmer with Microsoft-Decision-1 on the same 487 questions. The local model finished three answers behind in the original ordering and four behind after I reversed the options.
What I compared
I ran Microsoft-Decision-1 version 1 through Foundry on my personal Azure subscription. It is designed for bounded judgments such as classification and routing, and its native choice interface returns a selected option and option probabilities. Muse Glimmer is a roughly 30-billion-parameter generative model designed for local agent workloads. I ran its Q4_K_M version through Ollama on my personal workstation, using an 8,192-token context and the provider’s default sampling and high thinking settings.
The data came from IBM Research’s publicly available FailureSensorIQ benchmark. I used all 487 two-option questions from its original single-answer set. They cover sensor–failure relationships across ten equipment categories, including motors, pumps, transformers and turbines. These are written engineering questions, not live sensor readings. Of the 487, 476 ask about an irrelevant relationship and 11 about a relevant one.
I checked that both models received the same question wording, option content and question order, with expected answers withheld. Each used its own native interface and instructions. For the second presentation, I moved A’s answer content to B and B’s to A, and reversed the scoring key accordingly. Correctness means agreement with the released benchmark labels.
Results on the same questions
| Presentation | Microsoft-Decision-1 | Muse Glimmer | Gap |
|---|---|---|---|
| Original options | 457/487 — 93.84% | 454/487 — 93.22% | 0.62 percentage points |
| A/B contents reversed | 455/487 — 93.43% | 451/487 — 92.61% | 0.82 percentage points |
Glimmer came within one percentage point of Decision-1 in both presentations. That is a small observed gap on this particular quiz, rather than evidence that the models are equivalent. Their architectures, serving configurations and interfaces differ; hosting location was not isolated as an experimental variable.
This compares completed studies: one original and the first swapped run per model. Decision-1 also has four additional swapped runs; across all five its correct counts were 455, 452, 456, 454 and 453. Glimmer has one swapped run. I therefore cannot make a matched five-run repeatability comparison or attribute score changes solely to the option reversal.
What the errors reveal
Decision-1 was correct in both presentations on 448 questions; Glimmer on 441. Each was wrong in both presentations on 23 questions, but those were not identical sets: 13 questions overlapped, with ten additional repeated errors specific to each model. All answers in the four compared panels were valid.
Both models missed the same eight related steam-turbine questions in both presentations, leaving each at 18/26 in that equipment group. Those variants ask which failure is irrelevant when the speed sensor is abnormal; both models selected “damaged labyrinth” against the benchmark key. This is a shared error pattern, not eight independent demonstrations of a knowledge defect or an adjudication of the engineering labels. The complete equipment results and question IDs are retained in the supporting package.
Conclusion
On this fixed industrial quiz, a model running locally on my personal workstation approached the cloud-hosted model’s recorded accuracy, finishing within four answers in each presentation. That makes local models worth evaluating for bounded industrial judgment tasks. It does not establish general model parity, a cost or speed advantage, or readiness for autonomous maintenance decisions. The labels are unadjudicated, some questions are related, and training overlap is unknown.
I see a potential role alongside deterministic workflows, where established rules and validated logic still govern execution. That architecture was not tested here. A specific next question is whether domain-specific fine-tuning on separate material improves accuracy on held-out questions. I have not tested that improvement or fine-tuned either model in this experiment.