I am calling this a suggestion for the audience because I don't have the will/resources to do this.
> Base / architecture: Qwen/Qwen3.6-35B-A3B (Qwen3_5MoeForCausalLM, 256 experts, ~3B active). The "3.8" in the name refers to the teacher, not the base.
Not endorsement, haven't run it myself, just found the link.
I get 20 tokens/s on an M4 Max (larger GPU).
Go here: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
See the right hand side panel, you see a whole palette of quantizations and their respective sizes. Should give you an idea. Note that these are not the only quantizations available.
So it feels very fast.
But it does not seem to be better than Qwen 3.6 35B at coding. A bit worse, I think, though I will test it more.
If you have a machine that can fit a 35B model in VRAM, I would suggest testing Muse Glimmer with (from memory)
Reasoning strength: low
Despite being a dense model, this is actually capable of solving code problems faster than the Qwen MoE, despite having only one fifth of the raw token performance.
MTP is a trade-off, as it pushes some more of the model off the GPU.
I have managed to get usable quants of Laguna S2 and even DeepSeek V4 flash on this setup.
There is clearly some intelligence loss compared to similar sized dense models, but I feel like it stomps on the 9-12b models I could run fully on GPU
That's not true. For computers without unified memory architecture (which is the vast majority) VRAM capacity is the bottleneck for local models. In that case a dense model can deliver significantly more intelligence than an MoE model of the same size. And for a typical consumer/gamer Nvidia GPU, dense models are fast enough.
Most MoE architectures have a few experts that are always running; this, the router, KV, and whatever else you have space for can stay in fast VRAM; and the remaining experts can be offloaded.
With Qwen3.8 27B I could not get anywhere near 32k context window, that made it very unusable for agentic coding, although it was very smart.
tbh, I have stopped using MoE in the name of speed, the dense (with more active parameters) makes a real difference in output quality
I wonder what their angle is going to be; the scene is crowded, and they don't do serving.
Now I might try Ornith-1.5-35B first even though I don’t see an official MLX version.
They may have utility in trying to look at the whole landscape of models, but are very misleading when it comes to making 1:1 comparisons or in developing confidence at to how a given model will deliver on your workflow.
That is such an elegant way to put it.
Qwen 3.8-27B Ornith-1.5-35B
Terminal-Bench 2.1 73.0 67.8
SWE-bench Pro 61.7 59.6
DeepSWE (1.1) 42.2 22.0
NL2Repo 42.3 46.2
GPQA Diamond 89.2 89.2
Humanity's Last Exam 30.8 25.6I will definitely pass Ornith-1.5-9B through the gauntlet as well!
only mmproj can be downloaded right now
1. Is this self improvement at the model level (updated weights or memory, KV etc) or just by adding agentic code harnesess to guide the output better?
Thank you
What kind of hardware you’d need to run the 397B one at an acceptable speed?
That way, when not running inference, the machine can easily host VMs for other experiments and general housekeeping functions.
Their 9B model benchmarks competitively with Sonnet 4 which is pretty cool to have such a small model compared to one that came out 10 months ago.
I’m curious how providers will price their 397B model.
More generally, their current approach to constitutional AI pretty much only makes sense if they believe that they can first teach the model what the Claude character is like and also teach the model that the persona responding is Claude, so I figure that has to be part of the pipeline even if they're not very good at it.