Strata 0.1.45 Runs a 125-Billion-Parameter Model on a Gaming PC, Adds Vision
The latest release of the open-source local AI engine adds image support and up to 40% faster decoding. It still needs a real graphics card to pull it off.
Strata 0.1.45 landed today, and it can now see. That's the main news for what has become the default way to run a 125-billion-parameter model on a PC that would otherwise just be used for gaming. Maintainer Niko1221 pushed the release under today and, as usual, bundled the speedups together with the vision support.
The flagship model, Qwen3.8-Flash-Next, is unchanged. Answers are byte-identical to 0.1.44.1 across every GPU family tested, from CUDA to Mac M4. The new stuff centers on the smaller Qwen3.6-35B-A3B family, which is getting vision, parallelism, and faster decoding.
Why "run it locally" is suddenly doable
For most of the AI boom, running a state-of-the-art model meant two options: pay a cloud API or buy a rack of datacenter GPUs with hundreds of gigabytes of memory. Anything near 100 billion parameters sat firmly outside what a laptop could handle.
Two things changed that. First, Mixture-of-Experts models fire only a small slice of the total parameters per token, maybe 6 billion here instead of all 125 billion. Compute stays cheap even though the full weight set still has to ride along. Second, quantization squeezes 16-bit weights down to 4-bit or lower, which is what lets a model actually fit into the 12 to 24 GB of VRAM on a consumer card.
Strata lives right at that intersection. It's an inference engine built on pieces of llama.cpp / ggml, wrapped in a one-click installer that detects your hardware, picks a size that fits, downloads the model, and serves it through an OpenAI- and Anthropic-compatible API. Claude Code, Codex CLI, any OpenAI-style app, all offline, nothing leaving your machine.
As of this month the repo sits at roughly 26,600 stars, and it hit the Hacker News front page. That kind of attention tends to come with a proportionate number of open issues, though the maintainers have been unusually upfront about their process. mainline llama.cpp has since adopted its own expert-cache feature, which validates the whole direction and also splits the community's attention.
What 0.1.45 actually changes
Three changes matter most.
Vision comes first. Setup now asks whether you want images when you pick the Qwen3.6 family, downloads a roughly 0.9 GB image projector, and launches with --vision. It validates against llama.cpp on identical prompts and matched images, though the authors admit the first token only agreed in 5 of 6 test cases. On NVIDIA cards under 24 GB it uses the CPU encoder for pictures, which keeps a chunk of VRAM free. That's a real compromise, but the trade is clear. A GPU encoder is slower to read but steals nothing from your model's memory.
Faster decoding is second. New decode kernels were built "with the old kernels' bits" and verified against 18,360 real-activation comparisons with zero mismatches. That sounds paranoid, and I like it. On a 32 GB AMD R9700 decode jumped from 83 to 118 tokens per second, and long prompt reading climbed to 1.8× faster. One contributor's work even made a 500K-token prompt read 79% faster on AMD hardware.
Parallelism is third. Up to eight concurrent requests now stack with --parallel N, and the setup wizard tells you up front how many slots your card can spare at your chosen context. On the R9700, eight interleaved conversations ran 1.56× faster than llama.cpp alone. The honest disclosure is worth reading: if your card can't hold all the experts, the slots steal VRAM from the cache, and a single request actually runs a little slower. "Worth it for several users or agents at once, not for one chat," the notes say.
There's also a big set of opt-in STRATA_* environment variables for multi-GPU servers, plus a CUDA-only build option that reads resident experts in place. On a 4× R9700 rig those pushed time-to-first-token from 20.6 seconds down to 2.0. That's roughly 10× for a busy multi-user server. One switch, STRATA_SLOT_CKPT_MOVE=1, is experimental, and the maintainers literally tell you not to use it.
Benchmarks and the honesty problem
Strata's own README lists write and read speeds by quantization. On an RTX 5070 with a Q2_0 build you get about 94 tokens per second writing and 2,650 reading. On AMD's RX 9070 XT the same build drops to 60 tokens per second. Those are author measurements, not independent tests, so treat them as directional.
The model's benchmark scores are Qwen's own vendor-reported figures, a distinction worth holding onto. The 62.5 on SWE-bench Pro and strong math scores describe the full-precision model. They don't describe what you'll get running a Q2 or expert-pruned build locally. That's the single biggest community objection: aggressive quantization trades real quality for a model that fits your RAM.
Several caveats are worth flagging. First start can freeze your PC. Loading 35 to 55 GB into memory while pulling a 70 GB download will make the machine unresponsive for a couple minutes, and the first prompt takes roughly a minute per 30,000 tokens to read. Follow-ups start in seconds. The "paste this into your AI assistant" install is a genuine supply-chain quirk too: the README offers a prompt that tells an assistant to run the setup itself, and commenters compared it to piping a script into bash.
Platform gaps remain as well. In 0.1.45 the new work skipped Windows hardware, Intel Arc, and the RTX 5070, and Volta/Pascal got pushed to 0.1.46. If your machine has only 8 to 16 GB of RAM you're outside the useful range and better off with a smaller dense model.
License, caveats, and getting it
The Strata engine itself is MIT-licensed, which is permissive. The model weights are not. Qwen3.8-Flash-Next runs under the Qwen Community License 1.0, which means you need a separate license from Qwen if you host it as a service or build a commercial product on top. Keep that in mind before you plan to resell anything.
To run it yourself, clone the repo or grab main.zip, then follow the platform script. Run setup.sh, which builds the engine in about seven minutes. The installer detects your hardware, downloads the roughly 70 GB model, and opens the app at http://127.0.0.1:8080.
Want to see how your machine would fare? Run bench.sh for a shareable speed report that takes five to ten minutes.
The full release notes, model card, and setup docs are on the Strata GitHub repository.
