Performance of the Qwen Model
The newly released Qwen 3.8 27B model has demonstrated strong performance on standard benchmarks, scoring near much larger models. However, hands-on tests note that it can sometimes default to overthinking its reasoning processes.
The capability of mid-sized models to rival larger peers makes powerful AI local execution more accessible.
Source posts · 2
Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index — Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index That's the same score as GPT-5.6 Luna (max), and just one point behind GLM-5.2 (max) and DeepSeek V4 Pro 0813 (max) - that GLM is 753B and that DeepSeek is 1.6B parameters, and Luna is size unknown but presumably a whole lot bigger than 27B. Qwen 3.8 27B is a truly astonishing model . Via Hacker News Tags: ai , generative-ai , llms , qwen , ai-in-china , artificial-analysis
Simon Willison highlights Qwen 3.8 27B scoring 52 on the Artificial Analysis Intelligence Index, matching much larger models.
Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things — Friday's big release was Qwen 3.8 27B , an Apache 2 licensed 27B parameter vision-capable LLM from Alibaba's Qwen research lab. I've been looking forward to this one: 27B is an excellent size for running a model on a reasonably specced laptop, and its predecessor Qwen 3.6 27B was impressive. Qwen's self-reported benchmarks for this model are eye-opening. They show a boost from both Qwen 3.6 27B and the closed-weight Qwen 3.7-Plus, which was one of Qwen's strongest models of any size as recently as May this year . It will be interesting to hear what independent benchmarks have to say about the model. I've been running the model on two different machines: my 128GB M5 Max MacBook Pro, and an NVIDIA DGX Spark . On both machines I'm running LM Studio and their 17GB Q4_K_M quantized build . I also tried using llama-server directly on the Spark. The default of extra high results in spectacular over-thinking Qwen's documentation describes the model as defaulting to xhigh for the reasoning effort, and the LM Studio GGUF I've been trying preserves that default: Qwen3.8 comes with official support for reasoning_effort , which can be used to adjust reasoning depth and control cost: • xhigh (default): for complex tasks demanding thorough analysis • medium : balancing accuracy and speed • low : efficient reasoning optimizing for speed and cost This is a hilarious default. It's absolutely not a good way to run the model, especially on consumer hardware. I've been finding the results extremely entertaining. I quickly ran into problems with LM Studio's default context limit of 8,192 tokens - Qwen was using them all up thinking about even the most mundane of problems. I loaded the model with the full 262,144 maximum context length and that problem went away. Here's the pelican riding a bicycle SVG I got from my first attempt with that increased context length. It took 21 minutes to generate, using 22,276 reasoning tokens to produce 3,223 tokens of output. You can read the reasoning trace here . This is by far the best pelican SVG I've been able to generate with a model that runs on a local machine - and this Qwen is pretty small, just a 17GB file on disk. There's a lot to like about this: • The bicycle frame is the right shape • It has legs on each side of the bike - that's very rare • Good, clear pelican pouch • The wings extend to touch the handlebars! • The motion lines are behind, not in front • It has a tasteful background - nice sun, clouds, hill, flowers and grass. Was that worth waiting 21 minutes for? Absolutely not. Here's that same prompt run wi