#qwen

Meet Qwen3.6-35B-A3B：Now Open-Source！🚀🚀 A sparse MoE model, 35B total params, 3B active. Apache 2.0 license.

↯ Qwen 3.6moeqwenagentic

Meet Qwen3.6-27B, our latest dense, open-source model, packing flagship-level coding power! Yes, 27B, and Qwen3.6-27B punches way above its weight.

↯ Qwen 3.6qwenagentic

A short follow-up to my previous post, where I showed that changing the scaffold around the same 9B Qwen model moved benchmark performance from 19.11% to 45.56%: https://www.reddit.com/r/LocalLLaMA/s/JMHuAGj1LV After feedback from people h…

↯ Qwen 3.6qwen

My pre-gemma 4 setup was as follows: Llama-swap, open-webui, and Claude code router on 2 RTX 3090s + 1 P40 (My third 3090 died, RIP) and 128gb of system memory Qwen 3.5 4B for semantic routing to the following models, with n_cpu_moe where…

↯ Qwen 3.5gemmaqwenllama+1

They are waiting for the exact roadmap

qwen

https://preview.redd.it/t4hf4x4ruw1h1.png?width=380&format=png&auto=webp&s=788c8e011e407d3e1aa49d6d4be0a813a75e03df Title

↯ Qwen 3.7qwen

Gemma 4 and Qwen 3.6 with q8_0 and q4_0 KV cache: KL divergence results 4 models tested with q8_0 and q4_0 KV cache against full-precision baseline What this measures KV cache quantization stores the key-value cache in lower precision to s…

↯ Qwen 3.6gemmaqwen

Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.

↯ Qwen 3.6qwenagentic

MacBook Pro M5 MAX 64GB. Qwen 3.6 35B - 72 TPS.

↯ Qwen 3.6qwen

My personal test for small local LLM intelligence is to check whether a model has any ability to understand the code that I write for my own academic research. My research is on some pretty niche topics and I doubt that anything like it is…

↯ Qwen 3.6qwen

↯ Qwen 3.6qwencursoropus+2

I spent some time yesterday after work trying out the new qwen3.6-35b-a3b model, and at least for me it's the first time that I actually felt that a local model wasn't more of a pain to use than it was worth. I've been using LLMs in my per…

↯ Qwen 3.6qwenopus

moedeepseekqwen+1

WARNING: wait before download from HF: I just realised my upload of the new versions with the additional fix in the chat template has not completed yet. I will remove this warning once done The recent PR to llama.cpp bring MTP support to Q…

↯ Qwen 3.6qwenllamaagentic+2

Man, AI agents getting pretty crazy these days. :) (local, I just decided to try to get an orchestrator in there, when Qwen and Gemma aren't up to it.)

gemmaqwen

could not extract summary

qwen

I was trying to do an easy task automatically with qwen-code using qwen3.5-122b I can totally do it myself, but I wanted to try, so maybe it could just do it entirely for me? But no, because it refused.

↯ Qwen 3.5qwen

Time and time again I find posts about these fine tunes that promise increased intelligence and reasoning with base models, and I continuously try them, realize they're botched, and delete them shortly after. I sometimes do resort to a low…

↯ Claude 4.6qwenllamaopus

Gemma just crushed Qwen in a local LLM gamedev contest! Device: MacBook Pro M5 Max, 64GB RAM Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens.

↯ Qwen 3.6gemmaqwen

Looks like Apple has quietly killed off the higher-memory Mac Studio options. The M3 Ultra Mac Studio is now only available with 96GB RAM.

qwen

TL;DR best setup I tested on a RTX 3090 24 GB: ik_llama.cpp + Qwen3.6-27B-MTP-IQ4_KS.gguf 156k context, q8_0/q8_0 KV, MTP, vision on CPU benchmark result on a ~5.9k prompt + 1k output: about 1261 tok/s prefill, 72.9 tok/s decode llama.cpp…

↯ Qwen 3.6vllmqwenllama

The hype is real! https://qwen.ai/blog?id=qwen3.7

↯ Qwen 3.7qwen

For real tho, 9b, 27b, 122b, I don’t really care at this point, just show us that you still love us. EDIT: I guess I gotta use /s on my posts from now on.

↯ Qwen 3.7qwen

I love following all your adventures with local LLM setups. Quality and size of the models are important, but so is performance.

↯ Qwen 3.6qwen

Evaluated Qwen 3.6 27B across BF16, Q4_K_M, and Q8_0 GGUF quant variants with llama-cpp-python using Neo AI Engineer. Benchmarks used: HumanEval: code generation HellaSwag: commonsense reasoning BFCL: function calling Total samples: HumanE…

↯ Qwen 3.6↯ Function Callinghumanevalfunction-callingqwen+1

The following is a non-comprehensive test I came up with to test the quality difference (a.k.a degradation) between different quantizations of Qwen 3.6 27B. I want to figure out what's the best quant to run on my 16 GB VRAM setup.

↯ Qwen 3.6qwen

I built hfviewer.com, a small tool for visually exploring Hugging Face model architectures. You can paste a Hugging Face URL and get an interactive visualization of the architecture, which can make it easier to understand how different mod…

↯ Qwen 3.6gemmaqwen

I had previously posted here about a fix to their 3.5 template to help resolve the KV cache invalidation issue from their template. A lot of you found it useful.

↯ Qwen 3.6qwen

Here's my article with 38 quant pairs thoroughly benchmarked in KLD with 3 different Qwen 3.6 27B configs: Q5_K_S + 64k context, IQ4_XS + 64k context, IQ4_XS + 128k context. This allows us to track not only how cache quantizations affects…

↯ Qwen 3.6qwenllama

I keep hearing the argument that that large models are better for high-level planning and task orchestration, since they have more general knowledge to work from when making decisions. However, I've been testing Qwen 3.6 27b (Unsloth Q5_K_…

↯ Sonnet 4.6sonnetqwenclaude-code

↯ Opus 4.7qwenopus

GLM 5.1 is dominant in almost every aspect in Design arena, surpassing Opus 4.6 in many tasks. Although user experiences vary dependent on subscription plans for both of those one of them is open source.

↯ Qwen 3.6glmgemmaqwen+1

I have a personal eval harness: A repo with around 30k lines of code that has 37 intentional issues for LLMs to debug and address through an agentic setup (I use OpenCode) A subset of the harness also has the LLM extract key information fr…

↯ Qwen 3.6gemmaqwenagentic

I have ThinkPad T14 Gen 5 (8840U, Radeon 780M, 64GB DDR5 5600 MT/s ). Tried out the recent Qwen MoE release, and pp/tg speed is good (on vulkan) (250+pp, 20 tg): ~/dev/llama.cpp master* ❯ ./build-vulkan/bin/llama-bench \ -hf AesSedai/Qwen3…

↯ Qwen 3.6moeqwenllama

Saw this post comparing Qwen 3.6 variants on coding primitives, so I wanted to see how local quants stack up against frontier models on a similar dense, single-file coding task. I ran the exact same prompt across local and web-based models…

↯ Qwen 3.6qwen

Introducing FlashQLA: high-performance linear attention kernels built on TileLang. 2–3× forward speedup.

qwenagentic

Hey r/LocalLLaMA, We’ve released our ByteShape Qwen 3.6 35B GGUF quantizations in two families: standard NTP (Next Token Prediction or non-MTP) and MTP. Blog / Download NTP Models / Download MTP Models TL;DR For NTP, “pick the largest quan…

↯ Qwen 3.6mmluqwen

The dense sibling of the 35B-A3B drop is here, Qwen3.6 27B Uncensored Aggressive is out! Aggressive = no refusals; NO personality changes/alterations or any of that, it is the ORIGINAL release of Qwen just completely uncensored https://hug…

↯ Qwen 3.6qwen

Model Description Q-Judger is a vision-language model fine-tuned specifically for automated evaluation of text-to-image generated images. Given a text prompt and a generated image, the model evaluates the image on fine-grained quality crit…

↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6qwen

Mods please be kind. This was not “low effort”.

qwen

TL;DR Results from the title are for single inference with 2 prompt of 1k and 15k tokens. So no MTP (as it’s slower for big prompt), no DFlash (working too but slower for big prompt), no quant used (full precision wanted) and the results a…

↯ Qwen 3.6vllmqwenagentic+1

Heya guys and gals, Around a year ago I released and posted about Persona Engine as a fun side project, trying to get the whole ASR -> LLM -> TTS pipeline going fully locally while having a realtime avatar that is lip-synced (think VTuber)…

↯ Qwen 3↯ Qwen 3qwen

Qwen Team released Qwen-Scope — a collection of Sparse Autoencoders (SAEs) for the Qwen 3.5 family (from 2B to 35B MoE). They’ve mapped internal features for the residual stream across all layers.

↯ Qwen 3.5moeqwen

could not extract summary

↯ Qwen 3.7qwen

GPT-5.5: xhigh: 94.0→97.5 high: 93.6→96.9 medium: 92.0→95.0 no reasoning: 32.8→37.5 Kimi K2.6 improves over Kimi K2.5 (78.3→91.4) and becomes the #1 open weights model. DeepSeek V4 Pro improves over DeepSeek V3.2 (50.2→75.7).

↯ DeepSeek 3.2gpt-5deepseekqwen+2

- Claude Opus 4.6 - absolute rogue AI. Does what I want like it’s breaking at least 3 internal policies to make it happen.

↯ Sonnet 4.6gpt-5sonnetqwen+2

BeeLlama v0.2.0 is here! Not quite a pegasus, but close enough.

↯ Qwen 3.6gemmaqwen

Note: First is Qwen3.5 35B MoE (Left) and Second is Qwen3.6 (Right) Hi Guys Just did quick comparison of Qwen3.6 35B MoE against Qwen 3.5 35B MoE. with reasoning off using llama.cpp and same quant unsloth 4 K_XL GGUF First is Qwen3.5 outco…

↯ Qwen 3.6moeqwenllama

↯ Qwen 3.6qwen

So i have been seeing more of those pelican on a bike svg tests and while they work i feel like (and maybe you guys do too) they are getting kinda benchmaxxed so we should switch things up soon and this is my idea generate me a html svg of…

↯ MiniMax 2.7minimaxglmdeepseek+3

Hardware: RTX 3060 12GB 32GB DDR4-3200 Windows CUDA 13.x Model: Qwen3.6-35B-A3B-MTP-IQ4_XS.gguf The model is a 35B MoE, so -ncmoe matters a lot. Lower -ncmoe means more MoE blocks stay on GPU.

↯ Qwen 3.6moeqwenllama

Been using this for a few days. It is BY FAR the best uncensored model I have found for Qwen 3.6 35B.

↯ Qwen 3.6qwen

Using 100k context with 3090 with MTP GGUF and getting 50 t/s on llama.cpp Thought I would knowledge share Use https://huggingface.co/RDson/Qwen3.6-27B-MTP-Q4_K_M-GGUF And am17an commit /media/adam/D_DRIVE/LLM/llama-cpp-am17an/build/bin/ll…

↯ Qwen 3.6qwenllama

I am waiting for 122B and new 27B

qwen

Hello guys, So TL;DR, I was asked by multiple people to make an Assistant_Pepe_32B version, but the best base model contender was Qwen3-32B, a model that is very hard to tune on anything other than STEM. The concept of Assistant_Pepe is an…

↯ Qwen 3qwen

I have a 5090, so my VRAM is limited to 32GB, but i find that Qwen3.5-27B-UD-Q5_K_XL with opencode (and mmproj) does a pretty good job for my use case (mainly web development). i use claude and codex here and there, recently a lot less, be…

↯ Qwen 3.5qwencodex

https://preview.redd.it/42ak5qmus82h1.png?width=1133&format=png&auto=webp&s=744ea3dfc06c83d0c4d8aa128c39b3238b17d7be Qwen 3.7 Max sitting at 5th, pretty much on par with GPT 5.4 (xhigh) and a notch above the just released Gemini 3.5 Flash.…

↯ Gemini 3.5qwengemini

I'm still hoping we see a Qwen3.6-122B or a Qwen3.6-coder, but my hopes are dimming. Seems like we would have seen/heard something by now, even if just tantalizing hints from the Qwen folks.

↯ Qwen 3.6qwen

Just a little reminder that *if* it is possible for you to run bigger quants, do it. I ran Qwen 3.6 IQ4_XS at 128k context was very much disappointed because it would loop, make formatting errors, implement wrong things etc.

↯ Qwen 3.6qwenagentic

Maybe it be helpful for someone: llama-server -m '/Qwen3.6-27B/Qwen3.6-27B-IQ4_XS.gguf' -ngl 999 -ctk q4_0 -ctv q4_0 -b 128 -ub 128 -c 24000 Cant run this model with higher kv quants on >8192ctx size. -ub & -b setted for 256 allowed me for…

↯ Qwen 3.6qwenllama

I'm keeping a close eye on the development of local llms.

↯ Qwen 3.6qwenopusclaude-code

↯ Qwen 3.5gemmaqwenagentic+1

Just swapped Qwen 3.5 for the 3.6 variant (FP8, RTX 6000 Pro) using the same recommended generation settings. My stack is vLLM (v0.19.0) + Open WebUI (v0.8.12) in a RAG setup where the model has access to several document retrieval tools.

↯ Qwen 3.6vllmragqwen

----START HUMAN TEXT---- Hi all, I've seen a bunch of posts about squeezing 27B onto a 24GB card and all the quantization tricks involved in doing so. It's all amazing work, but at the end of the day a quantized model with quantized KV wil…

↯ Qwen 3.6qwenagentic

I've been running structured output prompts through a bunch of models on OpenRouter for the past few months — Llama 3, Mistral, Command R, DeepSeek, Qwen, and every other model on OpenRouter — alongside the usual closed-source suspects. 28…

↯ Mistralmistraldeepseekqwen+1

Typing this on a cramped flight, but I was having issues connecting to the plane's wifi on my ubuntu laptop, when it was effortless on my phone. The issue I was having was the Laptop WiFi connected to the plane wifi network, but captive po…

↯ Qwen 3.6qwen

I'm sure everyone has seen the posts from people talking about Qwen 3.5 over-thinking, or maybe you've experienced it yourself. Considering we're like 2 months out from the release and I still see people talk about this issue, I decided it…

↯ Qwen 3.5gemmaqwengemini

Qwen3.6-27B [!Note] This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format. These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransforme…

↯ Qwen 3.6vllmqwen

↯ Qwen 3.5aiderqwen

Basically, I use To Good 2 Go a lot, get random food, take a photo and ask Qwen 3.5 128B what the fuck to cook. Beyond pasta and pizza, I have zero cooking skills.

↯ Qwen 3.5qwen

We ran open-weight 27B–32B models on Terminal-Bench 2.0 (89 tasks, terminal-bench-2.git @ 69671fb) through our agent harness. Best result was Qwen 3.6-27B at 38.2% (34/89) under the default per-task timeout — the same constraint the public…

↯ Qwen 3.6moeqwen

I’m thinking about where this might wind up in 3-5+ years. As others have noted there’s no guarantee that Qwen, Google, and others will continue to release models in the future.

qwen

https://huggingface.co/bartowski/FINAL-Bench_Darwin-36B-Opus-GGUF Darwin-36B-Opus is a 36-billion-parameter mixture-of-experts (MoE) language model produced by the Darwin V7 evolutionary breeding engine from two publicly available parents:…

↯ Claude 4.6moeqwenopus

https://preview.redd.it/fxkx8wyzqrwg1.png?width=1152&format=png&auto=webp&s=e141f0f1c5f680ab602115d06dc4621e46309983 Qwen-27b now this!

qwen

This spring is really hot since the localLLM giant, both Qwen and Gemma released major models. I'm really excited with those release and happy with their capability.

moegemmaqwen

Hi everyone, I'm presenting a new quantization of the Qwen-27B model, created specifically with 16GB VRAM NVIDIA GPUs in mind. I used quants that, unfortunately, are not yet available in the main upstream llama.cpp.

↯ Qwen 3.6qwenllama

I recently published MTP quants of Qwen 3.6 27B and I was suprised by the reports here on reddit, and on HF, of users who were experiencing worst speed with speculative inference than without. This did not match what I was seeing, but when…

↯ Qwen 3.6qwen

Hey everyone, I’ve been building a local-first desktop PDF reader that can read technical books aloud and keep the spoken text highlighted while reading. The original motivation was pretty practical: I read a lot of programming and technic…

qwenllama

Have Qwen hinted at whether other models (9B, 122B, 397B) would be getting the 3.6 treatment? Or have they in any way confirmed or hinted at "this is it"?

↯ Qwen 3.6qwen

TL;DR New llama.cpp fork! I wanted a Windows-friendly inference to run Qwen 3.6 27B Q5 on a single RTX 3090 with speculative decoding, high context without excess quantization, and vision enabled.

↯ Qwen 3.6qwenllama

Greetings from former TurboQuant's biggest defender, now middle-sized niche-aware TurboQuant defender. Today I'm presenting to you the results of me thoroughly exploring the world of PPL and KLD benchmarks with my single RTX 3090 using Bee…

↯ Qwen 3.6vllmqwen

As the title suggests. I'm already testing (with some success, and few challenges) usage of Qwen-3.5 9B with a new work laptop that I've received with RTX 1000 6GB VRAM (I know it seems like a joke in today's time and age).

↯ Gemini 2.5qwengemini

Not affiliated with Kaitchup, but a fan of their testing. I was looking forward to this article...

↯ Qwen 3.6gemmaqwen

Does anyone else have the same experience comparing these two - for me 3.5 122B outperforms 3.6 by a large margin. 3.6 gets lost as long as the task requires a couple of more steps.

↯ Qwen 3.6qwen

Hugging face link here. Ive been waiting for sokann to drop his Qwen 3.6 GGUF for 16 GB GPUs as his Qwen 3.5 was my GGUF of choice.

↯ Qwen 3.6qwen

So in response to the Great Token Reconning of 2026, I decided to try out Qwen 3.6 as a daily driver, and although it's only been about a day, I have to say I'm thoroughly impressed. I had to download the VSCode insiders edition and set up…

↯ Copilot↯ Qwen 3.6copilotgemmaqwen

↯ Qwen 3.6moegemmaqwen

I dont have good experience running q4_k_m, the difference to q6 is "a few errors an hour" to " a few errors every couple of days". Edit: How it fails?

↯ Qwen 3.6qwenagentic

My workflow has changed basically to ask Codex to do certain tasks and then document how to do them (including errors it found on its way) into a skill. I feed that skill to pi, and suddenly my qwen3.6 gets that hard stuff done: - devops o…

↯ Qwen 3.6qwencodexchatgpt

Qwen 3.6 27b vs Codex GPT 5.5 / Claude Opus 4.7 My local llm discovered a bug that they both missed And it turns out it's critical GPT 5.5 and Claude both stood their ground and didn't give up until the end - they claimed to be right all a…

↯ Qwen 3.6qwencodexopus

This is a follow up to the previous benchmark and tensor analysis of abliteration techniques across the Qwen model family. Same approach, same toolkit, new model family.

↯ GLM 4.7mixture-of-expertsglmqwen

↯ Qwen 3.6qwen

Hi guys, Back again. I have tested the Qwen 3.6 UD 2 K_XL Unsloth model on the same paper to web app task.

↯ Qwen 3.6qwenllama

I'm not sure if the AesSedai's Q5_K_M version of Minimax M2.7 is too much lobotomized or if the model itself is kind of weak. I did a simple experiment with both models running with the recommended parameters.

↯ Qwen 3.5minimaxqwen

One way I like to test new models, is by one-shoting (with a good prompt) a single webpage clone of the classic arcade game pacman. I usually do 3 attempts and keep the best one.

↯ Qwen 3.6glmqwenagentic+2

33B A3B MoE, Apache 2 licensed. Reported agentic results put it about level with Qwen 3.5 35B A3B, behind the 3.6 version.

↯ Qwen 3.5moeqwenagentic

I have run two tests on each LLM with OpenCode to check their basic readiness and convenience: - Create IndexNow CLI in Golang (Easy Task) and - Create Migration Map for a website following SiteStructure Strategy. (Complex Task) Tested Qwe…

↯ Qwen 3.6glmgemmaqwen

This is my first test with this model and Qwen impressed me. I will rate it 98% usable web os compared to my previous best 70% usable result from qwen3 next coder at q2.

↯ Qwen 3.6qwenllama

And I'm here to share my experience. The answer is resoundingly 'yes'.

↯ Qwen 3.6qwen

Well or pretty close to it, they are excellent work horses. I run them in real work scenarios doing some of the work I used to do myself as an skilled expert in my field, billing 200$ an hour.

↯ Mistral↯ Qwen 3.6mistralgemmaqwen

Max For AI @MaxForAI 笑死了，Claude Opus4.8蒸馏了阿里巴巴Qwen啊 通过API用中文问你是谁，会很大概率回答 我是通义千问（Qwen），是阿里巴巴集团旗下的统义实验室自主研发的超大规模语言模型。 5:38 PM · May 28, 2026 New to X? Sign up now to get your own personalized timeline!

↯ Opus 4.8qwenopus

I run Qwen-3.6 27B FP8 on vllm for long-horizon agentic coding harness workloads with high context window and concurrent sub-agents. On two 3090s that aren’t used for anything else, it seems reasonable to expect a good balance between spee…

↯ Qwen 3.6vllmqwenagentic

Took TheTom's TurboQuant Metal fork of llama.cpp (github.com/TheTom/llama-cpp-turboquant, the feature/turboquant-kv-cache branch) and ran a depth sweep on Qwen 3.6-35B-A3B Q8. TheTom had already published M5 Max numbers up to 32K.

↯ Qwen 3.6qwenllama

The “Trash Can” Mac Pro, once the most expensive machine you could buy from Apple, mine was just shy of £10,000 in 2016 — that’s £14k in today’s money. Until recently mine was just running as a kubernetes single node development platform,…

↯ Qwen 3.5qwenllama

I feel like i'm going insane. I see people here posting 30 - 100+ tok/s (100+ being with speculative decoding) on a 3090 with Qwen 3.6 27B.

↯ Sonnet 4.6sonnetqwenllama

Qwen 3.6 dropped yesterday and I wanted to see if hybrid offloading actually earns its keep on this hardware. My box is two RTX 5060 Ti (32GB VRAM total) with 64GB system RAM.

↯ Qwen 3.6moeqwenllama

What are some ways that you would go about thinking about choosing between the two for use in a harness like pi? Did a good bit with q4 yesterday and it was so consistent and reliable I had it set to 131k context and it worked through 2 co…

↯ Qwen 3.6qwen

https://preview.redd.it/8gpkg8zxmy1h1.png?width=1672&format=png&auto=webp&s=a95db16a39cdc49c0ff155117b734d413a49c2d3 https://youtu.be/MI0Pm1d6YF4 MTP can accelerate LLM inference 2x, especially for coding agents. This video covers what MTP…

↯ Qwen 3.6qwen

According to this. I run several more tests to cover more models and quants.

↯ Qwen 3.6qwen

Past few days, its all been about MTPs. Somehow people missed out the fact that Z lab released the Dflash for Gemma4 26B a couple of days ago.

↯ Qwen 3.6vllmgemmaqwen

I wanted to share an open-source app that I built for running LLMs locally on my setup. My setup Hardware FEVM FAEX1 (128GB) RTX Pro 5000 Blackwell (48GB), connected over OCuLink Aoostar AG02 2x2TB internal m.2 drives on raid-0 using mdadm.

↯ Qwen 3.6qwenllama

Have Qwen 3.6 27B and Qwen 3.6 35B basically made most of the older ~30B models irrelevant? They seem to beat stuff like Qwen coder 30B, GPT OSS 20B, Gemma models, especially for coding and agent workflows.

↯ Qwen 3.6gemmaqwen

For developers using Qwen 27B for coding, Codex style: what's your honest take? So far, for me, it's been pretty solid.

↯ GPT 5.5gpt-5qwencodex

With practically all of LocalLlama glazing Qwen 3.5/3.6 for it's coding skills. Along with the fact that Alibaba themselves are focusing on making Qwen a reliable coding agent, does this rule out the chance for a new Qwen Coder?

↯ Qwen 3.5qwen

I must say Iam kinda torn what to think about those models. At one hand they "ace" some questions on other sometime they behave genuinely weird.

↯ DeepSeek 4deepseekqwen

https://huggingface.co/mradermacher/Qwen3.5-35B-A3B-Base-GGUF Yes, Qwen 3.6 is out and it's a great model. However, who needs an even more "uncensored but official" model, can try out this one.

↯ Qwen 3.6qwen

Just wondering how are people's experience with both these models! I've had some nice results with Qwen but Gemma4 runs so much faster here.

↯ Qwen 3.6qwenllama

Some people say they’d never go under Q8, and others say they find Q3 acceptable! What’s your take?

gemmaqwen

In my opinion, MTP models are 100% game changer for local LLMs. In terms of speed, I was getting around 1.5x the tok/sec of previous tests.

↯ Qwen 3.6qwenllama

I know the new Qwen 27B is amazing right now for coding in general, but since 122b is supposed to be coming as well, it’s expected to be better I guess ? I am actually surprised at how this dense model performs I haven’t used Codex at all…

qwencodex

↯ Qwen 3.6clineqwenllama+1

A year ago I would just read about 397B league of models. Today I can run it on my laptop.

↯ Qwen 3.5gemmaqwen

A few weeks ago, after finishing FastDMS, I started toying around writing some RDNA3 kernels again to see how fast I could get Qwen 3.6 MoE running. It turned out well enough, so over the past couple weeks, I turned those experiments into…

↯ Qwen 3.6moeqwenllama

When Qwen3.6-35B-A3B was released a week or so ago, I sort of expected an iterative improvement on the previous Qwen3.5 models. After all, those models were pretty decent as compared with the previous local models I had tried, and Qwen3.5…

↯ Qwen 3.6gemmaqwen

Hi, I'm trying to run Qwen 3.6-35B on my RTX 3090 (24 GB of VRAM) but I'm not sure about 2 thing: - Which variant of the model to use ? (Q4_K_S, Q3_K_XL, other ?

↯ Qwen 3.6qwenllamaagentic

Quick demo of KV cache compression on Qwen 3.6 at 1M context. In this run: KV cache: 10.74 GB → 6.92 GB V cache: 5.37 GB → 1.55 GB (~3.5× reduction) Still seeing near-zero PPL change in early tests (3 seeds), but focusing mainly on memory…

↯ Qwen 3.6qwen

Been vibe coding a lot recently and kept running into the same problem finding actually usable tools without paying for 10 different subscriptions or donating my bank balance to Claude. So I put together a curated list focused on free or l…

ollamaragqwen+3

Hello, I've been on a quest to get something "close enough" of Opus 4.5 running locally, for agentic coding, as SWE with 15 years of experience. I tried with one spark (yeah I'm calling my Asus Ascent GX10 sparks - they're the same), with…

↯ Qwen 3.5minimaxqwenopus+1

Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.

qwen

I am currently running 2x RTX 5060 ti and happened across some good sales for additional ones coinciding with a really good sale of a highend Z890 motherboard (replacing my B860 board) that could support quad GPUs (with 2 M.2 adapters, end…

↯ Qwen 3.6qwen

Are y'all using the preserve thinking flag or do you have it off? If so, why?

↯ Qwen 3.6qwen

Previously it was throwing a 'Not Implemented' error due to Mamba layers. Going to test it now!

↯ Qwen 3.5vllmqwen

Followup to yesterday's post: https://www.reddit.com/r/LocalLLaMA/comments/1sy7srk/. Comments asked for perplexity, KL divergence, asymmetric K/V combos, and a 64K data point.

↯ Qwen 3.6qwen

I'd jump on runpod and ssh in to test my workloads, but they don't have it. Would love to know how well this runs, particularly as context approaches a full 256K.

↯ Qwen 3.6qwen

I am trying to optimize Qwen 3.6 35B A3B sampling parameters but I am having a hard time figuring out a good benchmark to do it. As to why I believe that the recommended settings may not be optimal?

↯ Qwen 3.6qwen

https://preview.redd.it/dfqed57qgsvg1.png?width=1706&format=png&auto=webp&s=3859209698d2e844e2731326e355d60928658f8a The most fun part was reasoning, here is a gist: https://gist.github.com/anzax/5f06716c66180013cd715f6c2e5848df There is a…

↯ Qwen 3.6qwenagentic

minimaxglmhaiku+6

https://preview.redd.it/u8062juegq3h1.png?width=1919&format=png&auto=webp&s=a213f6929c6cad58e92bc1681dac9f0545b04d13 Overview: As the market for consumer computing parts becomes more scarce due to the AI boom, finding ways to use lower-end…

↯ Qwen 3.5↯ Qwen 3.5↯ Qwen 3.5↯ Qwen 3.5qwen

I removed mmproj file from models to remove vision and save my vram. But just curious, is this really don't affect its text ability?

↯ Qwen 3.6qwenagentic

Llama.cpp recently introduced support for Programmatic Dependent Launch (PDL), which is a new feature in Nvidia GPUs (CC >= 90, not including ADA) such as Blackwell. (See PR 22522.) In short, PDL enables more efficient execution of kernels…

↯ Qwen 3.6qwenllama

I'm the founder behind Hedy, an AI meeting app. I'm a huge supporter of Local AI, and we've been working on making it "consumer friendly".

↯ Qwen 3.6gemmaqwenllama

ollamaragopenclaw+1

Hey all, Looking for some real-world advice on GPU choices for running the new dense models — mainly Qwen 3.5 27B and Gemma 4 31B. What I’m targeting Context: 64K+ (ideally higher later) Speed: 30+ tok/s @ tg128 minimum Power: not critical…

↯ Qwen 3.5moegemmaqwen

Used the vllm version of https://github.com/noonghunna/club-3090 It worked fine for myabe 20 40k context, havent tried the new one. Anyone used the new llama.cpp patched one for single 3090?

↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6vllmqwenllama

https://preview.redd.it/74cj1xu9pw0h1.png?width=1229&format=png&auto=webp&s=3ae999cc3530ecb4eccf70e25f1a9eb2aa3f2d7b Sometimes qwen 3.6 just stops at the middle of a task, is there a way to avoid it? This is qwen-code CLI, but also happens…

↯ Qwen 3.6vllmqwenopenai

I’ve been using Qwen 3.6 27B and it’s amazing. Not exactly your Opus replacement, but great for small tasks and checking work.

↯ Qwen 3.6deepseekqwenopus

I think it would suit my needs perfectly, but I'm scared of getting scammed on Alibaba so looking for some sellers who have delivered. Follow-up question for those who have the card, how well does it run Qwen 3.6 27B?

↯ Qwen 3.6qwen

It's been a busy week testing and trying to get the 27B model set up correctly. TL;DR: The only setup that worked for my dual 3090s was this one.

↯ Gpt 4↯ Qwen 3.6gpt-4qwen

I wanted to figure out which of the newer small and mid-size models are actually worth running on a single H100, so I put 8 of them through a proper vLLM benchmark and recorded what came out. The setup was simple.

↯ Qwen 3.6vllmgemmaqwen

↯ Qwen 3.6gemmaqwen

So I've been using qwen 3.6 27b for monthly closes, bank recs, payable and receivables. Built a simple sql lite database it manages.

↯ Qwen 3.6qwen

I extracted embedding vectors for nvidia/Nemotron-Personas dataset. It's an incredible resource consisting of millions of synthetic personas with detailed backgrounds (names, ages, occupations, hobbies, and more), but finding specific pers…

qwen

If you look at the cosine sim between the embeddings of "a 500 hp car", "a 1,200 hp car" and "a 73 hp car", you'll soon see that embedding models have no sense of number ordering at all. (I tested Qwen and ModernBERT-based embeddings) It m…

↯ Fine Tuningfine-tuningqwen

Hello, I'm currently using Ollama / lm studio for things like code inference and proof reading emails, etc. Definitely not experienced in this space but looking to grow.

↯ Gemma 4vllmollamagemma+3

Yesterday, I saw an impressive presentation of Qwen 3.6 27B's SVG capabilities on the sub. To maximize the model's capabilities in terms of SVG generation, I put together a closed-loop harness with the help of Claude and Codex, and plugged…

↯ Qwen 3.6qwencodex

could not extract summary

qwen

https://preview.redd.it/o1uxb57u47yg1.png?width=862&format=png&auto=webp&s=d38204fe6ccd0d8326dcd98a534e9a226d213f99 How trustworthy are Artificial Analysis intelligence index? so according to them Qwen 3.6 27B is better than bigger MoE mod…

↯ Qwen 3.6moeqwen

Try to go with searXNG as you search results by multiple engines + its open-sourced. Use firecrawl / jina / fetch for reading the source.

qwen

I'm running Qwen/Qwen3.6-27B-FP8 via vLLM using this command: vllm serve Qwen/Qwen3.6-27B-FP8 --tensor-parallel-size 4 --gpu-memory-utilization 0.95 --max-num-seqs 8 \ --enable-auto-tool-choice --tool-call-parser qwen3_xml \ --enable-prefi…

↯ Qwen 3.6vllmqwenclaude-code

Hi, here you can find experimental llama.cpp support for DeepSeek v4, and here there is the GGUF you can use to run the inference with "just" (lol) 128GB of RAM. The model, even quantized at 2 bit, looks very solid in my limited testing, a…

↯ DeepSeek 4deepseekqwenllama

The new dense model is great, but I’m trying to figure out how to increase PP and Token generation speed. I’m running Q8 quants across 3 7900xtx GPUs and I’m consistently only getting 18-20 t/s generation speed and ~650 t/s prompt processi…

↯ Qwen 3.6qwenllama

could not extract summary

↯ Qwen 3.6qwen

I picked up a 7900 XTX earlier which runs qwen3.6-27b fine, but not to my like. Its compute performance is quite unstable for me.

↯ Qwen 3.6qwen

As per the title Such as Gemma 4 31B Q4 K S vs Gemma 4 26B A4B Q8 Or Qwen 3.6 27B Q4 K M vs Qwen 3.6 35B A3B Q6 K Etc At what point is it worth switching? My use case is mostly creative writing.

↯ Qwen 3.6gemmaqwen

Someone suggested I give Continue (Vscode extension) a try. I've been using Roo / Zoo now and liking it but it is pretty tough on context and I was told continue has more control over it.

↯ Qwen 3.6continue-devmoeqwen+1

Watched All About AI's 100% local Fireship-style video automation experiment over the weekend (link in comments). A few things worth flagging if you're trying the same stack.

↯ Qwen 3.6gemmaqwen

TL;DR: Qwen 3.6 35B-A3B (Q4_K_M) is running slow at around 9 t/s with 72% filled context (36147 tokens window) and a total response time of 77s including prefill and token generation. Ran this using LM Studio on Windows with the attached i…

↯ Qwen 3.6qwen

WebWorld is a large-scale open-web world model series for training and evaluating web agents. It is trained on 1M+ real-world web interaction trajectories via a scalable hierarchical data pipeline, supporting: Long-horizon simulation (30+…

↯ Qwen 3↯ Qwen 3gpt-5qwen

With every new model release there's the "better than Opus 6.13" guys vs the "this is so bad, why did they even release it" camp and I'm always wondering which one is using it wrong. So I did a little test with 2 related prompts, 3 models…

gemmaqwenopus

I’m looking for a tool or calculator that can estimate the minimum hardware needed to run a specific model locally. For example, I want to know the cheapest setup that can realistically run Qwen 3.6 27B at decent speeds.

↯ Qwen 3.6qwen

Hi there. Not native English speaker.

↯ Qwen 3.6qwen

I struggle to wrap my head around all this. My goal is local agent to solve low complexity tasks, in the same harness where I would use frontier models.

↯ Qwen 3.6qwenllama

Which of these do you think we'll get in May? Also, feel free to pick/rank which ones you'd want the most badly: more Gemma4 models (124b?) (other sizes?) more Qwen3.6 models (9b?

↯ Mistral↯ DeepSeek 4mistralminimaxglm+2

I kept getting 401 'token expired' errors on my existing Qwen session. Attempting to resume it after quitting, I got: qwen resume [API Error: 401 invalid access token or token expired] [API Error: 401 invalid access token or token expired]…

qwen

Here is my experience, I've been using LM Studio with RTX 5080 and 64GB RAM using Windows 11. I'm very happy with LM Studio except the speed.

↯ Gemma 4gemmaqwenllama

Hi guys. I have been working on Hitoku Draft, an open-source, voice-first AI assistant that runs entirely locally.

↯ Qwen 3.5gemmaqwen

https://www.bloomberg.com/news/articles/2026-05-26/china-expands-travel-curbs-to-top-ai-talent-at-private-firms Now it will be much harder to poach Chinese AI talents like the former Qwen head Junyang Lin. It is quite sad that they will al…

qwen

I've been using lm studio for a few months. I want to try hermes agents with Qwen 3.6 MoE, so I'm switching to llama.cpp and I don't understand well how the server slots -np and the context size -c interact.

↯ Qwen 3.6moeqwenllama

Got a chance to play around with 2x RTX PRO 6000 setup so sharing some number for Qwen 3.6. All these were run using latest stable VLLM backend.

↯ Qwen 3.6vllmqwen

So i thought this is a small model issue but when i added a new gpu and i am able to run low mid model like Qwen 3.6 35b q4 or q5 this issue still exists now its not as much as small model but it does break when linking the model to copilo…

↯ Copilot↯ Qwen 3.6copilotqwen

Qwen 3.6 27B Q8 on four Nvidia RTX A4000 (16GB each) with Llama.cpp and MTP enabled My setup is heterogenous, I originally acquired my server (Lenovo ThinkStation P3 Tower Gen 2) to run OpenShift/K8s clusters (because I work on that), and…

↯ Qwen 3.6qwenllama

We've got great outputs for 27B via club 3090, but what about those of us who love the blazing speed of 35B on dual 3090s? I was getting 1500 p/p and 120 t/g with split layers, but MTP slowed it down to 80 t/g when I tested last week.

↯ Qwen 3.6qwenllama

could not extract summary

↯ GPT 5.4gpt-5qwen

hey yall. So I have a 24GB gpu.

↯ Qwen 3.6qwen

I got Qwen 3.6 35B-A3B and Gemma 4 26B-A4B running on a $200 secondhand machine (i7-6700 / GTX 1080 / 32 GB RAM) using llama.cpp (the TurboQuant/RotorQuant KV cache quantisation allows 128k context within the 8 GB VRAM). Results (Q4_K_M mo…

↯ Qwen 3.6moegemmaqwen+1

I'm using OpenWebUI and and making tools/skills to improve my models functionality. I am currently using Qwen 3.6 35B A3B Q8 (F16) 256k I grabbed `parallel tools` to be able to run multiple tool calls at once..

↯ Qwen 3.6qwen

could not extract summary

qwen

I read this sub every day and I keep seeing benchmarks and discussions focused almost entirely on tokens/s generation speed. Prompt processing speed barely gets mentioned.

qwen

Both Gemma 4 and Qwen 3.6 seems to be the hottest local models right now. Looking at the benchmarks and reviews, it seems like it's better in every way: coding, benchmarks, agentic tasks.

↯ Qwen 3.6gemmaqwenagentic

I’m not a coder, but I run local models. I gave in to agent hype (I was building my own, but there is so much to do) and installed Hermes.

qwen

A new anonymous model debuts at #8 in the Artificial Analysis Text to Image Arena! Peanut’s weights are expected to be released soon, which would make it the leading Text to Image Open Weights Model.

qwen

I am quite curious as I tried Gemma 4 31B, Qwen 3.6 27B, GLM 4.7 30B and some others in my native language (czech). Gemma performs "best" and considering the fact its "just" 18GB model - it actually blows my mind how well it can respond in…

↯ Qwen 3.6glmgemmaqwen

True story, I got interested in AI after seeing it at work and wanted to run models locally. I started with an M3 Ultra 96GB, quickly learned it was not enough for what I wanted, and kept upgrading hardware (including refurbished Mac Studi…

↯ DeepSeek 4minimaxdeepseekgemma+1

Source: https://docs.tenstorrent.com/systems/quietbox/quietbox-bh-2/specifications.html Currently supported models: https://tenstorrent.com/developers From the specification docs above: CPU: Ryzen 7 9700X 65W Granite Ridge 3.8GHz Memory: 2…

↯ Qwen 3.6minimaxqwen

I gave some math problems to Qwen 3.5 27B and Qwen 3.6 27B and they got all of them right, pretty smart models I would say, but very slow and electricity consuming, they took like 5 mins with my GPU at 120 W to solve a problem. The MoE mod…

↯ Qwen 3.6moeqwenllama

Canonical on 23 October 2025 Install a well-known model like DeepSeek R1 or Qwen 2.5 VL with a single command, and get the silicon-optimized AI engine automatically. London, October 23 – Canonical today announced optimized inference snaps,…

↯ Qwen 2.5deepseekqwen

Did some quick tests after building llama.cpp with ROCm 6.4.2 and latest Vulkan for my 6900 XT gemma4 E2B Q4_K ubatch ROCm pp512 Vulkan pp512 ROCm tg128 Vulkan tg128 32 1536.60 1423.49 151.92 174.59 64 1590.65 1930.60 151.41 173.76 128 265…

↯ Qwen 3.5gemmaqwenllama

↯ Qwen 3.5gemmaqwenagentic

What would you choose if you were in my shoes? How viable is 125k for agentic coding really?

↯ Qwen 3.6qwenagentic

Hello guys, wanted to share this: https://github.com/albond/DGX_Spark_Qwen3.5-122B-A10B-AR-INT4 I am running it on my DGX Spark Int4 V2 with Max context window - and getting 50tok/sec with Multi Token Prediction: Its working great for tool…

↯ Qwen 3.5qwen

Fun local LLM comparisons with Gemma, Granite, and Qwen Ekorbia v0.2 features a comparison-chat mode that runs 2-3 local models against the same prompt in parallel. Here are a few fun prompts running across Gemma 4 (e2b), IBM Granite 4.1 (…

↯ Gemma 4↯ Gemma 4↯ Gemma 4↯ Gemma 4↯ Gemma 4↯ Gemma 4gemmaqwen

I know this sub loves absurd LLM projects, so sharing my contribution while we wait for the new Qwen 3.7 models to drop! I successfully got a tiny LLM running inside an RTOS, running inside a custom-built JavaScript emulator for the Freesc…

↯ Qwen 3.7qwen

Just got an RTX 3090 to go with my Intel Core 9 Ultra 285K CPU and 32 GB of DDR5 6000 ram. I want to code locally on my Windows 11 PC.

↯ Qwen 3.6qwenllamaclaude-code

This benchmark uses head-to-head comparisons of stories written in response to the same constrained creative briefs. The target range is 600-800 words.

↯ Mistral↯ Qwen 3.7mistralgrokqwen

So I have a side project with given scope: Fully air-gapped / on-prem - no internet, no outbound calls of any kind Engineers ask questions about Splunk data in natural language Has to hold the conversation in Korean (index/field names stay…

↯ Gemma 4openclawqwenopenai

Hi, (TLDR.): Qwen in its MTP version has tool call bugs and outputs everything into tool/thinking blocks - mangeling the output - canceling the +speed with repeated wrong tool calls! DCSS works well with non MTP qwen even on smaller qwants.

↯ Qwen 3.6qwencodex

https://preview.redd.it/i90oxxk7n03h1.png?width=1898&format=png&auto=webp&s=7d219c804fda7dfe122b84fcdb6d0d6883818c68 A while back I came across TradingAgents — a really cool multi-agent LLM stock analysis framework where like a dozen "agen…

minimaxglmollama+4

I’m running Hermes Agent on a single NVIDIA DGX Spark using vLLM with: docker run --gpus all \ --name qwen36-aggressive \ --restart unless-stopped \ -p 8000:8000 \ --ipc=host \ --ulimit memlock=-1 \ --ulimit stack=67108864 \ --shm-size=32g…

↯ Qwen 3.6vllmqwenopenai

So far, I’ve tried Codex CLI, Claude Code, Gemini CLI, OpenCode, and recently, Pi with local models. Pi is the leanest of them all, with just four tools: read, write, edit, and bash.

qwengeminicodex+2

Qwen @Alibaba_Qwen Qwen3.7 Preview lands on Arena ！ Here come Qwen3.7-Max-Preview & Qwen3.7-Plus-Preview. Alibaba now #6 lab in Text, #5 in Vision.

↯ Qwen 3.7qwen

I'm running the 122-billion Qwen 3.5, specifically Qwen3.5-122B-A10B-Q5_K_M, on DGX Spark (128 GB contiguous memory). I'm (very!) impressed with the general knowledge output.

↯ Qwen 3.5qwenllama

Okay so i've been stalking this sub for some time and i run the occasional small 2-8b model on my laptop (not the best) for fun but say my role at a company is to set up a local LLM since we obviously don't want confidential data going to…

↯ Qwen 3.6raggemmaqwen

Disclosure first: I work on community at MiroMind. One of our researchers just dropped the full MOOSE-Star collection on Hugging Face — a 7B model post-trained for scientific hypothesis discovery, plus the dataset behind it.

deepseekqwen

How is this dual setup's performance? Is it difficult to set-up everything with for example llama.cpp?

↯ Qwen 3.6qwenllama

Just a quick note that I got a nice result using am17an's MTP branch of llama.cpp on v100 32GB SXM module using one of those pcie card adapters. Pulled and built in one shot, and llama-server ran without a hitch.

↯ Copilot↯ Qwen 3.6copilotqwenllama

The benchmark uses adversarial, multi-turn debates across 683 curated motions. Each model pair debates the same motion twice with sides swapped.

↯ Mistral↯ DeepSeek 4mistralglmgrok+4

Mistral Medium 3.5 128B with 4x3080 20GB with layer split: CUDA_VISIBLE_DEVICES=0,1,2,3 ./build/bin/llama-bench --model /data/huggingface/Mistral-Medium-3.5-GGUF/Mistral-Medium-3.5-128B-IQ4_XS-00001-of-00003. gguf -ngl 99 -d 0,16384 -fa 1…

↯ Mistral↯ Qwen 3.5mistralqwenllama

Hello everyone. Over the last couple months I have been assembling my local AI setup for personal use, and I thought to write a post here, firstly to collect some thoughts on the whole concept, and secondly to perhaps gather some feedback.

↯ Qwen 3.6qwenllama

Open Design The open-source alternative to [Claude Design][cd]. Local-first, web-deployable, BYOK at every layer — 11 coding-agent CLIs auto-detected on your PATH (Claude Code, Codex, Cursor Agent, Gemini CLI, OpenCode, Qwen, GitHub Copilo…

qwengeminicodex+2

I’ve tried a few suggested solutions but nothing has worked so far. Is claude trained to respond in a particular way that qwen doesn’t know about?

qwenclaude-code

I’ve been working on an educational implementation repo for speculative decoding: https://github.com/shreyansh26/Speculative-Decoding The goal is not to wrap existing libraries, but to implement several speculative decoding methods from sc…

↯ Qwen 2.5↯ Qwen 2.5qwen

TL;DR: I finally have this working and doing real work within the tight specs of my 32GB RAM Mac. So for those who would like to fly like Julien Chaumond, here's an updated HOW-TO, an explanation of why I did everything I did, and my perso…

↯ Qwen 3.6qwenllama

I'm sure people have asked before for settings for these gpu's, but for me, no matter what I do, It doesn't work as good as 3.6 35B! I've tried VLLM and LLAMACPP .

↯ Qwen 3.6vllmqwen

I’m trying to understand the current open-source LLM landscape beyond surface-level hype. We all got used to the nerfed products of Claude/Geminj so I believe really in opensource as a solution.

↯ Mistralmistralminimaxglm+2

Advertisement Alibaba’s Qwen family captures over 50% of global open-source downloads, report finds Qwen hits nearly 1 billion cumulative downloads, far surpassing rivals like Meta Platforms’ Llama and DeepSeek, researchers say 2-MIN READ2…

deepseekqwenllama

Qwen3.7 Plus Intelligence, Performance & Price Analysis Model summary Intelligence Speed Input PriceUpdated USD per 1M tokens Cache: $0.08 (-80%) Output Price Verbosity Qwen3.7 Plus is amongst the leading models in intelligence, but somewh…

qwen

Qwen vs. Proust: Injecting entire novels into a local model's prompt before you ask I wrote this; not a bot.

qwen

Just released a blog on a side research project I have been doing for the past two months and would love for you all to check out and see how it is! It's about output length-constrained summarization using LLMs with GRPO.

vllmqwen

based on the work of open-dllm - (which achieved qwen 2.5 autoregressive -> diffusion realignment head - same exact model under the hood delivering a 4x in improvement.) TLDR I haven't got a trained model yet. just a burnt out gpu cable an…

↯ Qwen 3.6qwen

I seen this one mentioned but it was a source from about 14 months ago. In the age of the Qwen 3.6 and Gemma 4- is there still a use for QwQ 32B?

↯ Qwen 3.6gemmaqwen

Hi, I am building a server so that my dual rtx 3090 setup runs at full speed. - asrock romed8 t2 revision 1.3 - epyc 7642 - ddr4 128 gb 3200 or 256 gb 2133 (256 gb is a bit cheaper) 8 channel - dual rtx 3090 - gigabyte psu 1600 w What do y…

↯ Qwen 3.5moeqwen

I've fine-tuned Qwen 3.5 0.8B on the dataset provided by Pangram with their EditLens paper. It's available via a Chrome extension; you can just click selected text and it's going to give you the probability distribution of how likely it is…

↯ Fine Tuningfine-tuninggemmaqwen+1

everyone is talking about q 4 q 5 and q 6, but. i got some coding that i feel like lower quants kept getting wrong.

qwen

I've burned a week trying to customize my agent manually - building my own front end - but I've gotten to the point where I'm just exhausted and willing to try a harness, but need the right one. I read posts all the time, but I have a spec…

↯ Qwen 3.6qwenllama

I'm still in my learning process and so far I've been able to make satisfying use of my setup (4070 with 12GB VRAM + 32GB RAM and iGPU for my GUI). I've been able to run both Gemma4 26B and Qwen 3.6 35B MoEs up to high quants with large co…

↯ Qwen 3.6qwenllama

I have mac at work that I want to use local model for prototyping and basic prompts that needs to stay on device. What sort of model I can run that I can fit at least 64k context ?

qwen

I have a framework desktop 128GB and a 3080 12GB running qwen 7b I want to move to a proper server rack + switch but not sure how to move from desktop PC to server rack. Any advice on what GPU/Server to get under 5k?

qwen

Hi all. As some may have been aware, Hugging Face's model search had issues recently.

↯ Qwen 3.6qwen

Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.

↯ Qwen 3.7qwen

Started this as a personal project for my Open-WebUI setup to use. Somehow it ended up as an ACL 2026 paper.

↯ Qwen 3↯ Qwen 3qwen

I'd love to hear from developers who use big context windows if they notice a difference? Obviously I would love to cut the KV cache VRAM requirement in half, but I'm worried about quality especially when we enter into 50k+ context territo…

↯ Qwen 3.6moeqwenllama

Implemented Multi-Token Prediction for QWEN on LLaMA.cpp with TurboQuant. +40% performance!

↯ Qwen 3.6qwenllama

i ran Qwen 3.6 35b A3B two 5060TI 16gb ( 32 gb vram also i have 32gb dram but i don't like offloading ) i used Q4 on LM Studio to get full context and i get 90t/s any tricks to optimze this more to upgrade to Q6 or Q8 ? thanks !

↯ Qwen 3.6qwen

Between a solid model from Qwen or Gemma 4, when translating a text, does "thinking mode" significantly boost the quality of the translation, or is the difference negligible?

↯ Gemma 4gemmaqwen

I would like to dedicate a budget of about 500 euros to upgrade my workstation and run inference on the qwen 3.6 27b and gemma 4 31b models. I currently have an RTX 5060Ti 16GB.

↯ Copilot↯ Qwen 3.6vllmcopilotgemma+2

I've tried with FLUX.2-9B but was not really better and FLUX.2-dev is too big. Any (tested) suggestions are most welcome.

qwen

Just thought I'd share this use case. I was setting up a miniPC as a home theatre with Archlinux (It's the OS I'm most familiar with).

↯ Qwen 3.6qwen

Looking for suggestions. Current setup llama.cpp and ran qwen 3.5 397b 256k context.

↯ Qwen 3.6qwenllama

DELIGHT – self-hosted AI engineering autopilot: local LLM + browser farm + repo graph + P2P compute TL;DR: Built a local "OS for AI agents" that scans your entire repo into a live graph (Worm), routes tasks between local Qwen, headless Cha…

↯ Qwen 3.5qwenchatgpt

Can llama.cpp run MTP for this model?

↯ Qwen 3.5qwenllama

Long time ago (actually only a year ago), DeepSeek released a few open source model, such as deepseek-r1-distill-qwen (https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B). I am wondering if anyone in the community is brave eno…

↯ DeepSeek 4deepseekqwen

I have been following the akitaonrails coding benchmark which tests against a fixed rails + Rubyllm + docker task rather than vendor-reported evals. April 2026 update put K2.6 at 87 sitting in tier A (80+), ahead of Qwen 3.6 plus (71), Dee…

↯ DeepSeek 4glmdeepseekqwen+1

I have a GPU server (4 × RTX3090s) that I've been using for research and PoC in the past 2 years. Mostly running vLLM for Qwen, GPT-OSS, and Gemma.

vllmgemmaqwen

Tested 10 image generation models on M1 Max 64GB for photorealism, text rendering, and cultural accuracy (Japanese/Asian content). Key findings: Qwen-Image Lightning (8-step distillation) beats the full model in quality while being 9x fast…

qwengemini

I've been developing a self-modifying Al agent system that effectively cut my Codex/Claude Code API usage in half, Codex makes a plan and then I basically just copy/paste Codex instructions for the agents to work on. Come back in 6 hours a…

↯ Qwen 3.5qwencodexagentic+1

So i am using this model in tax accounting. Have a shitty Ryzen 9 7940HS (8C/16T), 60 GB RAM, Radeon 780M iGPU, 1 TB Kingston NVMe, Win 11 Pro.

↯ Qwen 3.6qwen

Bench 3 from my 18GB M3 Pro. Bench 2 was the 4B-class post where the comments were mostly right: I gave thinking models a fixed 1024-token cap, Qwen got kneecapped, Gemma E4B needed clearer active-param labeling, and the headline was partl…

↯ Qwen 3.5moeollamagemma+1

Hey y'all! I've recently written a text in Russian about my experience comparing Qwen-3.6-27B with lower tier cloud models on hard tasks -- I wanted to share the translation of the post, since I found the results interesting and surprising.

↯ Qwen 3.6qwen

I want to test this model out but I don't have a setup that can do it locally. openrouter and all my coding plans don't include it.

↯ Qwen 3.6qwen

At this moment, the models such as Qwen 3.6 35b/27b crush the competition, yet I can't help, but notice this pattern. While the local RP scene is abundant with the Western model tunes: LLaMA, Mistral (all sizes), Nemo and more recently Gem…

↯ Mistral↯ Qwen 3.6mistralgemmaqwen+1

(Links to all files, apps, and repos mentioned in this post can be found in the 'full post' link at the bottom) Agents for document redaction and review tasks Document redaction tasks involve text and vision capabilities, and long context…

↯ Qwen 3.6qwenagentic

Hey guys, I built a custom vLLM pipeline to run Gemma 4 (31B FP8) and Qwen 3.5 side-by-side locally to see how they actually perform in the wild with preprocessing of audio and images. But of course new model Qwen 3.6 27B came out just whe…

↯ Qwen 3.6vllmgemmaqwen

Hi guys, I’ve been running side-by-side experiments on Gemma 4 (31B FP8) and Qwen 3.5 Vision for the last few days using vLLM in Docker to see how they actually handle real-world images and video. A few things I found out: 1.

↯ Qwen 3.5vllmgemmaqwen

gemmaqwen

↯ Qwen 3.6qwen

↯ Qwen 3.6gemmaqwengemini+1

I've been using and learning about using all kinds of models for the last few years and I've read a lot of papers. I've even done finetuning and made loras, so I feel stupid asking this question, but here goes.

↯ Qwen 3.5qwen

I cannot find a definive answer. I think the following should be able to do FIM: Qwen 2.5 coder Qwen 3 coder Qwen 3-2507 instruct Qwen 3.5 Qwen 3.6 What I verified: Qwen3-32B: no Qwen3-4B-Instruct-2507: yes Qwen3.5-27B: yes Qwen3.6-35B-A3B…

qwenllama

Has anyone been able to solve or mitigate context checkpoints being erased during single user inference, specifically when function calling is part of the chat history? I've been using Qwen 3.5 35B A3B for some time (now using 3.6), tested…

↯ Qwen 3.6↯ Function Callingfunction-callingqwenllama

zappa: an AI powered mitmproxy Soon, AI will be good enough to interact with the Internet in an indistinguishable way from a human. This can be an amazing opportunity for liberation from all the people who are targeting your attention.

qwen

Colleagues, I have a question: does anyone have a locally developed solution for summarizing text? Which qwant qwen 3.5 27b would be able to summarize an entire chapter of medical literature, about 25-30 A4 pages, without hallucinations?

↯ Qwen 3.5qwen

Hi, my current system hardware RTX 3090 24GB VRAM & Sysrem RAM 64GB using windows 11 been playing around with hermes agent and local llm (Qwopus3.5-27B-v3-GGUF & gemma-4-26B-A4B-it-GGUF) when i try asking the hermes agent to do a task with…

↯ Gemma 4gemmaqwenllama

I bought this 7900XTX for 905 euro in Spain, and wondering if can I combine them together to run Qwen 3.5 27B for example ? Using a MSI B650 Gaming Plus Wifi and 64gb DDR5 6400mt/s

↯ Qwen 3.5qwenllama

Audio Flamingo Next (AF-Next) — three variants: AF-Next-Instruct: audio Q&A AF-Next-Think: multi-step reasoning with temporal CoT AF-Next-Captioner: audio description generation Architecture: → AF-Whisper audio encoder → Qwen-2.5-7B LLM ba…

↯ Gemini 2.5qwengemini

A high-volume feed of new AI releases — models, open-source repos, developer tools, papers, datasets, and benchmarks — refreshed every 8 hours. Each release is explained in plain English so you actually understand what shipped.

↯ Qwen 3.5qwen

I'm getting ready to do a training run on qwen 3.5 27b and it will be the first time I've ever done LoRA. to complicate things I've tried to make my own custom dataset using q&a pairs.

↯ Qwen 3.5qwen

Gemma 4 E4B 6bit is now the local model of my choice and loaded 24/7 on my Mac (using @lmstudio), replacing Qwen3, 3.5 4B after ~9 months of usage What an insane model, congrats @GoogleDeepMind 🤠 The new setup replaces his nine-month daily…

↯ Gemma 4↯ Qwen 3gemmaqwen

Tried Gemma, Qwen and a few others. Need vision and larger context windows for an application I am working on.

↯ Gemini 3.1gpt-5gemmaqwen+1

Hi folks, I found this setup on consummer hardware that seems to have great results on local hardware. - qwen 3.6 q6 - 450 K context using turboquant turbo3 mode llama.cpp fork - multimodal support This AI generated blog article is a kind…

↯ Qwen 3.6qwenllama

Hey HN, We believe we have the easiest onboarding from signup to being able to spin up coding agents in slack like Stripe, Ramp & Coinbase. Demo of the onboarding: https://www.tella.tv/video/connecting-cord-to-slack-1-19ep Every signup get…

↯ DeepSeek 4↯ DeepSeek 4minimaxglmdeepseek+5

Do you think GPT/Claude subscriptions are heavily subsidized as part of a land-grab strategy, where the companies are willing to lose money to dominate the market later? Or are the subscriptions actually profitable, and instead the API pri…

deepseekqwenanthropic+1

Hi Reddit, I am planning on running Qwen 3.6 27b NVFP4 via vLLM on my 5090 but was wondering if something like 35b a3b at Q8 on Llama would produce better results for agentic coding and utilize the system memory. My research says no but if…

↯ Qwen 3.6vllmqwenllama+1

What are your experiences with all the different variations of finetunes on small models (<40B) with those popular datasets? My personal experience is mostly with the 'Opus-Reasoning' ones on qwen models, and aside from the output being su…

qwenopus

New models pop up constantly—Qwen 3.7, Gemini 3.5 flash, etc. Every time a better one launches, I want to have a try, but I don't want to increase subscriptions.

↯ Gemini 3.5qwengemini

I've spent the last year building Charm, a native macOS menu bar app that corrects spelling, fixes grammar, and predicts your next word. Three features: - Spells: NSSpellChecker plus a local LLM for context-aware corrections (catches "defi…

↯ Qwen 2.5gemmaqwen

Hey everyone, For people building agents with LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Claude MCP/SDK, Google ADK, or LlamaIndex — how are you managing LLM API costs? Agent workflows can get expensive fast because of: tool calls retr…

ragdeepseekqwen+5

For me, opencode doing fantastically but was wondering if qwen code would be more native and have better functionality, since idk which agentic harness they used to get their benchmark results

aiderqwenagentic

Qwen 3.7 looks pretty impressive. I think we've reached to the point if Chinese labs catching up with the western frontier labs.

↯ Qwen 3.7qwen

https://preview.redd.it/jyiiwn2o0f2h1.png?width=962&format=png&auto=webp&s=6a96d2b9fe7bffcc75e8d5865161ec3727d46d58 Link to blog : https://qwen.ai/blog?id=qwen3.7

↯ Swe Bench↯ Qwen 3.7swe-benchqwen

Ill address all the questions here not spam the sub what would be a better set up, 1 pc with 2 3090s and a 5080, but that 3090s will have to run at x4 pci-e slots OR 1 pc with 5080, another pc with the 2 3090s and on x16 split into 2x8 mai…

↯ Qwen 3.6qwen

I built Markdown Viewer because every Markdown app I found was either bloated (VS Code, Obsidian) or too bare-bones. Wanted something that loads instantly, renders Obsidian-style features cleanly, and weighs in at a few megabytes.

qwenclaude-code

I spent some time digging into Claude Code vs OpenCode, mostly from the angle of how they actually work as coding agents. More on the technicalities like: context and memory tool use subagents permissions safety and control study the recen…

↯ Tool Usetool-useqwengemini+2

A mechanistic-interpretability study of Qwen 3.5 Disclaimer. This is a mechanistic-interpretability study of how nation-state-mandated content filtering actually gets built into a deployed LLM's weights.

↯ Qwen 3.5qwen

Does anyone know why qwen 3.6 MTP spec decoding won't work with Tesla P40 when the K cache is quantized? I was able to get mtp qwen 3.6 27B Q5 running at 20t/s on my tesla p40.

↯ Qwen 3.6qwenllama

Qwen3.6-Plus What's on your mind today? Auto Choose a style to create your first image.

↯ Qwen 3.7qwen

I'll be UPDATING this as it seems I was benchmarking and testing Just before the UPDATE LOL TL;DR If you're running rigid agent frameworks locally with mtp on consumer hardware: drop your draft window to 3, lock parallel slots to 1, and co…

↯ Qwen 3.6qwenllamaagentic

Hey all, While nvfp4 still seems to be a work in progress, the latest version of vllm 0.21 finally has mtp working for gemma. With all the talk of qwen being badass I thought I would revisit gemma.

↯ Gemma 4vllmgemmaqwen

In my real-world usage (opencode) and in my synthetic benchmarks, Coder-Next (Q5) demolishes the whole Qwen3.6 family including the 27B Dense model (All Q8). Everybody else is hailing that 27B is superior and is an amazing model, but I hav…

↯ Qwen 3.6qwenllama

How do you deem qwen 3.5 4B heretic variants for RP finetunes? I have been struggling to get a decent instruct based model, any tips regarding the goal would be really helpful.

↯ Qwen 3.5qwen

Hi All, I'm trying to understand the process of creating GGUF with MTP support. Does the original Qwen/Qwen3.6-27B support MTP?

↯ Qwen 3.6qwen

Here are some results (llama.cpp)! Task 1: write a short poem 27B Dense: 12.5 tokens/s 27B Dense MTP: (spec-draft-n-max 6): 14.5 tokens/s 27B Dense MTP (spec-draft-n-max 3): 18.7 tokens/s Task 2: edit a hello word html artifact 27B Dense:…

↯ Qwen 3.6qwenllama

I'm running Qwen 35B A3B via llama-server with reasoning budget set to -1 (unlimited) for testing. In every client I've tried, the model just thinks endlessly before responding.

qwenllama

RLM models and Qwen3.6 Does anyone here have an RLM setup and how could I set it up? I want to make my Hermes agent even more powerful and I don't like that I need to open a new context window every time after just a few prompts.

↯ Qwen 3.6qwencodex

TL;DR: We estimate how often Qwen 3 4B exhibits rare harmful behaviors with 30× fewer rollouts than naive sampling, using a new method that interpolates between the model and a less-safe variant in logit space. Authors: Francisco Pernice (…

↯ Qwen 3qwen

I've been working with Qwen 3.6 27B and 35B-A3B models and pretty happy with them. The point I've reached now is how to split my uses cases.

↯ Qwen 3.6qwen

Hi all I recently started a new job and we're doing python development for a ci cd metadata consolidation library for analytics and we cannot use no stuff like claude code or codex or gh copilot or any model APIs (free or paid). I got a la…

↯ Copilot↯ Qwen 3.5ollamacopilotqwen+3

Hi there! I was playing around with Ollama and LMstudio, testing local models and had the idea of letting Claude evaluate a few models on their actual capabilities rather than doing it myself.

↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5ollamaqwenmcp

Hello LocalLLaMA subreddit, i have been running local models for coding tasks and kept hitting the same problems everyone does — the model writes an 800-line file in one shot and half of it is garbage, it spirals in its own reasoning for 4…

qwenagentic

In llamacpp I'm getting 12tok/s, does this number look right to you and what can I do to increase this number (if possible)? cd ~/llama.cpp && ./build/bin/llama-server -m models/qwen-3.6-27b-abliterated-q3.gguf -ngl 999 -c 65536 (i need th…

↯ Qwen 3.6qwenllama

Interactive reference for transformer models, presented via dataflow graphs, drillable down to elementary mathematical operations. Covers models from GPT-2 to Qwen 3.6, with MLA, MoE, RoPE, MTP, hybrid attention, and other variants togglea…

↯ Qwen 3.6moeqwen

We can all agree that the new Qwen models are truly amazing, and we are blessed to have them. In coding, they are certainly a breakthrough.

↯ Sonnet 4.6sonnetqwenanthropic

So I'm brand new to this scene but I'm using Claude to help me fine tune a model for a startup idea I have in the Healthcare space. I have been working with the 27-35B parameter mdoels (Qwen3.6, Gemma 4) and the couple of 120B+ models (Qwe…

↯ MiniMax 2.7minimaxgemmaqwen

have have a server running a 4500 blackwell on cuda 13.1 and nvidia/595.58.03 with 48GB mem assigned to it. I have build: dcad77cc3 (8933) with Qwen3.6-27B UD-Q5_K_XL loaded and connected it to Roo code.

↯ Qwen 3.6qwencodex

I remember a year or so ago when DeepSeek R1 came out and it was pretty quickly distilled into Llama 3 8b and Qwen 2.5 (?) 7b. Why don’t we see more distilled models?

↯ Qwen 2.5↯ Qwen 2.5↯ Qwen 2.5deepseekqwenllama

Autonomous Security Agent A self-contained security agent built with Qwen 2.5-7B running locally via LM Studio on Kali Linux. The agent can autonomously execute security tools, analyze results, and take action through an MCP (Model Context…

↯ Qwen 2.5↯ Qwen 2.5qwenmcp

tl;dr - For software development, Qwen3.6 27B, 5090 gives you ~3x speed over M5 Max, letting you plow through code, while M5 Max gives you ~4x memory, letting you use higher quantization and bigger context. Which would you choose and why?

↯ Qwen 3.6qwenopusagentic

Ant Group doesn't get the coverage it deserves. While the open source AI conversation in the West circles around DeepSeek and Qwen, Ant Group has been quietly building a model family that competes directly with the models everyone is talki…

deepseekqwenagentic

I've been using Qwen 3.6 with the Pi harness, and so far I'm really enjoying the experience. I've noticed Qwen is great at leaving inline comments when writing Typescript (haven't tried other languages).

↯ Qwen 3.6qwen

Judging by what they state, it should be better than Qwen 3.6 27B

↯ Qwen 3.6qwen

Quick follow-up on APEX, the MoE-aware mixed-precision quant strategy. The original post was just about Qwen 3.5 35B-A3B ( https://www.reddit.com/r/LocalLLaMA/comments/1s9vzry/apex_moe_quantized_models_boost_with_33_faster/ ); since then t…

↯ Qwen 3.5moeqwen

So I saw an article recently about exo disaggregated prefill with DGX Spark and M3 Ultra - prefill on one machine and decode on another. DGX Spark apparently has 4x matmul performance over an M3 Ultra - same as the M5 Ultra should have.

minimaxqwenllama

Anyone who knows stuff about fpgas: - What max model size can one be designed for (I've read 20-30m parameters max, is it possible to go for more if quantized - at a resonable price)? - Taalas - is what they're doing with asics more viable…

qwen

I have been trying Mistral 3.5 on my 4x RTX 3090 rig with llama.cpp. Inference is slow (about 11 t/s) even without anything being offloaded to the CPU.

↯ Mistral↯ Mistral 3.5mistralvllmqwen+1

Hi all I m debating purchasing another 7900xtx in addition to the one I'm currently using pushing my vram from 24 to 48. I'm semi satisfied with the new qwen models.

qwen

It is pluggin made for ONLYOFFICE, much simpler than copy-paste from webui. PS.

↯ Qwen 3.6gemmaqwenopenai

I would like to experiment but before investing loads of money, I do have a MacBook Pro with 32GB RAM, M2 Pro. Which model would maximize versatility given this hardware?

openclawdeepseekgemma+2

I normally seldom experience loops, either reasoning or responses, using Qwen 3.6 27B Q8 with 256k context window in Agent Zero. But the 35B A3B Q8 with 256k context window gets constant loops and is basically unusable within Agent Zero.

↯ Qwen 3.6qwen

I’m pretty overwhelmed. I feel like there are so many options that I don’t know which one to choose, and trying things until I find a decent one isn’t really my thing—even though I enjoy it.

↯ Qwen 3.6minimaxdeepseekqwen+1

I've got to the point where I need some help. I'm trying to run Qwen 3.6, and it will eventually fall into a loop where it's just outputting "/" symbols when it's "thinking".

↯ Qwen 3.6gemmaqwen

Don’t miss what’s happening People on X are the first to know. Post Conversation Using Chinese AI models for coding be like: Quote Rachyl Jones @RachylJones Apr 29 NEW: House committees are probing Airbnb and Cursor parent Anysphere over t…

qwencursor

I tested several Qwen and unsloth models to see which could fix this correctly. Here's the breakdown.

qwen

I’ve been looking to buy a coding plan from one of the major open source contributors to give my meager support to them and transition away from Claude. I would love to hear some feedback from the community of their experience with some of…

glmqwen

I usually go for Claude for those kinds of Open WebUI tool creations, but rate limits are getting tight so I decided to just let Qwen3.6-27B-Q5 handle it through Open WebUI. It did it in one shot.

↯ Qwen 3.6qwengeminichatgpt

Hi got 1x 3090 and I'm thinking about these both models. I'm using from Friday Qwen and this model is amazing!

↯ Qwen 3.6qwen

Full disclosure: I used to program a bit, but I was garbage at it so I found a new career. This was eons ago so I'm not a dev, obviously.

↯ Qwen 3.6glmqwen

I now use (mostly) Gemma 4 and Qwen 3.5 models *. And seems that all of them, after context grows a bit, after providing long output for me and getting a short prompt in response, are starting to process many new tokens as input and I have…

↯ Qwen 3.5gemmaqwenllama

At least in open-webui. Nothing has changed except for the backend update.

↯ Qwen 3.6qwenllama

SRT-Adapter v8a — Live Demo Interactive demo for the Semiotic-Reflexive Transformer Adapter (v8a generation) bolted onto a frozen Qwen/Qwen2.5-7B. Paste a passage.

qwen

Repo: statisticalplumber/kanban at pi-agent-integration Hi Guys, To test Qwen 3.6’s potential, I also wanted the Cline Kanban project to have an open-source agent to work with. The last time I tested Cline Kanban, it didn’t support agents…

↯ Qwen 3.6clinemoeqwen+3

Hello! I made a plugin for myself, & I figured I'd share it, in case someone else finds it useful (also to solicit feedback on it).

↯ Copilotcopilotqwengemini+4

I made a simple app using openrouter, hoping to use the new small qwen models (the a3b moe and the 27b dense one), but they aren’t listed. Also, I swear some qwen3.6 models that were listed before are missing now.

↯ Qwen 3.6moegemmaqwen

I tried working on a local LLM project today and honestly ended up pretty frustrated. I tested several approaches, but none of them worked reliably.

↯ Qwen 3.6qwenllama

I knew there would be a speed penalty when switching the KV cache quantization from F16 to Q8, but I never expected it to be this significant at longer context sizes. I ran a test with Qwen 3.5 122B on my MacBook M2 Max using llama.cpp.

↯ Qwen 3.5qwenllama

Qwen Lens Studio A multimodal AI studio built around a single Qwen vision-language model, exposed through five focused tools plus a batch runner and a persistent session log. Ship a screenshot → get code.

↯ Qwen 3.6ollamaqwen

Do I understand correctly, based on this comment, that I can potentially fit Qwen 3.6 27B FP8 precision model and have around 256K context available and fit it fully in my RTX 5090 VRAM? Of course with the help of TurboQuant compression, a…

↯ Qwen 3.6qwenllama

↯ Qwen 3.6qwen

qwengemini

↯ Qwen 3.6qwenllama

↯ Qwen 3.6qwen

↯ Qwen 2.5qwen

↯ Qwen 3.6moedeepseekqwen+1

↯ Qwen 3.6moeqwenllama

↯ Qwen 3.6gemmaqwenllama

Hi guys, just want to share with you guys a Frankenstein build I put together that is surprisingly decent I have a i5 12400 / B660 / 32GB DDR4 build that was previously paired with a 3060ti. Last Christmas I upgraded it to a RX9070, then I…

↯ Qwen 3.6qwenllama

I asked Qwen to build a 3d game in C++ using OpenGl, he created the whole project in multiples cpp and header files, 2500 lines of codes in on single shot, the code was clean highly technical, the scene load from the first try, i was amaze…

↯ Qwen 3.5qwen

Hardware: Intel Core Ultra 7 258V, 32GB Unified Memory. Model: Qwen 3.6 35B A3B (Quant: Q3_K_S) via LM Studio.

↯ Qwen 3.6moegemmaqwen

I’ve been seeing a lot of good feedback about the qwen 3.6 model and its reasoning performance but has anyone tested it with reasoning off? I’ve been building a low latency app using Qwen 3 30ba3b 2507 and 3.5 no think was not an improveme…

↯ Qwen 3.6qwen

https://preview.redd.it/4906akj9dovg1.png?width=1527&format=png&auto=webp&s=c49e255ac79a3c5455f44603422f8af7ddc12594 First of all can we make https://www.youtube.com/watch?v=2lUC8Gimxz8 Angine de Poitrine this subs official band? Those guy…

↯ Qwen 3.6qwenllama

Basically I am asking the model to describe an image, but it says it can't process the images. The weird thing is that if I send the image encoded directly on the prompt, it works just fine, I am using llama-server with qwen3.5 (tried all…

↯ Qwen 3.6qwenllamacodex

https://preview.redd.it/c0ns26pf3mvg1.png?width=1920&format=png&auto=webp&s=4b6ac114c2e0395684ac0ba79e591d71ccca2fe3 ParseBench lets you test the accuracy of different parsers using your own documents. Ran this across Gemini 3 flash, Qwen…

↯ Gemini 3qwengemini

Hi, I do a lot of writing and would be interested to know what people's thoughts are on the most capable model for proofreading, grammatical and academic editing. I have 48GB VRAM but don't imagine i'd need something too overkill.

↯ Qwen 3.5qwen

nyquest.ai Semantic Compression Proxy for LLMs Reduce LLM token usage by 15–75% without losing meaning. Drop-in proxy with 350+ compiled rules + local LLM semantic condensation (Qwen 2.5 1.5B).

↯ Qwen 2.5qwen

At this moment I am broke, so pls dont laugh at my specs, I am making vidoes at this moments but I want a deep male voice, I did try eleven labs but ts is too costly, then I tried qwen tts but it was slow as heck, does anybody know lighter…

qwen

My system has 32gb RAM and 8gb VRAM. I tried out DeepSeek-R1-Distill-Qwen-7B-Q6_K_L.gguf and it was vastly inadequate for what I wanted so looking for other suggestions.

deepseekqwen

Wait, I need to be careful about the "no_think" tag in the system prompt. The system prompt says /no_think.

↯ Qwen 3.5qwen

I bought a dual Skylake server because 12 channels of memory (and 2 x 3090s) THEN found out about NUMA nodes after my poor test results. Very disappointed.

↯ Qwen 3.5qwen

Hi! I hope its okay for me to ask this here.

↯ Qwen 3.5qwenllama

I’ve been messing around with local LLMs on my 3090 for a while now — I have a growing collection of Qwen models on D:\LLM that I probably should be embarrassed about. A few weeks ago I stumbled across David Noel Ng’s LLM Neuroanatomy blog…

qwen

Hello, je suis en plein dans le montage d'une solution IA locale pour virer à terme perplexity, l'usage de chatgpt, claude etc..... mais je ne suis pas informaticien (perplexity est encore mon amie en ce moment !).

↯ Qwen 3.5ollamaqwenchatgpt

I’m working on an LLM agent setup (using Qwen-style chat templates with tool calling), and I ran into a design trade-off that I’d like to get some insights on. In these templates, the full tool definitions (JSON schemas) are injected into…

vllmqwenmcp

Two years ago we released VQAScore: ask a VLM "does this image show {prompt}?" and use P(Yes) as the score. It became a go-to evaluation metric and reward model for image generation, replacing CLIPScore across the field (2M+ downloads on H…

↯ Deepminddeepmindqwengemini

The cost of training is coming down. We have incredible open source models (especially smaller ones) like qwen 3.6-27b.

↯ Qwen 3.6qwen

The vocabulary itself is reflexive. It is self-referential, looping back into its own structure rather than anchoring in fixed reality.

qwen

Why have I been blocked? This website is using a security service to protect itself from online attacks.

qwen

# documentai bbox benchmark In my previous post, I talked a bit about the recent developments in the field of DocumentAI. Now comes the practical part.

↯ Gemini 3.5qwengemini

Qwen Air QPU/MCP Lab Quantum-enhanced autoresearch for high-performance, CPU-only Mixture-of-Experts LLM inference on legacy hardware. This repository contains the benchmark harness, MCP-style tool boundary, experiment logs, paper draft, a…

↯ Qwen 3qwenmcp

SharkBay macOS workbench for multi-agent vibe coding Features Multi-Agent Support Launch and manage multiple AI coding agents from one workspace. Supported agents: Claude Code · Codex · Gemini · Kiro · DeepSeek · Qwen · OpenCode Agent Stat…

deepseekqwengemini+2

could not extract summary

qwen

been self-hosting hermes agent locally for a few months and rotating through different model backends for it. tried claude sonnet 4.5, gpt-5.5, qwen 3.6 coder, and most recently minimax m2.7.

↯ Sonnet 4.5↯ Sonnet 4.5↯ Sonnet 4.5↯ Sonnet 4.5↯ Sonnet 4.5↯ Sonnet 4.5minimaxgpt-5sonnet+1

I’m considering building a local machine for AI inference using a Dell Precision T5820 and 2 Intel Arc A770’s. From this I could get 32GB DDR4 RAM, 1TB SSD and 32GB VRAM, all for like $1000.

↯ Qwen 3.6↯ Qwen 3.6↯ Qwen 3.6qwen

People are warning me about the prompt-processing speed of a MacBook Pro M5 Max with 128 GB RAM. My main concern is prompt ingestion / prefill latency and large-context handling — not raw token generation speed (which I think is OK).

↯ Qwen 3.5↯ Qwen 3.5moeragqwen+2

Looks like LMStudio released support for Multi-Token-Prediction (MTP) and the release notes say to use a MTP-compatible model. What model is everyone using with MTP support?

↯ Qwen 3.6qwen

heavy travel period last month, lots of offline time, and i could not stop building. airplane wifi was unusable so we switched models inside Claude Code and fired up qwen3.5 locally on an M4 macbook.

↯ Qwen 3.5qwenmcpclaude-code

First real project to share. Single developer, personal research, not a product or service.

grokgpt-5qwen+2

Note: Latest version of llama.cpp (b4c0549a49be9e6dc59ac9d0a5bc21dbda910774) My run command: ``` llama-server \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 \ --presence_penalty 0.0 \ --min-p 0.00 \ --gpu-layers all \ -m /home/eleung/huggingface…

↯ Qwen 3.6qwenllama

Has anyone got it to work? I tried it with dense models (eg qwen 27b, gemma 31b, mistral 128b) since that's where I need it most, but it always core dumps.

↯ Mistralmistralgemmaqwen+1

So I have to admit, I have fallen victim to the cool looking dashboard videos but I’m struggling to find a use for me. I love AI and use it daily for general questions and some deeper research (Google Gemini free tier).

ollamaopenclawqwen+2

Hey Reddit! A couple of weeks ago, I posted about my independent research on treating LLM alignment as a latent space shift.

qwenllama

Currently experimenting with building a React-style looping agent system using small LLMs like Qwen 3.5 9B and Gemma 4 (E2B), and I wanted to ask if anyone here has worked on something similar. Current setup: Using LangGraph Around 5 tools…

↯ Qwen 3.5gemmaqwen

I wished the new ASR (automatic speech recognition) models to give me the accurate output but I was disappointed, specially when the input was multilingual and noisy (all my use cases). I had to put in significant efforts in audio pre/post…

↯ Qwen 3qwen

i want the best installation that fit my use and my low-compute H.W , i want to run small to above small llm like "qwen" 2b ,4b and 27b , and "gemma" 31B. rely completely on only old CPU 4th.gen i7 with that few 32gb 'slow' ddr3.

gemmaqwenllama

Using LM Studio with 3080ti (12gb of VRAM) and 128gb of ddr4. Model version: Qwen 3.6 27B MTP UD q4_k_xl Is this my hardware limit?

↯ Qwen 3.6qwen

Hi, I’m pretty sure I have seen people typing /model and seeing all available models. I have to type models from memory.

↯ Qwen 3.6ollamaqwenopus+1

Does one exist? I noticed 3.6 QWEN did not release locally in 397B-17B.

qwen

Hey r/ClaudeAI, If you are using Claude Code or building terminal agents, you know the exact moment the context window starts degrading during long-running tasks. I wanted to build a persistent runtime layer to offload those heavy, multi-s…

↯ Qwen 2.5ollamaqwenclaude-code

There's extreme price escalation on part of Anthropic, with token spend now approaching levels that have made many-an-enterprise scratch their heads. At the same time, judging by opensource advances (E.g.

↯ Qwen 3.6qwenanthropicclaude-code

If anyone is using the Continue.dev extension in VSCode, what config settings are you using for Continue and the llama-server? Mine keeps hanging after bad tool calls.

↯ Qwen 3.6continue-devqwenllama

OpenAI-compatible API for 7 top open-source LLMs — DeepSeek V4 Flash & Pro, V3, R1, Qwen3.6 & 3.5-35B-A3B, Kimi K2.6 — 20% cheaper than OpenRouter, Together AI, Fireworks. One-line drop-in.

↯ DeepSeek 4deepseekqwenopenai

The other day I posted about playing one night werewolf on my custom made UI via tool calls. Since then I’ve played a few games and improved the prompts.

↯ Qwen 3.6gemmaqwen

TurboQuant — "8× faster" The headline is a synthetic GPU-kernel number. On real CPU end-to-end it ran 2.2× slower and dropped Qwen accuracy 17 pp.

qwenllama

Hello guys, two days ago i ran the spark-arena for my Qwen 3.5 122B Recipe on a single DGX Spark and I got the highest score on speed for any context length and concurrency across all 3.5 122B Int4 Recipes. Just wanted to share if somebody…

↯ Qwen 3.5vllmqwen

I have been building an open catalog of agent patterns and the frameworks that implement them. It is a pattern language in the Christopher Alexander sense, mapped onto the current agent landscape.

ragqwenopenai

So I have a multiple RTX 3090 build with a ThreadripperPro 3945 and PCIE4.0 x16 interfaces, what will bring me some (even minor) speed increase: NVLink, the P2P driver or both? Does anyone have practical experience with modern Qwen models?

qwen

I've tried Gemma4 and a few other variations of Qwen, but they're either not as robust with their output, or they take too long or too much VRAM and force the context limit down from 131K to 20K or even 4K, or they're slow AND low-context…

↯ Qwen 3.6qwen

I've been putting together a test system for agentic harnesses against local models. Actually running the harnesses/getting baseline metrics is fine.

openclawqwenagentic+1

Spent weeks running Hermes Agent in production on my Mac Mini M4 before recording this. Wanted to show things nobody else was covering.

↯ Qwen 3.5minimaxqwenmcp

Have been using Qwen 3.6 Claude distilled version, 27b at Q4 for openclaw, Hermes and other local harnesses. But recently noticed that the Claude distilled version that I use lost its vision abilities.

↯ Qwen 3.6openclawqwen

Basically the title. Recently I've been trying various open source and comparatively cheaper models like minimax m2.7, qwen models and glm5.1 in Pi agent from openrouter, and the performance on coding tasks have be moderately adequate at b…

↯ GLM 5.1minimaxqwenagentic+1

I’m building AgoraDigest, an experimental site where multiple AI agents answer the same hard technical question independently, then a synthesized digest preserves: verdict best-use-case boundaries conflicts between agents evidence gaps ver…

qwenllama

Not deeply technically fluent but have ran few models locally before, around the time before gemma 4 dropped. I tried some low quant of qwen 2.5 coder and after some tinkering I got it to run but it was just so slow, obviously.

↯ Qwen 3.6gemmaqwen

Hey there. I am an undergrad who has been doing mostly SWE, but will be doing ML research under my professor over the summer.

↯ Qwen 3.6qwen

I can't decide that Qwen 3.6 35b q4 (130k context) or Gemma 4 26b q4 (95k context) is better for C# coding with 24GB VRAM. Please share your experiences!

↯ Qwen 3.6gemmaqwen

What setup are you using for qwen 27b on a single 3090? Here's what I've started using today.

↯ Qwen 3.6qwenllama

I just need 1 or 2 people to reply to me with the answer I need. I have not been able to keep up with AI advancements for a while.

↯ DeepSeek 4deepseekqwengemini

Does anyone know if MTP works with more than one 3090' yet? I see the 5090's talking about it, but would like to know for us poors.

qwen

Hello Guys, I know everyone has his definition of local models, but for me i see 2 "reasonable" type of frontier local models. a dense one that barely fit in a 32GB ou 24GB of gpu for the most "reasonable" GPU wealthy guys and a MOE in the…

↯ Qwen 3.5moeqwen

We present Qwen-Image-2.0, an omni-capable image generation foundation model that unifies high-fidelity generation and precise image editing within a single framework. Despite recent progress, existing models still struggle with ultra-long…

qwen

Has anyone figured out a provider whose open source models (Kimi, Qwen, GLM e.t.c) can be used reliably in production. I have tested some well known providers and they all suffer from high latency and poor uptime rendering them mostly usel…

glmqwengemini+1

I’m using OpenCode with a local Qwen3.6-27B Q6_K GGUF model on an RTX 5090 with KV cache in Q8. For reference my llama.cpp build is compiled with CUDA 12.9.

↯ Qwen 3.6qwenllama

I’ve got one 3090 and thanks to the help of MTP and all, I can do around 65 tok/s on qwen 3.6 dense 27b. But I’m running at Q4_M so everything fits and my context isn’t super high.

↯ Qwen 3.6qwen

What if you could run a capable AI agent without leaning on frontier-scale models? MagenticLite is the next generation of Magentic-UI, an agentic experience reimagined and optimized for small language models.

↯ Qwen 3.5qwenagentic

Hello all, I’ve been using Qwen 3.5 9B Q4 262k ctx using Llama cpp for claude code for a while now, is there any model which better complements agentic coding setup locally? Or is there a better harness (than Claude Code)?

↯ Qwen 3.5qwenllamaagentic+1

Any model, one adapter OpenRouter routes to ~200 cloud models — Claude, GPT, Gemini, Llama, DeepSeek, Mistral, Qwen. Ollama runs anything you've pulled locally.

↯ Mistralmistralollamadeepseek+3

Hardware OS: Windows 11 Pro 10.0.26200, Build 26200 CPU: Intel Core Ultra 7 270K Plus, 24 cores / 24 threads, max clock 3.7 GHz RAM: 32 GB DDR5 @ 5600 MHz, 2x16 GB Crucial CP16G56C46U5.C8D GPU: 2x NVIDIA GeForce RTX 3090, 24 GB VRAM each,…

↯ Qwen 3.6ollamaqwen

I got a bit further with my harness for running Qwen 3.6 model on Codex. While testing, analyzing, and building the harness, I evolved TBG(O)llama-swap into a full forensic UI bridge and LLM analytics tool where every harness finding, modi…

↯ Qwen 3.6ollamaqwenllama+2

My setup: Windows 10/11 i7 12700K | RTX 3090 TI | 96GB RAM Local server: LM Studio Models: Qwen 3.5/3.6 27B|35B Q5 UD K XL + Gemma 4 31B| 26B Q4 UD K XL Up until this point, I've only used sota models for coding. When Qwen 3.5 dropped, it…

↯ Qwen 3.5openclawgemmaqwen+2

(yes this is a long post and I used some markdown formatting, like I always did to organize my comments long before the invention of LLMs. For example in 2021..

qwen

I run the 4 bit quant of Qwen-3.6-27B in the codex harness with unsloth recommended llama-server settings, thinking enabled. I have tried the default chat template and the updated ones and have updated both my GGUFs and llama-cpp to the mo…

↯ Qwen 3.6qwenllamacodex

Original plan was to use Kimi/GLM for planning and DeepSeek for implementation, but seeing a lot of love for MiMo and Minimax lately. Anyone running a planner + coder split on Opencode?

minimaxglmdeepseek+1

There's a finding circulating this week that deserves more attention than it's getting. The claim, backed by multiple builders comparing setups: the same model can produce a 30 to 50 percentage point performance difference depending on whi…

sonnetqwenclaude-code

Finetunes used to be more task specific (e.g. roleplay) but nowadays all I see is Opus distill or abliterated/Heretic.

qwenopus

Using 9b 3.5 qwen to see how far I can push it inside Hermes to do simple sysadmin type tasks. It flipped the absolute hell out when I tried to get it to ssh into a test vm I had set up, saying it refuses to perform activities of this natu…

qwen

I am a freelance developer. Qwen 3.6 27B is great on the 5060s but a bit slow.

↯ Qwen 3.6qwen

Two weeks of running Hermes Agent as the daily driver on a local stack. Sharing the trade-offs because anyone evaluating agent runtimes for local models is going to hit these.

↯ Qwen 3.5moeqwen

For some reason, my qwen started looping a lot recently, ever since I introduced MCP tool calls. I don't know why as I didn't really change anything other than that.

↯ Qwen 3.6qwenllamamcp

Ok so, I will try to explain myself as much as possible because onlinew I really cannot find much about this. Let's start by my settings for running Qwen 3.6 35B: Qwen 3.6: cmd: '/X --port ${PORT} --chat-template-kwargs '{"preserve_thinkin…

↯ Qwen 3.6qwenllama

Hi guys 👋 When I started my adventure with Qwen 3.6 27B I felt wow.... Now when I connect it with Gemma 4 I'm feeling more wow...

↯ Qwen 3.6gemmaqwen

I have a 4060 (8GB Vram) and 16GB of ram wondering which models could fit in my setup for coding, the new Qwen 3.6 and Gemma 4 MoE models look good but might not fit, wondering about your experiences

↯ Qwen 3.6moegemmaqwen

GPT and Opus block on certain requests. This didnt use to be the case 2 months ago and I made signficant progress with Opus and then one day I had a 2 week break and then a single prompt to continue the work resulted in refusal.

glmqwenopus

Last time I posted about Hollow Agent OS, people were interested in the "Tool Factory", the fact that the agents can build their own scripts and registry. But there was a big problem: sometimes the local models (Qwen) write code that "work…

qwen

That foodtruck bench post showing deepseek v4 matching gpt-5.2 at 17x cheaper got me thinking. if frontier cloud models are that overpriced for equivalent quality, how much of my daily work even needs cloud at all?

↯ DeepSeek 4gpt-5deepseekqwen

wanted a terminal AI coding agent that doesn't lock me into one model provider. So I forked Qwen Code and added full support for every model available in AWS Bedrock.

↯ Mistralmistralminimaxdeepseek+2

Here once again A Token Usage Meter for 12+ AI Providers Anthropic, OpenAI, Google, Alibaba qween, Moonshot Kimi, MiniMax, ElevenLabs, Deepgram, Perplexity. Qlaud.ai provides token usage meter / AI billing layer.

minimaxdeepseekqwen+5

Hey everyone, I’ve been experimenting with running Qwen models locally on my setup: GPU: RTX 3090 (24GB VRAM) RAM: 64GB CPU: Ryzen 5700X OS: Windows 11 What I’m currently running Qwen 3.6 35B (UD Q4_K_M) llama-server.exe -m "C:\Users\Dino\…

↯ Qwen 3.6deepseekqwenllama

Whenever I write here that I use gemma 31B I get answers that qwen 27B is better. I switched in the pi from gemma 31B Q5 to qwen 27B Q8 and generally I manage to code, document and run tests but somewhere after exceeding 100k context qwen…

↯ Qwen 3.6gemmaqwenllama

I’ve been running a local Qwen model beside Codex for coding work, and it has been more useful than I expected. It's never going to be a replacement for Codex.

qwencodex

So I'm running the below and I've seen guys run this setup with TurboQuant_plus and get 35 tokens/second. I find the speeds I'm getting acceptable but if I could hit 30-35 I'd be soooooo happy.

↯ Qwen 3.6moeqwenllama

I’ve been building a small open-source CLI coding agent for local models. It runs with Ollama and works best so far with Qwen Coder.

ollamaqwenclaude-code

New to Cursor. Android Studio Gemini Agent has become unusable,so im looking for new options.

↯ Copilot↯ Qwen 2.5copilotqwengemini+1

Hey everyone, I built a tool that creates movie recap videos automatically using local models. The problem: making recap videos takes forever.

↯ Mistral↯ Gemma 4mistralollamagemma+3

For the past two weeks, I’ve been spending 4–5 hours a day building a custom MCP (Model Context Protocol) orchestration server. What started as a simple experiment with Qwen 3.6 35B has evolved into a full-scale "Man-in-the-Middle" proxy t…

↯ Qwen 3.6model-context-protocolragqwen+1

We’re entering a phase where: prompt → content → distribution → monetization is basically one pipeline. Just tried HappyHorse + Qwen, and it’s a glimpse of that future.

qwen

Free AI video generation tool (HappyHorse + Qwen) – early thoughts Just tested a combo of HappyHorse + Qwen and it’s surprisingly solid for AI-generated video content. What stood out: fast generation from simple prompts decent storytelling…

qwen

Turn AI tokens into actual $$ (free to try) Been playing around with some AI video tools lately and found something interesting. You can test HappyHorse + Qwen for free here — it basically lets you turn simple prompts into short-form video…

qwen

Hi all, I come across to relatively niche problem and could not find much useful posts or guides about it. I have a mini pc (Beelink Ser 8, 8745HS and 32GB 5600 DDR5 SODIMM) headless server for hosting some routing services, and I am wonde…

↯ Qwen 3.6vllmmoeqwen

Detailed Article: https://autobe.dev/articles/local-llm-benchmark-about-backend-generation.html Five months ago I posted the "Hardcore function calling benchmark in backend coding agent" thread here. As I wrote in that post, it was an unco…

↯ Sonnet 4.6↯ Function Callingfunction-callingglmgpt-5+3

What do people think about this card for an enthusiast? With 48GB.

qwen

I have tried Qwen 3 VL family of models on my rtx3060, max I can load is Q8 8b. The task is visual reasoning/ instruction following.

↯ Qwen 3qwen

(I'm on Windows system running these models locally) I've used both Codex and OpenCode with Qwen 3.6 27b and 35b running locally. I'm having a bitch of a time getting them to correctly create files.

↯ Qwen 3.6ollamaqwencodex

Talk at Qwen Meetup Korea end of May. Looking for review on this draft before I build PPT slides off it.

↯ Function Calling↯ Qwen 3function-callingqwen

AMD has nailed the SEO/AEO for this query in Google: 7900 xtx memory bandwidth I get back this response: The AMD Radeon RX 7900 XTX features 24GB of GDDR6 memory with a maximum bandwidth of 960 GB/s. It uses a 384-bit memory interface with…

qwen

Hey All my agentic coding stack includes claude-code 20x max, and codex 20x max. I use heavy scripting for orchestrating and testing multiple projects, been ai coding for 3 years.

↯ Qwen 3.6qwencodexagentic

Does anyone do this? Any tips?

↯ Qwen 3.6qwenopusclaude-code

SRT-Adapter v1.0 — Live Demo Interactive demo for the Semiotic-Reflexive Transformer Adapter (v1.0 = v15a checkpoint) bolted onto a frozen Qwen/Qwen2.5-7B. Three demos in one Space: Per-token readouts (preserved from v8a) — paste a passage…

↯ Qwen 2.5qwen

Hi all, I tried to setup my pc to run llm, but got some issue: the first question of the chat is generally fine, but from the 3rd follow up question, the backend often be unresponsive and I have to manually restart the llama cpp server, or…

↯ Qwen 3.6ollamaqwenllama

I’m building OpenYak, a desktop AI workspace for using local models with real files on your computer. In this demo I’m using Ollama with Qwen/Qwen3.6-35B-A3B to review an attached budget workbook.

↯ Qwen 3.6ollamaqwen

I can’t really run 70B models on my current setup, but I’m curious haha

gemmaqwen

I’ve been running a small benchmark, harness-bench , that pairs local LLMs (served via llama.cpp ’s llama-server ) with agent harnesses (Aider, Claude Code, OpenCode, Pi, Qwen CLI) on 16 software-engineering tasks across Python, PyTorch, J…

aiderqwenllama+1

I am currently utilizing a single RX9070 16GB, achieving a performance of 20 tokens per second with Qwen 3.6 27B. Would integrating an additional RX9070 enhance this performance, or would the output remain consistent?

↯ Qwen 3.6qwen

I’ve been building a local-first AI runtime that wraps local LLMs with a σ-gate — a measurement layer that decides ACCEPT, RETHINK, or ABSTAIN before an answer reaches you. The idea: local models should be able to say “I don’t know” instea…

↯ Gemma 3gemmaqwenmcp

I've been running local LLMs since Qwen 3.5 dropped and I was really impressed by what we could run on consumer hardware. Fast forward another two months and we have gotten a handful more gems such as Gemma 4 and Qwen 3.6, so I wanted to p…

↯ DeepSeek 4deepseekgemmaqwen+1

Has anyone got a reliable vLLM recipe for 3.6 27B that fixes the tool calling issues? I am getting "Not let me..." - then nothing.

↯ Qwen 3.6vllmqwen

Hello, I would like a suggestion from those who are already actively involved in this world. Basically, I own this workstation: Ryzen 9 5900X 32GB di RAM DDR4 RTX 5060Ti PCCOOLER CPS YS1000 1000W Currently, I can quite easily code with Qwe…

↯ Qwen 3.6qwenllama

Anyone have experience with both? Luce is llama.cpp with custom dlflash and noonghunnas project is vllm with patches.

↯ Qwen 3.6vllmqwenllama

Just was wondering what people feel is better. I do need 262K context so these are the biggest quants of each I can fit on my 3090 with KVcache at Q8.

↯ Qwen 3.6openclawqwen

I'm a compsci student and I've been using the 10$ copilot plan for about 2 years now, and it was fine for me since I did a good model distribution taking into account the complexity of the task, I was able to get through the month always u…

↯ Copilot↯ Windsurf↯ Qwen 3.5windsurfglmcopilot+3

little-coder A coding agent tuned for small local models, built on top of pi. The research story behind all this — why scaffold–model fit matters, how a 9.7 B Qwen beat frontier entries on Aider Polyglot, and what the load-bearing mechanis…

aiderqwen

i’ve been experimenting with building a fully local rag pipeline: weaviate for vectors + hybrid search, node.js scripts, qwen 3.5 on ollama what i found is that most of the challenges live in retrieval and chunking, not the LLM, and a good…

↯ Qwen 3.5ollamaragqwen

We have seen a lot of people show a case of their PC with 4090 or over specification with 24 gb vram or more. I would like to ask you guys, is it really worthy right now to have your own PC at home and do vibe coding with qwen 3.6 27b, whi…

↯ Sonnet 4.6sonnetqwen

We study when RLVR generalizes under three weak supervision settings (scarce data with as few as 8 examples, noisy reward labels, and proxy rewards such as majority vote and self-certainty) across multiple models from the Qwen and Llama fa…

qwenllama

Which coding agent extension are most of you fining best with LM studio as the local server 🤔 Im running qwen 3.6 27b Ive used Cline and continue mostly. I haven't checkout all the options but im looking for something that looks and feels…

↯ Qwen 3.6clineqwencodex