model roundup
Qwen 3.5
-
Been testing DiffusionGemma 26B A4B for the last few days and the bottleneck profile is completely different from autoregressive models. With autoregressive models you are compute-bound during prefill and memory-bandwidth-bound during deco…
-
How do i prevent llama.cpp from offloading on Swap? (www.reddit.com via reddit)
I have tried preventing this issue by using llama.cpp flags. However, I still have the issue: whenever I'm close to my 96GB of RAM, llama-server / llama.cpp decides to offload the KV cache onto my swap.
-
NVFP4 with llama.cpp - FAQs? (www.reddit.com via reddit)
Lets clarify all things related to NVFP4 in this thread. Sharing few questions & links here.
-
Ask HN: Any Local LLM can I run without GPU for Local Agentic workflow AI? (news.ycombinator.com)
Claude Code like agentic workflow ai too costly for me.Any LLM can I run with VSCode at the below setup? 16ram Intel core i7 h processor 13gen 512gb NVMe SSD I want to run the ai as local agentic workflow with Vscode.I want use LLAMA agent…
-
Hot Take "Rigid code is better than Flexible code if you're on a budget" (www.reddit.com via reddit)
I've spent the last six months trying to build a fully local, agentic pipeline for a text_processing and extraction tool I use daily. Because I’m running everything on a single consumer GPU setup, my choices are limited to smaller, quanti…