Qwen 3.5

1. DiffusionGemma made me rethink what memory bandwidth means for local agent inference (www.reddit.com via reddit)

   2w agentic

   Been testing DiffusionGemma 26B A4B for the last few days and the bottleneck profile is completely different from autoregressive models. With autoregressive models you are compute-bound during prefill and memory-bandwidth-bound during deco…

2. How do i prevent llama.cpp from offloading on Swap? (www.reddit.com via reddit)

   2w qwenllama

   I have tried preventing this issue by using llama.cpp flags. However, I still have the issue: whenever I'm close to my 96GB of RAM, llama-server / llama.cpp decides to offload the KV cache onto my swap.

3. NVFP4 with llama.cpp - FAQs? (www.reddit.com via reddit)

   2w llama

   Lets clarify all things related to NVFP4 in this thread. Sharing few questions & links here.

4. Ask HN: Any Local LLM can I run without GPU for Local Agentic workflow AI? (news.ycombinator.com)

   +3 2w llamaagenticclaude-code

   Claude Code like agentic workflow ai too costly for me.Any LLM can I run with VSCode at the below setup? 16ram Intel core i7 h processor 13gen 512gb NVMe SSD I want to run the ai as local agentic workflow with Vscode.I want use LLAMA agent…

← all threads