Qwen 3.5

1. DiffusionGemma made me rethink what memory bandwidth means for local agent inference (www.reddit.com via reddit)

   2w agentic

   Been testing DiffusionGemma 26B A4B for the last few days and the bottleneck profile is completely different from autoregressive models. With autoregressive models you are compute-bound during prefill and memory-bandwidth-bound during deco…

2. How do i prevent llama.cpp from offloading on Swap? (www.reddit.com via reddit)

   2w qwenllama

   I have tried preventing this issue by using llama.cpp flags. However, I still have the issue: whenever I'm close to my 96GB of RAM, llama-server / llama.cpp decides to offload the KV cache onto my swap.

3. NVFP4 with llama.cpp - FAQs? (www.reddit.com via reddit)

   2w llama

   Lets clarify all things related to NVFP4 in this thread. Sharing few questions & links here.

← all threads