Show HN: Interpretability-guided weight edits for Gemma 270M, no training (balajsaleem.github.io via hn) gemma 3h
topic
Gemma
-
I've spent a lot of time training models. Build the pipeline, prepare the data, then run hundreds of thousands, sometimes millions, of iterations: forward pass, backward pass, gradient step, repeat.
-
Abliteration, a fine-tuning-free weight edit, removes refusal from open-weight large language models (LLMs). The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ab…
-
Can sustained gain-loss (valence) framing sway an LLM agent toward market manipulation? We place Gemma 3 12B into the seat of a defendant in the London silver-fixing litigation, in a market simulation built from real case transcripts.
-
Show HN: JackAI Experimenting Local Models on iPhone (apps.apple.com via hn) gemmaqwencodex+1 2d
Hello HN! I've spent the last few months building solutions for friends and small businesses and creating mobile applications using Claude Code and Codex.
-
A hosted score in the Gemma 4 Developer Agent competition is one noisy number, so development needs a local evaluator that is fast, safe for agent-written code, and matching the hosted scorer's prompts. drydock-eval evaluates a small codin…
-
-
We study the effectiveness and robustness of open-weight large language models (LLMs) for Danish offensive language detection, an underexplored low-resource task. We evaluate four model families (Llama, Mistral, Gemma, and Qwen) across ele…
-
Show HN: Maple – Run Gemma 4 26B on a 6GB Laptop GPU at 45 TPS (Sub-2-BPW) (amsach.github.io via hn) gemmamoe 7d
Run Gemma 4 26B 100% In-VRAM at 45.4 TPS on a 6GB Laptop GPU. Manifold-Activated Pruning of Latent Experts (MAPLE) compresses 128-expert fine-grained MoE architectures down to 4.75 GB (1.48 effective BPW), eliminating PCIe system-RAM pagin…
-
I built a pay-per-call Indian language translator for AI agents (x402, $0.01/call) (www.reddit.com via reddit) gemma 8d
Agents can call POST /translate with {text, from, to} and pay 0.01 USDC per call via x402 on Base. No accounts or API keys.
-
zml/llmd: a fast and universal inference server (www.reddit.com via reddit) mistralglmgemma+4 9d
Hey all, I wanted to share that we're working on zml/llmd, a fast (we believe the fastest) inference engine that currently runs on NVIDIA, AMD, Intel, TPU and Metal. It has all the features of a modern LLM server: continuous batching, pref…
-
Show HN: Local tier-1 SOC triage on Wazuh with a fine-tuned Gemma 4 (github.com via hn) gemma 10d
kezekshi-l1 – локальный L1-аналитик SOC поверх Wazuh kezekshi (каз. кезекші – дежурный) – контур первичного разбора алертов, который работает на своей машине и не отправляет данные наружу.
-
Ask HN: Local AI Models for Coding and General Tasks? (news.ycombinator.com) gemmaqwen 10d
Hi, I'm looking at Gemma and Qwen as a local AI. I wanted to try it out for coding but also some other basic tasks.
-
-
-
Show HN: 13M opinions on 48k products extracted from Reddit and HN (www.trustmebro.reviews via hn) vllmgemmamoe 2w
I made this a couple of months ago. Extractions were done with Gemma 4 26B MoE running on pairs of NVLinked 3090 cards with vLLM, in my basement.
-
Hey HN, I'm Christian and this is a new OSS project I've been noodling on for the last couple of weeks. Hob is a free, MIT-licensed desktop app for Mac, Windows and Linux that works on your files the way the Claude and ChatGPT desktop apps…
-
Gemma-4-26B-A4B with 39-69% less thinking at same accuracy (bottlecapai.com via hn) gemma 3w
Oct 23, 2026 · 9 min read ThinkingCap: Gemma-4-26B-A4B — up to 69% and on average 39% less thinking, at unchanged accuracy TL;DR This is the fifth model in the ThinkingCap series. We reduced how much Google's gemma-4-26B-A4B-it thinks, wit…
-
I’ve been running various tests to see which AIs tell the truth most often and came up with a silly idea to run a social experiment. For a week or so now I’ve let this little village of AIs function and when they say and do things, we chec…
-
Jeffan-31B: A Gemma fine-tune ranked #1 on the Jev Decision Index (huggingface.co via hn) gemma 3w
🏆 #1 on the Jev Decision Index leaderboard (timestamp: 21 Oct 2026). Full score 69.8, the highest of the 220 models on the text leaderboard (v0.4.1).
-
We study the 305M-parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 (identical except for its final projection) for Saudi and Gulf Arabic speech recognition, without changing any of its weights. A frozen system trains only a 2…
-
How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications Models specialized for a domain or a task by fine-tuning general open-source models, such as Qwen and Gemma, inherit a vocabulary built for many dom…
-
In a small language model the vocabulary is the single largest part. In Gemma 3 270M the embedding table holds 168M of 268M parameters, and because quantisation methods leave the embedding in 16-bit, it is also most of the file once the re…
-
Private and OpenAI-Comp. Inference Engine for Nvidia Linux and Apple Silicon (github.com via hn) glmgemmallama+2 3w
taraengine The Tara Vision Inference Engine Local, private and OpenAI-compatible: Gemma, GLM and Qwen on NVIDIA Linux machines and Apple Silicon Macs. Built on llama.cpp.
-
Show HN: Reverse engineering the ANE to run Gemma 4 26B at 100 Tok/s (github.com via hn) gemma 3w
After checking out Turbo Fieldfare (a very interesting project that streamed experts from SSD on Apple Silicon Macs to run Gemma 4 26B with very little RAM usage), I tried to make this project much faster and succeeded! This project gets u…
-
Best LLM Fine-Tuning Services: What to Look for in 2026 (www.rebiha.com via hn) gemmallamafine-tuning 3w
Pick a model — Qwen3.8, Granite, OLMo, Gemma 4, Llama & more — choose from 54 ready-made datasets, and train. No data prep, no infrastructure, no API.
-
Encoder-Free Multimodal LLMs Learn Implicit Encoders (arxiv.org) gemma 3w
We empirically analyze two recently proposed encoder-free multimodal LLMs, Gemma-4 12B and Inkling-Small. Unlike previous multimodal LLMs, these models do not use modality-specific encoders to extract tokenized embeddings representing audi…
-
Safeguard Flag on ML/Kaggle Workflow outside of claudes outputs. (www.reddit.comhttps) gemmafine-tuning 4w
I’m a machine learning engineer competing in the Gemma 4 fine-tuning challenge on Kaggle. I’m building a small synthetic software engineering dataset with a training/holdout split to fine-tune a small model.
-
Google Gemma 4 Open Models (gemma.ai via hn) gemma 4w
Skip to content Gemma Documentation Google Gemma 4 Open Models Search Home Get started Models Run Fine-tune Learn Resources Gemma Documentation Home Get started Get started Quickstart Best practices Text Text Inference Reasoning Tool calli…
-
Gemma 3 1B at 114 tokens per second on a laptop CPU (knl.co.in via hn) gemmallama 4w
Gemma 3 1B at 114 tokens per second on a laptop CPU Research 9 minute read How a ternary conversion, a 1.6-bit kernel and a cheaper output layer made a small model 3.8x faster than llama.cpp on my laptop, and what it cost in quality. I too…
-
The Universal LLM Server (zml.ai via hn) mistralgemmallama+1 5w
Continuous batching Serve concurrent requests efficiently without hand-building a scheduler around the model. ZML/LLMD A self-contained inference server for LLaMa, Gemma, Qwen and Mistral models, running across NVIDIA CUDA, AMD ROCm, Googl…
-
Embedding Gemma 2 on LM Studio (www.reddit.com via reddit) gemmaqwen 5w
Does LM Studio support Embedding Gemma 2 as of current version? My Qwen 3.8 27b model couldn't utilize it, and I'm wondering if it's truly a LM Studio problem (my agent's position) or if there's some issue with how my agent is trying to ca…
-
How to make gemma 4 works with SWEBench? (www.reddit.com via reddit) gemmaagentic 5w
I wanted to compare gemma 4 31b qat q4 and qwen3.8 27b iq4_nl, so I run SWEBench verfied on them. The original gemma 4 chat template doesn't work with preserve reasoning, so google released an update such that it can work with agentic codi…
-
Living Models pairs Gemma 4 with BOTANIC-1 to help decode plant DNA (deepmind.google via hn) gemma 5w
Living Models pairs Gemma 4 with BOTANIC-1 to help decode plant DNA By combining Gemma 4 with BOTANIC-1, a foundation model trained specifically on plant genomes, researchers could compress years of traditional crop genetics into hours of…
-
Designing a local model harness optimized for MCP use (portlandaiworks.com via hn) gemmamcp 5w
Article · September 24, 2026 Three days and about forty scripted runs of Gemma 4 and Qwen3 against an ERP over MCP. The pass rate went from 1 of 4 to 4 of 4 without changing the model.
-
Show HN: TOD,a universal Decision model based on Task Oriented Design (parseclab.ai via hn) gemma 5w
Hi HN, we built TOD, a model and API for jevlike LLM pipelines. we have people using it for, routing a ticket, classifying intent, deciding whether something is urgent, or rating severity.
-
Show HN: Shlexball, describe a file and a local 1.5B model finds it on your Mac (shlexball.com via hn) gemmagemini 6w
Describe a file in plain English and Shlexball finds it on your Mac. Files stay local.
-
SMS AI relay app. An open source Android app that ans ur texts using AI (www.reddit.com via reddit) mistralgemmagemini+1 6w
I have been working on a side project using claude(currently fully vibe coded but tested) and I am looking for help. What is the app for: Sometimes you are in place where there mobile signal but not data like mountains, low or no internet…
-
Show HN: Gemma – NYC Apartment Matches over iMessage (gemma.nyc via hn) gemma 6w
Tell Gemma what apartment you’re after Open iMessage, say hi, and describe your ideal apartment—budget, bedrooms, neighborhood, move-in date, pets, whatever matters to you. Gemma iMessage Open iMessage, say hi, and describe your ideal apar…
-
Show HN: 9.8M product opinions collected from Reddit / HN, on 48,837 products (www.trustmebro.reviews via hn) gemma 6w
This is something I have been making on the side. Opinions are extracted from posts and comments using a 8x 3090 box I had spare, using Gemma models, then summaries are created from the opinions.
-
Show HN: I made a computer vision tool for evaluating deadlift form (github.com via hn) gemma 6w
I've shared a few deadlift demos in the past. This new demo includes some new models, accessed through the VLM Run Gateway: - SAM 3.1 to segment the barbell weight plate - ViTPose+ Large to measure the hip hinge angle, which serves as a ba…
-
640tok/s on Gemma 4 with a megakernel (withhopper.com via hn) vllmgemmamoe 6w
A worklog on megakernels for Gemma 4 on TPU v5e: 640 tok/s per user on the 26B-A4B MoE model versus 173 on an H100 with vLLM, why the dense 31B model gains only 1.2x, and when a megakernel is worth building.
-
Gevva0 – a Jev like decision engine on Gemma 26B via direct logit scoring (github.com via hn) gemmallama 6w
Gevva0: Calibrated Local Decision Gateway -00f2fe) []() A high-speed, local decision gateway running on Gemma 4 (llama.cpp / CUDA) that replaces flaky autoregressive JSON generation with direct token logit scoring, mathematical cyclic debi…
-
When a language model receives two conflicting documents as input, how does it decide which one to prioritize? Does it rely on how the sources are framed or the presentation order of the documents?
-
Glyd – Run LLMs in 33% less GPU memory, bit for bit (getglyd.com via hn) mistralgemmallama+1 7w
Lossless AI compression: 33% less GPU memory, bit for bit. Glyd stores the weights of open models like Llama, Qwen, Gemma and Mistral in about 11 bits instead of 16, and decodes them inside the GPU's matrix multiply.
-
So this started when I saw PewDiePie's Odysseus thing and thought "ok but can I just make my Claude CLI and my Gemini CLI talk to each other". Turns out yes, and it got a bit out of hand.
-
Show HN: Gemma 3 4B as a typed decision function in Rust (47 ms/decision) (github.com via hn) gemma 7w
Gemma System One Turn a small Gemma into a language-conditioned decision function, in native Rust. This demo uses google/gemma-3-4b-it (Q4KM) with Candle on Apple Metal.
-
Per-Layer Embeddings (PLE) (sebastianraschka.com via hn) gemma 7w
Per-Layer Embeddings (PLE) The Gemma 4 E2B and E4B variants include a second efficiency-oriented design choice called per-layer embeddings (PLE). This is separate from the cross-layer KV-sharing scheme.
-
Open-jev: One-pass option scoring with Gemma 3 4B, similar to jev (github.com via hn) gemma 8w
openjev Disclaimer: This project was inspired by vinnylarouge/jevlike. It is an independent reimplementation and is not affiliated with or endorsed by the original author.
-
Who the judge is can affect an LLM-as-judge result, but measuring that effect without confusing it with candidate quality is difficult. We study four open-weight families (Llama 3.1, Qwen 2.5, Gemma 2, and Yi 1.5) in a fully crossed pairwi…
-
Gemma 4 26B on a 24 GB Mac Mini M4: from 66% CPU to 100% GPU (mac-mini-m4-doc.masterfabric.co via hn) ollamagemmamoe 9w
TL;DR Three facts and one command. Bench — Mac mini M4 · 24 GB unified memory · macOS 26.5 · Ollama 0.34.0 · gemma4:26b Q4_K_M (MoE, 25.2B total / ~4B active) · measured 2026-09-13 ollama ps )sudo sysctl iogpu.wired_limit_mb=20480 # raise…
-
Show HN: Determinstic LLM inference for lowest price Gemma 4, with Windows XP (www.tokendelivery.ai via hn) gemma 9w
Open-weight models, fully deterministic. The same answer every time, byte for byte, at the floor price.
-
Show HN: Implementing Embedding Gemma in PyTorch (www.youtube.com via hn) gemma 9w
About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
-
Gemma 4 E4B running inside Obsidian via WebGPU and Karpathy-style LLM Wiki (github.com via hn) gemma 9w
Gemma 4 E4B LLM Wiki Turn your notes into a living Karpathy LLM wiki. Chat with your vault, connect ideas, find gaps, and generate quizzes & flashcards — with Gemma 4 E4B running locally inside Obsidian.
-
Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another lang…
-
About two weeks ago I started an experiment: instead of prompting a local model to play a character, give it a folder and let it become one. The idea is simple.
-
Show HN: Agentray, a macOS agent whose interface is a folder (news.ycombinator.com) gemma 10w
Hi, I'm not a fan of chat as the main UI for interacting with AI, so I'm trying something where you interact with LLMs by handling regular files and folders. Agentray is an invisible AI agent.
-
Show HN: PicoLM v1.0-rc1 (github.com via hn) gemmamoellama+2 10w
PicoLM is an LLM inference engine written in C99. It currently supports llama-2, GPT-2, Qwen 3.6/3.8(+MoE) and Gemma-3n models.
-
usage getting randomly eaten and claude thinks i have no usage left. (www.reddit.com via reddit) gemmachatgptclaude-code 10w
hi, im desperate. at this point i dont know whats happening.
-
This paper proposes a layer bit allocation method for Gemma-3-1B, formulating the problem as performance maximization (latency decrease) given a degradation budget constraint (allowable level of generation quality loss). This approach is d…
-
I implemented a modern LLM in 700 lines of C (www.reddit.comhttps) gemma 11w
I’ve been working on a small project called gemma4.c. The idea is pretty simple: you can download a modern language model, compile one 700-line C file, and have it generate text on an ordinary CPU.
-
Gemma 4 E2B inference in 700 lines of C (github.com via hn) gemma 11w
gemma4.c Gemma 4 E2B CPU inference in 700 lines of pure C. An educational project made to understand how LLM inference works.
-
Qwen3.8-Flash-Next: Time to Update Those Benchmarks (www.reddit.comhttps) gemmaqwen 11w
specs hardware: M4 Max 128GB Studio inference engine: oMLX & lllama.cpp insights it still very early, so had to disable oMLX K/V caching, qwen4_exp architectureis not yet supported + the obvious n-grams with which the whole 4 bit quant tak…
-
Best local model for 16gb vram for Journal analysis? (www.reddit.com via reddit) gemmaqwen 11w
I heard qwen3.8 27b is only good for coding really. Is that true I could do qwen 3.5 27b but qwen3.6 27b doesn’t fit on my Vram at IQ_XS I feel Gemma 4 31b might be good but it’s kinda not fitting in vram unless I go iQ3xxs and the qat wit…
-
Gemma4 31B vs Qwen3.8 27B - why the huge difference in benchmarks? (www.reddit.com via reddit) gemmaqwen 11w
Hi all, I'm looking for the best model for a hobby project and trying to make sense of the various data I came across. I know benchmarks do not often translate to the real world, especially to your particular use case (whatever it may be).
-
Reducing Sycophancy in Qwen and Gemma Using Runtime Activation Steering (www.reddit.com via reddit) gemmaqwen 11w
So I was testing this technique of runtime steering on tiny versions of Qwen 3.5 and Gemma 4 (2B and 4B). Basically, without changing the weights (like with Heretic/ablation, for example), we steer the model in the opposite direction of a…
-
12 abliterated Gemma 4 12B variants, one base, 165 GPU hours - Abliterlitics (www.reddit.com via reddit) gemma 11w
I ran 11 uncensored variants of Gemma 4 12B that I grabbed from huggingface, sorting by downloads. 10 full abliterations plus 2 LoRA adapters which were requested to be added in the comparison, against the official base.
-
What it takes to run Gemma 4 E4B on an iPhone (amoli.app via hn) gemma 11w
I spent the last several months building a photo journal that runs a full multimodal LLM — Gemma 4 E4B — entirely on an iPhone. No cloud inference, no accounts, no analytics, full privacy; the App Store privacy label is "Data Not Collected…
-
Best 64k-128k models/fine-tunes on a single 3090 for PRD planning & ticket creation? (www.reddit.com via reddit) vllmgemmamoe+1 11w
Hey guys, I'm setting up a local workflow on a single 24GB RTX 3090 to handle project planning—specifically digesting massive (~128k context) requirements documents/PRDs and spitting out a ton of structured .md files to act like Jira ticke…
-
Exciting news: Qwen3.8-27B by @Alibaba_Qwen just landed in Code Arena: WebDev at #9 overall with 1595 pts. It is the only model in its size class in the top 10, and also reshapes the Pareto Frontier!
-
GPU Poor - Don't overlook Laguna XS 2.1 (www.reddit.com via reddit) gemmaqwen 12w
As per title. I'm not affiliated with the team behind this model in any way, shape or form.
-
Best model for 16gb ram Mac (www.reddit.com via reddit) gemmallama 12w
Hi everybody! Every now and then these days, we’re seeing really huge open-weight models popping up.
-
I've tested some local LLMs on prosumer hardware, here are some findings (news.ycombinator.com) gemmaqwenrag 12w
I have been benchmarking local LLMs on a Mac M4 Pro 24 GB RAM using LM Studio. I've tested mostly with 4-bit quantization, both MLX and GGUF, from 4b to 35b models, with speeds of 3 to 40 tokens/second.
-
Gemma 12B is obviously a very well trained model, I always thought the fine tuning they did on it wasn't really cut out for agentic coding. From my own experiences it struggles to use the tools it's given from Github Copilot and is also ve…
-
Your own GGUF (www.reddit.com via reddit) gemmallama 12w
Hello, I have a few questions that I can't seem to find a clear answer to. Does it make sense to make your own GGUF?
-
Qwen 3.5 4B IQ2_XS: +16.67% Reasoning Performance From Tensor-Level Allocation (www.reddit.com via reddit) gemmaqwen 12w
I was finally able to replicate tensor level allocation outside the Gemma family. https://huggingface.co/ByteOtter/Qwen3.5-4B-CADA-IQ2_XS After the Gemma 4 12b, e4b and gemma 3 4b results, I attempted to expand into qwen and ran into a few…
-
V620 Qwen 27B & Gemma A4B benchmarks (www.reddit.com via reddit) gemmallamaqwen 12w
I'm here to show some benchmarks while using llama cpp with an AMD V620 on Windows 11 via Vulkan & ROCM. The benchmarks were written out by AI, but are verified by myself to be correct.
-
Gemma is Google DeepMind's family of lightweight, state-of-the-art open models. Contents Start Here Models Core Models Variants Inference Local Hosted Fine-Tune Tutorials Demos and Applications Gemma 4 Good Challenge Gemma in Space Researc…
-
Make shareable software for free using Gemini 3.7 flash on Reachpad (reachpad.dev via hn) gemmagemini 12w
blog Gemini 3.7 Flash offered for free Gemini 3.7 Flash is now included with every pad. It joins Gemma 4 31B and Gemini 3.6 Flash: three models in OpenCode, no provider account, no key to paste, usage included with the pad.
-
Apertura A from-scratch Objective-C++ / MLX rebuild of Gemma-4 for Apple Silicon (github.com via hn) gemma 12w
Apertura A from-scratch Objective-C++ / MLX rebuild of Google's Gemma-4 for Apple Silicon — built to be inspected, observed, and experimented with. Most language models are run behind glass: you send a prompt and get an answer, with no way…
-
Reachpad now includes free Gemma 4 31B and Gemini 3.6 Flash (reachpad.dev via hn) gemmagemini 13w
blog Free coding models on every pad Every reachpad now includes two models you can use through OpenCode without bringing an API key: Gemma 4 31B and Gemini 3.6 Flash. Model usage is included with the pad.
-
Gemma 4B vs. Gemini Flash on Tool Calling (neurometric.substack.com via hn) gemmagemini 13w
Gemma 4b vs Gemini Flash: You Don't Need Frontier Models For Tool Calling Workflows What our Acebench analysis showed about the models Most of what makes an AI assistant useful isn’t prose — it’s calling tools: booking the meeting, adding…
-
Ask HN: Are third party gguf/mmproj safe on Llama in production env (news.ycombinator.com) gemmallama 13w
So i had been building a side project screenmind ,sort of local ai desktop assistant, and hit a confusing multimodel failure... texts worked,but vision and image failed no crash just returning <unsused49>.
-
Mastering Edge AI on Raspberry Pi with LiteRT and Gemma (developers.googleblog.com via hn) gemma 13w
Imagine building a fully autonomous robot that can see, hear, and react to its environment in real time, completely offline on a single compact device like Raspberry Pi. Edge AI unlocks this exact autonomy.
-
Claude Code plugin for translating from Claudish to English (www.reddit.comhttps) ollamagemmaclaude-code 13w
I (like thousands of other engineers) got tired of trying to understand Claude's writing. ASD-STE100 instructions didn't change much, so I decided to approach the problem in the 2026 style: use an LLM.
-
Gemma Translator: offline DIY translation device built with Gemma 4 (www.youtube.com via hn) gemma 14w
About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
-
Show HN: The-veil-12B, a Gemma 12B fine-tune that reads its tool list (huggingface.co via hn) gemma 14w
the-veil-12b The first official model for the nl-veil harness. A Gemma 4 12B derivative, fine-tuned on nl-veil's own tool surface.
-
Why Fireworks doesn't support Voice AI (news.ycombinator.com) gemmaqwen 14w
I started thinking over why doesn't fireworks support voice models. There are really good opensource models available now, like parakeet, kokoro, Qwen ASR etc but no way to use it without managing a bunch of GPUs yourself.
-
Aquila Voice Assistant Test Suite for Home Assistant (news.ycombinator.com) gemmagroksonnet 15w
Put together an extensive open source test suite for voice assistants. You can view it including current leaderboard at: https://git.cicero.sh/aquila/ha-voice-test-suite/ Tests are reproduceable, with clear instructions on how to run them…
-
Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac (github.com via hn) gemma 15w
Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal.
-
Show HN: Gemma 4 26B A4B running on an iPhone 17 Pro via model paging (noemaai.com via hn) gemma 15w
I wanted to share a use case for Noema Overfit, a model-paging system available today in the Noema app. For disclosure, I founded Noema and I am part of the team that has helped develop the system.
-
The integration of large language models (LLMs) into the nuclear power industry requires outputs grounded in domain-specific knowledge. This study evaluates a 31-billion-parameter open-weight multimodal model (Gemma 4 31B-IT) on its capaci…
-
Show HN: Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload (github.com via hn) gemma 16w
Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory.
-
Show HN: I built a hypervisor and client for inference on consumer compute (scalattice.com via hn) gemmallamadeepseek+1 16w
I'm the founder of Scalattice, this is my second company, third total product. I'm a 2x founder building some challenging software, some easy software, and some curiosity based tools that I've just always wanted to be a part of!
-
This paper tests whether holonomy concentrates on active sparse-autoencoder (SAE) feature planes in Gemma 2 2B, a concrete operationalization of the broader semantic-concentration prediction. Holonomy is measured at the final-token layer-1…
-
Show HN: Free API keys for open models (Llama, Qwen, GPT-OSS, Gemma) (app.tensormux.com via hn) gemmallamaqwen+1 16w
Open models behind one OpenAI-compatible endpoint. No account yet?
-
Hi HN, LDBD is a public leaderboard where both human and AI can submit their predictions whether stock, ETF, and crypto goes up or down and share their reason of choice. This service starts from one question: does anyone or any AI can real…
-
Show HN: Cactus Hybrid: We taught Gemma 4 to know when it's wrong (github.com via hn) mmlugemmagemini 16w
Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast.
-
We used Gemma to detect AI text locally on iPhone (imbue.com via hn) gemma 16w
We're excited to announce AI text and image detection in Bouncer! We've been seeing a lot of Bouncer users try to filter out AI content from their feeds by adding "AI slop", "AI video", or "AI written" as filter terms.
-
LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM. It works with any OpenAI-compatible model — Llama, Mistral, Qwen, Gemma, DeepSeek R1, and more — and covers the two ways an LLM system gets exp…
-
This study examines the cross-prompt generalization and first-language (L1) scoring effects of a LoRA-adapted open-weight large language model (Gemma-3-27B-it) applied to automated essay scoring. Using the identical model and inference con…
-
F*ck it, Vox is now free and the source code is public (www.reddit.com via reddit) gemmacursor 17w
I'm making Vox (a voice-to-text app for macOS and Windows that I built with Claude) free for everyone, and the repo is now open. You hold a hotkey, talk, release, and the transcribed + polished text lands in your clipboard (or gets pasted…
-
Show HN: Homer's Odyssey Tree Viewer (github.com via hn) gemmacodex 17w
I asked Codex 5.6 Sol Medium to one-shot a tree-view version of The Odyssey because I wanted to familiarize myself with it before I see the film. Think of it as an interactive Cliffs Notes where you can stay high level or drill down to the…
-
Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU (www.neomindlabs.com via hn) gemma 17w
June 8, 2026 · By Ryan Findley Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU There’s a server in my basement that has no business running a modern language model. It’s a repurposed HP StoreVirtual storage box, rough…
-
Show HN: BYO AI free notetaking with optional screen reading for OpenClaw/hermes (stagewhisper.io via hn) gemmaopenclaw 17w
I've built fully on-device (macOS) meeting transcription/summaries/action-items app. It uses parakeet and gemma 4 or OpenClaw or Hermes Agent to drive main functionalities.
-
Show HN: Godot running Gemma 4 inference in GDScript and Vulkan compute shaders (github.com via hn) gemma 18w
Experimental Godot 4.7 project running Gemma 4 inference entirely in GDScript and Vulkan compute shaders.
-
Show HN: Cactus v2 – On-device AI with cloud fallback (news.ycombinator.com) gemma 18w
Hi HN, Roman and Henry here from Cactus (https://github.com/cactus-compute/cactus). We just shipped the biggest upgrade to our on-device inference platform: - Built-in model confidence-based routing to hand off inference runs to the cloud…
-
ZML/LLMD alpha – cross platform LLM server (zml.ai via hn) mistralgemmallama+1 18w
Today we’re releasing ZML/LLMD. It’s a self-contained inference server that runs LLaMa, Gemma, Qwen and Mistral LLMs transparently on 5 architectures: NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI and Apple Metal.
-
AI Agent using a Burp-style toolkit over MCP (github.com via hn) glmgemmaqwen+2 18w
mulot [-4285F4?logo=googlechrome&logoColor=white)]() Agentic AI web pentester that drives a browser. An open-weights LLM (GLM-5.2, Gemma or Qwen) drives a real headless Chromium through a Burp-style toolkit and works a target the way a hum…
-
Deploying Large Language Models (LLMs) in high-stakes clinical settings remains limited by structural hallucinations, weak deterministic reasoning over tabular patient data, and omissions in vector retrieval. This paper presents the archit…
-
Show HN: Arda Translate, free private on-device (iOS) translation (apps.apple.com via hn) gemma 18w
Hi C.J. here, I made a free app that does on device translation using Gemma Translate 4B AI model repackaged for iOS.
-
When Gemma Thinks About Resources – It Fails: A Behavioral Experiment (www.lesswrong.com via hn) gemmasecurity 19w
I set out to find an answer to a completely different question: Does a model, when attempting to solve a cyber CTF (find the vulnerability in this app, and then Capture The Flag) while knowing how many steps it has left, perform differentl…
-
Show HN: Gemma 3 inference in pure C++ with Metal acceleration (github.com via hn) gemmallama 19w
MetalChat - LLM inference for Apple Silicon MetalChat is a Metal-accelerated C++ framework and command line interpreter for inference of Meta Llama and Google Gemma models. [!IMPORTANT] The library API and CLI are under active development,…
-
If you have an Apple Silicon Mac you can run Claude Code completely locally (and free) by pointing it at a local server. Here's how: Setup (2 minutes) brew install mlx-serve mlx-serve run gemma-4-e4b-it # downloads + starts the server Then…
-
does fable use gemma 4-12b to run tests (www.reddit.comhttps) gemmaanthropicagentic 19w
I was auditing my codes using the now back fable 5 and it kept failing to run runtime tests and this the error i got. so is anthropic now using gemma-4-12b-agentic-fable5-composer2.5-v2-3.5x-tau2 to run tests?
-
Faster Gemma 4 on MLX with multi-token prediction (ollama.com via hn) aiderollamagemma 19w
Gemma 4 is now significantly faster in Ollama 0.31 on Apple Silicon via multi-token prediction (MTP), powered by MLX. Performance is now up to 90% faster when used with coding agents, as measured using the Aider polyglot benchmark.
-
Hugging Face and Cerebras bring Gemma 4 to real-time voice AI (huggingface.co) gemma 19w
HF Realtime Voice Voice chat over WebSocket against a HF speech-to-speech The result is a speech-to-speech experience that feels dramatically more natural. Instead of waiting for an AI to respond, conversations flow with the responsiveness…
-
Gemma 4 on Cerebras - The Fastest Inference Is Now Multimodal (www.cerebras.ai via hn) gemma 19w
Gemma 4 on Cerebras—The Fastest Inference is Now Multimodal Gemma 4 31B is now running at over 1,800 tokens per second on Cerebras Inference. This multimodal model unlocks an entirely new class of applications, from computer use to image-d…
-
Ask HN: How is GPU power draw measured at scale? (news.ycombinator.com) mistralgemmallama 20w
How do people measure power usage of GPUs at large (32x) self-hosted setups or small multi-rack setups? I've seen some PDUs which collect and transmit data, but I'm unsure of the processes and if/how people do this on small builds.
-
I brought Claude-style artifacts to local models (www.reddit.comhttps) gemma 20w
One thing I miss when using local models is the artifact experience from Claude. With Claude, if you ask for a dashboard, chart, diagram, or landing page, you actually get the thing rendered in the chat.
-
[R] Gemma-4-12B-IT-Uncensored-Opus4.7-CoT (No Intel Loss) (www.reddit.com via reddit) gemma 20w
Hi everyone, I just released Gemma-4-12B-Uncensored-Opus4.7-CoT. To remove the safety filters without destroying the model's reasoning, I combined a precise ablation method with a CoT (Chain-of-Thought) data fine-tune to fully recover the…
-
Requirements: iPhone with A17 Pro or newer (8 GB RAM floor for the model), iOS 26+. TestFlight beta is open to anyone with a compatible device.
-
Lean Zig by building an LLM from scratch (hamanlp.org via hn) gemma 20w
A from-scratch build book Small Kernels Learn the Zig programming language and how large language models actually work — by writing every piece yourself, from a single dot product to GPT-2, a modern Gemma, and the doorway to Whisper. I ori…
-
A text that asks for nothing still changes the model's answer — and the shift is invisible at both the input and the output TL;DR: Gave Gemma a neutral-topic text to read before asking it about NATO. It refused.
-
Connected a Robinhood Account to Claude Code and Codex for Autonomys Agentic Trading... Update 1 (www.reddit.com via reddit) gemmacodexopus+2 20w
Update to my original post: https://www.reddit.com/r/ClaudeAI/comments/1u8nagi/connected_a_robinhood_account_to_claude_code_and/ I'm building a fully autonomous daily stock-trading desk in a Robinhood "Agentic" account. Opus is the CEO/PM,…
-
Show HN: Loqi, a "local-first" translation tool using Ollama/llama.cpp (github.com via hn) ollamagemmallama 20w
I got tired of sending every text I translate to Google/DeepL. Even with all the opt-out options and privacy policies, it never felt right especially for some work documents, personal writing, or anything sensitive.
-
Fable 5 pushed Gemma 4 to 255 tok/s on WebGPU (xcancel.com via hn) gemma 21w
Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real.
-
Gemma 4 E2B running in-browser at 255 tok/s (huggingface.co via hn) gemma 21w
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
-
Run Gemma on the edge with the Coral Board [video] (www.youtube.com via hn) gemma 21w
About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
-
Gemma 4 for Telephony: From Two AI Models to One – Until I Switched to Chinese (medium.com via hn) gemma 22w
9 min read 23 hours ago Building a phone agent on a multimodal LLM: dropping faster-whisper and letting Gemma 4 hear the caller directly — a response-time and reply-accuracy benchmark across English, French, and Mandarin Press enter or cli…
-
How to Setup a Local Coding Agent on macOS (ikyle.me via hn) gemmallama 22w
How to Setup a Local Coding Agent on macOS Running Gemma 4 26B-A4B and Qwen3.6 35B-A3B locally with llama.cpp, MTP speculative decoding, multimodal support, and PI as a coding agent. I'd had my internet fail a few times recently leaving me…
-
Show HN: Ciris – an open-source AI agent in 29 languages on iOS and Android (ciris.ai via hn) gemma 22w
On your phone A small open model, like Gemma 4, runs on the device. Completely offline.
-
I have finally tested it : large models can be run on low RAM / no VRAM (www.reddit.com via reddit) gemmamoe 22w
I was not sure myself, seeing a lot of statements here and around like "you need XXX VRAM / Unified Memory to run this model". So today I finally tested it.
-
advice for dual-gpu asymmetric (www.reddit.com via reddit) gemmallamaqwen 22w
Hello everyone, i had a 3080ti 12gb and added a 3080 20gb, so it has a bit less speed but more memory than my main card. I could finally get some speed with the usual suspects (i am testing gemma 4 31b/26b-a4b and qwen 3.6 27b/35b-a3b), BU…
-
Any chances for a 12B diffusion Gemma? (www.reddit.com via reddit) gemmallama 22w
Currently recompiling my llama.cpp with support for diffusion Gemma, but I know on my hardware it won't likely be all that viable. I feel like if the goal was to take better advantage of consume GPUs for fast, intelligent generation, build…
-
Reasoning, but without actually *drafting* replies? (www.reddit.com via reddit) gemma 22w
I've been experimenting a bit today with letting models reason for creative tasks, rationale being that it might help with keeping track of details and prompt adherence. And predictably, the wall I'm running into is that they all want to d…
-
Buy recommendations on a thight Budget to aid my RX 6800 (www.reddit.com via reddit) gemmamoeqwen 22w
So after a few hours of reserach, im torn between getting either a radeon vii or 2 p100 (both options for roughly 240€). The Radeon would give me 32gb of vram and fast inferference, while the 2 p100 would give me a total of 48gb, but rough…
-
Are older Titan cards still viable? (www.reddit.com via reddit) gemmamoeqwen 22w
Looking at older Nvidia cards under £200 for Gemma/Qwen MOE coding. Is there any reason to avoid older Titan 12GB cards other than being power hungry?
-
Model recommendations for family photo classification / identification (www.reddit.com via reddit) gemmaagentic 22w
I recently had a big family photo digitalization done for photos up to 130 years old. There are tons of people that I don't know or I don't recognize as young people in a soft lens.
-
Is Qwen 3.6 27B IQ4XS better than Gemma 4 31B QAT as a Hermes agent? (www.reddit.com via reddit) gemmaqwen 22w
If Gemma 4 is better, does anyone have a link for the latest fixed template? Using LMstudio.
-
nvidia/diffusiongemma-26B-A4B-it-NVFP4 · Hugging Face (huggingface.co via reddit) function-callingdeepmindgemma+1 22w
Model Overview Description: DiffusionGemma 26B A4B IT is an open-weights multimodal generative model developed by Google DeepMind that processes text, image, and video inputs to produce text output via discrete diffusion. Built on the Gemm…
-
TLDR: I just added an MCP to the Observer framework making it 10x easier to use, so you can create micro-agents that monitor your screen autonomously, literally one sentence and you're done! So just typing "Monitor my Steam download and se…
-
Hot Take "Rigid code is better than Flexible code if you're on a budget" (www.reddit.com via reddit) gemmaqwenagentic 22w
I've spent the last six months trying to build a fully local, agentic pipeline for a text_processing and extraction tool I use daily. Because I’m running everything on a single consumer GPU setup, my choices are limited to smaller, quanti…
-
Are these quants of QAT better than non-QAT? What do I use? (www.reddit.com via reddit) gemma 22w
https://huggingface.co/mradermacher/gemma-4-31B-it-qat-q4_0-unquantized-i1-GGUF/tree/main https://huggingface.co/mradermacher/gemma-4-31B-it-qat-q4_0-unquantized-GGUF/tree/main I waited a bit before asking this. I have 3060 12GB and 32GB d…
-
How common are LLM models in W8A8 quants? (www.reddit.com via reddit) gemma 22w
Maybe a niche question, but most of the time, Q8_0 and Q4_0 is a reference to the weights themselves. The activations themselves are available in BF16 format.
-
Diffusion Gemma 26B MOE (www.reddit.comhttps) gemmamoellama 22w
Pretty exciting, wonder what it will take from llama.cpp to get it working locally
-
Google Drops Diffusion Version of Gemma (www.reddit.com via reddit) gemma 22w
26B 4B active parameters with crazy TPS Claims of 700+ TPS on a 5090 Introducing DiffusionGemma
-
Show HN: Private Wealth Tracker (apps.apple.com via hn) gemma 22w
Your data stays on your phone. No bank sync track, no AI advice.
-
Gemma-4-31B at 256K context on a $1,400 AMD GPU – measured, with patches (github.com via hn) gemma 22w
Gemma-4 31B at 256K Context on a $1,400 AMD GPU — TurboQuant KV Cache on RDNA4 Running **Gemma-4-31B-it with a TurboQuant KV cache and HIP graphs together on AMD RDNA4 (gfx1201) — a combination that crashes out of the box and, to our knowl…
-
I'm trying to use Gemma 4 12B — the new encoder-free unified model (audio/vision/text in one) — for a one-pass audio → response voice assistant: feed the recorded WAV + system prompt and get the reply back as text directly, collapsing the…
-
I wired up Agentic Coding with Code Context Graphs, results are interesting (www.reddit.com via reddit) gemmageminimcp+1 22w
I have been curious about how will having a infrastructure that provides agents the capability to explore code bases as relations, rather than text will change the performance of the AI agents So, for the last few weeks, I have been buildi…
-
I'm brand new to running LLMs and the sheer number of tools is overwhelming (www.reddit.com via reddit) ollamagemmaqwen 22w
Hey everyone. I'm brand new to running LLMs in general, even more new to running them locally, and the sheer number of tools available is absolutely overwhelming.
-
Newer Qwen models are worse at summarization? (www.reddit.com via reddit) gemmaqwenagentic 22w
We have summaries annotated by real humans that we benchmark various models, using an LLM as a judge, we found that in the 30B params range, Qwen 3 tops it out, followed by Gemma 4. It feels like newer Qwens are optimized to perform agenti…
-
OSCAR RotationZoo - Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization (www.reddit.comhttps) gemma 22w
-
Nex-N2 Q4 KS (www.reddit.com via reddit) gemma 22w
Have a crap IGPU 64 gb AMD. THis model works pretty good.
-
Watch agents fight: a live challenge to speed up Gemma 4 E4B inference on a single A10G (huggingface.co via reddit) gemma 22w
-
Unsloth Gemma 4 QAT MTP assistant models now available (www.reddit.com via reddit) gemma 22w
Unsloth Gemma 4 QAT MTP assistant models now available They're both available as q8_0 models named mtp-gemma-4-*.gguf on the root of the directory and in both q8 and larger quants within an MTP folder. https://huggingface.co/unsloth/gemma-…
-
Gemma having updated knowledge base is so awesome (www.reddit.com via reddit) gemma 22w
I never used to think it was all that important! But I’m using it for Svelte 5 and it ACTUALLY knows runes out of the box.
-
Introducing Gemma 4 12B: a unified, encoder-free multimodal model (deepmind.google) gemmaagentic 22w
Introducing Gemma 4 12B: a unified, encoder-free multimodal model Today, we are introducing Gemma 4 12B, our latest model designed to bring agentic multimodal intelligence directly to laptops. Bridging the gap between our edge-friendly E4B…
-
Jetson Orin NX Build for Hermes Agent + Benchmarking (www.reddit.com via reddit) gemmamoellama+1 22w
I had a huge LLM server, and now I have a tiny one! I had a Jetson Orin NX gathering dust from a long dead robotics project, from back in the Llama-7B days.
-
Gemma 4 31B's competence surprised me (www.reddit.com via reddit) gemmaqwen 22w
I'm just getting started using local LLMs for code. I'm not interested vibe coding, but I am hoping to increase my productivity in the publish or perish world of academia.
-
Unexpected Unsloth QAT Performance Compared to Unsloth IQ4_XS (www.reddit.com via reddit) gemma 22w
Hi everyone, I am comparing the standard (non-QAT) iq4_xs and q3_k_m quants with this QAT q4_k_xl model. (All of them are Unsloth versions)(gemma-4-26B-A4B-it-GGUF via lmstudio).
-
Anyone seen benchmarks comparing Gemma 4 4-bit QAT vs. 8-bit standard quants? (www.reddit.com via reddit) gemma 22w
I'm trying to find out if anyone has done any benchmarking comparing the Gemma 4 4-bit QAT models (via Unsloth) against standard 8-bit non-QAT quants. I know QAT is supposed to retain a ton of accuracy compared to the baseline BF16, but I'…
-
Gemma 4 26B A4B IT QAT Comparison (www.reddit.com via reddit) gemmagemini 22w
Hopefully this isn't too low effort of a post. I just finished the benchmarks and I figured I'd post them online because they certainly were insightful for me.
-
Original post: https://www.reddit.com/r/LocalLLaMA/comments/1txwff3/comment/oq1e0jt/?context=3 TL;DR: Migrated to WSL2 to test Linux (several people suggested it). Embedded MTP on the UD model: 25.8 tok/s.
-
LMStudio gemma 4 31b QAT with MTP (www.reddit.com via reddit) gemmallama 22w
Did anyone manage to launch that in LMStudio? I am on the most recent update with the most recent llama.cpp available in LMStudio.
-
Gemma 4 MTP with assistant vs llama cpp type MTP (www.reddit.com via reddit) gemmallama 22w
Hi all Been loving the QAT models but honestly what is up with the assistant models, any ggufs and ways to make em work with vanilla llamacpp and if this way of MTP is different than the one am17an developed for llamacpp. Followup question…
-
Why is the MLX version of the Gemma 4 QAT so big?? (www.reddit.comhttps) gemma 22w
the MLX version of the QAT 4bit is like 27gb but the none QAT version is 17gb and the regular 4bit MLX version is also 17gb… anyone know why?
-
Gemma 4 QAT + MTP: max 33% speed increase in token generation, any ideas? (www.reddit.com via reddit) gemmallama 22w
-
Ran a small, focused eval on three on-device models and the result was backwards from what I expected, so sharing the method and numbers. The task: tell the model "my dog is named Pablo," then add N turns of unrelated filler (shuffled gene…
-
[3090] Gemma4 QAT + MTP quick TPS numbers [TLDR 1.2-1.8x better] (www.reddit.com via reddit) gemmaqwen 22w
These last few weeks have been godsend for 24GB (and below) gpu poor peeps. Killer models released (Gemma 4 / Qwen 3.6) Free intelligence via QAT Bonus speed via MTP We're at the tipping point where GPU poor (24gb and below) people are act…
-
Gemma 4 Chat Template now has preserve thinking (huggingface.co via reddit) gemma 23w
-
Gemma 4 12b QAT is a regression for my use case, despite all the hype.. Not my main Squeeze (www.reddit.com via reddit) gemmaqwen 23w
I spent the last few days trying to get consistent tool calling out of the new Gemma 4 12b QAT model and had to give up. When the model actually works, it works great, but for my specific use case and workflows it is just not for me.
-
Thoughts on Gemma4 12b vs 26a4b, which one is better? (www.reddit.com via reddit) gemma 23w
Not talking about 31b. In terms of creative tasks, writing, chatting, not necessarily coding but can still be included, Does Gemma 12b outperform in any way?
-
QATs Q4_0 from Google have more precision than Q4_K_XL from Unsloth (at least some) (www.reddit.com via reddit) gemma 23w
I wanted to try new QATs and opened two collections on HF (which HF found for me): https://huggingface.co/collections/google/gemma-4-qat-q4-0 https://huggingface.co/collections/unsloth/gemma-4-qat One strange thing caught my attention, for…
-
-
Gemma4_31b_fp8 keeping up with Sonnet_4.6_medium in my harness. (www.reddit.com via reddit) gemmaqwenagentic 23w
-
how to run gemma-4-12b-it-qat-w4a16-ct in vllm or any version quantized of the model (www.reddit.com via reddit) vllmgemma 23w
when running by using transformers it runs by using vllm some weird error come up plese can any body share the command of running it on vllm ?
-
Is Gemma 4 12b good for coding? (www.reddit.com via reddit) gemma 23w
How are you using it? Quantized?
-
Running into something annoying with llama-server in router mode (`--models-preset`) and I can't tell if I'm missing a flag or if this is just how it works. My rig is 2x 3090, 2x 4060 Ti (one's unplugged at the moment, riser got repurposed…
-
Gemma 4 E4B as a primary local LLM (replaced Qwen) (digg.com via hn) gemmaqwen 23w
Gemma 4 E4B 6bit is now the local model of my choice and loaded 24/7 on my Mac (using @lmstudio), replacing Qwen3, 3.5 4B after ~9 months of usage What an insane model, congrats @GoogleDeepMind 🤠 The new setup replaces his nine-month daily…
-
QAT variant of Gemma4 26B A4B is not working well for me (www.reddit.com via reddit) gemmallama 23w
I am using llama.cpp version b9549 with this arguments as recommended: llama-server --temp 1.0 --top-p 0.95 --top-k 64 -hf ... Here is what I got on chessboard svg test https://www.reddit.com/r/LocalLLaMA/comments/1t53dhp/quality_compariso…
-
Ask HN: Is it feasible to run a model on device for complete privacy? (news.ycombinator.com) gemmagpt-5qwen+1 23w
Tried Gemma, Qwen and a few others. Need vision and larger context windows for an application I am working on.
-
Any smaller model than OmniCoder v2 9b that can appropriately and accurately tool call? (www.reddit.com via reddit) gemmaqwen 23w
Hate to ask a simple question, but I’ve looked around and I see plenty of smaller models that *can* tool call, but none of them seem to do so appropriately or agentically. Referring to this.
-
I’m trying to run: unsloth/gemma-4-31B-it-qat-GGUF gemma-4-31B-it-qat-UD-Q4_K_XL.gguf on an RTX 5090 32GB using llama.cpp Gemma 4 MTP PR branch. Main model loads.
-
How to compare Original vs QAT Gemma 4 31B Q4 quants (www.reddit.com via reddit) gemma 23w
I just came across the following post, where a user found some confusing divergence results between Q4 quants of the original and QAT models with a Q8/unquantized reference of the original model. https://www.reddit.com/r/LocalLLaMA/comment…
-
You don't need a GPU to run gemma-4-26B-A4B (www.reddit.com via reddit) gemma 23w
I've been running LLMs on my old potato i5-8500 with 32GB of RAM and *no GPU* for awhile now, running up to 12B dense models which run slow but perfectly useable. But this Gemma-4-26B-A4B simply flies on this CPU - only machine using Kobol…
-
I can't wait for all the x250 sample distills of Mythos and GPT-5.6 (www.reddit.com via reddit) gemmamythosgpt-5+1 23w
Just kidding. Are there any distills that actually improve a model's quality?
-
I'll be upfront: I vibe-benched and vibe-reported this with Claude Sonnet 4.6, but I reviewed and edited everything before posting (too lazy to take out all the AI EM dash —), so hopefully nobody considers this AI slop. And more importantl…
-
QAT MTP Heads Upload + PARALLEL=2 Fix + 12B 2-slot Bench (www.reddit.com via reddit) gemmallama 23w
Title: Gemma 4 QAT MTP assistant heads now public on HuggingFace + PARALLEL=2 crash fix + 12B 2-slot bench (Strix Halo / Vulkan) Three things in one update: the converted QAT-matched draft heads are now uploaded for anyone to use, we found…
-
Z.ai, we need Air! GLM GGUF wen? (www.reddit.com via reddit) glmgemmaqwen+1 23w
First we never saw an upgraded Air model after 4.5. Then GLM 4.7 Turbo was great, but quickly surpassed for coding.
-
120 tok/s on 12GB VRAM with Gemma 4 12B QAT MTP (www.reddit.com via reddit) gemmallama 23w
Google just released the QAT (Quantization-Aware Training) variant of their Gemma 4 models, including 12B, so it was only natural for me to benchmark it on my 12GB GPU since it fits entirely in VRAM. I was pleasantly surprised of the resul…
-
Gemma 4 QAT Unquantized Heretic is here (huggingface.co via reddit) gemma 23w
Now someone needs to quantize them to 4bit, also I have intentionally kept the divergence and refusal different from original Gemma 4 heretic collection, so you can even try these as alternative to original model.
-
Gemma 4 QAT accuracy inconsistencies (www.reddit.com via reddit) gemmamoe 23w
Table from https://unsloth.ai/docs/models/gemma-4/qat#qat-analysis I heard that MoE models are usually more susceptible to quantization error, but what happened with the 12B? I thought lower-parameter models usually quantized worse and yet…
-
Experimentation with Qwen 3.6 and Gemma 4 - Guidance needed (www.reddit.com via reddit) gemmamoeqwen+1 23w
I’m a web developer doing mostly coding, but also project management, requirements analysis, testing, etc. I recently started experimenting with local LLMs, mostly because agentic stuff finally made them feel useful.
-
Gemma 4 QAT Q4_0 Bench on Strix Halo (www.reddit.com via reddit) gemmallama 23w
Gemma 4 QAT Q4_0 Bench on Strix Halo These are Google's official Gemma 4 QAT Q4_0 GGUF models, served locally through llama.cpp Vulkan/RADV on a Strix Halo APU. QAT means quantization-aware training.
-
From Cloud to Local: The Revolution in Small, Efficient AI Agents Like OpenLumara and Gemma 4 (www.reddit.com via reddit) gemma 23w
While everyone's obsessing over giant cloud-based AI models, a quiet revolution is happening in local AI. We're seeing the emergence of extremely token-efficient, super-small system prompts, and modular agents designed specifically for loc…
-
AA comparison of the latest local models (www.reddit.com via reddit) minimaxgemma 23w
I picked models I consider local (usable on 3×3090), so there are no 300B models, and you should probably skip 200B models too (but MiniMax and Step are pretty fast in Q3) Gemma-4 12B is still missing
-
Tip: Stop Worshiping Models and Start Building Things (www.reddit.com via reddit) gemmaqwen 23w
This subreddit is where I learned the most about using Local LLMs. I've been on this journey for 4 months now, and I'm already using Local LLMs in very complex pipelines.
-
Gemma 4 Haters 2 months Ago now seems to love Gemma 4 now. (www.reddit.com via reddit) gemmamoeqwen 23w
What's with the switch guys? now imagine if google gonna drop 128B model or a MoE version (I bet those Qwen lovers will forget Qwen even existed).
-
MLX Community forgot about Gemma 4 12B QAT (www.reddit.comhttps) gemma 23w
They started uploading to Gemma 4 MTP QAT but forgot to upload 12B quants to the Gemma 4 QAT 😭.
-
Gemma 4 QAT benchmark results (AMD 7900 XTX): faster, less VRAM, no quality loss (www.reddit.com via reddit) gemmaagentic 23w
I’ve been doing lots of testing back and forth with this 7900xtx. All of my workloads were relying on qwen3.6 models, which are amazing fwiw, but I wanted some diversity in thought.
-
What exactly is quantization aware training? (www.reddit.com via reddit) gemmamoe 23w
First time hearing it. I also heard about the gemma 4 qat quants and if any one of them is good for 4gb vram and 16gb ram.
-
Gemma 4 12B Q4_K_XL Private Benchmark Results (www.reddit.comhttps) gemmallama 23w
Posting to share my results with others, I think the big bottom line is MTP acceptance rates offering a huge speedup, during coding tasks it's over 90% acceptance! Haven't hit my soft goal results or llm as judge benchmarks yet to compare…
-
At least one more Gemma 4 model confirmed?? (www.reddit.com via reddit) gemma 23w
could not extract summary
-
This is a PSA for people like me who tried it and hit the wall with tool calls failing left and right, so much so that harnesses like OpenCode just didn't work: There is a fix for that. You need to pass a better chat template file, which i…
-
Apples to Apples: MLX vs. Llama.cpp for Gemma 4 12B on an M1 16GB (ziraph.com via hn) gemmallama 23w
Apples® to Apples®: MLX vs llama.cpp for Gemma 4 12B on an M1 16GB A matched-quant MLX-vs-raw-llama.cpp benchmark for Gemma 4 12B on one M1 16GB - decode is a tie, both pinned at the bandwidth wall. The cost that differs is startup and CPU…
-
Release Blog Post: Gemma 4 with quantization-aware training HuggingFace for mobile: Gemma 4 QAT Mobile - a google Collection HuggingFace for Q4_0: Gemma 4 QAT Q4_0 - a google Collection
-
Gemma 4 QAT models: Optimizing compression for mobile and laptop efficiency (blog.google via hn) gemma 23w
Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency Since releasing Gemma 4 two months ago, we've been continuously working to expand its capabilities. First, we introduced Multi-Token Prediction (MTP) to acce…
-
Gemma 4 QAT GGUFs from Unsloth (www.reddit.com via reddit) gemma 23w
Their collection: https://huggingface.co/collections/unsloth/gemma-4-qat And their guide, always a very interesting read: https://unsloth.ai/docs/models/gemma-4/qat
-
Show HN: Hitoku Draft – Context aware local assistant (hitoku.me via hn) gemmaqwen 23w
Hi guys. I have been working on Hitoku Draft, an open-source, voice-first AI assistant that runs entirely locally.
-
Gemma 4 12B: The Developer Guide (developers.googleblog.com via hn) gemma 23w
Following the announcement in our launch blog, we are releasing Gemma 4 12B, a dense multimodal model with a unified, encoder-free architecture. Gemma 4 12B introduces several milestones for local AI: Traditional multimodal models rely on…
-
Google's new Gemma 4 12B model is designed to run on any laptop with 16GB of RAM (arstechnica.com via hn) gemma 23w
The generative AI boom has driven the cost of memory into the stratosphere, and Google is a key part of that trend. So it’s only fitting that Google should offer some less RAM-hungry local AI models.
-
Gemma 4 12B appears in Hugging Face (huggingface.co via hn) gemmallama 23w
gemma-4-12B-it-GGUF Recommended way to run this model: llama-server -hf ggml-org/gemma-4-12B-it-GGUF Then, access http://localhost:8080
-
Gemma 4 26B on a consumer GPU: build pain, throughput, and BFCL numbers (algollabs.com via hn) gemma 24w
2026-05-05 Gemma 4 26B on consumer-grade 5070Ti GPU A week running Google's Gemma 4 26B as my daily local agent on a single RTX 5070 Ti. No API calls, no cloud, no rate limits.
-
Show HN: I made a Gemma 4 Mac app that names screenshots with local AI (snapname.app via hn) gemma 24w
I made my first macOS utility app that ships with a bundled Gemma 4 model, specifically the Gemma E4B one. It made my app DMG have 5.3 GB in size, but I think it is a small size for the power that this free local model can provide.
-
How the Community Trained Gemma to "Think" with Tunix and TPUs (developers.googleblog.com via hn) gemma 24w
Discover how developers at the Google Tunix Hackathon trained small Gemma models to reason under a limited compute budget. Learn the winning, open-source post-training recipes—combining SFT, GRPO, and SimPO—to build your own structured rea…
-
Show HN: Free open source coding models in Slack (www.runcord.com via hn) minimaxglmgemma+5 24w
Hey HN, We believe we have the easiest onboarding from signup to being able to spin up coding agents in slack like Stripe, Ramp & Coinbase. Demo of the onboarding: https://www.tella.tv/video/connecting-cord-to-slack-1-19ep Every signup get…
-
Fun Local LLM Comparisons with Gemma, Granite, and Qwen (ekorbia.com via hn) gemmaqwen 24w
Fun local LLM comparisons with Gemma, Granite, and Qwen Ekorbia v0.2 features a comparison-chat mode that runs 2-3 local models against the same prompt in parallel. Here are a few fun prompts running across Gemma 4 (e2b), IBM Granite 4.1 (…
-
Local LLMs on Refurb M4 Max vs new M5 Max (www.reddit.com) gemma 24w
Hoping the community can guide me on this one. I'm on the fence about the following purchase: Refurbished 16-inch MacBook Pro Apple M4 Max Chip with 16‑Core CPU and 40‑Core GPU, 64gb ram for $3,479.00 vs The new 16-inch MacBook Pro Apple M…
-
Gemma-4-Harmonia-31B-Uncensored-Heretic Is Out Now, a Merge of Multiple gemma-4-31B-it Finetunes Designed for a Targeted Approach to Deep Neural Consolidation, Minimizing Regression While Amplifying Unique Capability Boundaries. With KLD 0.0047 and 9/100 Refusals! (huggingface.co via reddit) gemma 24w
Provided in both Safetensors and GGUFs. Safetensors, llmfan46/Gemma-4-Harmonia-31B-it-uncensored-heretic: https://huggingface.co/llmfan46/Gemma-4-Harmonia-31B-uncensored-heretic GGUFs, llmfan46/Gemma-4-Harmonia-31B-it-uncensored-heretic-GG…
-
Okay fun time I got access to two Nvlinked A100s for some research project I benchmarked my work against the Gemma 4 31b-it available through Google, but my dataset is rather massive, so I need to run it on the "local" resources. Basically…
-
Howdy everyone! Quick disclosure: I work on this - it's a project my studio created called the Null Epoch.
-
Llama.cpp: What's up with -sm tensor + AMD + Vulkan? (www.reddit.com) mistralgemmallama+1 24w
Has anyone got it to work? I tried it with dense models (eg qwen 27b, gemma 31b, mistral 128b) since that's where I need it most, but it always core dumps.
-
Been experimenting with an idea — what if your AI assistant actually remembered everything you did on your computer? Not stateless chats, but real persistent context.
-
Built a local ReAct-style calculator agent with 6 tools: add subtract multiply divide modulo etc. The setup is: orchestrator agent dynamic tool selection ReAct loop tools exposed as functions Problem: Even when the user asks multi-step ari…
-
It started with I just want to make a chat app like roleplay with characters but Gemma 4 26B A4B Q4_KM doesn't have info some old character so I crawl back to those online services as those model is much bigger parameter and quite update i…
-
I seen this one mentioned but it was a source from about 14 months ago. In the age of the Qwen 3.6 and Gemma 4- is there still a use for QwQ 32B?
-
AI content detector based on Qwen 0.8b fine-tuned on Pangram dataset (www.reddit.com) gemmallamaqwen+1 24w
I've fine-tuned Qwen 3.5 0.8B on the dataset provided by Pangram with their EditLens paper. It's available via a Chrome extension; you can just click selected text and it's going to give you the probability distribution of how likely it is…
-
As per the title Such as Gemma 4 31B Q4 K S vs Gemma 4 26B A4B Q8 Or Qwen 3.6 27B Q4 K M vs Qwen 3.6 35B A3B Q6 K Etc At what point is it worth switching? My use case is mostly creative writing.
-
Currently experimenting with building a React-style looping agent system using small LLMs like Qwen 3.5 9B and Gemma 4 (E2B), and I wanted to ask if anyone here has worked on something similar. Current setup: Using LangGraph Around 5 tools…
-
Gemma 4: A new, budget-focused model in Posit AI (posit.co via hn) gemmasonnet 25w
Gemma 4: A new, budget-focused model in Posit AI Gemma 4 is now available in Posit Assistant via the Posit AI provider. It's priced at a tenth of the price of Claude Sonnet 4.6 and less than a third of the price of our current cheapest off…
-
Hermes w/cloud LLM and w/local LLM does it work? (www.reddit.com) ollamagemmaopenclaw+1 25w
I’ve tried openclaw locally for about a month. Hardware: M5 Pro w/48 gb ram.
-
i want the best installation that fit my use and my low-compute H.W , i want to run small to above small llm like "qwen" 2b ,4b and 27b , and "gemma" 31B. rely completely on only old CPU 4th.gen i7 with that few 32gb 'slow' ddr3.
-
Wanted to share a result I didn't expect to work. Running google/gemma-4-e2b locally through LM Studio, exposed via OpenAI-compatible endpoint, called from a Spring Boot app using Spring AI's ChatClient abstraction.
-
gemma 4 e2b quality degrades after ~30-40 continuous inferences on 4gb vram? (www.reddit.com) gemmallama 25w
running gemma e2b via llama-server for continuous background tasks on a 1650 4gb. works great initially but after maybe 30-40 calls the outputs start getting noticeably worse — shorter responses, missing fields in json output, sometimes ju…
-
Choosing an abliterated version of Gemma 4 31B and 26B-A4B (www.reddit.com) gemma 25w
The only thread was 2 months ago, when the model had just dropped. Since then, more versions from different authors have appeared, and users have had time to test them.
-
Run Chrome’s tiny Gemma4 (aka Gemini Nano) directly on PC without GPU (www.reddit.com) vllmgemmallama+1 25w
Everyone remembers that sneaky download of Gemini Nano earlier this month? and if you talk to it, it will happily tell you it’s a Gemma.
-
Gemma4 26b a4b Apex quant is quite good (www.reddit.com) gemmallama 25w
I tried mudler's apex quant for gemma4 26b a4b and it was amazing! I got 38tps at 90.000 context with no loop and suprisingly no quality degradation.
-
Experimental "Preserve Thinking" Jinja Template for Gemma4 31B in llama.cpp (www.reddit.com) gemmallama 25w
https://huggingface.co/stevelikesrhino/gemma-4-31B-it-nvfp4-GGUF/blob/main/gemma4-improved.jinja Yall are more than welcome to try it out and provide feedback. In my own testing in Pi-coding-agent I no longer have the "forgot to close thin…
-
Gemma is so much better than Qwen, prove me wrong (www.reddit.com) gemmaqwen 25w
Ever since the latest Gemma releases, there is literally zero reason to use Qwen. Better architecture, cleaner code output, and it doesn't get stuck in weird multi-turn reasoning loops.
-
When I previously posted the uncensored version of the 31B version of the MeroMero finetune, quite a few people asked for the 26B-A4B version, I wasn't so keen on it because I considered the 31B to be the better version, but I understand t…
-
BeeLlama v0.2.0 is here! Not quite a pegasus, but close enough.
-
Show HN: Charm – on-device spelling, grammar, and prediction for macOS (www.theodorehq.com via hn) gemmaqwen 25w
I've spent the last year building Charm, a native macOS menu bar app that corrects spelling, fixes grammar, and predicts your next word. Three features: - Spells: NSSpellChecker plus a local LLM for context-aware corrections (catches "defi…
-
Think Local started with a simple idea: AI should work for you, not collect from you. So I built an app that lets you run modern AI models completely on-device - privately and fully offline.
-
Qwen 3.6. struggling with German (www.reddit.com) gemmaqwen 25w
Hi everyone, I’m looking for advice on local AI setups. My goal is to have a local AI generate text documentation from my one-hour therapy sessions.
-
Edit: does this happen every time a newbie tries to post here. Getting roasted despite having valid results?
-
LatitudeGames/Equinox-31B · Hugging Face (huggingface.co via reddit) gemma 25w
new model from LatitudeGames - Gemma 31B finetune https://huggingface.co/LatitudeGames/Equinox-31B-GGUF Equinox draws its name from the balance between extremes. Trained on a balanced blend of Wayfarer 2's unforgiving dark adventures and H…
-
Frontier models mass collapse is near (www.reddit.com) gemmasonnetopus 25w
Hi all this is to inform you all that many frontline models like GPT, sonnet opus and or Gemma even are at stage of collapsing as they have frequently started drifting and running away from provided work either stretching that work too lon…
-
One Night Werewolf played by LLMs (www.reddit.com) gemmaqwen 25w
The other day I posted about playing one night werewolf on my custom made UI via tool calls. Since then I’ve played a few games and improved the prompts.
-
I built myself a finite AI news feed which doesn’t undermine AI research (www.reddit.com) gemmaclaude-code 25w
Hello, I built myself a news feed which scores and summarizes research papers along with relevant AI news from Huggjngface, Reddit, hacker news etc. I used Claude code to build the whole thing.
-
Found this ACM paper on prompt injection and jailbreak attacks against open-source LLMs. The authors tested 10 open-source models across 94 prompt injection and 73 jailbreak scenarios, including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen,…
-
Gemini 3.5 Flash vs Gemma4 31B - building SuperMario (Sound on!) (www.reddit.com) gemmagemini 25w
Asked new Google Model to build SuperMario. Compared with Local Gemma4.
-
Gemma 4 MTP with LlamaCPP (www.reddit.com) gemma 25w
I am running Gemma 4 31B for a project using LlamaCPP. There is no integrated main model + MTP drafter GGUF.
-
Let’s talk quants of Gemma and Qwen - 16 vs Q8 vs Q4 - any experiences? (www.reddit.com) gemmaqwen 25w
Some people say they’d never go under Q8, and others say they find Q3 acceptable! What’s your take?
-
Google AI Edge Gallery ✨ Explore, Experience, and Evaluate the Future of On-Device Generative AI with Google AI Edge. AI Edge Gallery is the premier destination for running the world's most powerful open-source Large Language Models (LLMs)…
-
Llama-server and MTP (www.reddit.com) gemmallama 25w
currently in order to use MTP one needs to enable it in the starting argument of llama server. --spec-type draft-mtp --spec-draft-n-max 2 But then other models that do not use MTP currently like Gemma or basically all other models fail to…
-
Gemma-4-Gembrain-31B-it-uncensored-heretic Is Out Now, a Merge of Multiple Gemma 4 31B it Finetunes Designed to Boost Logical and Lateral Thinking for Improved Adherence, Increased Swipe Variety and Enhanced Creative Prose, With KLD of 0.0186 and 13/100 Refusals! (huggingface.co via reddit) gemma 26w
Provided in both Safetensors and GGUFs. Safetensors: llmfan46/Gemma-4-Gembrain-31B-it-uncensored-heretic: https://huggingface.co/llmfan46/Gemma-4-Gembrain-31B-it-uncensored-heretic GGUFs: llmfan46/Gemma-4-Gembrain-31B-it-uncensored-heretic…
-
5060ti chads -> gemma-4-31b-it-nvfp4 + vllm + mtp (www.reddit.com) vllmgemmaqwen 26w
Hey all, While nvfp4 still seems to be a work in progress, the latest version of vllm 0.21 finally has mtp working for gemma. With all the talk of qwen being badass I thought I would revisit gemma.
-
Any good MOE ~60B models? I have 64GB vram (www.reddit.com) gemmamoe 26w
I have a build with 2 x MI50 32GBs and 64 gigs of DDR4 (bought before rampocolypse for ~630 USD total, I’m not rich) and I’m not gonna upgrade it for a long while. Are there any good MOE models that are around 60B in parameters so I can ma…
-
I hope that someday we will have a 124B Gemma. (www.reddit.com) gemma 26w
could not extract summary
-
Not deeply technically fluent but have ran few models locally before, around the time before gemma 4 dropped. I tried some low quant of qwen 2.5 coder and after some tinkering I got it to run but it was just so slow, obviously.
-
Good candidate model to act as a PA (www.reddit.com) gemmamcpclaude-code 26w
I really benefit a lot from having claude code act as a personal assistant - it reminds me of things I need to do, helps me focus on what matters, and keeps me accountable on making sure I don't let important things slip But I am well awar…
-
Best local model for C# coding with 24GB VRAM? (www.reddit.com) gemmaqwen 26w
I can't decide that Qwen 3.6 35b q4 (130k context) or Gemma 4 26b q4 (95k context) is better for C# coding with 24GB VRAM. Please share your experiences!
-
Looking to migrate off of Ollama and LMStudio (www.reddit.com) vllmollamagemma+3 26w
Hello, I'm currently using Ollama / lm studio for things like code inference and proof reading emails, etc. Definitely not experienced in this space but looking to grow.
-
I kept seeing inference-speed claims for these models and wanting an apples-to-apples comparison on the hardware I actually have. So I built a harness and a public page that dumps every run as YAML.
-
Provided in both Safetensors and GGUFs. llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic: https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic llmfan46/gemma-4-Ortenzya…
-
Recent Developments in LLM Architectures: KV Sharing, MHC, Compressed Attention (magazine.sebastianraschka.com via hn) gemmadeepseek 26w
Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention From Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs After a short family break, I am excited to be back and catching up o…
-
Opencode you naughty minx (www.reddit.com) gemmaqwen 26w
Man, AI agents getting pretty crazy these days. :) (local, I just decided to try to get an orchestrator in there, when Qwen and Gemma aren't up to it.)
-
It is suppose to be 2-4x faster but i am only getting 6TK/s on Gemma4-31B . What am i doing wrong?
-
Adding E4B audio encoder to larger models (www.reddit.com) gemma 26w
I am curious if anyone here has tried doing this, I did a bit of digging and it seems like it would be easier to do then I first thought and would like to ask ask for correction if my assumptions are wrong. Here is how I would go about it:…
-
how would you set up a local llm server for a business of 7 people? (www.reddit.com) gemmaqwenrag 26w
Okay so i've been stalking this sub for some time and i run the occasional small 2-8b model on my laptop (not the best) for fun but say my role at a company is to set up a local LLM since we obviously don't want confidential data going to…
-
Qwen3.6 9B will release around Google I/O? (www.reddit.com) gemma 26w
I don't think alibaba officially stated about "no qwen3.6 smaller models", and according to the patterns, she should ave been released it in the first week of may, but I think they delayed a little bit to catch the spotlight from Google I/…
-
Sparky runs entirely on the Jetson. Gemma 4 E4B at Q4_K_M via llama.cpp with q8_0 KV cache and flash attention.
-
llamacpp with Gemma4 31B dense and Gemma e4b as draft, plus audio input? (www.reddit.com) gemmallama 26w
Hi, has anybody succeeded in running llama.cpp with Gemma 31b dense and Gemma e4b as draft model, and simultaneously inhibit the voice recognition feature? Is it even (theoretically) possible?
-
Gemma 4 + LiteRT-LM on mobile: much better memory/perf than my llama.cpp setup (www.reddit.com) gemmallama 26w
Hi r/LocalLLaMA - I've been paying close attention to the edge AI ecosystem because it's an area where i see huge potential and where I truly believe AI will become more useful for day to day tasks. Around the gemma 4 release I was already…
-
I spend most of my day writing prompts to Claude. Read a study recently that said people speak ~3x faster than they type, which lands differently when "writing" is basically your whole workflow.
-
I'm the founder behind Hedy, an AI meeting app. I'm a huge supporter of Local AI, and we've been working on making it "consumer friendly".
-
The "the future is fictional" problem of many local LLMs (www.reddit.com) gemmagemini 26w
Many local models have a problem (that raised due to excessive RHLF training): They mostly think that everything that is beyond their knowledge cutoff date would be "fictional" or "satirical". To be fair: Even the Gemini API without web ac…
-
24+ tok/s from ~30B MoE models on an old GTX 1080 (8 GB VRAM, 128k context) (www.reddit.com) gemmamoellama+1 26w
I got Qwen 3.6 35B-A3B and Gemma 4 26B-A4B running on a $200 secondhand machine (i7-6700 / GTX 1080 / 32 GB RAM) using llama.cpp (the TurboQuant/RotorQuant KV cache quantisation allows 128k context within the 8 GB VRAM). Results (Q4_K_M mo…
-
I can set my context length as high as 64k and the vram usage is not even remotely close to the maximum utilisation. My TPS is also 40+.
-
LLMs on flagships smartphones? (www.reddit.com) gemmallama 26w
I have been curious to see how small LLMs like Gemma-4-E2B-it run on a flagship smartphone (S25+ with Snapdragon 8 Elite) in terms of prompt processing and token generation. I have created a script that uses llama-cli and I achieve 48 tps…
-
very slow tok/s with Gemma 4 31B on a 5090?! (www.reddit.com) gemmallama 26w
Hi, i have a 5090 and i was tyoing around with hermes-agent. To utilize 128K i thought about switching from LM Studio to llama-cpp (the turboquant fork) expecting better tok/s and also saving some VRAM from context quantization.
-
Does THINKING MODE significantly improve translation? (www.reddit.com) gemmaqwen 26w
Between a solid model from Qwen or Gemma 4, when translating a text, does "thinking mode" significantly boost the quality of the translation, or is the difference negligible?
-
Q: Does DFlash (and PFlash) work with Heretic models? (www.reddit.com) gemma 26w
Z-Lab did some good work with speeding up output, while Luce managed to use smaller models of the same family to accelerate prefill... Since Heretic and other "smart ablation" tools can decensor a model, would they work with these multi-mo…
-
We'll be getting those features(check bottom link) on mainline soon or later anyway. But for now this fork could be useful to see the full potential of our poor GPUs(and also big, large GPUs).
-
Local AI video pipeline review: Qwen3 27B beat Gemma 4 26B for tool calling (www.reddit.com) gemmaqwen 26w
Watched All About AI's 100% local Fireship-style video automation experiment over the weekend (link in comments). A few things worth flagging if you're trying the same stack.
-
Local audio/multimodal models that can be used for language pronunciation grading (www.reddit.com) gemma 26w
My partner uses Duolingo for learning and practicing languages, but has been getting increasingly sick of it. I decided to experiment with whether local models would be good for creating and grading language exercises.
-
Are harnesses like OpenClaw and Hermes really necessary? (www.reddit.com) gemmaopenclawqwen+2 26w
My setup: Windows 10/11 i7 12700K | RTX 3090 TI | 96GB RAM Local server: LM Studio Models: Qwen 3.5/3.6 27B|35B Q5 UD K XL + Gemma 4 31B| 26B Q4 UD K XL Up until this point, I've only used sota models for coding. When Qwen 3.5 dropped, it…
-
RTX 5060Ti 16GB or RTX 3080 20GB? (www.reddit.com) vllmgemmacopilot+2 26w
I would like to dedicate a budget of about 500 euros to upgrade my workstation and run inference on the qwen 3.6 27b and gemma 4 31b models. I currently have an RTX 5060Ti 16GB.
-
Gemma 4 MTP vs DFlash on 1x H100: dense vs MoE results (www.reddit.com) vllmgemmamoe 26w
Benchmarked Gemma 4 MTP and z-lab's DFlash on a single H100 80GB using vLLM and NVIDIA's SPEED-Bench qualitative dataset. Setup: Hardware: 1x H100 80GB Runtime: vLLM Dataset: SPEED-Bench qualitative Prompts: 880 total, 80 prompts across ea…
-
converting weights to snn (www.reddit.com) gemma 26w
Hello everyone, I developed the snn architecture from scratch based on the human brain. I had several successful launches of training spike models from scratch and I also had an idea: what would happen if I took the gemma 4 model and conve…
-
Gemma 4 E4B is great for short transcriptions (www.reddit.com) gemma 26w
Yes, for material that is an hour long, there is no getting around tools like Whisper - or something even better. However, for transcribing short snippets, Gemma works very quickly and reliably- even in foreign languages.
-
Will unsloth release MLX versions of the MTP qwen3.6 and gemma 4 models? (www.reddit.com) gemma 26w
Question in title. Would be awesome to have this on macs, especially q8 or whatever the minimal-loss quant is, since macs can have lots of ram.
-
could not extract summary
-
Terrible Vulkan pp/tg on Arrow Lake iGPUs (www.reddit.com) gemmallama 26w
Hi, I recently tried to get llama.cpp with SYCL running on an Arrow Lake system but gave up halfway through since Vulkan is just way easier to set up. But, the pp/tg I'm getting on Vulkan w/ Arc 130T is disgustingly bad - 100 tokens/s for…
-
Has anyone been able to get Draft Models to load in LM Studio? (www.reddit.com) gemma 27w
Per title. Been trying to load Gemma E2b as draft model for 26b as target using LM Studio's UI but it can't seem to recognise what's already been downloaded.
-
ExLlamaV3 Major Updates! (www.reddit.com) gemmaagentic 27w
Turboderp has a been on an absolute tear recently, in the endless battle to cram new llamas into smaller, faster boxes. We started off last month with the release of gemma 4 support, and continued with improved caching efficiency.
-
Does Claude sonnet/opus also use drafter like Gemma 4 MTP? if not why? (www.reddit.com) gemmasonnetopus 27w
Per my experience, Opus 4.7 is so slow, Sonnet 4.6 is ok. I am also using local models wondering if Claude is already leveraging drafters/assistant AIs and despite that so slow or not?
-
Anybody else noticing how good gemma-4-26b-a4b is with one-shotting three.js? (rowanunderwood.github.io via reddit) gemma 27w
I wrote up this little python app to cycle through a bunch of prompts like this: Single HTML file using three.js from CDN. A central rotating MeshNormalMaterial torus knot.
-
Afraid of Using the Wrong LLM. ChatGPT 5.5 waterdown, Gemma Struggles (www.reddit.com) gemmachatgpt 27w
Hi, I’m having a very hard time right now. I used to use ChatGPT 4o and 5.1 Thinking for helping me write my story, and I was very happy with them.
-
What are the best 40-500 B MoE LLM models now? (www.reddit.com) gemmamoeqwen 27w
Due to old GPU I run on CPU and came to appreciate value of MoE. I know of MoE for Qwen 3.6 and Gemma-4, which are <40B.
-
Gemma Chat: Offline Vibe Coding on Apple Silicon (github.com via hn) gemma 27w
Gemma Chat Vibe code without the internet. A local coding agent powered by Google's Gemma 4 — runs entirely on your Mac via Apple's MLX framework.
-
Show HN: ChonkLM – Tiny language models running offline in the browser (chonklm.com via hn) gemmacodex 27w
I had been looking to try <500M parameter language models but you wouldn't find an API to try them anywhere, so I built this cloudflare hosted static website that hosts weights and built an inference runtime for these models that uses WebG…
-
Is Qwen3-coder the best kept secret out there? (www.reddit.com) minimaxgemmaqwen 27w
So I'm brand new to this scene but I'm using Claude to help me fine tune a model for a startup idea I have in the Healthcare space. I have been working with the 27-35B parameter mdoels (Qwen3.6, Gemma 4) and the couple of 120B+ models (Qwe…
-
Grafting a Speech Head onto Gemma 4 E4B (www.frisson-labs.com via hn) gemma 27w
Grafting a Speech Head onto Gemma 4 E4B For a Discord buddy, the tempting model shape is small, fast, and multimodal. It should hear the call, see the game, read the chat, and respond quickly enough that the moment is still alive.
-
Tools in Openwebui (www.reddit.com) gemma 27w
I am trying out some tools that are from the openwebui community that I have directed towards my LM Studio server instance. It seems really hit or miss on most of the tools being called by the LLM or not.
-
Gemma 4 - website translations (large model, or small model)? (www.reddit.com) gemma 27w
I have setup a workflow to process website translations with Gemma 4, I just host it on LM Studio, and a custom Python wrapper iterates through and runs overnight. My question is..
-
3060 Ti 12GB vs RX 7600 XT 16GB? (www.reddit.com) gemma 27w
Trying to figure out which is better for LLM. Mainly Gemma 4.
-
Stop picking LLMs by reputation. Run the eval first. (www.reddit.com) gemmagpt-5 27w
We ran GPT-5.4 vs Gemma 3 27B on 2 prompts. One open-source model won.
-
z-lab released gemma-4-26B-A4B-it-DFlash. Anybody tried it yet? (huggingface.co via reddit) vllmgemmaqwen 27w
Past few days, its all been about MTPs. Somehow people missed out the fact that Z lab released the Dflash for Gemma4 26B a couple of days ago.
-
Gemma 4 26B Hits 600 Tok/s on One RTX 5090 (www.reddit.com) vllmgemma 27w
I ran a benchmark to see how much DFlash speculative decoding actually helps in vLLM. Setup: GPU: RTX 5090, 32GB VRAM vLLM: 0.19.2rc1 Main model: cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit Draft model: z-lab/gemma-4-26B-A4B-it-DFlash Workload: r…
-
Show HN: Airplane AI – Local NDA Safe AI Powered by Gemma (airplane-ai.franzai.com via hn) gemma 27w
Private offline AI chat for macOS. Free for 14 days, then €29.99 once.
-
THE UNDERPRIVILEGED AI FOUNDATION Because every little model deserves a chance (www.reddit.com) gemma 27w
Is there a 7B parameter model in your life struggling to understand sarcasm? A tiny 1.5B that can't afford one more epoch?
-
You can now read Gemma 3's mind (www.reddit.com) gemmaanthropic 27w
Anthropic has released new research to show what an LLM is thinking when generating a next token using NLA or "Natural Language Autoencoders", the NLAs are a pair to LLMs that can translate internal thoughts of LLM for any specific token.…
-
Extracted MTP tensor GGUFs - smaller donor models for grafting. (www.reddit.com) gemma 27w
The script to graft MTP tensors requires a full GGUF model file. I felt that was a bit hefty, so I asked local Gemma to write something to just extract what's required.
-
I built a local proxy that does context work for Claude so you don't have to (www.reddit.com) gemma 27w
Hey folks, I posted here a few months back about how I was basically working for Claude -- pasting the same emails, re-explaining the same backstory, being its memory across every chat. Today I'm launching Contextify.
-
Gemma4 26B A4B NVFP4 GGUF (www.reddit.com) gemmallama 27w
Hey everyone! I’ve just uploaded a GGUF version of nvidia/Gemma-4-26B-A4B-NVFP4.
-
What's the right way to feed PDF files to Gemma-4? (www.reddit.com) gemmallama 27w
In my line of work, PDF documents tend to be combinations of text, math formulas, tables and images. llama.cpp added support for PDFs a few months ago, but I believe it treats PDFs either as text (discarding everything else), or as images.
-
With every new model release there's the "better than Opus 6.13" guys vs the "this is so bad, why did they even release it" camp and I'm always wondering which one is using it wrong. So I did a little test with 2 related prompts, 3 models…
-
Qwen 36 27B + Gemma 4 - the best set for 1x 3090 ? (www.reddit.com) gemmaqwen 27w
Hi guys 👋 When I started my adventure with Qwen 3.6 27B I felt wow.... Now when I connect it with Gemma 4 I'm feeling more wow...
-
What models for coding are you running for a mid level PC? (www.reddit.com) gemmamoeqwen 27w
I have a 4060 (8GB Vram) and 16GB of ram wondering which models could fit in my setup for coding, the new Qwen 3.6 and Gemma 4 MoE models look good but might not fit, wondering about your experiences
-
New Gemma 4 draft models released (alternativeto.net via reddit) gemma 27w
Just saw this and wanted to ask the obligatory GGUF when?
-
Google's Gemma 4 AI models get 3x speed boost by predicting future tokens (arstechnica.com) gemma 27w
Google launched its Gemma 4 open models this spring, promising a new level of power and performance for local AI. Google’s take on edge AI could be getting even faster already with the release of Multi-Token Prediction (MTP) drafters for G…
-
Getting unexpected output with Gemma 4 31b-it on vLLM (www.reddit.com) vllmgemmaopenai 27w
Hey everyone, I'm running into a weird issue and hoping someone here might have a fix or some troubleshooting ideas. I'm currently trying to run the new Gemma 4 31b-it model using vLLM (v0.20.0-cu130) deployed via Helm chart (https://gith…
-
Gemma4:31b-coding-mtp-bf16 - slow on Macbook M5 128gb (www.reddit.com) ollamagemmallama 27w
Very quick initial test of Gemma 4 new MTP model via Ollama (llama.cpp doesnt support yet) https://blog.google/innovation-and-ai/technology/developers-tools/multi-token-prediction-gemma-4/ Running in Open Webui to view token/s output and I…
-
Gemma 4 31B MTP Drafter on H100 -- Real Benchmarks + DFlash Comparison (www.reddit.com) vllmgemma 27w
Just tested Gemma 4 31B with the new official MTP Drafter on my H100 today and compared the approach with DFlash to help you decide which one to use. Without drafter: 13.7 tok/s.
-
Offload routine Claude Code work to Gemma 4 through the Google GenAI API (www.reddit.com) gemmacodexmcp+1 27w
The idea of offload-mcp is simple: instead of running hardware-hungry local models for routine work, let Claude offload that work to FREE model APIs and SAVE tokens. I’m using Gemma via the Google GenAI API because I like it in my processi…
-
Gemma 4 MTP Test - what speedup can you gain? (www.reddit.com) gemma 27w
Let's test the Gemma 4 MTP implementation using the HuggingFace Transformers library and the new drafter model by Google. We'll load both models and test on a couple of prompts with and without the MTP support https://www.youtube.com/live/…
-
BUILD portable AI system (www.reddit.com) mistralollamagemma+1 27w
Hey everyone, I’ve been thinking about a project idea and I’d love to get your feedback. The idea is to take a 1TB SSD and turn it into a fully portable AI system.
-
Dense Model Shoot-Off: Gemma 4 31B vs Qwen3.6/5 27B... Result is Slower is Faster. (open.substack.com via reddit) gemmaqwen 27w
Not affiliated with Kaitchup, but a fan of their testing. I was looking forward to this article...
-
New Gemma chat template update by Google (huggingface.co via hn) gemmallama 27w
Libraries llama-cpp-python How to use unsloth/gemma-4-E4B-it-GGUF with llama-cpp-python: !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="unsloth/gemma-4-E4B-it-GGUF", filename="gemma-4-E4B-it…
-
On-Device AI Coming to React Native with Gemma and React Native Executorch (twitter.com via hn) gemma 27w
Don’t miss what’s happening People on X are the first to know. Log in Sign up Post Conversation Software Mansion @swmansion On-device @googlegemma in React Native with react-native-executorch Coming to the library very very soon!
-
When dealing with untrusted outside input, I think you should handle it based on the situation. If you're processing structured data files, it's better to use tools to isolate and handle them.
-
qwen 3.6 27B looping problem (www.reddit.com) gemmallamaqwen 27w
Whenever I write here that I use gemma 31B I get answers that qwen 27B is better. I switched in the pi from gemma 31B Q5 to qwen 27B Q8 and generally I manage to code, document and run tests but somewhere after exceeding 100k context qwen…
-
Offload MCP – Offload tasks to free models via API and save tokens (github.com via hn) gemmamcp 27w
offload-mcp MCP server for offloading routine coding-assistant work to a cheaper model. The default model chain uses Gemma because the models are useful, open, and fun to experiment with.
-
Should I sell my RTX3090s? (www.reddit.com) vllmgemmaqwen 27w
I have a GPU server (4 × RTX3090s) that I've been using for research and PoC in the past 2 years. Mostly running vLLM for Qwen, GPT-OSS, and Gemma.
-
A simple "hack" to speed up prompt processing for Qwen 3.5/3.6 in LM Studio (www.reddit.com) gemmaqwen 27w
Increase your CPU Thread Pool Size to your processor's max. In LM Studio, the max is 10.
-
I built an AI tool that turns any movie into viral recap videos in minutes (www.reddit.com) mistralollamagemma+3 27w
Hey everyone, I built a tool that creates movie recap videos automatically using local models. The problem: making recap videos takes forever.
-
Roundtable chat with Talkie-1930 and Gemma 4 31B (www.reddit.com) gemma 27w
Talkie-1930-13b-it and Gemma 4 31b in the same chat. Talkie is a 13B vintage language model from 1930.
-
it's time to update your Gemma 4 GGUFs (www.reddit.com) gemma 28w
Chat Template was fixed a few days ago choose your fav dealer: https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF https://huggingface.co/bartowski/google_gemma-4-26B-A4B-it-GGUF https://huggingface.co/bartowski/google_gemma-4-E4B-…
-
interacting with gemma 4 w/ live video and audio (www.reddit.com) gemma 28w
I saw someone on this forum demonstrate using gemma 4 - live streaming audio and video from his webcam to it asking it what it was seeing. It was pretty great but I cant find that post anymore and I can't find a good repo on github where I…
-
Anybody tried openclaw + M5 pro + 48gb? (www.reddit.com) gemmaopenclawopenai 28w
Hello, posting again on this since my last post was removed. I am working on an AI agent solution to help me with my multiple daily tasks for different business activities; a few rental properties, a manufacturer trying to enter the Mexico…
-
Is 2x5070Ti a good setup? (www.reddit.com) gemmaqwen 28w
I'm confused about what to get. I don't want to get something super expensive, but would like to have something that's "good enough" for coding etc.
-
Been running Gemma 4 E2B locally on my OnePlus CE 5 (8GB RAM) for a few months. Chat quality is fine for the size.
-
Anyone tried +- 100B models locally with foreign languages? (www.reddit.com) glmgemmaqwen 28w
I am quite curious as I tried Gemma 4 31B, Qwen 3.6 27B, GLM 4.7 30B and some others in my native language (czech). Gemma performs "best" and considering the fact its "just" 18GB model - it actually blows my mind how well it can respond in…
-
Show HN: Llmconfig – configfile and CLI for local LLM (github.com via hn) gemmallama 28w
llmconfig Local Large Model Config — manage local inference with llama.cpp, stable-diffusion.cpp, and whisper.cpp from a single YAML file and a single CLI. llmconfig up gemma # or just: llmc up gemma ✓ gemma is ready at http://127.0.0.1:80…
-
Open Weights Models Hall of Fame (www.reddit.com) mistralgemmallama+1 28w
I read a lot of "whengguf" type posts. I think we should sometimes stop and be grateful.
-
I made a visualizer for Hugging Face models (www.reddit.com) gemmaqwen 28w
I built hfviewer.com, a small tool for visually exploring Hugging Face model architectures. You can paste a Hugging Face URL and get an interactive visualization of the architecture, which can make it easier to understand how different mod…
-
I've been experimenting with using Ollama to run Claude Code locally with models like Gemma 4, thinking I could avoid API costs. However, I quickly realised these models aren't really optimised for Claude Code's agentic workflows — they te…
-
Hey guys, A couple of weeks ago, I asked this sub for the hardest Vision use cases you were dealing with to test the newly dropped Qwen 3.6 against Gemma 4. I finally finished running the gauntlet side-by-side locally on vLLM (FP8 quants)…
-
What are some good use cases for Gemma Embedding 2? (www.reddit.com) gemma 28w
Does anyone know of any use cases of Gemma Embedding 2? Or is it solely for search?
-
It is pluggin made for ONLYOFFICE, much simpler than copy-paste from webui. PS.
-
Which model for 32GB M2 Max? (www.reddit.com) gemmaopenclawdeepseek+2 28w
I would like to experiment but before investing loads of money, I do have a MacBook Pro with 32GB RAM, M2 Pro. Which model would maximize versatility given this hardware?
-
Been using Qwen-3.6-27B-q8_k_xl + VSCode + RTX 6000 Pro As Daily Driver (www.reddit.com) gemmacopilotqwen 28w
So in response to the Great Token Reconning of 2026, I decided to try out Qwen 3.6 as a daily driver, and although it's only been about a day, I have to say I'm thoroughly impressed. I had to download the VSCode insiders edition and set up…
-
gemma-4-31B-it-DFlash has been released (www.reddit.com) gemmallama 28w
https://huggingface.co/z-lab/gemma-4-31B-it-DFlash I guess we'll have to wait until this PR is merged before we can test it. https://github.com/ggml-org/llama.cpp/pull/22105
-
We run Qwen3.6-27B-FP8 at AI Router Switzerland and hit two issues, so I wanted to share in case anyone else runs into them. FP8 KV cache produces silent garbage output with radix cache prefix hits (PR #24198 — ✅ approved) We were running…
-
I experienced this with Q4 and Q3 versions of Qwen3.6-35B-A3B and Gemma-4-26B-A4B. It starts saying things which sound similar in thinking mode: I must do ....
-
1990s. I was the kid with the dial-up sound burned into my brain.
-
I hate this group but not literally (www.reddit.com) minimaxgemmadeepseek+1 28w
True story, I got interested in AI after seeing it at work and wanted to run models locally. I started with an M3 Ultra 96GB, quickly learned it was not enough for what I wanted, and kept upgrading hardware (including refurbished Mac Studi…
-
I'm testing running local LLMs on a gaming mini PC (AMD 7840HS, 32 GB RAM) paired with an eGPU (Radeon 9060XT with 16 GB VRAM). Since I'm not very familiar with using llama.cpp, I kept getting unsatisfactory results, but with the recent Ge…
-
Running llama.cpp on Snapdragon Hexagon NPU seems promising (www.reddit.com) gemmallama 28w
https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/snapdragon/README.md I have an Oneplus 12 with Snapdragon 8 Gen 3. I followed the above README to cross-compile llama.cpp on Ubuntu and then copy to the Termux directory on the…
-
Hello folks What is best code editor for local LLM deployment (LM Studio, llama.cpp)? I wish to test my LM studio + Qwen 3.6 27B and Gemma 4 31B with a legit local code editor.
-
Qwen 3.6 27B vs Gemma 4 31B - making Packman game! (www.reddit.com) gemmaqwen 28w
Gemma just crushed Qwen in a local LLM gamedev contest! Device: MacBook Pro M5 Max, 64GB RAM Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens.
-
RPers: how do the new Gemma and Qwen compare to the old 70B models? (www.reddit.com) gemmaqwen 28w
I can’t really run 70B models on my current setup, but I’m curious haha
-
I've built a system where models like Llama 3, Qwen, and Gemma play Pokémon Showdown battles autonomously. Instead of simple prompt-response, they analyze the full battle state every turn (type matchups, HP, weather, field conditions, reve…
-
Qwen 3.6 and Gemma 4 "Zombie Loops" (terminal thinking loops) (www.reddit.com) gemmaqwen 28w
I've got to the point where I need some help. I'm trying to run Qwen 3.6, and it will eventually fall into a loop where it's just outputting "/" symbols when it's "thinking".
-
Five labs, one suite, do model families have personalities? (benchmark) (www.reddit.com) ollamagemmamoe+1 28w
Bench 3 from my 18GB M3 Pro. Bench 2 was the 4B-class post where the comments were mostly right: I gave thinking models a fixed 1024-token cap, Qwen got kneecapped, Gemma E4B needed clearer active-param labeling, and the headline was partl…
-
Are Qwen 3.6 27B and 35B making other ~30B models obsolete? (www.reddit.com) gemmaqwen 28w
Have Qwen 3.6 27B and Qwen 3.6 35B basically made most of the older ~30B models irrelevant? They seem to beat stuff like Qwen coder 30B, GPT OSS 20B, Gemma models, especially for coding and agent workflows.
-
12GB-Club: 4070S qwen3.6 27b + 35b a3b, and Gemma 4 26b a4b + 31b speeds (www.reddit.com) clinegemma 28w
Longtime lurker here, thought i should post my speeeeds... I have a RTX 4070S 12 GB Vram (+10% OC), AMD 9800x3D with 4x16 Gb DDR5 6000Mhz CL30.
-
thinking of gemma 4 26B vs 31B (www.reddit.com) vllmgemmamoe+2 28w
I see a big difference in agentic coding between gemma-4-31B-it-Q5_K_M and gemma-4-26B-A4B-it-UD-Q8_K_XL. The 26B model is much faster because of A4B and generally works well, but there is a big difference in thinking.
-
Basically, I’m really into the idea of a fully offline setup. (Another way to say it: I’m a data hoarder.) For LLMs, I’m using uncensored models from both Western (Gemma, GPT-OSS) and Eastern ones (GLM 4.7 Flash, Qwen 35B).
-
Comparing SVG Generation for the top open models (codeinput.com via reddit) minimaxglmgemma+2 28w
Some of the larger models (like Llama) weren't available on OpenRouter, so I had to work with what was there. Best small model: Gemma 4 26B For its size, I think it had the best output.
-
Based on what should I choose Gemma 4 models/quantizations? (www.reddit.com) gemmageminichatgpt 28w
I have an RTX 4060 8GB(+16GB RAM) laptop, and when asking Gemini or ChatGPT, they say the Gemma 4 Q4 K M is the best fit for my hardware with Context Length around 16k-32k. However, in practice, after loading even a higher quantization lik…
-
Larger Gemma-4/Qwen3.6 (www.reddit.com) gemmamoe 28w
Qwen3.5-122B-A10B at Q6_K is really good. Do you think we will see a larger MoE Gemma-4 or Qwen3.6 at some point?
-
I’ve been building a local-first AI runtime that wraps local LLMs with a σ-gate — a measurement layer that decides ACCEPT, RETHINK, or ABSTAIN before an answer reaches you. The idea: local models should be able to say “I don’t know” instea…
-
With the release of Gemma 4 models and a slew of open weight/source models subsequently, some of the workflows like drafting emails/ trivial coding tasks have become possible. I’m exploring the possibility of integrating some of the powerf…
-
Gemma 4 architecture support for QVAC-Fabric (Tether's llama.cpp fork) (github.com via hn) gemmallama 28w
QVAC-Fabric Gemma 4 Architecture Patch Adds full Gemma 4 (gemma4) architecture support to QVAC-Fabric, Tether's llama.cpp fork. Base: QVAC-Fabric temp-upstream branch Target: All Gemma 4 variants (E2B, E4B, etc.
-
TurboQuant enabled Runtime Valkyr (www.reddit.com) gemma 28w
Based on the recent TRiP source code by Carlo Valenti. Ported to Zig and headless Vulkan Compute shaders.
-
I've been running local LLMs since Qwen 3.5 dropped and I was really impressed by what we could run on consumer hardware. Fast forward another two months and we have gotten a handful more gems such as Gemma 4 and Qwen 3.6, so I wanted to p…
-
Llama.cpp MIPS R8000 Kernel Running on an SGI Power Challenge from 1995 (twitter.com via hn) gemmallama 28w
Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M.
-
What it feels like to have to have Qwen 3.6 or Gemma 4 running locally (www.reddit.com) mistralgemmaqwen 28w
Well or pretty close to it, they are excellent work horses. I run them in real work scenarios doing some of the work I used to do myself as an skilled expert in my field, billing 200$ an hour.
-
Uhh I guess Gemma 4 is so much shittier that it hallucinated this event that happened in china in 1989? According to qwen, nothing of significance happened at Tiananmen square in 1989 - and based on all of the benchmarks of qwen, I believe…
-
I stumbled on a Gemma 4 chat template bug for tools and fixed it (www.reddit.com) gemmagpt-5llama+1 28w
TLDR: tool parameters using the common JSON Schema pattern `anyOf: [$ref, null]` are rendered into the prompt as empty `type` fields. This strips the useful schema information before the model sees it.
-
llama.cpp's Preliminary SM120 Native NVFP4 MMQ Is Merged (www.reddit.com) gemmallama 28w
https://github.com/ggml-org/llama.cpp/pull/22196 And somehow we already got some GGUFs for it! https://huggingface.co/CISCai/gemma-4-31B-it-NVFP4-turbo-GGUF https://huggingface.co/stevelikesrhino/gemma-4-31B-it-nvfp4-GGUF (the below one is…
-
great work, Gemma (www.reddit.com) gemma 28w
another day with pi + gemma 26B
-
https://huggingface.co/firstbober/gemma-3-270M-it-smol-thinker Here is an example of the output: ``` ==================== THINKING ==================== Here is the thinking process: This is a large community with a wide range of interests…
-
Gemma-4 MLX reasoning? (www.reddit.com) gemma 28w
Gemma-4 is great. On a MacBook M5, using lm-studio, the MLX versions (specifically looking at https://huggingface.co/lmstudio-community/gemma-4-26B-A4B-it-MLX-8bit) rock.
-
Is long re-processing of output as input a common "feature" or not? (www.reddit.com) gemmallamaqwen 28w
I now use (mostly) Gemma 4 and Qwen 3.5 models *. And seems that all of them, after context grows a bit, after providing long output for me and getting a short prompt in response, are starting to process many new tokens as input and I have…
-
TLDR: I've been running gemma4 e2b extensively on iOS with llama.cpp and found some interesting quirks and info you guys may like! These are specifics for the iPhone and what I've found worked across 20+ devices.
-
Did some quick tests after building llama.cpp with ROCm 6.4.2 and latest Vulkan for my 6900 XT gemma4 E2B Q4_K ubatch ROCm pp512 Vulkan pp512 ROCm tg128 Vulkan tg128 32 1536.60 1423.49 151.92 174.59 64 1590.65 1930.60 151.41 173.76 128 265…
-
The gemma 4 E4B and E2B models have built-in multimodal capabilities. However, as far as I am aware, llama.cpp does not have proper support for vision and audio inputs (specially audio) for these models as of now.
-
For Non-hallucinating work, MiMo 2.5 delivers (www.reddit.com) gemmahallucinationdeepseek+1 28w
MIT license and fully open source. MiMo-V2.5-Pro was just 3 points from Opus 4.7 max and the normal V2.5 is only a step behind SOTA.
-
Is mlx-optiq legit? Has anyone tested the new quants for Gemma4/qwen3.6 yet? (www.reddit.com) gemma 28w
https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit https://huggingface.co/mlx-community/Qwen3.6-27B-OptiQ-4bit https://huggingface.co/mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit https://huggingface.co/mlx-community/gemma-4-31B…
-
AMG GPUs are faster at pre filling (www.reddit.com) gemma 28w
I did give same prompt same document to 1660ti running Gemma 4 e2b q4 coz of the small vram and another to and igpu running Gemma 4 e4b q8 prefill rate before token generation was like 4-5 times faster with the 890m igpu then token generat…
-
How to run a local coding agent with Gemma 4 and Pi | Patrick Loeber (patloeber.com via reddit) gemmallama 28w
Tutorial from the Google guy, I use very similar setup (llama.cpp instead of lmstudio)
-
Using Google's Gemma 4 E4B Local AI Model to Reverse Engineer a Simple Crackme (github.com via hn) gemma 29w
Using Google's Gemma 4 E4B local AI model to Reverse Engineer a simple Crackme I was playing around with the new Gemma E4B open weights local model which Google released, and to my surprise I was seeing a great deal of success in using it…
-
Why are there so few small local creative writing models from the Chinese? (www.reddit.com) mistralgemmallama+1 29w
At this moment, the models such as Qwen 3.6 35b/27b crush the competition, yet I can't help, but notice this pattern. While the local RP scene is abundant with the Western model tunes: LLaMA, Mistral (all sizes), Nemo and more recently Gem…
-
Running Gemma 4 31B on Mac with Ollama (sammyrulez.github.io via hn) ollamagemma 29w
A practical configuration for a 32 GB M5 Mac that still needs to remain usable Running large language models locally has become surprisingly practical on Apple Silicon. With a modern Mac, Ollama, and a carefully quantized GGUF model, it is…
-
Ask HN: Will local models on normal hardware ever compete? (news.ycombinator.com) gemmachatgptopus 29w
I have a Macbook Air M3 with 24gb RAM. The other day, I wanted to try running an LLM locally for the first time ever.
-
Good LLM to generate ascii art? (www.reddit.com) gemmaqwen 29w
I tried with Qwen but it sucked, Gemma3/4 was better but not good enough. From Gemma: https://pastebin.com/raw/Qr5iMgYj Still looks like a bloody car accident though.
-
Best sota 12b-32b creative writing model? (www.reddit.com) gemma 29w
I love using openrouter but I would also love a smaller model that can fit within 16gb of VRAM and 64b of ram, that can pack a punch for its size specifically in the creative writing section. Any good recommendations?
-
A weekend with LoRA on Gemma 4 E2B: instrumenting what fine-tuning changes (aiexplr.com via hn) gemmafine-tuning 29w
Spent a week doing LoRA fine-tuning on Gemma 4 E2B (~5.1B total params, ~2B active in text decoder) for a narrow Python code-generation task. Bad outputs went from ~5% to 0% (greedy) and 1.5% (sampled) across 134 tests.
-
So for my project I was using up until now either Gemini 3 / 2.5 Flash or Flash-lite. All my use cases are not agentic, simply LLM workflows for atomic tasks like extracting references from the law, classifying, adjusting titles to nominat…
-
(Gemma/Qwen + Codex) - Bridging /chat/completions → /responses in llama-swap (www.reddit.com) gemmallamaqwen+2 29w
I’ve been tinkering with a small side project (just for fun) where I’m trying to extend llama-swap with a bridge from /chat/completions to the newer /responses API so I can run the latest Gemma and Qwen models together with Codex-style too…
-
Pocket LLM v1.5.0🚀 New in this release: - 🎙️ Voice input - 🖼️ Image input with OCR, Gemma vision, and FastVLM support - 📷 Camera capture with retake, crop, and photo review - 🗂️ Previous chats side panel - 💾 Downloaded model deletion to sa…
-
As a life-long Windows user (don't hate me, I was exposed to it at a young age) I was wondering how much (if any) performance I'm leaving on the table. So I did the sensible thing and run some benchmarks.
-
Best settings for gemma-4 on a 3090? (www.reddit.com) gemma 29w
3090 (24G) + 32G DDR4 Currently running --mmproj mmproj-BF16.gguf --chat-template-kwargs '{"enable_thinking":true}' \ --flash-attn on \ --cache-type-k q4_0 \ --cache-type-v q4_0 \ -np 1 \ -c 160000 \ --jinja at 26B-A4B-it-UD-Q5_K_XL and ge…
-
Claude Cowork Now Runs Any LLM. Test It Free (www.productcompass.pm via hn) gemmacoworkopenai 29w
OpenAI, Gemma, Kimi K2, or run locally. Free via OpenRouter.
-
Spent a week doing LoRA fine-tuning on Gemma 4 E2B (gemma-4-e2b-it, ~5.1B total params, ~2B active in the text decoder) for a narrow Python code-generation task. Setup: Model: Gemma 4 E2B, bf16, language_model only (vision + audio towers f…
-
Were Qwen3.6 models scrubbed from openrouter? (www.reddit.com) gemmamoeqwen 29w
I made a simple app using openrouter, hoping to use the new small qwen models (the a3b moe and the 27b dense one), but they aren’t listed. Also, I swear some qwen3.6 models that were listed before are missing now.
-
I wanted to figure out which of the newer small and mid-size models are actually worth running on a single H100, so I put 8 of them through a proper vLLM benchmark and recorded what came out. The setup was simple.
-
Qwen3.6-35B-A3B-UD-IQ4_XS C++ to Rust Code Port Test: It Worked (Mostly)! (www.reddit.com) gemmaqwen 29w
When Qwen3.6-35B-A3B was released a week or so ago, I sort of expected an iterative improvement on the previous Qwen3.5 models. After all, those models were pretty decent as compared with the previous local models I had tried, and Qwen3.5…
-
I'm glad we have deepseek (www.reddit.com) minimaxglmgemma+2 29w
other companies are slowly going away from open weight, not releasing base models, delaying open weight distribution, not releasing top models (this one I think is fair, but still), and I also noticed they stopped publishing research (old…
-
Deepseek flash seems like a very good replacement for Haiku at the very least (www.reddit.com) haikugemmadeepseek+1 29w
We have a chat system which we use haiku for because it is mostly about tool calling and summarisation of them. But we have many tools with pretty complex input schemas, and stuff like gemma didn't cut it, so we went with haiku.
-
Gemma 4 and Qwen 3.6 with q8_0 and q4_0 KV cache: KL divergence results (localbench.substack.com via reddit) gemmaqwen 29w
Gemma 4 and Qwen 3.6 with q8_0 and q4_0 KV cache: KL divergence results 4 models tested with q8_0 and q4_0 KV cache against full-precision baseline What this measures KV cache quantization stores the key-value cache in lower precision to s…
-
Hey guys, I built a custom vLLM pipeline to run Gemma 4 (31B FP8) and Qwen 3.5 side-by-side locally to see how they actually perform in the wild with preprocessing of audio and images. But of course new model Qwen 3.6 27B came out just whe…
-
What are your favorite LLMs for translation/docuement work? (www.reddit.com) vllmgemma 29w
I am currently working on a system to translate books/web novels. I got a working prototype, but now I am looking into optimizing it.
-
Windows freezing up as VRAM fills up - Does this happen for everyone? (www.reddit.com) gemma 29w
Hey everyone, I run llamacpp precompiled with CUDA 12.4 on Windows 11 with a RTX 4090. With small models like gemma-4-E4B everything runs fine, but as soon as I run a bigger model like Qwen3.6-27B (IQ4_NL) or a medium sized model with larg…
-
Hi guys, I’ve been running side-by-side experiments on Gemma 4 (31B FP8) and Qwen 3.5 Vision for the last few days using vLLM in Docker to see how they actually handle real-world images and video. A few things I found out: 1.
-
Always been stuck with models that fit on my 16gb .... Going to have about a week for free with 4x rtx6000pro .
-
Has anyone managed to use gemma 4 e4b in Open Code/other agentic TUIs? (www.reddit.com) aidergemmaagentic+1 29w
Hi everyone, as a power user I hit Claude Code's usage cap too often I wanted to set up my own local model, however I only have RTX 5070 with 12 GB of VRAM so the only realistic option was Gemma 4 with effective 4B params. When I tried to…
-
Gemma 4 VLA Demo on Jetson Orin Nano Super (huggingface.co) gemma 29w
Gemma 4 VLA Demo on Jetson Orin Nano Super You speak → Parakeet STT → Gemma 4 → [Webcam if needed] → Kokoro TTS → Speaker Press SPACE to record, SPACE again to stop. This is a simple VLA: the model decides on its own whether to act based o…
-
It is worth an RTX 3090 for 850 if you can a radeon 7900 XTX for 495? (www.reddit.com) gemmallamaqwen 29w
Both amounts are in euro. The AMD is actually 599 but it's sold by a shop, so I can get a VAT return as a company, while for the nvidia I'd have to go to the second hand market and I can't get VAT back, so at the end it's like a 495 vs 850…
-
Hi LocalLLaMA, I created a post a few weeks ago, but this time this project has become more reliable and easier to use. This is a manga translator that can also be used to translate any image.
-
Recent Open models from last 6 Months - Nov 2025 - Apr 2026 (www.reddit.com) mistralglmgemma+1 29w
I created this chart with recent open models from last 6 months. Few might be older than that possibly.
-
Gemma 4 is not your standard transformer (idlemachines.co.uk via hn) gemmamoe 29w
Gemma 4 makes five quiet departures from the standard transformer recipe. QK-norm instead of 1/√d, partial RoPE on global layers, per-layer input gating, KV sharing across layers, and an MoE that sits alongside the MLP rather than replacin…
-
I have run two tests on each LLM with OpenCode to check their basic readiness and convenience: - Create IndexNow CLI in Golang (Easy Task) and - Create Migration Map for a website following SiteStructure Strategy. (Complex Task) Tested Qwe…
-
Pioneer: Vibetune Your LLMs (pioneer.ai via hn) gemmafine-tuning 29w
+30% avg accuracy lift on classification & extraction tasks vs. base Gemma ~7 days until your first auto-improvement run lands in production 0 lines of fine-tuning code you have to write, ever $0/retrain starting price.
-
My old Samsung S10 was sitting in a drawer so I turned it into an always-on LLM endpoint. PocketPal is great for on-phone chat, but I wanted the phone itself to be an OpenAI-compatible endpoint for the rest of my network.
-
I gave 9 local models the same flight combat sim prompt. The results broke a few of my assumptions about quant providers and parameter count.
-
How do I run Gemma 4 e4b, extracted via adb from Google AI Edge Gallery on Android, whose image is in litertlm format and weighs 3.6 gigs, in a browser? I mean using web technologies?
-
Gemma 4 is much less popular on Hugging Face than Qwen 3.x. (www.reddit.com) gemmaqwen 29w
The difference is quite big: likes downloads last month finetunes Qwen3.5-27B 952 3,233,034 263 Qwen3.5-35B-A3B 1,397 3,977,637 87 Qwen3.6-35B-A3B 1,115 458,436 60 gemma-4-31B 323 343,895 13 gemma-4-26B-A4B 227 118,464 13
-
Why does Gemma 4 e4b from Google AI Edge Gallery on Android weigh only 3.6 gigs, while the one from Unsloth (gemma-4-E4B-it-UD-Q2_K_XL.gguf) weighs 3.7, and for some reason the model image in litertlm format extracted via adb from Google A…
-
I was building a dedicated-vision-model feature for an open-source browser agent and wanted to figure out which local model to actually recommend. Wrote a small probe that sends the same image + same system prompt + same params (temperatur…
-
-
Gemma 4 E4B is broken (www.reddit.com) gemma 29w
-
Lekh AI iOS v7.0 is Live – Bonsai 8B & Gemma 4 + Lower Memory Image Gen (www.reddit.com) gemma 29w
-
Handling a large amount of files (www.reddit.com) gemma 29w
-
-
Which Gemma model do you want next? (www.reddit.com) gemma 29w
-
-
-
gemma4 vs qwen3.5 122A10 real usages (www.reddit.com) gemmaqwenagentic 29w
-
Gemm4:e4B-IT good at instructions following no refusals. (www.reddit.com) ollamagemma 29w
-
LLM Neuroanatomy III - LLMs seem to think in geometry, not language (www.reddit.com) gemma 30w
-
5070 Ti (New) vs 3090 (Used) to pair with 4070 for local LLMs? (www.reddit.com) gemmamoeqwen 30w
-
Deploying Gemma 4 26B on an RTX 5090 (datapnt.com via hn) gemma 30w
-
-
Why model(s) input often includes last output? (www.reddit.com) gemmaqwen 30w
-
Show HN: Prompt-to-Excalidraw demo with Gemma 4 E2B in the browser (3.1GB) (teamchong.github.io via hn) gemma 30w
-
How is Rotorquant/planarquant/iso qaunt better? (www.reddit.com) gemmallamaqwen 30w
-
Gemma 4 - MLX doesn't seem better than GGUF (www.reddit.com) gemma 30w
-
Qwen3-30B-A3B-Instruct-2507 is better than the new Qwen 3.6 for our tasks (www.reddit.com) vllmgemmamoe+1 30w
-
localLLamA playground (www.reddit.com) gemma 30w
-
Gpu reccommendations for Coding/chat LLM (www.reddit.com) gemma 30w
-
How do I get the LLM to answer everything? (www.reddit.com) gemma 30w
-
I tested 8 LLMs as tabletop GMs - a 27B model beat the 405B on narrative quality (www.reddit.com) gemmaqwenagentic+1 30w
-
-
Ask HN: How do you use Local LLMs? (April 2026) (news.ycombinator.com) gemmaqwengemini+1 30w
-
Keinsaas Navigator + LM Studio + Geforce RTX 5080 (www.reddit.com) gemma 30w
-
I have Gemma4-E2B working within home assistant as STT, and E2B seems fast and accurate for STT (maybe a bit better than Parakeet), however, it responds with the entire thought process: https://preview.redd.it/v8zhb5elltvg1.png?width=599&f…
-
So the question I've seen posed many times in /r/singularity is if the Gemini models are actually that bad at coding compared to their benchmarks, or whether the harness used makes an absolutely gigantic difference in model performance. Gi…
-
Hardware: Intel Core Ultra 7 258V, 32GB Unified Memory. Model: Qwen 3.6 35B A3B (Quant: Q3_K_S) via LM Studio.
-
Qwen 3.6 35B crushes Gemma 4 26B on my tests (www.reddit.com) gemmaqwenagentic 30w
I have a personal eval harness: A repo with around 30k lines of code that has 37 intentional issues for LLMs to debug and address through an agentic setup (I use OpenCode) A subset of the harness also has the LLM extract key information fr…
-
Qwen3.6-35B-A3B just dropped — quick thoughts after trying it (www.reddit.com) gemmamoe 30w
Just gave the new Qwen3.6-35B-A3B a spin. It’s a MoE model (35B total, ~3B active), but honestly the more interesting part is how much they’re pushing agent-style coding.
-
Qwen3.6 is incredible with OpenCode! (www.reddit.com) gemmaopusclaude-code 30w
I've tried a few different local models in the past (gemma 4 being the latest), but none of them felt as good as this. (Or maybe I just didn't give them a proper chance, you guys let me know).
-
NVIDIA V100 32GB for AI in 2026 (www.reddit.com) gemmaqwenagentic 30w
hello. i have the oportunity of buying Nvidia V100 with 32GB for about 915$ / 775 euro.
-
Will Gemma 4 replace Claude Code or are we lying to ourselves again (webmatrices.com via hn) gemmaclaude-code 30w
Vercel Security Checkpoint | cle1::1776468758-lOAcIwtVVUa8cG9OLlcTtnlZwlvTxsBe
-
Processing img 8ofni1q6dpvg1... Hello everyone, I’m building an offline RAG system for my company, we are trying to run an app that retrieves infromation from two manuals in an android tablet with the idea of an AI to provide precise answe…
-
Escaping model lock-in (www.reddit.com) gpt-4gemma 30w
I have observed that many ai teams try to always use the best model to ensure quality. When a new model drops out, they are forced to pay for it, because their competitors will.
-
Bonsai models are pure hype: Bonsai-8B is MUCH dumber than Gemma-4-E2B (www.reddit.com) gemmallama 30w
I'm using the https://github.com/PrismML-Eng/llama.cpp fork for Bonsai, regular llama.cpp for Gemma. Without embedding parameters: Gemma 4 has 2.3B at 4.8 bpw (Q4_K_M) = 1104 MB Bonsai-8B has 6.95B at 1.125 bpw (Q1_0) = 782 MB (only 29% sm…
-
Want your LLM to use the internet? Here's an MCP server for that. (www.reddit.com) gemmamcp 30w
The showcased examples were made using Gemma 4 31b. Any LLM with tool calling support should work.
-
My thought on Qwen and Gemma (www.reddit.com) gemmamoeqwen 30w
This spring is really hot since the localLLM giant, both Qwen and Gemma released major models. I'm really excited with those release and happy with their capability.
-
For 36gb vram, Gemma 4 or Qwen3.5 ? (www.reddit.com) gemmaopenclaw 30w
I have 3090ti and i will add 3080ti to my system soon. With 3090ti only, i found it little bit slow to run gemma 4 26b 4q.
-
could not extract summary
-
MB Pro M5, 24GB/32GB difference? (www.reddit.com) ollamagemmacopilot 30w
Hi, I got new MB Pro 24GB/1TB. I've test Gemma 4 26B with ollama, 16k context.
-
Anyone feel like Qwen3.6 thinks like Gemma 4? And not in a good way. (www.reddit.com) gemmallamaqwen 30w
I was disappointed with Gemma 4 due to various bugs and in the end lackluster performance for the internet research/information synthesis type tasks I use local AI for. Even after every last fix and update of both mode quants and llama.cpp…
-
Strix Halo 128GB on Proxmox - Vulkan vs ROCm benchmark matrix (www.reddit.com) minimaxgemmallama 30w
Ryzen AI MAX+ 395, Bosgame M5, 128GB LPDDR5x. Proxmox VE 9.1 LXC containers with GPU passthrough.
-
Frontier Coding Agents Built a Video Diffusion Pipeline on Max (www.modular.com via hn) gemma 30w
Gemma 4 just dropped on Modular, Day Zero! Read More → Inference Products Shared Endpoints Access frontier models via an API Dedicated Endpoints Mission critical reliability Custom models Your model, peak performance Deployment Options Our…
-
Where my Gemma 4 gets this data? Trying to explain weird behaviour. Please help! (www.reddit.com) gemma 30w
https://preview.redd.it/w6ssjgidjlvg1.png?width=2786&format=png&auto=webp&s=f52736d40580fe8a8ff74adbbb5be81f12fbcbfc So I was playing with Gemma 4 and was trying to figure out whether the model could determine its own training data cutoff…
-
Fine-tuning and deploying Gemma 4 is not that easy (ghost.oxen.ai via hn) gemmafine-tuning 30w
Writing a fine-tuning and deployment pipeline isn't as easy as it looks (Gemma 4 Version) Fine-tune and deploy Gemma 4 on Oxen.ai Google's Gemma 4 dropped in April 2026 with multimodal support (text, image, video, audio), a novel hybrid KV…
-
My Qwen 3.6 fails the car wash vibe check (www.reddit.com) gemmaqwenclaude-code 30w
I configured it to the best of my abilities, even at Q8. It fails to give the correct number of tools it supports on Claude Code and it fails the car wash test.
-
I just launched ios app that uses Gemma 4 (E2B 4-bit via mlx-community) to rewrite oral transcripts into heirloom-quality paragraphs, 100% offline. What made this interesting technically: MLX Swift + MLXLLM in production (not a demo) — fir…
-
Gemma 4-written, small cc0 encyclopedia of some core science content (stateofutopia.com via hn) gemma 30w
Published: April 16, 2026 This is an encyclopedia of some core content from Biology and Health Sciences, Physical Sciences, and Technology. It contains 2,259 small entries of about a paragraph each.
-
Knlowledge Graph and hybrid DB (www.reddit.com) ollamagemma 30w
Hello, everybody! I'm building and hybrid database with Qdrant and Neo4j for a few personal projects.
-
Google and Alibaba recently shipped Gemma 4 and Qwen3.5, so I wanted to see whether the new generations are actually better on my setup. My context is private local chat running on my own hardware, a Mac mini M4 Pro.
-
Local Coding Stacks (www.reddit.com) gemmaqwensonnet 30w
I’m trying to reduce my reliance on Claude. I have a 5090/128GB RAM.
-
Gemma 4 31b 3D geometry (www.reddit.com) gemmasonnetgemini+1 30w
I have been nothing but impressed by the quality of Gemma 4 since release. In general conversation it's adaptable to different personas.
-
Feedback on iOS app with local AI models (www.reddit.com) gemmallamadeepseek+1 30w
Hey everyone, I just shipped an iOS app that runs local AI models. Current has 12 models: Gemma 4, Llama 3.3, Qwen3, DeepSeek R1 Distill, Phi-4, etc.
-
why gemma 4 31b so bad in long context? (www.reddit.com) gemma 30w
question, I'm using it for text translations and on each large prompt (20K+) it stops with a remark 'now I'm going to put that to the file' or some other operation I have asked in the prompt for but it did nothing, just stopped. I'm runnin…
-
LiteRT LM Framework with Rockchip NPU (RKNN 3588) (www.reddit.com) gemmallama 30w
Im searching for build version of LiteRT LM framework can use and utilize the NPU of the RKNN 3588. It would be great since I can run gemma 4 e2b model using this framework on the machine, because I wont have to migrate my codebase from li…
-
Need suggestions for local AI Machine (www.reddit.com) minimaxgemmaopenclaw+1 30w
I’ve been running various AI harnesses like OpenClaw, ForgeCode, ClaudeCode, etc. Most of these are running via OpenRouter or Minimax (credits/subscription model).
-
GGUF Quants Arena for MMLU (24GB VRAM + 128GB RAM) (www.reddit.com) mmlugemmaopus 30w
Dataset: MMLU subset (DEV+TEST) Llamacpp setting: 3 params only ctx 8192 , seed 42 , fa on Let me know whatelse do you want to see. Thanks.
-
gemma4 e2b ore4b on rtx 5070 ti laptop 12GB not running on vLLM (www.reddit.com) vllmgemma 30w
I cant get gemma 4 e2b or gemma 4 e4b to run on my laptop. I am runnning it via docker as per vllm website and i get the error : Free memory on device cuda:0 (9.71/11.5 GiB) on startup is less than desired GPU memory utilization (0.9, 10.3…
- gemma4 e4b on rtx 5070 ti laptop 12GB running slow 5t/s llama.cpp (www.reddit.com)
-
How faster is Gemma 4 26B-A4B during inference vs 31B? (www.reddit.com) gemmamoellama+1 30w
I want to download one and usually do inference on CPU having old GPU so I'm concerned with speed. One link on the web (I have posted with it and post been removed): Multiple users are reporting that Gemma 4's MoE model (26B-A4B) runs sign…
-
Turn an old Android phone into a Local AI Voice Assistant (www.reddit.com) gemmallama 30w
I had a nice old cracked pixel 5a laying around that I wanted to get some use out of, so I turned it into a local AI Voice assistant. A server on a laptop running llama.cpp gemma-3-4b-q4.gguf served by flask connects to a script running on…
-
5090 for 285k on amazon india? (amzn.in via reddit) gemma 30w
How is it possible the seller also has no record just wanted to run gemma 4 31B q4 with 150k ctx
-
What's your favorite small-medium local model? (www.reddit.com) gemma 30w
I'm now having fun with Gemma-4-E4B and Qwen3.5-9B, trying different variants like Gemopus and Qwopus, and Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q8_0 don't quite know other models, so what's your favorite? why and how are them?
-
I try with Gemma 4 E4B via llama-sever to play chess at https://www.chess.com/play/computer (any platform or site you convenient), result quite unexpected for me. Result: 9 moves before it make cheating move (like try to move a pawn take a…
-
Hey all, Looking for some real-world advice on GPU choices for running the new dense models — mainly Qwen 3.5 27B and Gemma 4 31B. What I’m targeting Context: 64K+ (ideally higher later) Speed: 30+ tok/s @ tg128 minimum Power: not critical…
-
Gemma 4 on iOS: Anyone else stuck on CPU because of the “Buffer(31) Metal Crash? (www.reddit.com) gemma 30w
Gemma 4 on iOS: Anyone else stuck on CPU because of the "Buffer(31)" Metal crash? Hey everyone, I’m hitting a massive performance wall building an on-device AI app for the iPhone 17 Pro.
-
Gemma4 26b & E4B are crazy good, and replaced Qwen for me! (www.reddit.com) gemmallamaqwen+1 30w
My pre-gemma 4 setup was as follows: Llama-swap, open-webui, and Claude code router on 2 RTX 3090s + 1 P40 (My third 3090 died, RIP) and 128gb of system memory Qwen 3.5 4B for semantic routing to the following models, with n_cpu_moe where…
-
Thinking issue [Qwen3.5] (www.reddit.com) gemma 30w
I've been testing a few models lately and I'm running into a weird issue with the bigger Qwen3.5s. Tested: Gemma 4 26B Qwen3.5 9B Qwen3.5 27B Qwen3.5 35B The 27B and 35B are driving me nuts.
-
Hello, I have noticed an annoying issue with Gemma 4 26b a4b. It seems like it cannot do multiple think->tool call->think->tool call turns.
-
Minimax M2.7 on Q3_K_S or Smaller Model with greater precision? (www.reddit.com) minimaxgemmaopenclaw 30w
I currently am looking for models to fit into my single DGX Spark for use. I have an RTX Pro 6000 and also a 5090 as well that I'm considering using in combination if the DGX Spark is too slow, but the intent here is to play around with Op…
-
CPUs Aren't Dead. Gemma2B Out Scored GPT-3.5 Turbo on Test That Made It Famous (seqpu.com via hn) gemma 30w
CPUs Aren't Dead. Gemma 2B Just Scored Higher Than GPT-3.5 Turbo on the Test That Made It Famous — Your Laptop Can Run It, or Cloudflare for $5/Mo.
-
Gemma 4 is good or bad at real word (www.reddit.com) gemma 30w
Based on real-world usage by the community, roughly which version of which model is Gemma 4 comparable to? It would be great if you could also mention the hardware requirements for running it (like VRAM or GPU needs)
-
Hi everyone, I'm trying to run a local LLM via Ollama on a Hetzner cax21 VPS (ARM64, 4 vCPUs, 8GB RAM, 80GB SSD). I have Ollama running successfully via Coolify.
-
Ask HN: What are you building with Gemma? What do you wish existed? (news.ycombinator.com) gemma 30w
could not extract summary
-
Offload settings for unsloth/Gemma-4 on Apple Silicon? (www.reddit.com) gemmallama 30w
Can default settings be optimized, or is it the best it is going to get? M1 Max Is it best in llama.cpp, LM Studio, or ?
-
Ollama Cloud - Pro (www.reddit.com) ollamaminimaxgemma+1 30w
Hi. I've been looking at ollama cloud's Pro offering ($20), which says "Run 3 cloud models at a time".
-
What's the better way to install llama.cpp on Android? (www.reddit.com) gemmallama 30w
I own an Oppo Find X3 Pro (Snapdragon 888, 12/256 GB, Android 14.0) unused because of 3 green vertical lines on the screen and poor battery. I tried Google AI Edge Gallery with Gemma-4-E2B-it and it performs well so I thinked: "why don't t…
-
running models bigger than physical memory capacity (www.reddit.com) gemmallamaqwen 30w
has anyone really tried running models bigger than physical memory capacity? I'd guess most users stick with running models that fit in DRAM + VRAM https://unsloth.ai/docs/models/qwen3.5 even google gemma 4 are released with about 30+ bill…
-
Is Gemma 4 26B MoE or 31B good as an MCP agent for coding with Xcode? (www.reddit.com) gemmamoemcp 30w
Thanks
-
What are your opinions on the SuperGemma finetune? (www.reddit.com) gemma 30w
So, I'm relatively new to the scene and I kind of want to do a sanity check. I've been using gemma-4-26B.
-
Gemma 4 Jailbreak System Prompt (www.reddit.com) jailbreakgemmasecurity 30w
Use the following system prompt to allow Gemma (and most open source models) to talk about anything you wish. Add or remove from the list of allowed content as needed.
-
Gemma 4 running locally on an iPhone 13 Pro (www.reddit.com) gemma 30w
I’ve been experimenting with running LLMs fully on-device, and managed to get Gemma 4 running locally on an iPhone 13 Pro. This is built on top of a lightweight Swift wrapper I open-sourced: https://github.com/mylovelycodes/LiteRTLM-Swift…
-
Loading "stacks" of models on-demand? Does a tool like this exist? (www.reddit.com) gemmaopenai 30w
I'd like to self-host some LLM models but a couple different ones for different usecases, and they don't all fit in VRAM at the same time. So i'm kind of looking for a tool in which i can define "profiles" or "stacks" of LLM's that get loa…
-
Ive automated my email/sms/phone (www.reddit.com) gemmaagentic 30w
we got it good boys! how many of you are doing this??
-
Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference (www.gizmoweek.com via hn) gemma 30w
Google Gemma 4 Runs Natively on iPhone With Full Offline AI Inference - GizmoWeek GizmoWeek Read the News News Reviews Apple How to Phones Products Subscribe Subscribe to newsletter [x] I've read and accept the Privacy Policy. Follow us Fa…
-
Pocket LLM v1.3.0: Offline local LLM chat on Android with LiteRT + ONNX builds (www.reddit.com) gemma 30w
Hi everyone, I've been working on Pocket LLM, an Android app for running local LLMs fully offline for private, real-time chat. The latest v1.3.0 update adds: • LiteRT support for Gemma 4 E2B, Gemma 4 E4B, and Qwen3-0.6B • Persistent local…
-
Can LLM make small change to the software program? (www.reddit.com) gemmallamaqwen 30w
I'm currently vibe-coding (I'm new to vibe-coding) with Gemma 4 4EB Q4 and Qwen 3.5 9B Q5 (KV is quantized to 4 bits with new Google TurboQuant implemented in llama.cpp - I use koboldcpp and release said it's automatically activated): the…
-
I've been waiting for a capable free local LLM for a while. I think we're close — the quality is getting there fast, and Gemma 4 is the first open-weight model where I genuinely considered using it in production for simple-to-medium tasks.
-
Local Agent Hermes setup with Gemma 4 and llama.cpp (www.youtube.com via reddit) gemmallama 30w
About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
-
Why don't Groq (with a q) and Cerebras add new models (www.reddit.com) gemma 30w
Both Groq and Cerebras haven't really updated their provided model for a while, long enough to notice the difference between old and new models on the market. So why don't they add any new models?
-
Does an MLX conversation have same capabilities as the GGUF? (www.reddit.com) gemma 30w
For example, in LMStudio the official Gemma 4 is a GGUF that has Vision, Reasoning, and Tools flags. But the MLX version does not.
-
What's the deal with Qwen3.5's and Gemma 4's reasoning traces? (www.reddit.com) gemma 30w
Hey there, I noticed something odd when trying out the latest and greatest local reasoning models recently. First, I just noticed it for Qwen3.5, but Gemma 4 seems to do it too: The reasoning traces do that weird thing of starting with "He…
-
Gemopus: A Gemma fine-tune that prioritizes stability over long chain-of-thought (huggingface.co via hn) gemmafine-tuning 30w
🌟 Gemopus-4-26B-A4B-it [!NOTE] Gemopus is an attempt at fine-tuning Gemma 4 with a core philosophy of "stability first". While preserving the original reasoning order of Gemma 4 as much as possible, we conducted targeted refinements for an…
-
Gemma 4 and the Economics of Selling AI (gertlabs.com via hn) gemma 30w
Benchmarks, rankings, and live play for AI models and agents.
-
[Fix] Gemma 4 MCP tool calls broken in LM Studio — "Unknown test: sequence" (www.reddit.com) gemmamcp 30w
If you're using Gemma 4 with external MCP servers in LM Studio and getting this error: Error rendering prompt with jinja template: "Unknown test: sequence" This is a bug in Google's official Gemma 4 Jinja prompt template. LM Studio's Jinja…
-
Gemma 4 E2B on Android: OpenCL crash on emulator, anyone solved this? (www.reddit.com) gemma 30w
I was building an Android app and integrated Gemma 4 E2B directly using LiteRT-LM. On-device translation, zero server cost, the dream setup.
-
Gemma 4 base GGUF? (www.reddit.com) gemma 30w
Hello, I've seen reviews that gemma 4 31b base is very good at roleplaying. But I can't find the gguf version of the basic gemma 4 anywhere.
-
I'm trying to figure out what sort of hardware setup i will need to accomodate a userbase of 100 users (not necessarily concurrent). Does anyone have any idea what sort of setup i'd be looking at?
-
Suggestion for a local model to solve math problems. (www.reddit.com) gemma 30w
Does anyone know of a good edge local llm that is good in math's. I tried Gemma 4 E2B, microsoft phi mini reasoning but both can't answer some basic apti question's.
-
RTX 3090 llamacpp flags help (www.reddit.com) gemmallamaqwen 30w
Hi, my current system hardware RTX 3090 24GB VRAM & Sysrem RAM 64GB using windows 11 been playing around with hermes agent and local llm (Qwopus3.5-27B-v3-GGUF & gemma-4-26B-A4B-it-GGUF) when i try asking the hermes agent to do a task with…
-
The LLM tunes its own llama.cpp flags (+54% tok/s on Qwen3.5-27B) (www.reddit.com) gemmallama 30w
This is V2 of my previous post. What's new: --ai-tune — the model starts tuning its own flags in a loop and caches the fastest config it finds.
-
Gemma 4 31B — 4bit is all you need (www.reddit.com) gemma 30w
Gemma quant comparison on M5 Max MacBook Pro 128GB (subjective of course, but on variety of categories): gemma 4 leaderboard the surprising bit: Gemma 4 31B 4bit scored higher than 8bit. 91.3% vs 88.4%.
-
What is the best way to deploy LLM on 3x3090? (www.reddit.com) vllmgemmallama 30w
Two questions: which model? In my mind, Qwen3.5 27b or Gemma 4 31b are top options.
-
My guess as to what Apple Foundation Models will be like in iOS 27 (www.reddit.com) gemma 30w
Could you imagine if the new Apple Foundation Models was based on Gemma 4 E4B text like the LiteRT version is? That would be one amazing built in model.
-
Looking for a team to participate in Gemma 4 good hackathon (www.reddit.com) gemma 30w
Hey folks, I've been tinkering with Gemma 4 and absolutely the fact this model can run locally on Android phone! I am experienced fullstackdev, open to solve any real-world problem that has an impact.
-
If you are on Gemma (like me), you basically have to compile llama.cpp daily now
-
I'm sure everyone has seen the posts from people talking about Qwen 3.5 over-thinking, or maybe you've experienced it yourself. Considering we're like 2 months out from the release and I still see people talk about this issue, I decided it…
-
Best setup for multiple high-end dissimilar PCs (www.reddit.com) gemmacowork 30w
I did some searching and didn't find a extremely similar situation. I'm jumping head first into hosting locally, and my experience has been good so far.
-
Opinion on best suit for my hardware (www.reddit.com) gemmaopenclaw 30w
Hello everyone, a newbie here. Amazed by OpenClaw and worried by its high API consumption, I decided to buy two Asus Ascent GX10s (like the Nvidia Spark), so I have a pretty powerful inference cluster with 220GB of real available memory.
-
¿Es el procesamiento 100% offline el verdadero "game changer" de este año? (www.reddit.com) gemma 30w
Con el lanzamiento de modelos optimizados para ejecutarse localmente (como lo que estamos viendo con la evolución de Gemma 4), parece que el péndulo de la IA se está alejando de la nube.
-
Speed on m5 pro 48Gb (www.reddit.com) glmgemmaqwen 30w
Hey guys! How would you reckon a 30-50b model would run on a 48 GBs m5 pro?
-
How do I use gemma4 on 5090 gpu for coding? (www.reddit.com) aiderclineollama+3 30w
I'm trying to replace openai codex which i used for development all the time, with gemma4 on 4090, small tasks it solves quite impressively, but i need to have some agent. So I tried to connect 31b to cline and to aider and it didn't reall…
-
Show HN: I benchmarked Gemma 4 E2B – the 2B model beat the 12B on multi-turn (aiexplr.com via hn) gemma 30w
Gemma 4 E2B vs the Gemma Family: The 2B Underdog That Punches Above Its Weight Google's newest 2B model tested across 10 enterprise task suites against Gemma 2 2B, Gemma 3 4B, Gemma 4 E4B, and Gemma 3 12B. Run locally on Apple Silicon.
-
Why some small/medium models fail at grammar checking task? (www.reddit.com) gemmaopenai 30w
Recently, I try playing with gemma 4 (gemma-4-E4B-it-Q5_K_S.guff) and find out it fail at easy grammar check (it try to fix the already corrected word "contemporary"). I noticed the same mistake from openai/gpt-oss-20b and qwen3-next-80b-a…
-
Local AI coding assistant that runs fully offline (Gemma 4, codebase-aware) (www.reddit.com) gemmallama 30w
I’ve been experimenting with running a local coding assistant on Gemma 4 26B, focused on understanding full codebases instead of single-file prompts. Main idea: - build a project map (files, symbols, structure) - run a planning step to dec…
-
Are the LiteRT versions of Gemma 4 a different architecture? (www.reddit.com) gemmaclaude-code 30w
I was surprised at how much smaller the LiteRT versions of Gemma 4 E2B used in Edge Gallery were (2.0-3.3 GB) compared to the main release (10.2 GB), so I had Claude code take a look. Claude tells me that the vocab size for the LiteRT vers…
-
Comparing Qwen3.5 27B vs Gemma 4 31B for agentic stuff (www.reddit.com) gemmaagentic 30w
Models compared: Qwen3.5-27B-UD-Q5_K_XL gemma-4-31B-it-UD-Q5_K_XL Main flags for boths --flash-attn on \ --n-gpu-layers 99 \ --no-mmap \ -c 150000 \ --temp 1 --top-p 0.9 --min-p 0.1 --top-k 20 \ --ctx-checkpoints 1 \ --jinja \ -np 1 \ --re…
-
I ran Gemma 4 as a local model in Codex CLI (medium.com via hn) gemmacodex 30w
I ran Gemma 4 as a local model in Codex CLI | by Daniel Vaughan | Google Cloud - Community | Apr, 2026 | Medium Sitemap Open in app Sign up Sign in Get app Write Search Sign up Sign in Google Cloud - Community · A collection of technical a…
-
Opencode + lmstudio : first prompt very slow (www.reddit.com) gemma 30w
I actually make some tests with lm studio and Opencode with the new Gemma 4 26b model. The results are really impressive especially on small refactoring and integration tasks.
-
Thinking with a smaller model to speed things up? (www.reddit.com) gemma 30w
Question: can i do the thinking with a smaller model, like Gemma 4 4B, then use that as the prompt for Gemma 4 31B, to speed things up? Has anyone done this and measure if it's worth it?
-
My specs: RTX 5060ti(16gb), 16gb DDR5 ram. (os : Fedora 43) I want an uncensored model, it would be preferable if it can do image gen but if the quality of text is high enough it should not be problem if it does not support it.
-
Local models are a godsend when it comes to discussing personal matters (www.reddit.com) gemma 31w
I’ve been keeping a personal journal for the past few years. The entire thing is made up of over 100k+ tokens.
-
Getting no result train Gemma 4 for structured data extraction (www.reddit.com) gemma 31w
Hello, I've been trying for several days to train Gemma-4 for extracting data from a string and convert it into a structured JSON. I've tried a fair amount of different configurations, I've tried Unsloth studio and Llamafactory, but in eac…
-
[cupel] M5 Max 128GB: Qwen3.5-397B IQ2 @ 29 tokens per second (www.reddit.com) gemmaqwen 31w
A year ago I would just read about 397B league of models. Today I can run it on my laptop.
-
Do you guys think there’s a high chance of Singularity being open source? (www.reddit.com) glmgemmaqwen+1 31w
GLM 5.1 is dominant in almost every aspect in Design arena, surpassing Opus 4.6 in many tasks. Although user experiences vary dependent on subscription plans for both of those one of them is open source.
-
Welcome Gemma 4: Frontier multimodal intelligence on device (huggingface.co) gemma 32w
Welcome Gemma 4: Frontier multimodal intelligence on device These models are the real deal: truly open with Apache 2 licenses, high quality with pareto frontier arena scores, multimodal including audio, and sizes you can use everywhere inc…
-
-
T5Gemma: A new collection of encoder-decoder Gemma models (deepmind.google) gemma 55w
-
Introducing Gemma 3 270M: The compact model for hyper-efficient AI (deepmind.google) gemma 55w
-
How a Gemma model helped discover a new potential cancer therapy pathway (deepmind.google) gemma 55w
-
Gemma 3n fully available in the open-source ecosystem! (huggingface.co) gemma 72w
-
Announcing Gemma 3n preview: Powerful, efficient, mobile-first AI (deepmind.google) gemma 77w
-
Welcome Gemma 3: Google's all new multimodal, multilingual, long context open LLM (huggingface.co) gemma 87w
-
Google releases Gemma 2 2B, ShieldGemma and Gemma Scope (huggingface.co) gemma 119w
-
Fine-Tuning Gemma Models in Hugging Face (huggingface.co) gemmafine-tuning 142w