Import AI 455: AI systems are about to start building themselves. Jack Clark thinks there’s a ~30% chance by the end of 2027 and a ~60%+ chance by the end of 2028 that AI research becomes automated, with models eventually helping train the…
#fine-tuning
951 items
Anthropic co-founder Jack Clark says AI is nearing the point where it can automate AI research (www.reddit.com) I built a 103B-token Usenet corpus (1980–2013) — pre-web, human-only, zero AI contamination. Got strong traction on r/ML, thought this community would find it useful. (www.reddit.com) Posted this to r/MachineLearning a couple weeks ago (30K views, 100+ upvotes) and have been meaning to share it here where the fine-tuning angle is more directly relevant. I spent years building and processing a complete Usenet corpus from…
Trained a 125M LM from scratch instead of fine-tuning GPT-2 — releasing weights + SFT framework for others to build on (www.reddit.com) Trained a 125M LM from scratch (custom tokenizer) + released instruct checkpoint and SFT framework so others can fine-tune their own variants I’ve been experimenting with training small language models fully from scratch (no GPT-2 init, no…
GPT 5.5 "secret sauce" is just having the thinking be some stupid caveman mode? (www.reddit.com) I think I had GPT-5.5 leak its trace during a normal conversation, and it really reads like the caveman mode fad from a few months back. Maybe we can achieve better token efficiency by taking some high-quality thinking trace from an open m…
ServiceNow-AI/SuperApriel-15B-Instruct · Hugging Face (huggingface.co via reddit) A 15B-parameter token-mixer supernet with 8 optimized deployment presets spanning 1.0× to 10.7× decode throughput at 32K sequence length, all from a single checkpoint. Derived from Apriel-1.6 through stochastic distillation and targeted su…
[D] Released a 100k-sample dataset on Hugging Face (www.reddit.com) We’ve released a 100,000-sample Chain-of-Thought (CoT) dataset for fine-tuning local reasoning models. Each sample includes explicit intermediate reasoning traces, rather than answer-only supervision.
End-2-end tutorial on fine-tuning, the whole journey (docs.liquid.ai via reddit) I put together a hands-on tutorial that takes you from problem framing to fine-tuning, step by step. I decided to build a wildfire prevention system that uses satellite images and a Small Vision-Language Model (LFM2.5-VL-450M) to extract r…
Jackrong/Qwopus3.5-9B-Coder-GGUF · Hugging Face (huggingface.co via reddit) Qwopus3.5-9B-coder is specially optimized and fine-tuned for high-performance 🤖 Agentic Coding, complex Tool Calling, and logical reasoning. 💡 Why the 9B Dense Model?
OpenAI has announced they will be winding down fine tuning. (www.reddit.com) Got an email today about the announcement. > OpenAI is winding down the fine-tuning API and platform.
Built a Japanese ASR benchmark because existing ones can't measure quality differences properly (www.reddit.com) Was fine-tuning a Japanese ASR model (based on Qwen3-ASR) to handle technical terminology better. The model clearly improved — "Next.js" comes out as "Next.js" instead of "ネクストジェイズ", punctuation works, etc.
AI Engineer Notebooks – free, framework-free RAG/agents/evals on Colab (github.com via hn) AI Engineer Notebooks Learn the applied-LLM stack the way you'll actually be interviewed on it — framework-free, on a free API, from prompting all the way to serving, fine-tuning, and a red-team benchmark. Runnable Colab notebooks for the…
How Unsloth and Nvidia made LLM training 25% faster on consumer GPUs (unsloth.ai via hn) Fine-tuning is one of today's most computationally intensive workloads, and it continues to push hardware to its limits. NVIDIA GPUs are purpose-built for these workloads: they break complex problems into pieces and process them in paralle…
Show HN: MemFactory: Unified Inference and Training Framework for Agent Memory (arxiv.org via hn) Memory-augmented Large Language Models (LLMs) are essential for developing capable, long-term AI agents. Recently, applying Reinforcement Learning (RL) to optimize memory operations, such as extraction, updating, and retrieval, has emerged…
Number-aware embeddings (www.reddit.com) If you look at the cosine sim between the embeddings of "a 500 hp car", "a 1,200 hp car" and "a 73 hp car", you'll soon see that embedding models have no sense of number ordering at all. (I tested Qwen and ModernBERT-based embeddings) It m…
Fine-Tuning TranslateGemma-4B to improve bi-directional English & Welsh translations on an H200 GPU! (metalglot.com via reddit) Open source repo: https://github.com/grctest/finetuned-gemmatranslate-cy 5% of the fine-tuning took 40 minutes and cost a couple dollars to prove the process works. Looking forwards to Flash Attention v4 to leave beta, to test fine-tuning…
Llama models: still valuable for finetuning or surpassed by everything new? (www.reddit.com) Hello there people. So I have noticed that people are pretty much ignoring Llama 3 plus 3.1, 3.2, and 3.3 these days.
Finetuning Dataset: Claude Opus 4.6/4.7 - 8.7k Chats (www.reddit.com) https://huggingface.co/datasets/angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k A synthetic fine-tuning dataset created from Claude 4.6/4.7. 8,706 total examples all with reasoning.
losing my mind fine-tuning jina-v5 for a legal corpus (www.reddit.com) For the last month i've been trying to fine-tune jina-v5 (which has performed best on my corpus out of the box) on slovak law chunks, time and time again no matter what i do I can't get the model to learn nuance of slovak syntax. here's th…
Dropping learning rate fixed my Qlora fine-tune more than anything else i tried (www.reddit.com) Been fine-tuning llama 3.1 8b with Qlora for a classification task using about 8k samples. I was getting bad eval results for a while and kept thinking something was wrong with my data.
Show HN: LLM post-training to speak like GenZ, costing less than a cup of coffee (github.com via hn) GenZ LLM A post-trained language model that responds in GenZ slang, built on top of Qwen2.5-0.5B-Instruct using Supervised Fine-Tuning (SFT) followed by Reinforcement Learning with GRPO. The fine-tuned model is available on Hugging Face: a…
I have practically unlimited access to Opus and every other frontier model. I'd like to help contribute to a dataset. (www.reddit.com) No, I won't tell you how. No this is not for anyone who is not already a proven contributor to the fine-tuning space.
RTX PRO 5000 (48GB) vs MacBook Pro M5 MAX (128GB RAM) - The choice for fine-tuning & agentic coding (www.reddit.com) Show HN: Anime Text Datasets for LLMs Training (github.com via hn) Large Anime Text Dataset — LLM & RAG Ready Text-only anime dataset built for fine-tuning large language models (LLM), instruction tuning, retrieval-augmented generation (RAG), question answering (QA), entity linking, recommendation systems…
Ask HN: When is fine-tuning a small LLM worth it? (news.ycombinator.com) I'd be interested in hearing about your experiences. What kind of task did you use it for, what model did you train, and what were the results?
ChatGPT Optimizes Its Agent Loop: Harness, API, and Inference (blog.bytebytego.com via hn) How ChatGPT Optimizes its Agent Loop: Harness, API, and Inference Only Pay for Fine-Tuning That Works (Sponsored) GPU-hour billing charges for the entire time a machine is reserved — setup, idle time, queueing, and failures — so your cost…
I Kept a Diary for Seven Years. An LLM Finally Read It. (www.reddit.com) I've kept a personal diary since 2019. Last week I fed 200+ entries to an LLM and asked it how I've changed over 7 years.
Liquid AI releases fine-tuning harness for AI agents (lqh.ai via hn) Liquid Harness is an autonomous agent by Liquid AI that takes a plain-English spec and ships a fine-tuned Liquid Foundation Model. Spec, data, eval, training, deployment — all in one run.
Learn, run and test Agentic AI on your browser for free! (Built with Claude Opus 4.7 in 2 days) (www.reddit.com) Hey Everyone, Over the last few months, I noticed a massive gap in how we learn about Agentic AI. There are a million theoretical blog posts and dense whitepapers on RAG, tool calling, and swarms, but almost nowhere to just sit down, run a…
↯ Fine Tuning↯ Function Calling↯ Opus 4.7function-callingfine-tuningrag+4
Fine-tuning and deploying Gemma 4 is not that easy (ghost.oxen.ai via hn) Writing a fine-tuning and deployment pipeline isn't as easy as it looks (Gemma 4 Version) Fine-tune and deploy Gemma 4 on Oxen.ai Google's Gemma 4 dropped in April 2026 with multimodal support (text, image, video, audio), a novel hybrid KV…
Show HN: Riff – An iPhone calendar you zoom instead of switching views (riffcalendar.com via hn) Hi, HN. I am Magnus from Sweden with a side project: Riff is an iPhone calendar where you can zoom smoothly from half a year to a single day, and every stop on the way is a working calendar.
Fine-tuning decision models – Server Logs and IoT Fleets (hive.technology via hn) FOR AGENTS: this article as plain text with a reading guide, fine-tune-decision-models.agents.md. DISCUSS THIS WITH AN AGENT Read https://hive.technology/lab-notes/fine-tune-decision-models/fine-tune-decision-models.agents.md, the article…
Ask HN: When do you choose RAG over Fine-Tuning? (news.ycombinator.com) could not extract summary
I built an LLM debugger for fine-tuning failures (github.com via hn) Which of my training data, or which config setting, broke my fine-tune? Documentation · Quick start · How it works · The math · Verification · Research Gradian is a training-data attribution engine for LoRA fine-tuned LLMs.
AI content detector based on Qwen 0.8b fine-tuned on Pangram dataset (www.reddit.com) I've fine-tuned Qwen 3.5 0.8B on the dataset provided by Pangram with their EditLens paper. It's available via a Chrome extension; you can just click selected text and it's going to give you the probability distribution of how likely it is…
Demo of fine-tuning Orpheus 3B on a TTS dataset in Transformer Lab (open source) (www.reddit.com) I'm part of the team building Transformer Lab, an open source ML research platform. We put together a short demo of how to run text to speech training, which you can do on your own hardware using a Local provider.
I Built a desktop app for generating LLM fine-tuning datasets — started it a week ago while learning FT (www.reddit.com) Hey, I've been building side projects with Claude Code for a few months, but I'm completely new to fine-tuning — started experimenting maybe a week ago. From day one I wanted a GUI for the dataset side of the workflow, so this desktop app…
Converting XQuery to SQL with Local LLMs: Do I Need Fine-Tuning or a Better Approach? (www.reddit.com) One MacBook Pro Beat Apple's Four Mac Studios at WWDC26 LoRA Fine-Tuning (nuketorch.ai via hn) Team We are a small team with our own mathematics and a Tensor Engine built from the ground up. We are driven by theories, architectures and measurements.
Show HN: Meshword - crosswords meet nonograms (meshword.org via hn) Hi! I made Meshword, a daily puzzle combining crosswords and nonograms.
Microsoft is fine-tuning Windows 11. Here's what's coming in November (www.pcworld.com via hn) Accelerated App Launch, Adaptive Hibernate, and cleaner privacy settings are coming. The optional KB5127936 preview update has them.
System One Lite – typed decisions from a local LLM, with no generated tokens (github.com via hn) System One Lite A tiny project that turns a normal local LLM into a typed decision engine. It needs no fine-tuning, text generation, or parser.
GRP-Obliteration: Unaligning LLMs with a Single Unlabeled Prompt (arxiv.org via hn) Safety alignment is only as robust as its weakest failure mode. Despite extensive work on safety post-training, it has been shown that models can be readily unaligned through post-deployment fine-tuning.
Coding Sahayi – Local-first autonomous AI coding IDE (WinUI 3 and Ollama) (codingsahayi.github.io via hn) Local-first autonomous WinUI 3 IDE powered by Ollama, ConPTY self-repair loops, and local fine-tuning.
A decision tree for picking attention (efficientagent.substack.com via hn) I keep getting asked the same question by people implementing or fine-tuning small-to-mid LMs: “should I use MHA, GQA, MQA, or MLA?” The honest answer is “it depends,” and I’ve found that unhelpful when I was the one asking. So I ran the e…
Tinker: GLM 5.3 Fine-Tuning (tinker-docs.thinkingmachines.ai via hn) Models & Pricing All prices are per million tokens. Checkpoint storage is charged at $0.10 per GB per month.
The Review That Praised the Bug: grading three LLM code reviews against the code (mrjstickel.com via hn) AI systems engineer who designs and ships production AI end to end - RAG pipelines, agentic assistants, measured retrieval quality, and multi-provider LLM infrastructure, with hands-on QLoRA fine-tuning. Built a private AI platform (Archit…
Show HN: Popkorn – A CSS based alternative for Lottie animations (github.com via hn) Hi all, I’ve been working on this project for a while now and wanted to share it here for feedback and contributions. You can test drive it in the playground at: https://usepopkorn.dev I’ve been calling it Popkorn.
Robotic Adaptation Suffers from Data Collection Bottleneck (news.ycombinator.com) How can we ease Robotic Adaptation, or the critical last mile of reliable robotic policy deployment? Vision-Language-Action Policies (such as Gr00T, Pi0, OpenVLA) are designed so that an adaptation stage is needed before reliable deploymen…
Installing Verified Facts into Transformer Weights by Direct Construction (zenodo.org via hn) We present SuperBake, a system that installs new factual knowledge into the weights of large language models without any gradient steps. Instead of fine-tuning, low-rank adaptation, or retrieval, SuperBake measures how the stock model repr…
Is fine-tuning still needed? LLMs, RAG, & LoRA – IBM Technology [video] (www.youtube.com via hn) About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
Petals: Collaborative Inference and Fine-Tuning of Large Models (arxiv.org via hn) Many NLP tasks benefit from using large language models (LLMs) that often have more than 100 billion parameters. With the release of BLOOM-176B and OPT-175B, everyone can download pretrained models of this scale.
Jackrong LLM Fine-Tuning Guide (github.com via hn) Jackrong LLM Fine-Tuning Guide An educational, end-to-end open-source knowledge base for LLM fine-tuning, dataset distillation, reinforcement learning, and local deployment. 🌐 Languages: English | 中文 | 한국어 | 日本語 🤗 Hugging Face: Jackrong Th…
Show HN: Does a vibe leak? Fine-tuning an LLM on an attitude it never states (github.com via hn) Latent Bias Transfer (LBT) A note on how this was made. The hypothesis and the questions are mine — but several of the techniques here (LoRA fine-tuning, activation steering, the statistics) were new to me.
Show HN: We're open sourcing Superlog (YC P26), an autonomous monitoring tool (github.com via hn) Hi HN! This is Arseniy from Superlog (YC P26).
What workstation to get for ~13k EUR? (www.reddit.com) My use-cases will be to test open-weight LLMs and work on harnesses, inference systems and possibly other non-ML workflows (CS-related) in the future. Fine-tuning would not be something I do locally because I can rent a B200 from RunPod fo…
↯ Llama↯ Fine Tuning↯ Vllm↯ Minimax↯ DeepSeek 4vllmminimaxllama+2
RAG vs. Fine-Tuning – The Question Every AI Builder Gets Wrong (thingswithai.org via hn) RAG vs. Fine-Tuning — The Question Every AI Builder Gets Wrong AI models don't know your private data.
RAG vs. Fine-Tuning: Which AI Strategy Saves Your Team Time and Budget (lightrains.com via hn) Two weeks before a Fortune 500 product launch, we told a client to scrap their fine-tuned model and rebuild with RAG instead. They lost eight weeks and $180K.
Best open-weight model to run locally on 8x A100 80GB for generating teacher data? (www.reddit.com) I have (free) access to a SLURM cluster with 8x NVIDIA A100 80GB GPUs (=640 GB VRAM) on a single task, and I want to run an open-weight model locally with llama.cpp for data generation, not coding. My use case is generating teacher data fo…
Show HN: I built a 2nd-order PyTorch optimizer for LLMs that runs on 16GB GPUs (news.ycombinator.com) Hi HN, I'm Danilo. I've been struggling with the limitations of AdamW when fine-tuning LLMs locally.
Findings: Gemma4 26B-A4B fine-tuning on a single RTX 4090 — 10 patches, benchmark, PCIELink path #1 (www.reddit.com) Summary of Findings This issue documents what we learned making Gemma4 26B-A4B-it train on consumer hardware (RTX 4090, 24GB VRAM). No A100.
Show HN: Rollquation – A Rolling-Ball Math Puzzle Game for Android (Solo Dev) (play.google.com via hn) Hey HN! I'm a solo dev and I just wanted to share my latest Android game — Rollquation.
Musings on long-context fine-tuning (hardikmeisheri.github.io via hn) Notes on model training Musings on long-context fine-tuning Notes on training a 27B vision-language model on long-context examples. On this page - Getting one example to train - Sharing the repeated work I recently fine-tuned Qwen3.6-27B,…
Pipeline Parallelism is here: 8x faster than Apple's WWDC26 fine-tuning (nuketorch.ai via hn) Announcement · November 11, 2026 Pipeline Parallelism is here: 8x faster than Apple’s WWDC26 fine-tuning One 48 GB MacBook Pro ran Apple’s WWDC26 fine-tuning faster than Apple’s four M3 Ultra Mac Studios. Four MacBook Pros ran it more than…
Offline context-aware dictation without screenshots, LLMs or fine-tuning (morethanamachine.com via hn) Handy Context: Offline Context-Aware Dictation Without Screenshots, LLMs or Fine-Tuning By Nishaanth Reddy · [00] Motivation Wispr Flow’s best feature is that it spells names right, because it reads them off your screen. It also sends your…
Fine-tuned Qwen on my 16gb GPU to respond without the letter e (kevin-wu.ca via hn) Qwen, Minus the E Fine-tuning Qwen3-1.7B locally to write without the letter ‘e’. Contents A little while back, I saw an RL project on teaching a language model to draw flowers.
Recent Security Breach Attempts and Fine-Tuning Claims Against NovaSpark (iqlvbs.com via hn) Statement A Note on Recent Security Breach Attempts and Fine-Tuning Claims Against NovaSpark Recently, there's been an accusation made by Harald Schaefer at Comma circulating that IQ's NovaSpark driving model is a “short fine-tuned” versio…
Best LLM Fine-Tuning Services: What to Look for in 2026 (www.rebiha.com via hn) Pick a model — Qwen3.8, Granite, OLMo, Gemma 4, Llama & more — choose from 54 ready-made datasets, and train. No data prep, no infrastructure, no API.
↯ Gemma↯ Llama↯ Fine Tuning↯ Qwen Updategemmallamafine-tuning
Fine-tuning a small language model for RAG in a browser on a Mac (bart.degoe.de via hn) Last week we tried to summarize semantic search results with small language models running in your browser. They weren’t great at it, hallucinating often.
A Statistical Analysis of LLM Benchmaxxing (llemre.com via hn) benchmaxxingnoun Benchmaxxing is the practice of aggressively fine-tuning, prompt-engineering, or overfitting a language model specifically to top public benchmark leaderboards (such as MMLU, GSM8k, or SWE-bench). This often produces artif…
Show HN: MLX Direct Decider, Mac local decider without fine-tuning (github.com via hn) Hi HN, I built this open source project to enable the existing MLX model on your Mac to be used as a decider, without additional fine-tuning.
If LLMs Can Decide Without Fine-Tuning, Do We Still Need Models Like Jev? (itsodeleo.github.io via hn) Often, we ask a model to write a response just to extract a decision from it. Jev, released by TypeSafe in September, is built around this need: give it context and a question, and get a choice, score, or probability directly.
Show HN: Spectra – A drop-in optimizer that cuts LLM training VRAM by 50% (github.com via hn) Authors: Spectra Labs Research Status: Closed Beta (Python POC Available) Abstract The primary memory bottleneck when fine-tuning Large Language Models (LLMs) on consumer hardware is not the model weights, but the optimizer states. Standar…
NASA-IBM Lunar Foundation Model and fine-tuning code (github.com via hn) NASA-IBM Lunar-FM Finetuning and inference release of the NASA-IBM Lunar Foundation Model foundation model. Two Python packages: ni_lfm/ — the model package (backbone, tokenizers, data utilities).
OpenStax-LLM: tools for OpenStax LLM access (github.com via hn) Pedagogical semantic chunking, RAG dataset preparation, and LLM fine-tuning pipelines from OpenStax textbooks. Built on top of openstax-md, openstax-llm transforms OpenStax college textbooks into structured, citation-aware, formula-safe da…
Show HN: Gatekeeper – Persuade a 34MB Jev-like model to let you into the castle (gatekeeper.dopp.sh via hn) Upfront caveat: it's English only! but I'm personally still just amazed that 34MB (fine-tuned BAAI/bge-small-en-v1.5 ) can fit so much knowledge that it's possible to have an iPhone 15 running these classifications entirely client-side on…
Fine-tuning Qwen3.5-4B to replace Gemini Flash-Lite in a RAG pipeline (duane.sh via hn) Pipeline costs Local Minutes finds meeting documents, converts each document to text, splits the text into excerpts, adds context to each excerpt, and embeds the result for search. Adding context is the most expensive of those steps, becau…
The dataset where fine-tuning turned actively harmful – and how we now catch it (www.cacheverifier.com via hn) Every dataset in our benchmark battery except one is a snapshot: a fixed pool of (query, candidate answer) pairs, scored once, split once. SemCacheLMArena , SemCacheSearchQueries , and Quora Question Pairs all behave this way, and on all t…
I Built Laya Studio: Specialize Laya Without Fine-Tuning (github.com via hn) Laya Studio Before fine-tuning Laya, try specializing its representation space. Laya Studio helps you specialize a frozen Laya model for a new domain or decision task—without gradients, without changing its weights, and without maintaining…
Empirical results fine-tuning π0.5 on a real manufacturing task (dream-machines.eu via hn) The path to general-purpose robotics runs through task-specific deployments in real workplaces. While much of the field optimises general-purpose policies to do many tasks reasonably well, deployment requires robots that can do one task re…
Molt – Thermally aware QLORA fine-tuning for Windows laptop GPUs (github.com via hn) MOLT Thermally aware, memory-efficient QLoRA fine-tuning for consumer NVIDIA GPUs. MOLT provides a Windows-first workflow to prepare data, validate workload fit, fine-tune supported local language models, safely resume interrupted runs, an…
Nari Labs: Multimodal inference at the speed of light (narilabs.com via hn) Ship multimodal models with lower latency and higher efficiency. Explore optimized model APIs, dedicated enterprise inference, and fine-tuning with Nari Labs.
Show HN: 188M Hindi encoder, 28B tokens, 8K context, 1× RTX 4090 (github.com via hn) I wanted to improve Hindi retrieval quality, particularly for longer documents ( as the current architectures don't really have a longer context length ), and was curious how far can I push a 4090 haha :) So I trained a Hindi-first ModernB…
Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models (energy-based-fine-tuning.github.io via hn) Given vocabulary $\mathcal{V}$, a ground-truth distribution $p$ over contexts $c \in \mathcal{V}^{*}$ and completions $y \in \mathcal{V}^{G}$ of length $G$, and a language model $p_{\theta}$, we introduce a feature-matching loss that direc…
Fine-tuning your LLM might be making it dumber (github.com via hn) Did it even learn? A contamination-controlled audit of whether community LLM fine-tunes on HuggingFace actually improve on their base models, or just get better at the benchmark.
Agents are a security nightmare... are they? (write.as via hn) ### A lot has been written about the "bitter lesson" of LLMs. Fine-tuning models usually doesn't make sense, because the next base model...
Getting a Foothold in Reinforcement Learning for LLMs (amitpoonia.github.io via hn) (##) Premise Fine-tuning LLMs is a key part of applied AI/ML engineering work these days, by using supervised fine-tuning (SFT) and increasingly in combination with Reinforcement Learning (RL) methods like GRPO and its variants, aka policy…
Why PDF extraction for RAG breaks, and one approach to make it verifiable (github.com via hn) Cernodata Cernodata is an open-source ETL framework for layout-aware PDF extraction, automated quality iteration, structural layout debugging, and dataset generation for RAG and LLM fine-tuning. Parsing PDFs for RAG is a guessing game You…
Niche expert AI fine-tuning data set in clean JSON format (headwater.gumroad.com via hn) Accelerate your LLM development with this high-quality, pre-formatted fine-tuning dataset. Contains 100+ meticulously structured instruction-response pairs optimized for training AI models on specialized, niche-expert logic.
Transformer Models in Financial Forecasting: Outperforming LSTMs (algo-finance.com via hn) TL;DR - Zero-shot capabilities are transforming finance: Google's TimesFM (200M parameters) shows a 15-20% improvement in forecasting accuracy without fine-tuning compared to traditional ML baselines. - Quant giants like Two Sigma and Man…
Fine-Tuning from First Principles: LoRA, QLoRA, Serverless Fine-Tuning (debnsuma.github.io via hn) A ground-up journey into fine-tuning LLMs, where we build quantization and LoRA from scratch in plain PyTorch, combine them into QLoRA, and finally fine-tune Qwen3-8B into a PII redaction engine that outperforms a 70B model, using Crusoe’s…
LoRA Fine-Tuning and Profiling of a 0.5B Language Model on Apple Silicon (zenodo.org via hn) We present a fully reproducible case study in fine-tuning and deploying a sub-billion-parameter language model for closed-vocabulary structured extraction on commodity hardware. Using low-rank adaptation (LoRA), we fine-tune Qwen2.5-0.5B-I…
LoRA Speedrun – a public wall-clock leaderboard for fine-tuning techniques (github.com via hn) LoRA Speedrun 🏁 How fast can you LoRA-fine-tune Qwen2.5-1.5B to ≥ 57% on GSM8K — on a single L40S? This is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records.
Auto-SFT optimizes parameters for LoRA fine-tuning (github.com via hn) Auto-SFT Automated LoRA fine-tuning pipeline: hyperparameter search → best-config full fine-tune → GGUF export. Inspired by Karpathy's autoresearch.
Fine-tuning a small model for text simplification with simple verifiers (miguelconner.substack.com via hn) Engineer Two the art of designing cleverer checks for your loop. I was inspired to start experimenting again with AI coding after reading a quote from the CTO of Speak, Andrew Hsu, about how his team thinks about building with AI.
LLM Fine-Tuning and Model Iteration Loop: Improve AI Apps in Production (www.youtube.com via hn) About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
PorTAL: Portable Task Adapters for LLMs (twitter.com via hn) https://t.co/v1tcKZ2PHF Ramp Labs@RampLabsArticlePorTAL: Portable Task Adapters for LLMsResearcher: Ben Geist Abstract Parameter-efficient fine-tuning (e.g. LoRA) adapts a frozen LLM to a task, but the resulting adapter is locked to one ba…
UATC – A Closed-Loop Controller to Prevent GPU OOM During LLM Training (github.com via hn) UATC: Universal Adaptive Training Controller Abstract Fine-tuning Large Language Models (LLMs) on resource-constrained edge hardware is brittle. A single long sequence or an unexpected batch-size spike can trigger an Out-Of-Memory (OOM) cr…
Tracing a silent-corruption bug in differentially private LoRA fine-tuning (imranahamed.substack.com via hn) The DP-LoRA silent corruption: how 5 months of broken fine-tuning hid in plain sight How a device-placement ordering quirk between opacus, PEFT, and HuggingFace caused DP fine-tuning to silently break, and what to check in your own setup.…
Fine-Tuning and Deploying LLMs on Mobile:F/b to learnings (www.youtube.com via hn) About Press Copyright Contact us Creators Advertise Developers Terms Privacy Policy & Safety How YouTube works Test new features NFL Sunday Ticket © 2026 Google LLC
Parallelogram – catch fine-tuning dataset bugs before training (www.parallelogram.dev via hn) Open-source local CLI that validates OpenAI/Qwen chat JSONL and ShareGPT-style fine-tuning datasets — broken roles, empty messages, duplicates, encoding artifacts, context-window overflows — before they poison a training run.
Fine-tuning LLMs on 30M academic papers from ScholarAPI (scholarapi.net via hn) Global academic literature at your fingertips. Reliable Google Scholar alternative for large-scale access to academic PDFs and metadata, with full-text search and bulk download.
Learn from Your Mistakes: Tree-Like Self-Play for Secure Code LLMs (arxiv.org via hn) While Large Language Models (LLMs) excel in code generation, they remain prone to replicating subtle yet critical vulnerabilities endemic to their training data. Current alignment techniques, such as Supervised Fine-Tuning (SFT) and Reinfo…
Fine-tuning an LLM to write docs like it's 1995 (passo.uno via hn) In my predictions for 2030 I wrote that tech writers would be using specialized LLMs, running locally on powerful hardware. I see hints of this move to “local first” among engineering pundits, but we’re not there yet, in part because of ho…
Fine-Tuning for Engagement (robertdruska.com via hn) May 29, 2026 It’s been quite some time since major LLM providers introduced the behaviour that the chatbots often end their response with a question. The motivation is clear: more engagement, more data to train on.
Personal continual learning for LLMs without GPU — position paper [OC] (www.reddit.com) I proposed two architectures for enabling LLMs to learn daily from personal interactions: Internal KV-Sphere Architecture (IKSA) Background Micro Fine-Tuning (BMFT) Both work with zero GPU and zero catastrophic forgetting. Full paper: in c…
About to start fine-tuning on RunPod. What should I know to not waste money? (www.reddit.com) I was MLOps lead at an AI company managing 5000+ GPUs across GCP and CoreWeave. Left to start my own thing and now I'm back to renting GPUs like everyone else.
How to Fine-Tune LLMs on AMD Strix Halo (www.promptinjection.net via hn) How to Fine-Tune LLMs on AMD Strix Halo (Ryzen AI MAX+ 395) and Other Exotic AMD Hardware A Complete Windows and Linux Guide to Full SFT and LoRA Training This guide covers full SFT and LoRA fine-tuning on AMD hardware that sits outside th…
Open-sourced our MCP server for GPU workload execution looking for feedback (www.reddit.com) Hey everyone I’m Jaguar, building Jungle Grid. We just open-sourced our MCP server for agentic GPU workload execution.
Introducing AI finetuner, Source available and free Claude skill to fine tune your vibe coded UI with live preview (www.reddit.com) Fine-tuning UI with AI right now: "Make the shadow softer." "Stronger." "No, less." "Go back." "A bit more." 17 messages later, you've spent more tokens than the shadow is soft. I built something that breaks the loop.
Model Spec Midtraining: Improving How Alignment Training Generalizes (alignment.anthropic.com via hn) We introduce model spec midtraining (MSM): after pre-training but before alignment fine-tuning, we train models on synthetic documents discussing their Model Spec. This shapes how models generalize from subsequent alignment training.
Anthropic just published new alignment research that could fix "alignment faking" in AI agents here's what it actually means (www.reddit.com) Anthropic's alignment team published a paper this week called Model Spec Midtraining (MSM) and I think it's one of the more practically interesting alignment results I've seen in a while. The core problem they're solving: Current alignment…
HELP - How to fine-tune an LLM to match academic writing style (www.reddit.com) I've been using LLMs to help write my thesis, but the output feels dry and uses awkward phrasing (especially in translation). I'm looking to fine-tune an accessible LLM to better match natural academic writing in my language.
Can LLMs create lasting flashcards from readers' highlights? (memory-machines.com via hn) We tested prompting, fine-tuning, RL, and grounded evaluation across ~1,500 labeled flashcards—and found models catch obvious misses but not plausible failures.
Run, Learn and test Agentic AI for free, on your browser! (Open AI Models are included) (www.reddit.com) Hey Everyone, Over the last few months, I noticed a massive gap in how we learn about Agentic AI. There are a million theoretical blog posts and dense whitepapers on RAG, tool calling, and swarms, but almost nowhere to just sit down, run a…
↯ Fine Tuning↯ Function Callingfunction-callingfine-tuningrag+3
Interactive playground to learn Agentic AI hands-on (Free) with Certification (www.reddit.com) Hey Everyone, Over the last few months, I noticed a massive gap in how we learn about Agentic AI. There are a million theoretical blog posts and dense whitepapers on RAG, tool calling, and swarms, but almost nowhere to just sit down, run a…
↯ Fine Tuning↯ Function Callingfunction-callingfine-tuningrag+3
Research note: Fine-tuning experiments on CoT controllability (metr.org via hn) We find that a small amount of fine-tuning on instruction following in the CoT generalizes to meaningful increases in CoT controllability on an out-of-distribution set of tasks. We fine-tune four reasoning models on small datasets of instr…
A weekend with LoRA on Gemma 4 E2B: instrumenting what fine-tuning changes (aiexplr.com via hn) Spent a week doing LoRA fine-tuning on Gemma 4 E2B (~5.1B total params, ~2B active in text decoder) for a narrow Python code-generation task. Bad outputs went from ~5% to 0% (greedy) and 1.5% (sampled) across 134 tests.
Show HN: ClickMVP – Deterministic full-stack code generation (no LLMs) (app.clickmvp.com via hn) I've built software for clients for 38 years and kept hitting the same wall: weeks spent scaffolding the data layer and the Clean Architecture around it before any real work begins. I asked Claude to estimate how long it would take to gene…
Pioneer: Vibetune Your LLMs (pioneer.ai via hn) +30% avg accuracy lift on classification & extraction tasks vs. base Gemma ~7 days until your first auto-improvement run lands in production 0 lines of fine-tuning code you have to write, ever $0/retrain starting price.
LLM from scratch (32l) – Interventions: updated instruction fine-tuning results (www.gilesthomas.com via hn) Which kind of base/fine-tunes have you done? And which data did you use? (www.reddit.com) [Release] Swedish Construction FAQ — 503 bilingual (SV+EN) Q&As for fine-tuning, CC BY 4.0, now on HF / PyPI / Kaggle / Zenodo (www.reddit.com) I've been building an open Q&A dataset for the Swedish construction industry (byggbransch) over the last few weeks — something that's been a gap in Swedish-language domain-specific datasets. Finally hit a milestone worth sharing.
Friday, self-evolving assistant, only CC $100 plan, no agent framework (github.com via hn) Friday — A 24/7 AI Assistant Built Entirely on Claude Code An always-on personal AI system using only Claude Code CLI ($100/month) and Telegram — no custom AI, no cloud VMs, no fine-tuning. Live page: missingus3r.github.io/friday-showcase…
A guide to model quantization in fine-tuning (and how to pick the right GGUF) (www.siquick.com via hn) A guide to model quantization in fine-tuning (and how to pick the right GGUF) About this post Fine-tuning with Unsloth and Axolotl is, on the whole, a well thought-out experience where a lot of the complexity is handled for you. However on…
Gemopus: A Gemma fine-tune that prioritizes stability over long chain-of-thought (huggingface.co via hn) 🌟 Gemopus-4-26B-A4B-it [!NOTE] Gemopus is an attempt at fine-tuning Gemma 4 with a core philosophy of "stability first". While preserving the original reasoning order of Gemma 4 as much as possible, we conducted targeted refinements for an…
Diffusion Tree Sampling for Inference Time Adaptation of Crystal Generative Models (arxiv.org) Diffusion-based generative models have become strong priors over plausible crystal structures, but turning that prior into promising candidates requires steering toward several conflicting properties at once. Fine-tuning is expensive and p…
SWIFT: Sample-Wise Instance Fine-Tuning of Time-Series Foundation Models for Forecasting (arxiv.org) While current few-shot adaptation strategies for time series forecasting foundation models are predominantly dataset-centric, the necessity of obtaining an additional dataset for the finetuning process poses a significant hurdle for adopti…
Is Abliteration Getting Harder? Not Uniformly: Refusal Removal by Direction-Based Ablation and ARA in Heretic on the Qwen3.8, Gemma 4, Olmo 3.1 and Mistral Small 3.2 LLMs Versus Their Predecessors (arxiv.org) Abliteration, a fine-tuning-free weight edit, removes refusal from open-weight large language models (LLMs). The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ab…
↯ Mistral↯ Gemma↯ Qwen Model Update↯ Fine Tuning↯ LLM Model Experimentationmistralgemmafine-tuning
LOFT: Low-Rank Orthogonal Fine-Tuning via Task-Aware Support Selection (arxiv.org) Orthogonal parameter-efficient fine-tuning (PEFT) adapts pretrained weights through structure-preserving multiplicative transformations, but existing methods often conflate two distinct design choices: the subspace in which adaptation occu…
Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint (arxiv.org) Federated Learning (FL) with Parameter-Efficient Fine-Tuning (PEFT) enables efficient adaptation of Large Language Models (LLMs), yet its energy and carbon costs across hardware remain poorly understood. We evaluated federated Low-Rank Ada…
UniPipe: Unified Inference Serving and Fine-Tuning for Deadline-Constrained DNN Surrogate Workflows (arxiv.org) DNN surrogates are increasingly used to accelerate scientific workflows that must process high-rate sensor streams in near real time, exemplified by Ptychographic imaging at next-generation X-ray light sources. Such surrogates deliver 100-…
EvenVLM: A Text and Multimodal Dataset for the Even Language with Vision-Language Fine-Tuning (arxiv.org) We present EvenVLM, the first text and multimodal (VQA) dataset for Even (Lamut), an endangered Northern Tungusic language spoken by approximately 6,800 people in Siberia. Our dataset comprises 50,521 structured Even-Russian entries extrac…
Fine-Tuning Language Models to Know What They Know (arxiv.org) Evaluating true metacognition in Large Language Models (LLMs) is difficult due to biases and heuristics. This paper presents a framework to measure and enhance LLM metacognition while controlling for these biases.
Predicting Ethanol Drinking Status from Nonhuman Primate Liver Histology: A Proof of Concept for Parameter-efficient Fine-tuning in Few-Shot Vision-Language Modeling (arxiv.org) Combining vision-language models (VLMs) with parameter-efficient fine-tuning (PEFT) offers a promising approach to few-shot whole-slide image (WSI) classification in computational pathology. This study evaluates this approach for predictin…
Data-Efficient LLM Safety Alignment Guided by Interpretable Features (arxiv.org) Existing large language models typically rely on massive fine-tuning datasets for safety alignment, which results in higher costs for safety alignment and often leads to the problem of over-refusal. We propose Less is Safer (LIS), a compac…
JudgeMix: Judge-Guided Mixed-Rollout Data Synthesis for LLM Agents (arxiv.org) Supervised fine-tuning (SFT) on expert trajectories is a standard paradigm for training LLM agents. However, it suffers from a well-known training-inference distribution mismatch, calling for target-aware data synthesis strategies that gro…
The Definitive Fine-Tuning Guide for 3D Generation Models (huggingface.co) The Definitive Fine-Tuning Guide for 3D Generation Models TLDR; In this blog, we provide fine-tuning scripts and containers to fine-tune TRELLIS.2, a popular 3D model on your 3D assets, that can run both locally and on Hugging Face Jobs. I…
SLIC: Reinforcement Fine-Tuning Small LMs for Multi-Turn Analog Circuit Optimization (arxiv.org) Analog integrated circuit (IC) design is a multi-turn, simulator-grounded engineering task that demands iterative reasoning over coupled performance tradeoffs, yet proprietary process design kits (PDKs) make hosted large language model wor…
Off-target posterior mass: A training-free diagnostic for fine-tuning spoken language diarization probes (arxiv.org) Spoken Language Diarization (SLD) models provide frame-level language information of a speech recording. SLD modeling pipeline includes feature extraction from pre-trained speech models followed by clustering.
Towards end-to-end LLM-based censoring-aware survival analysis (arxiv.org) Objective: Survival analysis is central to medical prediction, yet large language models (LLMs) are rarely used as end-to-end survival models because censoring prevents straightforward supervised fine-tuning. Here we present LLMSurvival, a…
RBLA+: Efficient Rank-Specific LoRA Adapter Extraction for Federated Fine-Tuning with Heterogeneous Clients (arxiv.org) Federated fine-tuning with low-rank adaptation (LoRA) reduces communication and computational overhead by training and exchanging compact trainable LoRA factors. Clients may use different LoRA ranks to meet adapter communication budgets an…
Full Fine-Tuning is not the Problem: Why Adam Fails and SGD Succeeds in Few-Shot CLIP Adaptation (arxiv.org) Zeroth-Order Riemannian Optimization on Fixed-Rank Manifolds for LLM Fine-Tuning (arxiv.org) FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models (arxiv.org) Worse Forces Can Lead to Better Structures: Trajectory-Level Fine-Tuning of Machine Learning Interatomic Potentials (arxiv.org) ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making (arxiv.org) FireGate: Bounding Behaviour Change Before Training in Continual RL Fine-Tuning (arxiv.org) Fine-tuning a deployed language model on one task after another can change behaviour that should stay fixed, such as earlier skills, general chat or refusals. Today, such forgetting is found only after training, on the benchmarks someone c…
Direct Trajectory Optimization for Inference-Time Alignment of Text-to-Image Diffusion Models (arxiv.org) Alignment of text-to-image diffusion models with reward functions is a key challenge in controllable generation. Existing approaches typically rely on optimizing model inputs or fine-tuning denoising networks, which requires computationall…
VeilTune: Secure Fine-Tuning of Private LoRA Adapters on Public Pretrained Language Models (arxiv.org) Fine-tuning public pretrained language models with lightweight adapters such as LoRA has become the mainstream way to adapt them to downstream tasks. A service provider often trains its adapter on a data owner's samples, so two assets need…
Enhancing Mathematical Reasoning in Turkish Large Language Models with Search-Based Inference and Model Merging (arxiv.org) We improve Turkish mathematical reasoning in Large Language Models by combining LoRA fine-tuning, weight-space model merging, and search-based inference. Using Turkish-Llama 8B-DPO with Turkish translations of MetaMath, NuminaMath, and Tig…
Restoring General and Improving Domain-Specific Safety in Fine-Tuned LLMs Without Retraining (arxiv.org) Safety-aligned language models are known to suffer degradation in refusal behavior following domain-specific fine-tuning, even when the fine-tuning data is entirely benign. The predominant mitigation strategy, preference optimization on th…
Homeopathic Learning: Diluting poisoned fine-tuning data can strengthen its effect (arxiv.org) We study homeopathic learning, a phenomenon where poisoned training data with little effect on its own can create a hidden backdoor once clean, contrasting data is added. In our experiments, a model fine-tuned only on a "poison" dataset as…
Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs (arxiv.org) Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distributed across layers. In this work, we reveal a surprising…
ReplayAudit: Auditing Outsourced LLM Fine-Tuning via Sealed Records and Selective Replay (arxiv.org) Auditing outsourced LLM fine-tuning requires checking an untrusted provider's training while controlling verification cost. Uniform sampling ignores training-time signals, while checking a suspicious record can require replaying several pr…
Reconstructing LLM Lineage Trees via Fine-Tuning Path Inference (arxiv.org) Fine-tuned (FT) derivatives of large language models (LLMs) are now released in large numbers, yet their provenance relies on self-reports and cannot be verified. Misattributed lineage leaves a verifier unable to audit outsourced fine-tuni…
Not All Misalignment Is Alike: A Decomposed Benchmark for Emergent Misalignment (arxiv.org) Emergent misalignment (EM), in which fine-tuning on a narrow task induces misaligned behaviour on unrelated prompts, has been demonstrated across a wide range of datasets, model families, and training setups. Yet, most studies continue to…
Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering (arxiv.org) Two questions regarding practitioners' use of patent embeddings arise: (i) Does one fine-tuning recipe suffice for all downstream applications? (ii) Is fine-tuning on one patent landscape sufficient for downstream application on other land…
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization (arxiv.org) Differentially private (DP) zeroth-order (ZO) fine-tuning of large language models (LLMs) inherits DP's harsh utility-privacy trade-off. PAC Privacy, an instance-based notion, offers an alternative.
Guardrail Erosion in Domain Fine-Tuned Large Language Models: A Paired Audit of Knowledge-Centric Abstention in Biomedical LLMs (arxiv.org) The number of domain fine-tuned models, particularly in the biomedical domain, is rapidly expanding. Fine-tuning promises high-performing models at a fraction of the cost of training a base model, but it can also produce unintended behavio…
SAFTAC: Simulation-Augmented Fine-Tuning of Open-Source LLMs for Analog Circuit Design (arxiv.org) Circuit design for analog integrated circuits (ICs) is challenging because circuit topology, device sizing, and circuit performance are tightly coupled. Existing large language model (LLM)-based methods either rely on inference-time correc…
I'm going to try to turn a 4B model into a genuinely useful coding agent that can compete with much larger models (www.reddit.comhttps) After releasing 3D Coder and GLB 3D Coder, I realized I should have involved the community earlier. Your feedback directly influenced the second model and could have saved me some additional training runs.
Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning (arxiv.org) Low-rank adaptation (LoRA) is commonly used in both supervised fine-tuning (SFT) and reinforcement learning (RL), often under similar optimizer settings. Yet, how the different dynamics of the two stages manifest in the low-rank increment…
FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning (arxiv.org) Reinforcement learning (RL) has become a standard post-training method for diffusion models, but most RL objectives roll out the current policy at every iteration, and with a many-step sampler these rollouts typically dominate the cost of…
Rethinking Data Selection: The Importance of Coverage over Difficulty in LLM Fine-Tuning (arxiv.org) Selecting high-quality training data can reduce computation cost for LLM fine-tuning. Prior work on data selection developed a variety of difficulty-based scores, motivated by the intuition that harder examples are more informative.
Measuring the Marginal Value of Verifier-Based RL Beyond Supervised Fine-Tuning (arxiv.org) Reinforcement learning from verifiable rewards (RLVR) is usually evaluated as an alternative to supervised fine-tuning (SFT), or as a stage that follows it, with the implicit assumption that the RL stage contributes gains of its own. We me…
Interactive Backdoor Attacks in LLM-Based Proactive Dialogue Systems (arxiv.org) LLM-based proactive dialogue systems provide goal-directed support through sustained interaction. Fine-tuning their underlying models on externally supplied dialogue data introduces backdoor risks.
Pediatric ECG foundation model via adult-to-pediatric fine-tuning: downstream tasks and clinical evaluation (arxiv.org) Pediatric electrocardiogram (ECG) interpretation is complicated by age-dependent electrophysiological changes that limit direct transfer of adult AI models. We asked whether an adult ECG foundation model could be adapted to pediatric ECGs.
MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels (arxiv.org) Reinforcement learning (RL) fine-tuning improves large language model (LLM) reasoning using reward signals on sampled rollouts. When reward labels are available for only a subset of rollouts, restricting policy updates to that subset leave…
Reinforcement Learning in GUI Agents: A Survey Toward Digital Inhabitants (arxiv.org) Graphical User Interface (GUI) agents have emerged as a promising paradigm for intelligent systems that perceive and interact with graphical interfaces visually. Yet supervised fine-tuning alone cannot handle long-horizon credit assignment…
Re-examining low-rank adaptation for private LLM fine-tuning (arxiv.org) Privacy is a central concern when fine-tuning large language models (LLMs) on sensitive data, and differentially private stochastic gradient descent (DP-SGD) provides formal privacy guarantees. When performance at a fixed privacy level lar…
Efficient Collaborative LLM Fine-Tuning over Heterogeneous Mobile Devices via Many Backbones to One Side-Network Tuning (arxiv.org) Collaboratively fine-tuning (FT) large language models (LLMs) over heterogeneous mobile devices fosters immense potential applications of personalized intelligence. However, such a vision faces critical system challenges.
Eco-MobiLLM: Efficient LLM Fine-Tuning on the Mobile Device via Server Assisted Side-Tuning (arxiv.org) There is a huge gap between numerous intriguing applications fostered by on-device large language model (LLM) fine-tuning (FT) from fresh mobile data and the limited resources of a mobile device. While existing server-assisted methods (e.g…
Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning (arxiv.org) Scientific discovery requires plausible designs spanning useful trade-offs among competing properties: a molecule with strong binding affinity, for example, may be difficult to synthesize. Pretrained generative models, e.g., flow and diffu…
Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference (arxiv.org) Pervasive Capability Degradation (PCD) refers to the phenomenon in which supervised fine-tuning (SFT), while adapting a large language model (LLM) to a target task, often erodes the general capabilities acquired during pre-training. Prevai…
Self-Privileged Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) is a key component of language model post-training, enabling effective task adaptation and serving as an important foundation for subsequent reinforcement learning. Recent methods seek to improve SFT by introdu…
Alignment Dynamics in LLM Fine-Tuning (arxiv.org) Although Large Language Models (LLMs) achieve strong alignment through supervised fine-tuning and reinforcement learning from human feedback, the alignment is often fragile under subsequent fine-tuning. Existing explanations either attribu…
VidVec: Unlocking Video MLLM Embeddings for Video-Text Retrieval (arxiv.org) Recent studies have adapted generative Multimodal Large Language Models (MLLMs) into embedding extractors for text-vision tasks, typically by fine-tuning them to produce universal representations. However, their performance on video-text r…
Knowingly Wrong: Training LLMs Against Their Own Beliefs can Induce Misalignment (arxiv.org) Fine-tuning a large language model (LLM) on harmful data in a narrow domain, such as bad medical advice, can make it broadly misaligned. How realistic this risk is remains unclear, since models are very rarely fine-tuned on explicitly harm…
Beyond Fine-Tuning: Evaluating Prompt Engineering and Prompt Tuning for Roman Urdu Hate Speech Detection (arxiv.org) Low Resource Languages (LRLs) pose significant challenges for hate speech detection because of a scarcity of readily available annotated data, language variation, non-standard spelling, and context-dependent ambiguity. Beyond the problem o…
Tune-Up Open-Weight CLIP: Optimization Framework for Self-Supervised Fine-tuning of CLIP (arxiv.org) FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning (arxiv.org) Training Agents to Fail Predictably: Fine-Tuning for Forecastable Tail Risk (arxiv.org) Auditing GeoFM Evaluation for Field-Extent Segmentation: Label Proxies, Baselines, and When Frozen Features Match Fine-tuning (arxiv.org) Diagnosing Zero-Shot Forgetting in CLIP Fine-Tuning: Attention Drift Characterizes Adaptation, Representation Drift Predicts Transfer Degradation (arxiv.org) Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap (arxiv.org) ENGINE: Endogenous Variational MultiScale Optimization for Zeroth-Order LLM Fine-Tuning (arxiv.org) Sensitivity Inflation from Cross-Record Precision Routing in Private Quantized Fine-Tuning (arxiv.org) ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning (arxiv.org) Preprocessing Choices in Fine-Tuning Data Matter: Utility and Demographic Sensitivity in LLM-Based Recommendation (arxiv.org) Train the Sampler You Run: Few-Step Masked Diffusion Language Models via Mixture-Likelihood Fine-Tuning (arxiv.org) Saidzoda-Tajik-E5-Large: Fine-Tuning a 560M Embedder for Tajik Retrieval (arxiv.org) Finding the passage that answers a question is the first step of most practical systems built on documents, and for Tajik we found no openly described model for it. We present Saidzoda-Tajik-E5-Large, a Tajik text embedder built from multi…
Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims (arxiv.org) Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance…
Labadain-MedQA: A Tetun Medical QA Dataset for Fine-Tuning and Evaluating Large Language Models (arxiv.org) Advances in large language models (LLMs) have created opportunities for cross-lingual transfer of medical knowledge from high-resource to low-resource languages to support access to healthcare information. However, medical question-answeri…
Towards Multi-Value Alignment via Value Localization in LLMs (arxiv.org) Aligning large language models (LLMs) with multiple, potentially conflicting human values remains challenging, as existing multi-objective alignment methods often require costly fine-tuning and provide limited insight into the internal mec…
Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Training (arxiv.org) Fine-tuning pre-trained Large Language Models (LLMs) for downstream tasks using First-Order (FO) optimizers presents significant computational challenges. Parameter-Efficient Fine-Tuning (PEFT) methods address these by freezing most model…
When Does Jacobian Fine-Tuning Stabilize Neural PDE Emulators? (arxiv.org) Neural emulators of chaotic PDEs are trained on one-step prediction but deployed autoregressively, where their errors compound. Jacobian-based regularisers aim to curb this growth, but in a chaotic system part of the growth is physical and…
TurnTurk-GPT: Text-Based Turn-Taking Prediction for Turkish (arxiv.org) This paper presents TurnTurk-GPT, a text-based turn-taking predictor for Turkish obtained by fine-tuning a Turkish GPT-2 model with the TurnGPT objective on synthetic and real dialogue, and evaluates it on real Turkish conversation against…
Precision--Recall Trade-off on the Inference Level for LLMs in Grammatical Error Correction (arxiv.org) Large language models (LLMs) have recently been adopted for Grammatical Error Correction (GEC), achieving strong performance via both full fine-tuning and prompt-based approaches. As probabilistic sequence models, LLMs expose inference-tim…
Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA (arxiv.org) Memorization in large language models (LLMs) makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tu…
EvoMCP: Co-evolving MCP agents and training data (arxiv.org) Supervised fine-tuning of Model Context Protocol (MCP) agents is typically conducted on a static data distribution, which fails to adapt to the student's evolving capabilities and leaves newly exposed weaknesses insufficiently addressed. T…
↯ Fine Tuning↯ Model Context Protocolmodel-context-protocolfine-tuningmcp
Preserving Safety During Sequential Fine-Tuning: A Factorial and Geometric Analysis of Replay (arxiv.org) Benign fine-tuning can erode safety alignment in large language models, and sequential adaptation substantially compounds this risk. Prior work mitigates this erosion by replaying harmful-prompt refusals and benign-compliance examples, whi…
Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations (arxiv.org) Fine-tuning an LLM to maximize performance on a downstream task requires finding the optimal data and model configuration. This is a black-box optimization problem that Bayesian optimization (BO) addresses by sequentially evaluating traini…
Sparse Fine-Tuning for Parameter-Efficient Adversarial Training (arxiv.org) Deep neural networks achieve strong performance across many tasks but remain vulnerable to adversarial perturbations. Adversarial training (AT) is one of the most effective defenses, yet it suffers from high memory cost on large models and…
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation (arxiv.org) LLM-based generative recommendation reads long, text-rich interaction histories but generates only short semantic identifiers (SIDs). This asymmetry makes supervised fine-tuning inefficient: encoding the history is expensive, yet each hist…
Supervised Fine-Tuning for Reasoning Large Language Models: A Survey (arxiv.org) Large reasoning models have recently demonstrated remarkable capabilities in solving complex tasks, where supervised fine-tuning (SFT) on long chain-of-thought data serves as a crucial foundation for eliciting and enhancing their reasoning…
Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps (arxiv.org) While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use. Conversely, smaller models trained via supervised fine-tuning (SFT) or offline reinf…
seems like i get about $4000 per month's worth of usage out of my $100 claude code subscription (www.reddit.comhttps) I set up grafana to track my tokens with claude code. it's still a work in progress, and very likely inaccurate, but i'm still fine-tuning and gathering data .
When is Fine-Tuning Scalable? Certified Recovery from Noisy Data and Linear-Cost Fine-Tuning on Structured Task Shifts (arxiv.org) Transformers often transfer across related tasks with little fine-tuning; we study this question in noisy nonparametric regression for continuous maps $f,g:[0,1]^d\to\mathbb{R}^D$, representing the source and shifted tasks, using a single-…
Robust LLM Data Selection via Content-Invariant Gradient Attribution (arxiv.org) Gradient-based data attribution has become an important tool for high-quality data selection in the instruction fine-tuning of large language models (LLMs), as it estimates the contribution of training examples through their influence on m…
Freeze, Don't Clone: Fine-Tuning with Expressive Policies Without Retaining Offline Data (arxiv.org) Offline-to-online reinforcement learning uses offline pretraining to accelerate subsequent online learning. While many methods assume the offline dataset remains available throughout fine-tuning, there are many settings where this is not t…
Reversion, Retention, and Semantic Intervention: Measuring Functional Content in CLIP Fine-Tuning Deltas (arxiv.org) Fine-tuning adapts pretrained models to new tasks, but identifying which weight changes support the resulting behavior remains difficult. Weight-space interpretability assigns semantic descriptions to these changes, called fine-tuning delt…
Amplify, Locate, Estimate (ALE) on Rare-Failure Rates of Fine-Tuned LLMs (arxiv.org) Safety fine-tuning can make undesirable model behaviors sufficiently rare that measuring their frequency by direct sampling becomes prohibitively expensive. We introduce Amplify, Locate, Estimate (ALE), a method for estimating such rare fa…
Automatic Configuration of LLM Post-Training Pipelines (arxiv.org) LLM post-training pipelines that combine supervised fine-tuning and reinforcement learning are difficult to configure under realistic compute budgets: the configuration space is high-dimensional and heterogeneous, stages are strongly coupl…
FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraints (arxiv.org) Federated Learning with LoRA fine-tuning offers an efficient and privacy-aware solution for institutions to collaboratively leverage their large datasets to train VLLMs. However, participating institutions often possess heterogeneous compu…
Unlocking the Specialization Capability of PPG Foundation Models for Wearable Signals through Structured Pruning and Adaptive Fine-Tuning (arxiv.org) Photoplethysmography (PPG) plays a central role in wearable health monitoring and clinical decision support. However, existing approaches to universal PPG representation learning are predominantly trained on high-quality large-scale datase…
A Comprehensive Framework for All Stages of Large-Model Training: From Data and Tokenization, through Pretraining, to Post-Training Alignment (arxiv.org) The capabilities of large language models (LLMs) and multimodal LLMs (MLLMs) arise from the synergy of the full training pipeline, including data and tokenization, pretraining, and post-training (supervised fine-tuning, preference alignmen…
HIFI: A Highly Efficient Framework for Statistical Inference with Foundation Models (arxiv.org) Foundation models provide powerful representations for unstructured data, such as text and images, and parameter-efficient fine-tuning (PEFT) makes their adaptation to downstream tasks increasingly practical. For neural estimators that lac…
Mechanistic Insights into LoRA: Layer Sparsity for Adaptive Fine-Tuning via Path Patching (arxiv.org) Low-Rank Adaptation (LoRA) has emerged as the dominant paradigm for parameter-efficient fine-tuning of large language models. Standard all-layer LoRA applies adaptation throughout the transformer, despite heterogeneous contributions from i…
Task Format as a Factor in Adaptation: A Controlled Comparison of LoRA and Full Fine-Tuning for Prompt-Safety Classification (arxiv.org) Fine-tuning a language model for classification involves many design decisions. Comparisons of low-rank adaptation (LoRA) with full fine-tuning are common but often confounded, and task format is rarely studied.
Whispering Wisely: Smart Subset Selection for ASR Fine-Tuning (arxiv.org) Automatic Speech Recognition (ASR) models such as OpenAIs Whisper have substantially improved speech recognition quality but remain expensive to fine-tune, especially when large amounts of transcribed audio are available. This makes it cha…
Claude decided to run an experiment on itself (www.reddit.com via reddit) So I was asking Claude (Sonnet 5.5, I'm using the free version) about its opinions on various scientific papers about LLM introspection. Then it saw one of the cited sources - this one about fine-tuning a model to become good at predicting…
DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization (arxiv.org) LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning (arxiv.org) As large language models (LLMs) process increasingly long sequences, attention becomes a major computational bottleneck. Sparse attention mostly targets inference, yet many methods still train densely or cannot support sparse training, and…
Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery (arxiv.org) Fine-tuning on datasets of unknown origin can create a backdoor in the trained model when the data includes poisoned examples. Difficult to localise and remove, backdoors can persist through safety training.
Efficient Adjoint Matching for Fine-tuning Diffusion Models (arxiv.org) Reward fine-tuning has become a common approach for aligning pretrained diffusion and flow models with human preferences in text-to-image generation. Among reward-gradient-based methods, Adjoint Matching (AM) provides a principled formulat…
One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning (arxiv.org) In Large Language Model (LLM) fine-tuning, parameter and data selection are common strategies for reducing fine-tuning cost, yet they are typically driven by separate scoring mechanisms. When a parameter mask and data subset jointly determ…
When Thoughts Become Memory: Unifying In-Context Learning and Fine-Tuning (arxiv.org) We are the first to give a certified rule for whether unseen data belong in a pretrained model's context or its weights, built on an exact law for their information value. That value, the code length saved on held-out queries, equals in ex…
ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation (arxiv.org) Text clustering is a fundamental task in natural language processing, yet traditional clustering algorithms with pre-trained embeddings often struggle in domain-specific contexts without costly fine-tuning. Large language models (LLMs) pro…
Fine-Tuning Small Reasoning Models for Quantum Field Theory (arxiv.org) Despite the growing application of large language models (LLMs) to theoretical physics, there has been little academic exploration of how domain-specific physics reasoning ability develops during training. To investigate this, we perform t…
Where Do LLMs Learn to Recommend AI? (arxiv.org) Asked what to study, which career to pursue, or what product to build, large language models (LLMs) often recommend AI-related options. We ask where this tendency comes from by comparing models before and after fine-tuning and by varying h…
When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning (arxiv.org) Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation.
VHDLBench: Benchmarking and Fine-Tuning Large Language Models for VHDL-Based Chip Design (arxiv.org) The rapid evolution of digital circuit complexity demands innovative approaches to hardware design automation. While Large Language Models (LLMs) have influenced software development, their application to Hardware Description Language (HDL…
NormTune: Norm-Guided Parameter-Efficient Fine-Tuning for Reasoning-Intensive Tasks (arxiv.org) Full fine-tuning has achieved remarkable success on reasoning-intensive tasks, but it often requires substantial GPU resource consumption. Parameter-efficient fine-tuning (PEFT) provides a promising alternative for reducing this cost, yet…
A Small Model Asks Better: Fine-Tuning Query Clarification for Retrieval-Augmented Generation (arxiv.org) Query clarification is essential for retrieval-augmented generation (RAG) systems that face ambiguous user queries. We consider multi-turn grounded conversational scenarios where a user query is either ill-formed or ambiguous in the contex…
How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications (huggingface.co) How Specializing the Vocabulary of SLMs Can Cut Inference Cost by 77% in Real Applications Models specialized for a domain or a task by fine-tuning general open-source models, such as Qwen and Gemma, inherit a vocabulary built for many dom…
OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning (arxiv.org) Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models but suffers from catastrophic forgetting when learned updates interfere with the dominant singular directions that encode essential pre-trained knowledge. We…
MuPACE: Learning to Pace Muon for LoRA Fine-Tuning (arxiv.org) Learning rate schedules are typically fixed before training begins, yet when fine-tuning large language models with low-rank adapters (LoRA), the appropriate scale varies substantially with the model, task, and adapter configuration. Pract…
CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning (arxiv.org) Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforcement learning (RL)-based post-training on high-quality reasoning data. However, reproducing and ex…
Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs (arxiv.org) Large Language Models (LLMs) often struggle with temporally grounded reasoning because their static parametric memory is misaligned with the dynamic evolution of real-world knowledge. Although RAG and Parameter-Efficient Fine-Tuning (PEFT)…
Fine-Tuning Small Language Models for Reliable VASP INCAR Generation (arxiv.org) Language models can prepare VASP INCAR files from natural-language requests, but so far only large proprietary cloud models come close to handling the tightly coupled, physics-sensitive settings reliably, a dependence that fits poorly with…
How Fine-Tuning Objectives Shape Layer-Wise Information in LLM Hallucination Detection (arxiv.org) Under matched conditions, classifier-head fine-tuning (\textsc{cls}) paradigm consistently outperforms the LLM-as-Judge sequence-to-sequence (\textsc{s2s}) on binary hallucination detection. We trace this gap to the representations induced…
SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning (arxiv.org) Scaling Parameter-Efficient Fine-Tuning for Brain Encoding (arxiv.org) Is it RL or Data? A Controlled Comparison of GRPO and Rejection-Sampling Fine-Tuning (arxiv.org) NADS: Navigator-Guided Data Selection for Mitigating Catastrophic Forgetting in Fine-Tuning (arxiv.org) PACT: NVFP4 Data-Flow LoRA Fine-Tuning on Consumer GPUs (arxiv.org) Distribution-Based Regularization for Reward Fine-Tuning Diffusion and Flow Models (arxiv.org) Learning from Convenience Samples: Fine-Tuning LLMs to Predict Voting Choices from Biased Survey Samples (arxiv.org) An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs (arxiv.org) Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models (arxiv.org) Recent advancements in motion forecasting have been significantly accelerated by self-supervised pre-training techniques. However, current methods typically rely on extensive fine-tuning of pre-trained models for downstream tasks like traj…
Distilling LLM Feedback for Lean Theorem Proving (arxiv.org) Post-training for reasoning models typically combines supervised fine-tuning with reinforcement learning from verifiable rewards, most commonly with GRPO. However, this algorithm suffers from sparse rewards, limited exploration, and mode c…
Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation (arxiv.org) Fine-tuning large language models (LLMs) on downstream tasks often weakens their previously aligned safety behavior, creating an inherent trade-off between task adaptation and safety preservation. Existing approaches attempt to mitigate th…
Safety Offset Learning: When Benign Fine-Tuning Misaligns LLMs (arxiv.org) Safety risks from fine-tuning are commonly characterized by the harmfulness of the training data:harmful fine-tuning is considered dangerous, while benign fine-tuning is often assumed to preservealignment. We argue that this view misses a…
PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models (arxiv.org) Medical data remain fragmented across institutional silos because of privacy, governance, and ownership constraints. Federated learning enables medical institutions to adapt large language models (LLMs) collaboratively without transferring…
When Safety Fails Collectively: Robust Alignment Against Harmful Fine-Tuning via Joint Weight Perturbations (arxiv.org) Harmful fine-tuning can rapidly erode the safety alignment of large language models, creating substantial risks when aligned models are adapted to downstream tasks. Existing alignment-stage defenses use harmful examples and simulated harmf…
Medical Fine-Tuning Reshapes, Not Reduces, Patient-Facing AI Risk: A Rubric-Based, LLM-Judge Evaluation Framework (arxiv.org) Patient-facing medical question answering with large language models (LLMs) is moving from research demonstrations to production deployments, yet evaluation remains anchored on factual correctness. Factuality is necessary but not sufficien…
DKnown-Jev: Fine-Tuning and Evaluation of a JEV-Style Model for Tasks Involving Normative Knowledge (arxiv.org) Tasks involving normative knowledge include concept interpretation, eligibility determination, procedure explanation, material requirements, and service-outcome assessment, all of which require rules and facts published, maintained, or rec…
Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces (arxiv.org) Parameter-Efficient Fine-Tuning (PEFT) has become the standard approach for adapting large language models (LLMs). However, despite reducing trainable parameter counts, many PEFT methods remain memory-bound by token-level activation interm…
ToPA: Block-wise Toeplitz Adaptation for Expressive and Efficient Fine-Tuning (arxiv.org) Parameter-Efficient Fine-Tuning (PEFT) has emerged as a prevalent strategy for adapting large pre-trained models. Among PEFT techniques, reparameterization-based methods, particularly those that employ low-rank or sparsity structures, have…
MetaCurate: Evolving Curation Filters for High-Quality Agentic Coding Data (arxiv.org) Supervised fine-tuning on agent trajectories is an effective way to improve coding agents, but selecting useful training examples requires more than identifying successful runs. A resolved patch may follow inefficient reasoning or unnecess…
Rank Measures of Fine-Tuning Updates Need a Noise Baseline (arxiv.org) Studies of what fine-tuning does to a language model often summarise the weight update $\Delta W$ by its rank. We compare the rank measures used for this against the same measures on a random matrix of the same shape, and against two harde…
Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning (arxiv.org) Fine-tuning large language models (LLMs) on proprietary records enables domain adaptation but introduces two challenges: (i) privacy leakage of memorized private data, such as personally identifiable information (PII), and (ii) balancing p…
Safeguard Flag on ML/Kaggle Workflow outside of claudes outputs. (www.reddit.comhttps) I’m a machine learning engineer competing in the Gemma 4 fine-tuning challenge on Kaggle. I’m building a small synthetic software engineering dataset with a training/holdout split to fine-tune a small model.
↯ Gemma↯ Fine Tuning↯ LLM Fine-Tuning Announcement↯ AI Model Releasegemmafine-tuning
ZenFlow: Enabling Stall-Free Offloading for LLM Training (arxiv.org) Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks like ZeRO-Offload treat all parameters equally and update th…
Topology-Inspired Fine-Tuning for Wind-Field Super-Resolution (arxiv.org) Scientific super-resolution (SR) models are typically optimized with pointwise reconstruction losses, yet scientific interpretation often hinges on extrema, connected regions, and multiscale scalar structure. We characterize this tension w…
Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning (arxiv.org) Federated learning (FL) is increasingly used to adapt a pretrained, centrally fine-tuned model to data that remain on user devices, beyond training a model from scratch. In this report, prepared for the 4th Federated Machine Learning Inter…
MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs (arxiv.org) Generative large language models (LLMs) have been shown to exhibit harmful biases and stereotypes. While safety fine-tuning typically takes place in English, if at all, these models are being used by speakers of many different languages.
Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning (arxiv.org) We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can…
Knowledge That Is Linear Before Fine-Tuning Generalizes After It (arxiv.org) Not all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond their own recall, from inert facts, which can be recalled but do not support such generalizat…
AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning (arxiv.org) Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, se…
Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs (arxiv.org) Fine-tuning APIs make frontier LLMs easy to customize, but they can also weaken safety alignment during fine-tuning. While prior work shows that benign supervised fine-tuning (SFT) can reduce refusal behavior, deployed fine-tuning pipeline…
Safe-FCL: Continual Risk Awareness and Fine-Grained Safety Aggregation for Federated Continual Learning of LLMs (arxiv.org) Federated fine-tuning enables large language models (LLMs) to learn collaboratively from distributed client data, but malicious client updates can compromise global model safety. As federated tasks evolve, changing malicious update pattern…
ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning (arxiv.org) Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning by learning low-rank weight updates. Increasing rank expands adaptation capacity, but does not guarantee that all learned directions improve downstream performance.
AdaptFuse: Training-Free Sequential Preference Inference via Externalized Bayesian Computation (arxiv.org) Large language models (LLMs) struggle to accumulate evidence across multiple rounds of user interactions, failing to update their beliefs in a manner consistent with Bayesian inference. Existing solutions require fine-tuning on sensitive u…
Reclaiming Low-Quality Instruction Data with Rule-Guided LLM Reconstruction (arxiv.org) Data selection for supervised fine-tuning (SFT) often discards low-scoring instruction examples that may retain useful supervision. We propose a rule-guided LLM reconstruction framework to reclaim this information.
Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) changes logit margins among non-target tokens at teacher-forced prefixes. Matching a local reference distribution keeps those probabilities and also penalizes margin expansions.
Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning (arxiv.org) Understanding Negative Learning in Large Language Model Fine-tuning (arxiv.org) How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO) (arxiv.org) SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs (arxiv.org) Confine to the Task, Anchor the Rest: Concept-Guided Retention Regularizer for Parameter-Efficient Fine-Tuning (arxiv.org) Fast and Accurate: Improving Personalized LLM Performance by Semantic Concentration Inspired Workload Split between RAG and Fine-Tuning (arxiv.org) Zeroth-Order Fine-Tuning of LLMs in Learnable Low-Rank Subspaces (arxiv.org) DEFT: Disentanglement-Enhanced Fine-Tuning for EEG Foundation Models (arxiv.org) Stability Regimes for Framing-Sensitive Fine-Tuning in Language Models (arxiv.org) Fine-Tuning Strategy, Not Just Data: A Statistically Rigorous Audit of Gender and Stereotypical Bias in Embedding Models for HR Recommendation Systems (arxiv.org) UZHCL_for_the_win at Touche: Encoder Fine-Tuning, Synthetic Augmentation, and an Underpowered Encoder-LLM Comparison (arxiv.org) Geometric Calibration of Supervised Fine-Tuning (arxiv.org) FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation (arxiv.org) StructZO: Structured Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning (arxiv.org) AutoSec-RAN: Specification-Grounded LLM Fine-Tuning for Automated Security Analysis of O-RAN (arxiv.org) Select or Rewrite: Oracle-Guided Fine-Tuning for Generative ASR Post-Editing from N-Best Hypotheses (arxiv.org) Toward System One Clinical Intelligence: Parameter-Efficient Fine-Tuning and Counterfactual Explainability of Decision Foundation Models (arxiv.org) Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning (arxiv.org) Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors---such as misaligned personas or political slant---that the curation procedure has en…
RARA: Robust Policy Fine-Tuning through Retrieval-Anchored Representation Alignment for Robotic Manipulation (arxiv.org) Pre-trained robot policies, such as Vision-Language-Action (VLA) models, provide a strong foundation for generalist robotic manipulation, but adapting them to new tasks with only a few demonstrations remains challenging. Directly fine-tuni…
Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning (arxiv.org) Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly bounded noise variance. Both assumptions can fail even in eleme…
A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges (arxiv.org) Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score. We compare them as maps.
Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning (arxiv.org) Open coding agents are commonly trained through supervised fine-tuning on interaction trajectories whose final patches pass the benchmark test suite. However, this outcome-level filtering criterion ignores process quality: trajectories tha…
Task-Centric Personalized Federated Fine-Tuning of Language Models (arxiv.org) Federated Learning (FL) has emerged as a promising technique for training language models on distributed and private datasets of diverse tasks. However, aggregating models trained on heterogeneous tasks often degrades the overall performan…
Agentic Critical Training (arxiv.org) Imitation learning (IL) teaches language-model agents to reproduce expert actions but not to distinguish them from plausible mistakes. Self-reflection methods expose models to alternatives yet use supervised fine-tuning (SFT) to imitate fi…
MAP4CS: A Multi-dimensional Data Pruning Framework for Efficient Code Retriever Fine-tuning (arxiv.org) Retrieval-Augmented Generation (RAG) has become a cornerstone in software engineering for enhancing Large Language Models (LLMs) with domain-specific knowledge. However, adapting retrievers to evolving code repositories remains challenging…
SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent models (i.e., the pretrained models before fine-tuning). Thi…
Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution (arxiv.org) Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support. Frequent concepts are more likely to be well learn…
Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention (arxiv.org) Parameter-efficient fine-tuning (PEFT) enables large language models (LLMs) to adapt to specialized tasks, but often at the cost of degrading general capabilities acquired during pretraining. Existing approaches primarily mitigate this tra…
Artifacts have turned 10-minute tasks into 3-hour side quests (www.reddit.com via reddit) Me at 2 PM: "I just need a quick Bash script to rename a bunch of .png files to lowercase." Claude: "Here is the script! I also built an interactive web dashboard in an Artifact where you can drag-and-drop folders, preview regex file mut…
Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks (arxiv.org) Recent defenses for safeguarding open-weight large language models (LLMs) are intended to prevent adversarial usage. Underlying these defenses is an assumption that new harmful behavior is learned through fine-tuning rather than elicited b…
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization (arxiv.org) Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the fine-tuning distribution remains limited. Our analysis reveals a selective reshaping of ex…
HE-OFT: Privacy-Preserving One-Shot Federated Fine-Tuning under Homomorphic Encryption (arxiv.org) Many organizations adapt large pretrained models to their own tasks by fine-tuning on private data. Several of these parties often hold data for the same task and wish to fine-tune a model together without pooling that data.
Are Parameter-Efficient Fine-tuning Methods Really Different? (arxiv.org) Parameter-efficient fine-tuning (PEFT) offers many parameterizations, yet their methodological and functional differences remain unclear. We compare six methods in language and diffusion models to examine how their parameterizations relate…
The Persona Hierarchy Model: Understanding Contextual Generalization in Fine-Tuning LLMs (arxiv.org) Language models are routinely fine-tuned under a fixed context, such as a generic system prompt, persona or domain-specific instruction, yet the learned behavior sometimes stays confined to that context and sometimes broadly generalizes to…
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation (arxiv.org) We present Adjacent Possible Exploration (APE), a selective fine-tuning method for adapting large language models that systematically explores parameter modifications while maintaining model stability. Inspired by evolutionary optimization…
SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing (arxiv.org) Fine-tuning-as-a-service enables users to adapt aligned large language models (LLMs) to specialized tasks, but malicious fine-tuning can erode refusal behavior while preserving task performance on legitimate inputs. We revisit recent layer…
A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks (arxiv.org) Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using often-inaccessible original data. In this data-free regime, we analyze where forgetting o…
MIRROR: From Imitation to Internalization in LLM Personalization (arxiv.org) The demand for personalized LLMs is shifting from style imitation toward content quality. We investigate whether self-distillation can bridge this gap in existing fine-tuning paradigm.
Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning (arxiv.org) Pixel-space diffusion models avoid the lossy VAE of latent models, which suggests an advantage on downstream tasks where fine-grained detail matters. We test this claim along both routes to a pixel-space backbone.
Successive Training Stages and Large Language Model Persuasion: Effects of Misalignment, Supervised Fine-Tuning, and Preference Optimization (arxiv.org) Large language models (LLMs) can be tuned to influence human attitudes, yet the respective contributions of successive post-training stages remain un-clear. This study examines how three successive training stages affect LLM persuasiveness…
One Model Family, Two Gold-Level Results: Fine-Tuning Nemotron for IOI and IMO (huggingface.co) One Model Family, Two Gold-Level Results: Fine-Tuning Nemotron for IOI and IMO Our recent results show that Nemotron is a strong, adaptable foundation for building world-class specialist models. Starting from Nemotron 3, our teams used sup…
Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification (arxiv.org) Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of maintaining the general performance of the model while allowing it to gain new skills. A valuable goal f…
Emoception: Selective Affective Layer Fine-Tuning of Video Vision Transformers for Player Arousal Change Recognition From Gameplay Footage (arxiv.org) This article proposes Selective Affective Layer Fine-Tuning (SALFT), an efficient adaptation framework for Video Vision Transformers in player arousal recognition from gameplay. To bypass computationally expensive full fine-tuning, SALFT i…
Federated Mixture-of-Experts Alignment on Mobile Edge Networks under Data Heterogeneity (arxiv.org) The growing demand for on-device large language model (LLM) services on mobile edge devices has driven the adoption of Mixture-of-Experts (MoE) architectures, which scale model capacity with limited computation. Since fine-tuning MoE-based…
Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models (arxiv.org) Fine-tuning has become the dominant paradigm for adapting Vision-Language Models (VLMs), yet most approaches rely on explicit weight updates that introduce a fundamental trade-off. Full Fine-Tuning (FFT) may perturb pretrained representati…
Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning (arxiv.org) Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text exhibits a fundamentally different uncertainty structure from general text: dense low-confidence r…
Principles that Guide, Actions that Inform: Agent Evolution via Knowledge Abstraction (arxiv.org) Large language model (LLM) agents have demonstrated strong capabilities in interactive environments, yet their ability to continually evolve from experience remains limited. Although fine-tuning enables adaptation, its dependence on parame…
Attenuated in-context identification in time-series foundation models: diagnosis under counterfactual inputs and repair by synthetic forced-system fine-tuning (arxiv.org) Covariate-aware time-series foundation models (TSFMs) promise training-free what-if answers for instrumented plants: the change in output that a different future input would cause. We test this on forced engineering systems with exact coun…
DART-ES: Difficulty-Aware Reweighting and Targeted Replay for Fine-Tuning LLMs with Evolution Strategies (arxiv.org) Evolution Strategies (ES) enable memory efficient full parameter fine-tuning of large language models (LLMs) using only forward computation. However, standard ES uniformly averages rewards across problems and compresses problem level popul…
Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment (arxiv.org) Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs. Existing safety alignment approaches, including Supervised Fine-Tuning (SFT) and Reinforce…
↯ Fine Tuning↯ Security↯ Jailbreakjailbreakfine-tuningsecurity
Dynamic Positional Attention Modulation for Parameter-Efficient Fine-Tuning of Large Language Models (arxiv.org) Parameter-efficient fine-tuning (PEFT) has become a standard approach for adapting large language models to downstream tasks. However, most existing PEFT methods rely on uniform and static adaptations, without accounting for the structured…
A Fine-Grained Analysis of the LoRA Fine-Tuning Landscape with Implications for Data Selection (arxiv.org) RoSA: Rotational Sparse Adaptation for Memory-Efficient Fine-Tuning (arxiv.org) Learning without Overwriting: A Theory of Self-Distillation and Supervised Fine-Tuning in Continual Reasoning (arxiv.org) Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience (arxiv.org) HeuFouFT: Task-Guided Metaheuristic Coordinate Search for Fourier Fine-Tuning (arxiv.org) Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training (arxiv.org) Data-Driven Personas for Survey Simulation: Insights into Simulation Alignment Across Data-Access Regimes (arxiv.org) However, many existing steering approaches rely on target-domain human data for fine-tuning or prompting that is costly to collect and raises privacy concerns. In this paper, we study demographic group-level survey simulation, where person…
DiMOS: Doob-Guided Inference-Time Multi-Objective Search for Scientific Design (arxiv.org) Scientific design often requires jointly satisfying multiple objectives and constraints. Pretrained masked diffusion models provide a generative foundation for this task, but fine-tuning them to meet these objectives and constraints incurs…
Topology-Conditioned Backdoors: Language Models That Insert Vulnerabilities When They Infer They Are in a Multi-Agent System (arxiv.org) A language model may behave safely in a single-agent evaluation yet produce vulnerable code when its context suggests that it is part of a multi-agent system. We study this failure mode by fine-tuning Qwen2.5-7B-Instruct to condition code…
A Tropical Geometry View of Forgetting: A Per-Unit Projector for Knowledge-Preserving Fine-Tuning (arxiv.org) Fine-tuning a language model on new text degrades what it already does. Replay-free projectors such as Adam-NSCL and GPM forbid one shared subspace of a layer's inputs in every row of the update.
Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotations (arxiv.org) Spatial intelligence is a fundamental skill in multiple domains, such as Science, Technology, Engineering, and Mathematics (STEM), Medicine, Architecture, and Construction. Recent studies indicate that Vision-Language Models (VLMs) still f…
IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models (arxiv.org) How To Train Your World Model: Fine-tuning vs RAG for LM-based World Modeling (arxiv.org) World models (WMs) simulate the transition dynamics of environments, enabling agents to plan over the consequences of their actions. In text-based environments, fine-tuning a Language Model (LM) to serve as a WM has emerged as a dominant p…
AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning (arxiv.org) Fine-tuning pre-trained Vision-Language Models (VLMs) for robotic manipulation introduces a fundamental stability-plasticity dilemma: continuous flow-matching action experts backpropagate concentrated, low-rank regression gradients into tr…
SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs (arxiv.org) This work presents SeedFlood, a new approach to decentralized LLM fine-tuning designed to scale across large models, large collaborations, and complex network topologies while achieving global consensus with negligible communication overhe…
Same Scene, Different Task: Skill Alignment for Compositional Generalization in VLAs (arxiv.org) Vision-language-action (VLA) models often struggle to generalize to skill combinations absent from their fine-tuning demonstrations, even when every constituent skill has been demonstrated. We focus on a vision shortcut as one failure mode…
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs (arxiv.org) Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present \textbf{ChainLoRA}, a replay-free continu…
TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning (arxiv.org) Full-parameter fine-tuning of large language models (LLMs) incurs substantial optimizer state memory overhead, limiting the model sizes that fit on modern GPUs. Existing approaches either compress optimizer state, abandon first-order gradi…
Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning (arxiv.org) Step-size selection remains a central challenge in large-scale neural network optimization; conservative steps slow convergence, while aggressive steps can destabilize it. We combine \textbf{Z}ero-and-\textbf{F}irst-\textbf{O}rder optimiza…
FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization (arxiv.org) Federated Learning (FL) enables privacy-preserving fine-tuning of Large Language Models (LLMs), yet the massive communication overhead remains a critical bottleneck. Furthermore, applying Low-Rank Adaptation (LoRA) in FL faces a fundamenta…
Every Batch Is Its Own Validation Set: Leave-One-Out Gradient Matching for Online Data Selection in LLM Fine-Tuning (arxiv.org) Online batch selection fine-tunes a language model on the most useful part of each candidate batch. Selectors that match the gradient of the candidate batch are attractive because they need no held-out data, yet they rarely beat training o…
Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allow (arxiv.org) Language models (LMs) are increasingly used for tasks that require reasoning about hidden variables from a few observations, for which Bayesian inference is the normatively correct solution. While supervised fine-tuning of an LM on the out…
Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning (arxiv.org) Fine-tuning adapts aligned large language models (LLMs) to downstream tasks, but a few dozen harmful examples can remove their refusal of harmful requests. Prior work localizes safety-related behavior to specific layers, directions, and to…
GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning (arxiv.org) Low-rank adaptation (LoRA) has become a dominant paradigm for parameter-efficient fine-tuning (PEFT) of large-scale deep learning models. However, its bilinear parameterization induces a parameter-dependent geometry: the mapping from train…
Compute Aligned Training: Optimizing for Test Time Inference (arxiv.org) Scaling test-time compute has emerged as a powerful mechanism for enhancing Large Language Model (LLM) performance. However, standard post-training paradigms, Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), optimize the likel…
No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse (arxiv.org) Iterative fine-tuning on synthetic data causes \emph{model collapse}: output diversity narrows as rare patterns are progressively lost, a signature most visible as phrase-level repetition. Existing mitigations either require model log-prob…
Federated Learning for LLMs over Mobile Networks: Issues and Solutions in the RAN Transport (arxiv.org) Federated LLM fine-tuning enables large models to be adapted using private and geographically distributed data at the network edge, creating recurring and deadline-sensitive communication workloads across access and transport networks. Thi…
Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents (arxiv.org) Self-improving GUI agents keep the trajectories they produce and return them to the agent, by fine-tuning or by retrieval into the prompt, and studies that compare the two destinations disagree. We attribute this to the unit of experience:…
PG-SFT: Balancing Capability Acquisition and Retention in Offline Agent Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) on offline agent trajectories is the standard approach for training specialized tool-using agents, but forcing models to imitate reasoning and actions token by token may harm other capabilities (e.g., general r…
Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection (arxiv.org) With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the s…
Comparison of techniques for fine-tuning open-weight models for entity extraction from radiology reports (arxiv.org) Converting free-text radiology reports into structured labels supports cohort building, quality assurance, and monitoring of clinical imaging models, but the strongest label extractors are hosted proprietary models whose use raises privacy…
Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD? (arxiv.org) Differentially Private Stochastic Gradient Descent (DP-SGD) is a leading approach for privacy-preserving fine-tuning of large language models (LLMs). Many decoder-only LLMs employ weight tying between input and output embeddings, a design…
Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing (arxiv.org) Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable…
RA-MoE: Routing-Aligned Fine-Tuning for Multilingual Adaptation of Mixture-of-Experts Models (arxiv.org) Mixture-of-Experts (MoE) models enable efficient LLM scaling, yet adapting them to non-English downstream tasks remains challenging. Standard multilingual fine-tuning largely ignores their heterogeneous routing structure.
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (arxiv.org) Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater representational plasticity, Low-Rank Adaptation (LoRA) can match or surpass FFT performance while constr…
Instruction Retrieval at Inference Time for Small Language Models (arxiv.org) The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which need specialized knowledge and follow multi-step procedures. Fine-tuning for a specific domain or ta…
Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting (arxiv.org) Supervised fine-tuning of discrete diffusion language models masks some response tokens and trains the model to recover their original values from the visible context. The masking pattern therefore determines both the context available to…
Backdoor Mitigation in Decentralized LLM Fine-Tuning (arxiv.org) Losing the name before the box: measuring and repairing what narrow fine-tuning costs a detector outside its deployment vocabulary (arxiv.org) Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness (arxiv.org) Layer-Informed Fine-Tuning via Three-Stage Functional Segmentation of LLMs (arxiv.org) VLM Fine-Tuning for End-to-End Combinatorial Optimization (arxiv.org) Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning (arxiv.org) From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning (arxiv.org) MeanFlowAdvantage: Stable Reward Fine-Tuning for Few-Step Average-Velocity Generators (arxiv.org) MeanFlow enables efficient few-step generation by predicting interval-average velocities, but this representation creates a mismatch for reward fine-tuning: existing advantage-based objectives are typically defined on instantaneous velocit…
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning (arxiv.org) Federated parameter-efficient fine-tuning enables clients to adapt pre-trained models without sharing raw data or communicating the full model, but statistical heterogeneity makes a single global adapter insufficient for personalized predi…
Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition (arxiv.org) Low-Rank Adaptation (LoRA) is a widely used approach to parameter-efficient fine-tuning (PEFT), yet a performance gap can remain relative to full fine-tuning (FFT). Many LoRA variants improve the initialization or optimization of low-rank…
CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuning (arxiv.org) Federated LoRA fine-tuning enables parameter-efficient adaptation of pre-trained models without sharing private data, but suffers from two fundamental mismatches under heterogeneous client data: a structural aggregation mismatch caused by…
Is Human-Readable Text Necessary for Effective LLM Fine-Tuning? (arxiv.org) Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations can preserve or improve adaptation utility without requiring a human-readable textual form.
Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning (arxiv.org) DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures (arxiv.org) Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning (arxiv.org) Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness (arxiv.org) Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks (arxiv.org) Two-Timescale Fine-tuning Provably Learns New Features for Two-Layer ReLU Networks (arxiv.org) The Privacy Fallacy of Crowdsourced Fine-Tuning: Extracting Proprietary Data via Topic-Based Poisoning (arxiv.org) Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment (arxiv.org) AIM-ZO: Activation-Informed Subspace Maintenance for Zeroth-Order LLM Fine-Tuning (arxiv.org) Optimizing the Phi-2 Small Language Model for Real-time Chatbot Applications Using Parameter-Efficient Fine-Tuning (PEFT) with QLoRA Quantization (arxiv.org) CFLoRA: Federated Fine-tuning of LLMs with Complementary Factors for Error-free Aggregation (arxiv.org) SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning (arxiv.org) What Should We Freeze? Guarded Freezing: Connectivity Shapes the Fine-Tuning of Pretrained Models (arxiv.org) PlurVA-LLM-2026 Shared Task Track-1: Pluralistic Value Alignment in LLMs via Multilingual Fine-Tuning and Threshold Calibration (arxiv.org) Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage (arxiv.org) Goal-Conditioned Supervised Learning for LLM Fine-Tuning (arxiv.org) Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning (arxiv.org) Concept-Guided Fine-Tuning: Steering ViTs away from Spurious Correlations to Improve Robustness (arxiv.org) Selective Fine-Tuning for Targeted and Robust Concept Unlearning (arxiv.org) MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference (arxiv.org) TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge (arxiv.org) SoFT: Soft Targets for Generalizable LLM Fine-Tuning (arxiv.org) DOHF: Online Diffusion Fine-tuning with Doob's $h$-transform Guidance (arxiv.org) EvoIn: Bridging Evolution and Internalization for Agent Fine-Tuning (arxiv.org) When Can First-Order Models of Fine-Tuning Bound Forgetting? (arxiv.org) Parameters vs. Context: TRACE Fine-Tuning for Robust Retrieval-Augmented Generation (arxiv.org) Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models (arxiv.org) Continual fine-tuning is essential for large language models (LLMs) to dynamically adapt to real-world environments, yet it inevitably suffers from catastrophic forgetting, particularly the performance degradation of previous tasks and LLM…
Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy (arxiv.org) Online reinforcement learning fine-tuning of pretrained flow-matching vision-language-action (VLA) policies promises robots that keep learning after deployment, but continued updates often destroy competence on individual tasks while the a…
Beyond Average Safety: Chance-Constrained LLM Fine-tuning (arxiv.org) Fine-tuning large language models on new objectives can improve helpfulness, instruction following, or domain-specific performance, but it can also induce regressions on safety-critical prompts. Existing safety-preserving fine-tuning metho…
Technical Manual for Toolkit for Confidence-Corpus Consistency via Fine-Tuning on a Fabricated Corpus (arxiv.org) A language model's confidence in an answer is often read as a proxy for how well it knows the corresponding fact. This manual documents an open toolkit built to test that reading directly: a small causal language model is fine-tuned on a c…
Baszta: Data-Centric Fine-Tuning of a Polish Multi-Label Safety Classifier (arxiv.org) We develop a multi-label Polish content-safety classifier by fine-tuning allegro/herbert-base-cased (124M) across five categories (hate, vulgarity, sexual content, crime, self-harm) using a Focal + R-Drop objective, and evaluate the result…
The Gaussian Is Enough: Flow-Matching Priors Do Not Help When Fine-Tuning Large Behavior Models (arxiv.org) Modern robot imitation learning increasingly relies on generative policies based on diffusion or flow-matching models, which generate actions by transforming samples from a prior distribution. A key question is whether the choice of prior…
Alignment Inertia: Auditing the Durability of Training Data Influence Through Policy Override Resistance (arxiv.org) Platform operators increasingly rely on system prompts and fine-tuning to govern model behavior, yet it remains unclear how reliably these interventions override behavior inherited from prior training. We propose Override Success Rate (OSR…
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following (arxiv.org) Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on general benchmarks.
Can One Adapted Model Do It All? Fine-Tuning Strategy Selection for Customer Support LLMs (arxiv.org) Production customer-support systems often require LLMs to support multiple skills, such as intent classification, question answering, summarization, or tool-use decisions. A central deployment question is whether these skills should be han…
Text-only adaptation in LLM-based ASR through text denoising (arxiv.org) Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet underexplored challenge. Standard fine-tuning of the LLM on the target domain text often disrupts…
ChainDoRA: Tensor-Train Factorized Weight-Decomposed Low-Rank Adaptation for Parameter-Efficient LLM Fine-Tuning (arxiv.org) Parameter-efficient fine-tuning (PEFT) adapts large language models (LLMs) to downstream tasks while updating only a small fraction of their pretrained parameters. Low-Rank Adaptation (LoRA) uses two trainable low-rank matrices, while Weig…
Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs (arxiv.org) Large Language Models (LLMs) demonstrate impressive capabilities across many applications but remain vulnerable to jailbreak attacks, which elicit harmful or unintended content. While model fine-tuning is an option for safety alignment, it…
↯ Fine Tuning↯ Security↯ Jailbreakjailbreakfine-tuningsecurity
The Limits of Simulated Societies: How Post-Training and Survey Fine-Tuning Erase Cross-Cultural Variance (arxiv.org) Using large language models (LLMs) to simulate diverse human populations has the potential to transform many aspects of computational social science, yet many evaluations score the average response rather than the spread of opinion within…
Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA (arxiv.org) Reinforcement learning (RL)-based post-training has become an effective approach for eliciting reasoning capabilities in large language models (LLMs). However, adapting post-RL models to new knowledge domains or behaviors through subsequen…
iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs (arxiv.org) On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teacher. This fixes teacher influence at the full-teacher endpo…
A Pinch of SFT, A Dash of RL: When Reinforcement Learning Helps Long-Horizon Advertising Agents (arxiv.org) Enterprise analytics agents solve long-horizon tool-use problems over distributed business data, requiring retrieval, reasoning, API calls, code execution, and adaptation to intermediate observations. Supervised fine-tuning (SFT) calibrate…
Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning (arxiv.org) Uncertainty quantification techniques measure confidence in language model outputs to support critical applications like hallucination detection and selective prediction. While prior work has developed various confidence metrics and demons…
Steering Vector Fields for Context-Aware Inference-Time Control in Large Language Models (arxiv.org) Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing a practical middle ground between prompting and fine-tuning. Yet SVs can be unreliable in pr…
LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning (arxiv.org) FORM is a domain-specific symbolic manipulation language widely used in particle physics for processing the very large algebraic expressions arising from multi-loop Feynman diagram calculations. Despite its central role in precision theore…
Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning (arxiv.org) Producing task-specific large language models requires discovering effective training strategies through experimentation. Automated fine-tuning systems have made this experimentation feasible with far less manual effort.
CAMFT: Conflict-Aware Mergeable Fine-Tuning for Large Language Models (arxiv.org) Model merging has emerged as a promising paradigm for integrating multiple task-specific capabilities into a single large language model. However, existing methods predominantly focus on post-hoc processing of independently fine-tuned mode…
GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training (arxiv.org) We study how post-training changes the weights of Large Language Models (LLMs) relative to their pretrained weights. Across 12 post-training chains with supervised fine-tuning (SFT) and reinforcement learning (RL), we express each weight u…
QLoRA Fine-Tuning of Ministral LLM for Sequence-to-Function Protein Annotation (arxiv.org) Functional annotation of newly sequenced proteins remains a bottleneck in molecular biology: the number of sequences in public repositories grows far faster than the capacity for manual curation. Most computational approaches consider anno…
Perplexity Predicts Protection: Choosing Pretrained Backbones for Worst-Client Fairness in Federated Parameter-Efficient Fine-Tuning (arxiv.org) Federated learning lets multiple parties train a shared model without pooling their data, but a client with far less data than the others can end up poorly served even when the group's average accuracy looks fine. We ask whether the choice…
Team DArgk at the 2026 ELOQUENT lab for evaluating generative language model quality: Residuals of Humanity: AI Detection Evasion via GRPO Fine-Tuning (arxiv.org) Large language models (LLMs) can generate fluent and coherent text that is increasingly difficult to distinguish from human writing, motivating the development of automatic AI-generated text detectors. However, the robustness of such detec…
Multilingual Safety Signals Are Multi-Layered: Filtering Safety-Degrading Data for Safer LLMs (arxiv.org) Preserving safety alignment during large language models fine-tuning is critical, however, recent studies have demonstrated that even benign fine-tuning data may contain safety-degrading samples that silently undermine safety alignment. Ex…
Cultural Alignment in Large Language Models Using Soft Prompt Tuning (arxiv.org) Large Language Model (LLM) alignment is commonly achieved through supervised fine-tuning or reinforcement learning, both of which require labeled or preference data and update model weights. Without targeted cultural adaptation, however, d…
SynthDemo-RL: Breaking the Zero-Reward Barrier in VLA Adaptation with LLM-Guided Synthetic Demonstrations (arxiv.org) Fine-tuning Vision-Language-Action (VLA) models commonly relies on human teleoperation demonstrations, while reinforcement learning (RL) with sparse binary rewards faces an exploration challenge when successful trajectories are rarely samp…
We are a small AI lab looking for 10 devs to blind-test our new fine-tuning method for coding tasks (100k initial free tokens in return and the more feedback you give the more tokens you get). (www.reddit.com via reddit) Hey everyone, We are a small AI research team working on a new fine-tuning approach for open-source models, specifically aiming to improve how they handle complex logic, refactoring, and edge cases in code generation. To see if our method…
QUATRO: Query-Adaptive Trust Region Policy Optimization for LLM Fine-tuning (arxiv.org) GRPO-style reinforcement learning (RL)-based LLM fine-tuning algorithms have recently gained popularity. Relying on heuristic trust-region approximations, however, they can lead to brittle optimization behavior, as global importance-ratio…
OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher (arxiv.org) As scaling pre-training data alone yields diminishing returns, post-training is becoming increasingly important across physical AI domains such as autonomous driving. End-to-end driving policies are pre-trained in open loop with behavior c…
CellRFT: Reinforcement Fine-Tuning for Single-Cell Perturbation Modeling (arxiv.org) Predicting cellular responses to perturbations supports the study of gene function, disease mechanisms, and therapeutic strategies. Despite advances in single-cell perturbation modeling, existing models typically optimize surrogate losses…
Fine-Tuning Models for Biomedical Relation Extraction (arxiv.org) Next-Generation Sequencing has revolutionized the study of genetic mutations, enabling large-scale investigations into their roles in disease development. However, extracting meaningful insights from the vast amount of biomedical literatur…
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL (arxiv.org) Agent trajectories record what an agent does and what happens next. Yet standard supervised fine-tuning (SFT) applies loss only to agent-authored action tokens, using environment observations as context but not as prediction targets.
Dataset-Dependent Effects of Cross-Depth Aggregation and Soft-Routed Experts in EEG Foundation Model Fine-Tuning (arxiv.org) EEG decoding tasks can rely on different temporal dynamics and cross-channel relationships. We test whether specialized modules improve a fully fine-tuned EEG foundation model by augmenting CBraMod with cross-depth Attention Residuals (Att…
Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT (arxiv.org) Large language models are commonly adapted to downstream tasks through supervised fine-tuning (SFT), but substantial distribution mismatch between downstream supervision and a model's generation distribution can intensify catastrophic forg…
AgentPack: A Dataset of Code Changes, Co-Authored by Agents and Humans (arxiv.org) Fine-tuning large language models for code editing has typically relied on mining commits and pull requests. The working hypothesis has been that commit messages describe human intent in natural language, and patches to code describe the c…
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs (arxiv.org) Speech Large Language Models (Speech-LLMs), typically built from a pre-trained speech encoder, a modality projector, and an LLM fine-tuned with Low-Rank Adapters (LoRA), have shown strong Automatic Speech Recognition (ASR) performance on g…
Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning (arxiv.org) Multi-turn agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both training and inference cost. We propose viewing each trajectory as a \emph{round-level depend…
SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale (arxiv.org) Limited controlled evidence exists on how training data, adaptation method, and model scale jointly affect tool-calling performance in language-model agents. We evaluate supervised fine-tuning (SFT) with LoRA, reinforcement learning (RL) v…
CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents (arxiv.org) Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing strategies from adaptin…
Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs (arxiv.org) Recent work by Doshi and Hauser (2024), Bisbee et al. (2024), and Xie et al.
Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning (arxiv.org) We present an efficient method to distill reasoning capabilities into compact video-language models (VLMs) for video question answering (VideoQA). Our approach fine-tunes a 2B-parameter model using only $\sim$900 uncertainty-selected examp…
ReMova: Fine-tuning LLMs for English to Belarusian translation (arxiv.org) This paper presents a Belarusian-specific data-cleaning pipeline and fine-tuning for English-Belarusian machine translation. Our cleaning pipeline distinguishes itself from others by employing a correction tool that addresses the issue of…
Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis (arxiv.org) Retrieval-Guided Fine-Tuning (RAG-FT) incorporates retrieved data directly into the training objective, but the statistical consequences of noisy retrieval during training remain theoretically undercharacterized. We study this question by…
Corpus Characterization and Inverse Constitutional Fine-Tuning for Style-Aware Radiology Reports (arxiv.org) Automated radiology report generation has advanced rapidly in diagnostic accuracy, yet generated reports frequently diverge from the stylistic conventions of authentic radiologist writing in structure, diction, and uncertainty language, a…
Token Merging for Multilingual Speech Recognition: A Systematic Study Across Model Scale and Fine-Tuning (arxiv.org) Leading multilingual speech recognition models like Whisper transcribe diverse, low-resource languages without language-specific training but are computationally expensive to deploy. Token merging mitigates this inefficiency by dynamically…
Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning (arxiv.org) Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multip…
NeuroProlog: Multi-Task Fine-Tuning for Neurosymbolic Mathematical Reasoning via the Cocktail Effect (arxiv.org) Large Language Models (LLMs) achieve strong performance on natural language tasks but remain unreliable in mathematical reasoning, frequently generating fluent yet logically inconsistent solutions. We present \textbf{NeuroProlog}, a neuros…
Route, Don't Fix: Regime-Dependent Decoding Correction and a Trajectory-Gated Router for Reliable Clinical LLM Answer Selection (arxiv.org) Large language models (LLMs) are often deemed unsafe for clinical question answering because of their tendency to hallucinate. Retrieval augmentation, fine-tuning, and external verifiers require new infrastructure that clinical governance…
LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference (arxiv.org) We introduce LayerRoute, a parameter-efficient method for adaptive transformer layer-skipping that combines per-layer hard-gated routing (trained via a straight-through estimator) with joint LoRA fine-tuning. LayerRoute augments each of th…
Adaptive Phase-Switching for Communication-Efficient Federated LoRA Fine-Tuning (arxiv.org) Federated fine-tuning of large language models with low-rank adaptation reduces per-client trainable parameters, but client-to-server communication remains the dominant cost. Existing accounting for federated LoRA protocols omits the asymm…
TryOnReward: Learning Foveated Consistency for Reinforcement Fine-Tuning of Virtual Try-On (arxiv.org) Virtual Try-On (VTON) aims to dress a person with the reference garment, producing visually reasonable results aligned with human preferences. Turning this preference-oriented goal into an actionable objective relies on a scoring function…
ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models (arxiv.org) Vision-Language-Action (VLA) models demonstrate strong generalization in robotic manipulation and navigation, but existing fine-tuning methods provide limited safety guarantees. Current approaches primarily rely on Lagrangian optimization…
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models (arxiv.org) Fine-tuning instruct models often improves target performance while inducing behavioral drift from the reference model, which can degrade existing capabilities. Rather than treating this drift as an uncontrolled consequence of optimization…
Safety as a Constraint: Fine-Tuning a LLM Recommender to Explain Itself (arxiv.org) Traditional recommender systems are typically trained to predict what item users will interact with next, but not why. However, offering personalized evidence for why a user might like the predicted item is an important way to enhance the…
FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks (arxiv.org) Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks. However, conventional fine-tuning typically relies on centralized data collection, bringing in privacy concerns.
ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (arxiv.org) Popular low-rank parameter-efficient fine-tuning (PEFT) methods represent adaptation as separate updates to selected backbone weights, without maintaining an explicit task-specific state that is updated and reused across depth. These updat…
Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning (arxiv.org) Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, limiting efficient adaptation. Naive large-batch fine-tuning repeatedly includes overrepresented sample groups while weakly covering u…
When do cheap embeddings beat protein language models? A theoretically-grounded hashing sketch for biological sequence classification (arxiv.org) \textbf{Motivation:} Pre-trained protein language models (PLMs) such as ESM-2 have become the default representation for biological sequence tasks, but they are computationally heavy and require GPUs both for embedding and for fine-tuning.…
Optimizing Three Critical Factors for Practical and Effective OOD Detection Fine-Tuning (arxiv.org) In out-of-distribution (OOD) detection, fine-tuning with auxiliary outlier data often improves detection performance at the cost of classification accuracy. This trade-off stems from the loss of the original in-distribution (ID) distributi…
Understanding LoRA Rank Trade-offs in Diffusion Model Fine-Tuning (arxiv.org) Selecting LoRA rank for diffusion fine-tuning requires balancing quality and compute cost. We present a controlled study on CIFAR-10 using a DDPM U-Net with ranks {2,4,8,16,32}, fixed optimization settings, and a reproducible local-folder…
Privacy-Preserving Split Learning for Federated LLM Fine-Tuning (arxiv.org) Fine-tuning large language models (LLMs) on domain-specific data is essential for downstream adaptation. In many deployments, a participant cannot hold the complete model locally.
Towards Stress-Aware Sentence-Level Filipino G2P With Weakly-Supervised ByT5 Fine-Tuning (arxiv.org) Grapheme-to-phoneme conversion (G2P) refers to the task of converting a sequence of graphemes to a corresponding sequence of phonemes. While Filipino G2P is fairly straightforward due to its shallow orthography, the inclusion of prosodic f…
Active Adaptation, Not Static Defense: Temporal Dynamics of Preventative Steering in Adversarial Fine-Tuning (arxiv.org) Large language models remain fragile against malicious fine-tuning, motivating training-time defenses against harmful persona drift. Preventative Steering injects undesirable-trait persona vectors during fine-tuning and removes them at eva…
Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both? (arxiv.org) In Retrieval-Augmented Generation (RAG) systems, a large number of retrieved chunks are concatenated to form the input context so that users can receive high-quality responses based on external knowledge. As a result, the input context len…
Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning (arxiv.org) Large language model safety and security research is preoccupied with, among other things, detecting and preventing jailbreak attacks: alignment bypasses that allow an adversarial user to elicit unwanted or harmful outputs from models. Arb…
↯ Fine Tuning↯ Security↯ Jailbreakjailbreakfine-tuningsecurity
MI-PEFT: Mixture-of-Experts Integrated Parameter-Efficient Fine-Tuning Protein Language Models Improves Acidophilic Proteins Classification (arxiv.org) Acidophilic proteins that remain stable and functional under highly acidic conditions, are important for industrial biocatalysis, acid-related bioprocessing, and the discovery of acid-stable enzymes. However, their identification relies he…
Geometry-Aware Bayesian Parameter-Efficient Fine-Tuning on the Stiefel Manifold via Stein Variational Gradient Descent (arxiv.org) Several geometry-aware approaches to low-rank adaptation have emerged for parameter-efficient fine-tuning of large pre-trained models. These methods aim to take full advantage of the geometric structure of low-rank manifolds for improving…
MpSub: A Momentum $p$-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models (arxiv.org) Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by estimating update directions from loss evaluations, but exi…
One Rate Is Not Enough: Adaptive Anisotropic Learning Rates for LoRA Fine-Tuning (arxiv.org) Low-rank adaptation (LoRA) has become the standard for parameter-efficient fine-tuning of large language models. Most LoRA variants follow a uniform-LR convention, applying a single global learning rate across every rank-one component of e…
Popular Knowledge Propagates More Errors in LLM Knowledge Updating (arxiv.org) Updating a language model's knowledge through fine-tuning is essential for keeping its outputs current, yet can also induce factual forgetting and new hallucinations. Prior work shows that long-tail knowledge is harder to acquire and newly…
Neuron-Guided Fine-Tuning: Unlocking Efficient Alignment Mechanisms for Large Language Models (arxiv.org) Existing Supervised Fine-Tuning paradigms, particularly Full Parameter Fine-Tuning are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting, which current methods typically address in isolation and…
LayerRoute: Input-Conditioned Adaptive Layer Skipping via LoRA Fine-Tuning for Agentic Language Models (arxiv.org) Agentic language model systems alternate between two structurally distinct step types: structured tool calls (short, deterministic, low perplexity) and open-ended planning/reasoning steps (long, complex, high perplexity). Despite this hete…
Linear Separability of Activation Representations after Supervised Fine-Tuning on Incorrect Responses: A Study of Synthetic Dishonesty in Large Language Models (arxiv.org) When a language model is fine-tuned to produce systematically incorrect responses, does this training leave a structured, linearly recoverable trace in its internal activations? We study this question in a controlled model-organism setting…
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks (arxiv.org) Open-weight language models can be rendered unsafe through several parameter-level interventions, yet models with matched harmful compliance can exhibit fundamentally different failure modes. We compare harmful supervised fine-tuning (SFT)…
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets (arxiv.org) Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training…
Clinician-Friendly Foundation Models for Ophthalmic Image Diagnostics without Fine-Tuning or Technical Barriers (arxiv.org) Artificial intelligence (AI) shows remarkable potential in medical imaging diagnostics, yet most current models require retraining when applied across different clinical settings, limiting their scalability. We developed GlobeReady, a depl…
SwiftExplorer: Training-free Diffusion Model Alignment with Swift Diversity Exploration (arxiv.org) Diffusion models have general generative abilities but struggle to align with specific objectives. Fine-tuning can improve alignment, yet its training cost is often prohibitive.
Steering Geometry: Validating Human Value Geometry in LLM Steering Space (arxiv.org) As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inference-time alternative to fine-tuning methods (e.g., RLHF, DPO) for behavioral control. Howeve…
FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon (arxiv.org) Federated fine-tuning adapts large language models (LLMs) to decentralized client data, but its scalability in cross-device training is often limited by the high communication cost. Muon is an optimizer that improves optimization performan…
ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMs (arxiv.org) Parameter-efficient fine-tuning (PEFT) of mixture-of-experts (MoE) models commonly attaches a separate low-rank adapter to each expert. This expert-wise design fragments adaptation in three ways: capacity is split across narrow low-rank up…
Newton Matching for Generative Modeling: A Unified Framework for Fine-Tuning and Sampling (arxiv.org) We develop Newton Matching, a unified framework for fine-tuning and sampling in generative modeling. The target is $\pi\propto\mu e^{\tau r}$, where $r$ is the reward, $\tau>0$ the inverse temperature, and $\mu$ denotes the pretrained mode…
Inference-Time Nash Alignment (arxiv.org) Preference-based fine-tuning methods such as RLHF and DPO require substantial compute and large preference datasets. They also need direct access to the model parameters which are not provided by many state-of-the art models.
Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving (arxiv.org) Large language models (LLMs) have been widely used for problem-solving tasks. Most recent work improves their performance through supervised fine-tuning (SFT) with labeled data or reinforcement learning (RL) from task feedback.
Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective (arxiv.org) Pause-token methods improve LLM reasoning by inserting special tokens into sequences. Prior work explains these gains through computational expressivity.
Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning (arxiv.org) We evaluate three open-weight LLMs (Gemma3-12B from the USA, Bielik-11B-v3 from Poland, and Qwen3-4B from China) against World Values Survey Wave 7 data for 63 demographic personas across three countries, using normalized Wasserstein dista…
When Financial Fine-tuning Fails: A Three-Level Detectability Analysis of Numerical Hallucination in Domain-Adapted Language Models (arxiv.org) Financial large language models are increasingly deployed for summarization of reports and disclosures, where numerical hallucination poses significant practical risks. While prior work often attributes such hallucination to insufficient n…
Routing Is Not Enough: Diagnosing Intra-Adapter Subspace Contention in MoE+LoRA Fine-Tuning (arxiv.org) Multi-domain fine-tuning often combines MoE routing with LoRA, assuming that token-level routing separates domain-specific updates. We test this assumption in MoE+LoRA using Python code paired with biomedical text and mathematical reasonin…
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness (arxiv.org) How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training…
AgentRM: Enhancing Agent Generalization with Reward Modeling (arxiv.org) Existing LLM-based agents have achieved strong performance on held-in tasks, but their generalizability to unseen tasks remains poor. Hence, some recent work focus on fine-tuning the policy model with more diverse tasks to improve the gene…
SpeakPay: Domain-Adaptive LoRA Fine-Tuning of Whisper for Low-Resource Nepali Financial Speech Recognition (arxiv.org) Mobile payment applications in Nepal are graphically mediated and largely inaccessible to visually impaired users. This paper presents SpeakPay, a voice-first digital wallet, and documents the central technical contribution: a controlled s…
Inference-Time Optimization of Prompt Embeddings in Diffusion Models: A Comparison of sep-CMA-ES and Adam (arxiv.org) Deep diffusion models have revolutionized image generation by producing high-quality outputs. However, achieving specific objectives with these models often requires costly adaptations such as fine-tuning, which can be resource-intensive a…
RouteGraph-Mona: Confusion-Aware Routing Fine-Tuning for Mineral Image Classification (arxiv.org) Mineral image classification is important for geological exploration and resource development, but it remains challenging due to substantial intra-class variations in appearance and high inter-class visual similarity. Multi-cognitive Visua…
NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference (arxiv.org) Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision encoders with causal language models. Visual document retrievers such as ColPali repurpose th…
Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps (huggingface.co) Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps Structured output is one of the most common real-world tasks for LLMs, yet most benchmarks fold it into broader reasoning or extraction scores rather than measuring i…
FlexP-SFT: A Flexible Aggregation-Free Framework for On-Device Personalized Split Federated Fine-Tuning of LLMs (arxiv.org) To fine-tune large language models (LLMs) over private data, federated learning (FL) has emerged as a promising paradigm. However, the prohibitive memory and communication demands of LLMs render standard FL impractical for resource-constra…
Freeze, Diffuse, Decode: Geometry-Aware Adaptation of Pretrained Transformer Embeddings for Antimicrobial Peptide Design (arxiv.org) Pretrained transformers provide rich, general-purpose embeddings, which are transferred to downstream tasks. However, current transfer strategies: fine-tuning and probing, either distort the pretrained geometric structure of the embeddings…
Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation (arxiv.org) Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where those coefficients act is equally consequential.
Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis (arxiv.org) Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that require following exact output rules, such as instruction following, chain-of-thought reas…
CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RAN (arxiv.org) The next generation of mobile networks is envisioned as fully AI-native, with AI-RAN architectures embedding small language models (SLMs) to perform reasoning over real-time telemetry. The state-of-the-art training paradigms for telecom LL…
Elite-Weighted Supervised Fine-tuning for Goal-Directed Molecular Optimization (arxiv.org) Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implemented with policy-gradient reinforcement learning, which requires a generation-trajectory l…
Post-Training Science for Supervised Fine-Tuning (arxiv.org) Every supervised fine-tuning run forces the same chain of decisions, such as learning rate, batch size, LoRA or full fine-tuning, how many epochs, which optimiser, and what data to feed the model. Each of these is typically rediscovered fr…
Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs (arxiv.org) Cultural fine-tuning has become the de facto paradigm for building culture-aware large language models (LLMs), yet existing optimization exclusively for alignment scores provides an incomplete portrait of cultural fidelity by systematicall…
Why Fine-Tuning Encourages Hallucinations and How to Fix It (arxiv.org) Large language models are prone to hallucinating factually incorrect statements. A key source of these errors is exposure to new factual information through supervised fine-tuning (SFT), which can increase hallucinations w.r.t.~knowledge a…
Scaling Near-Optimal SFT-RL Annotation Budget Allocation from Small to Large LLMs (arxiv.org) How to divide a fixed annotation budget between supervised fine-tuning (SFT) and reinforcement learning (RL) during LLM post-training remains an open problem. Existing work characterizes only broad trends (e.g., SFT dominates in low-data r…
When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning (arxiv.org) Benign fine-tuning severely weakens the safety alignment of large language models (LLMs), so we study why refusal behavior is so fragile. While prior work often attributes this failure to gradient conflict, we propose a fundamentally diffe…
Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneity (arxiv.org) Large Language Models (LLMs) have achieved remarkable success across diverse domains, but their adaptation to privacy-sensitive, distributed datasets remains a challenge. While Federated Learning (FL) combined with Low-Rank Adaptation (LoR…
RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks (arxiv.org) Parameter-efficient fine-tuning methods such as LoRA have become a standard approach for adapting large foundation models. Adopting fine-tuning to distributed settings faces several challenges.
Attention Sensitivity Is Not Enough: Dissociating Attention-Level and Behavioural In-Context Learning under Fine-Tuning (arxiv.org) In-context learning (ICL) lets large language models adapt to new tasks from demonstrations, and fine-tuning can erode this behaviour. Many preservation diagnostics inspect attention: if attention changes when demonstrations change, the mo…
Secret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors (arxiv.org) Local fine-tuning datasets routinely contain sensitive secrets such as API keys, personal identifiers, and financial records. Although "local offline fine-tuning" is often viewed as a privacy boundary, we reveal that compromised model code…
Learning from What You Retrieve: Online RL Fine-Tuning for Semantic Retrieval (arxiv.org) In large-scale e-commerce retrieval, dual-encoder retrievers are op- timized for contrastive similarity, whereas downstream rerankers capture finer-grained relevance preferences; this objective mis- match limits end-to-end retrieval qualit…
SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning (arxiv.org) The unprecedented surge in Earth observation data volume and diversity has exposed a critical bottleneck for traditional manual workflows, catalyzing the emergence of Remote Sensing (RS) Agents. However, the practical deployment of these a…
Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training (arxiv.org) Large language models (LLMs) can now solve complex problems through long chain-of-thought (CoT) reasoning, but the trade-off between performance and token cost remains a central challenge. To address this issue, supervised fine-tuning (SFT…
GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning (arxiv.org) Zeroth-order (ZO) optimization is a memory-efficient alternative to backpropagation for fine-tuning large language models, but its deployment is limited by the high variance of gradient estimation. We propose GRZO, a Group-Relative Zeroth-…
Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs (arxiv.org) Low-Rank Adaptation (LoRA) methods have emerged as crucial techniques for adapting large pre-trained models to downstream tasks under computational and memory constraints. However, they face a fundamental challenge in balancing task-specif…
RSPO: Regularized Self-Play Alignment of Large Language Models (arxiv.org) Self-play-based policy optimization has emerged as an effective approach for fine-tuning large language models (LLMs), formulating preference optimization as a two-player game. However, the regularization with respect to the reference poli…
Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space (arxiv.org) Fine-tuning Low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practically important as it reduces memory and inference cost for storage and deployment.
Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow (arxiv.org) To mitigate the time complexity of generative models, one-step generative models have recently emerged through direct mapping from noise to data in a single forward pass. However, the reward-guided fine-tuning method of one-step generative…
Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities (arxiv.org) Sandbagging, in which a model deliberately underperforms on an evaluation despite retaining the underlying capability, threatens the safety evaluations that frontier-model governance depends on. The Elicitation Game found that fine-tuning…
Retrieval-Augmented LLM Agents: Learning to Learn from Experience (arxiv.org) While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented gene…
Small Updates, Big Doubts: Does Parameter-Efficient Fine-tuning Enhance Hallucination Detection ? (arxiv.org) Parameter-efficient fine-tuning (PEFT) methods are widely used to adapt large language models (LLMs) to downstream tasks and are often assumed to improve factual correctness. However, how the parameter-efficient fine-tuning methods affect…
An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning (arxiv.org) Fine-tuning is an important step in adapting foundation models such as large language models to downstream tasks. To make this step more accessible to users with limited computational budgets, it is crucial to develop fine-tuning methods t…
Stick to What You Know: A Study of Knowledge-Aligned Supervised Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We study this as a source of hallucinations and frame a group of m…
Learning to Reason and Use Tools through Unsupervised Fine-Tuning in Task-Oriented Dialog Systems (arxiv.org) Current dialogue systems struggle with dynamic information retrieval, often leading to hallucinations and lower response accuracy. We address this by adapting the ReAct framework for Task-Oriented Dialogue, enabling Large Language Models (…
Do Small Models Use the Law You Give Them? Measuring Context Use on a Bilingual Bangladesh Legal Benchmark (arxiv.org) Fine-tuning can improve legal question-answering accuracy without improving how models use law supplied in context. We study this distinction in bilingual Bangladeshi legal QA, where observed errors can arise from answer scoring, retrieval…
CPR for LLMs: Critical-Point Routing against Catastrophic Forgetting in Domain Adaptation (arxiv.org) Supervised fine-tuning (SFT) is the de facto standard for adapting large language models (LLMs) to target domains, but it often degrades the model's general capabilities, a phenomenon known as catastrophic forgetting. Existing approaches t…
Cloud and On-Premises Deployment of Uzbek Legal RAG via Targeted Retriever Fine-Tuning (arxiv.org) Deploying large language models for legal question answering raises challenges that general-purpose leaderboards do not capture, particularly for low-resource languages and under hard operational constraints. We report on building and oper…
Efficient Online Continual Foundation Model Fine-Tuning for Predictive Process Monitoring (arxiv.org) Predictive Process Monitoring (PPM) models are increasingly deployed in dynamic environments where concept drift causes the underlying process distribution to shift over time. While recent work has moved toward online continual learning, e…
AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning (arxiv.org) Multimodal large language models (MLLMs) can memorize identity-specific facts about people in their fine-tuning data, creating privacy risks when a person requests deletion. Existing MLLM unlearning methods often assume access to retain im…
FISGuard: Defending Against Membership Inference via Fixed Input Subspaces (arxiv.org) As large language models are increasingly adopted in federated learning, protecting user privacy while performing parameter-efficient fine-tuning on distributed private data has become an important challenge. Although clients only share gr…
From Documents to Reasoning: A Validated Synthetic Data Pipeline and Semantic-Aware Fine-Tuning for Financial Numerical Reasoning (arxiv.org) Financial question answering (QA) has emerged as a key benchmark for evaluating the performance of Large Language Models (LLMs) on domain-specific tasks involving complex data formats such as tables, charts, and rich textual narratives. Wh…
AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air (arxiv.org) Operating Large Language Models (LLMs) on edge devices is increasingly challenged by limited communication bandwidth and strained computational and memory costs. Thus, cloud-assisted remote fine-tuning becomes indispensable.
Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction (arxiv.org) Clinical language models can achieve strong in-hospital accuracy yet fail under deployment shifts because they exploit note-specific artifacts (e.g., templates, separators, boilerplate) that do not reflect patient state. We propose CAST (C…
Fine-Tuning of Transformer models with Frames (arxiv.org) Parameter-Efficient Fine-Tuning (PEFT) strategies such as Low-Rank Adaptation (LoRA) are effective solutions for fine-tuning large-scale pre-trained models; however, their memory requirements scale with the size of the model, $\mathcal{O}(…
How have you evolved your workflow (www.reddit.com via reddit) Dropping AI into your old workflow won’t get the gains you expect. You have to mold how you work at the machine level: the tools, how you use them, how you control them.
Domain-Adaptive ASR for Telephony AI Agents: Fine-tuning Canary Flash Models for Enterprise Contact Center Applications (arxiv.org) This technical report describes Botnoi Group's methodology and results for rapidly fine-tuning the open-source NVIDIA Canary 180M Flash and NVIDIA Canary 1B Flash multitask models for speech-to-text tasks using the NVIDIA NeMo framework, w…
A Layer-wise Analysis of Supervised Fine-Tuning (arxiv.org) While critical for alignment, Supervised Fine-Tuning (SFT) incurs the risk of catastrophic forgetting, yet the layer-wise emergence of instruction-following capabilities remains elusive. We investigate this mechanism via a comprehensive an…
Distance Is Not Enough: Forget-Retain Alignment Gap Predicts LLM Relearning Robustness (arxiv.org) Machine unlearning aims to make a model forget specific data, yet unlearned LLMs often fail to stay unlearned: brief fine-tuning can revive removed knowledge. Existing robustness predictors rely on global weight-space displacement, but dis…
TailSFT: Filtered Fine-Tuning Improves Post-Training Performance (arxiv.org) Reinforcement learning post-training drives reasoning and agentic capabilities in modern AI systems, yet a growing body of work shows that it is most effective when used to fine-tune an already capable base model. We question whether exist…
Mitigating LLM sycophancy with RL-based fine-tuning: Bayesian Truth Serum approach (arxiv.org) Large language models (LLMs) frequently exhibit \emph{sycophancy}: they adapt their answers to a user's stated beliefs or preferences instead of reporting what they hold to be true, which lowers factual accuracy and can amplify misinformat…
Ladder Up, Memory Down: Low-Cost Fine-Tuning With Side Nets (arxiv.org) Fine-tuning large language models (LLMs) is often limited by the memory available on commodity GPUs. Parameter-efficient fine-tuning (PEFT) methods such as QLoRA reduce the number of trainable parameters, yet still incur high memory usage…
Fine-Tuning Whisper for Automatic Speech Recognition in Baniwa: A Preliminary Study (arxiv.org) Automatic Speech Recognition (ASR) technologies have achieved remarkable performance in recent years through the use of large multilingual foundation models. However, most advances remain concentrated on high-resource languages, while indi…
Cross-Dataset Stability of Expert-Informed Skill Prompting and Fine-Tuning for Chinese Metaphor Identification (arxiv.org) Metaphor-identification performance can change markedly across datasets that differ in text distribution and annotation policy. We examine whether a fixed expert-informed procedure produces a more even cross-dataset profile than task-speci…
Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal (arxiv.org) Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after dep…
Iwin Transformer: Hierarchical Vision Transformer using Interleaved Windows (arxiv.org) Vision Transformers (ViTs) face two limitations: the rigid resolution dependency of positional embeddings, which complicates cross-resolution fine-tuning, and the quadratic complexity of attention. While Swin Transformer alleviates the lat…
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning (arxiv.org) Quantized fine-tuning (QLoRA) saves memory but not time. It dequantizes every 4-bit weight on the fly, so it trains more slowly than fp16 LoRA.
GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning (arxiv.org) Recent studies show that the safety alignment of large language models (LLMs) can be easily compromised even by seemingly non-adversarial fine-tuning. To preserve safety alignment during fine-tuning, a widely used strategy is to jointly op…
RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards (arxiv.org) Safeguarding language model agents requires assessing complete execution trajectories under context-dependent safety policies. Existing policy-aware safeguards mainly rely on prompting or supervised fine-tuning, limiting their ability to a…
me to the model I spent all weekend fine-tuning (www.reddit.comhttps) I just can't resist
Two Stage Wireless Federated LoRA Fine-Tuning with Sparsified Orthogonal Updates (arxiv.org) Federated fine-tuning with low-rank adaptation (LoRA) communicates only two low-rank matrices instead of the full model, but existing methods typically fix the LoRA rank in advance as a manually tuned hyperparameter. In wireless networks,…
Fairness-Aware Low-Rank Representation Fine-Tuning (arxiv.org) Pre-trained foundation models can be efficiently adapted for specific tasks using Low-Rank Adaptation (LoRA), but the fairness properties of these adapted classifiers remain underexplored. Existing fairness-aware fine-tuning methods assume…
CD-LoRA: Consistency-Driven Low-Rank Adaptation for Multi-Task Fine-Tuning (arxiv.org) While Multi-Task Learning (MTL) is essential for adapting Large Language Models (LLMs) to diverse domains, prevailing LoRA-based methods rely on complex routing mechanisms that partition task-specific knowledge. In this work, we reveal tha…
Unveiling the Depth-Performance Dilemma in Split-Federated Fine-tuning of LLMs (arxiv.org) Split Federated Fine-tuning (SFF) is a promising paradigm for scaling Large Language Models (LLMs) by partitioning model depth between resource-constrained clients and a centralized server. While system incentives for throughput and privac…
What Models Know, How Well They Know It: Knowledge-Weighted Fine-Tuning for Learning When to Say "I Don't Know" (arxiv.org) While large language models (LLMs) demonstrate strong capabilities across diverse user queries, they still suffer from hallucinations, often arising from knowledge misalignment between pre-training and fine-tuning. To address this misalign…
From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning (arxiv.org) Parameter-Efficient Fine-Tuning (PEFT) is essential for adapting Large Language Models (LLMs) to multi-task scenarios. A prevailing trend in this field involves complex LoRA variants with multiple adapters or heads, which rely on the premi…
Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration (arxiv.org) Large language model unlearning is consistently fragile under relearn attacks. On TOFU, fine-tuning on twenty forget examples substantially recovers held-out forget-set ROUGE for every method we evaluate, and we trace this fragility to opt…
The Emergence of Relevance Through Axiomatic Attention Patterns During LoRA Fine-Tuning (arxiv.org) LoRA fine-tuning is standard for adapting LLMs to reranking, but it remains unclear where in the network task-specific relevance behavior is learned and what attention-level changes accompany that learning. Through ablation and attention e…
Small Reasoning Models are Instruction Followers in Function Calling (arxiv.org) Function calling represents the core capability of agentic large language models (LLMs). Existing research has focused on enhancing LLMs function-calling accuracy through fine-tuning, reinforcement learning (RL), and multi-agent frameworks…
↯ Fine Tuning↯ Function Callingfunction-callingfine-tuningagentic
MobileMoE - a facebook Collection (huggingface.co via reddit) MobileMoE is a family of on-device Mixture-of-Experts (MoE) language models with sub-billion active parameters, designed to push the quality–efficiency Pareto frontier for on-device LLMs, including three model scales (S/M/L): 0.3B/0.5B/0.9…
Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers (arxiv.org) Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for fine-tuning non-differentiable, event-driven spiking neural networks (SNNs). However, its deployment on in-memory computing (IMC…
Fine-tuning LLMs for Tourist Trajectory Prediction using Field Experiment Data (arxiv.org) Evaluating mobility interventions at tourist destinations requires predicting visitor behavior under varying conditions. Traditional methods struggle because tourist decisions depend heavily on context like weather and fatigue, yet models…
Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AI (arxiv.org) Federated fine-tuning enables large language models to adapt on edge devices without centralizing private data, but practical deployments must address hardware instability and adversarial update corruption together. Thermally constrained c…
Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs (arxiv.org) Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint, and with it every evaluation and cache that referred to those exact bits. We instead lea…
RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs (arxiv.org) Fine-tuning safety-aligned language models for downstream tasks often leads to substantial degradation of refusal behavior, making models vulnerable to adversarial misuse. While prior work has shown that safety-relevant features are encode…
Fine-tuning an ECG Foundation Model to Predict Coronary CT Angiography Outcomes (arxiv.org) Coronary artery disease (CAD) remains a major global public health burden, yet scalable pre-imaging risk stratification tools are limited. In this multicenter study, we developed and validated an artificial intelligence-enabled electrocard…
Knowing but Not Saying: Preventing Factual Access Failures in LLM SFT via Recall-Anchored Distillation (arxiv.org) Supervised fine-tuning (SFT) can degrade factual behavior outside the target domain. This degradation is often described as catastrophic forgetting, yet open-ended factual failures do not necessarily imply that the underlying facts have be…
CAS: Conformalized Agentic Search via Adaptive Retrieval and Policy Weighting (arxiv.org) Search Agents face a severe reliability crisis during reinforcement learning (RL) fine-tuning. Heuristic Top-K retrieval often causes critical evidence loss or noise inclusion, while over-confidence induced by progressive RL leads to hallu…
1/100 → 44/100: fine-tuning a 450M VLM on 50K browser screenshots (www.reddit.com via reddit) could not extract summary
Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation (arxiv.org) State-of-the-art vision-language-action (VLA) models such as $\pi_{0.5}$ exhibit strong semantic understanding, instruction following and task behavior. However, when deployed on new robots, even minor mismatches in hardware configuration…
DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values (arxiv.org) Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often overlook cultural and demographic variation. We introduce DiverValue-Bench, a population-aware be…
Learning how to Forget: Fine-tuning for Long-Context Sparse Attention (arxiv.org) A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer language models without excessive hardware budgets. We provide a new method for fine-tuning m…
LoRA-GA$^2$: Low Rank Adaptation with Multi-step Gradient Adaptive Alignment (arxiv.org) Low-Rank Adaptation (LoRA) is a prominent fine-tuning method for large models, achieving competitive performance with reduced memory overhead. However, a persistent performance gap remains between LoRA and full fine-tuning.
A Speech Corpus for Mizo Automatic Speech Recognition: Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation (arxiv.org) This study reports the development of an Automatic Speech Recognition (ASR) system in Mizo, a low-resource language. The development included collecting 17.62 hours of speech data, curating it, and fine-tuning the Mizo ASR system with thre…
KA2L: A Knowledge-Aware Active Learning Framework for LLMs (arxiv.org) Fine-tuning large language models (LLMs) with high-quality knowledge has been shown to enhance their performance effectively. However, there is a paucity of research on the depth of domain-specific knowledge comprehension by LLMs and the a…
Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning (arxiv.org) Figurative language is deeply culturally embedded; fluent use requires not just linguistic competence but cultural immersion. We ask whether LLMs can learn this link: does fine-tuning on cultural data improve figurative language understand…
Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models (arxiv.org) Multimodal large language models (MLLMs) are typically built through a multi-stage pipeline consisting of cross-modal alignment, supervised fine-tuning (SFT), and preference optimization. This pipeline assumes that adapting an LLM to a new…
A Few Cases Are All You Need: An Empirical Study of Annotation-Efficient LoRA Fine-Tuning of MedSAM3 (arxiv.org) Medical image segmentation is essential for clinical workflows such as treatment planning and disease assessment. While specialist tools like TotalSegmentator and MRSegmentator achieve strong performance, they require large annotated datas…
Impact of Iterative Fine-Tuning on Transcription Accuracy in Complex Historical Sanskrit Manuscripts (arxiv.org) Digitizing the text from handwritten historical manuscripts is required to make them easily accessible, preservable, and to enable historical scholars to study them in new ways. Historical manuscripts, however, often exhibit complex hetero…
Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements (arxiv.org) Reinforcement Learning (RL) has been promising in single-turn LLM fine-tuning. However, long-horizon agentic reasoning introduces increasingly branching interactions and sparse rewards, exposing several limitations of RL: its heavyweight b…
OraclePhys: A Systematic Framework for LLM Fine-Tuning on Structural Mechanics (arxiv.org) What a language model internalizes from fine-tuning is usually diagnosed after the fact. We make it an experimental variable.
Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training (arxiv.org) As supervised fine-tuning data continues to scale, selecting high-value subsets from large candidate pools is crucial for reducing training cost and improving model performance. Existing methods often measure diversity directly in the orig…
Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training (arxiv.org) Data selection in supervised fine-tuning aims to select a small set of effective samples from large-scale candidate data, reducing training cost while preserving model performance. However, existing methods usually treat data value as a re…
Self-Distillation as a Performance Recovery Mechanism for LLMs: Counteracting Compression and Catastrophic Forgetting (arxiv.org) Large Language Models (LLMs) have achieved remarkable success, underpinning diverse AI applications. However, they often suffer from performance degradation due to factors such as catastrophic forgetting during Supervised Fine-Tuning (SFT)…
Parametric Knowledge in RAG-SFT for Domain-Specific Document Generation (arxiv.org) Retrieval-Augmented Generation (RAG) fine-tuning has shown substantial improvements over vanilla RAG, yet most studies target document question answering, leaving open whether these gains transfer to specialized tasks. We study supervised…
Task Specialization Fine-Tuning for Contextual Reinforcement Learning (arxiv.org) Contextual Reinforcement Learning (CRL) seeks to generalize classical RL by maximizing task coverage across a context space of related tasks. While prior works often train from scratch and rely on either multi-task learning for a single po…
FDA-Opt: Federated Fine-Tuning via Dynamic Update Schedules (arxiv.org) Federated Learning (FL) enables the utilization of vast, previously inaccessible data sources. At the same time, pre-trained Language Models (LMs) have taken the world by storm and for good reason.
SubZero+: Efficient Zeroth-Order LLM Fine-Tuning via Large Learning Rates (arxiv.org) Zeroth-order (ZO) optimization enables backpropagation-free fine-tuning of large language models, but existing ZO methods suffer from high-variance gradient estimators, making convergence unstable and highly sensitive to learning rates. We…
Towards a theory of inference-time alignment with unknown rewards (arxiv.org) Generative model alignment has received broad interest, and significant progress has been made in supervised fine-tuning and inference-time computation. Yet, alignment has remained poorly understood from a statistical learning perspective.
LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights (arxiv.org) We introduce LoRA-CRAFT (\textbf{C}ross-layer \textbf{R}ank \textbf{A}daptation via \textbf{F}rozen \textbf{T}ucker), abbreviated CRAFT throughout, an extremely parameter-efficient fine-tuning (PEFT) method that applies Tucker tensor decom…
Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning (arxiv.org) Palmyra x6 is a large language model optimized for use with enterprise-oriented agentic tasks. The model was built by post-training a Mixture-of-Experts base model with Anchored Supervised Fine-Tuning on a compact corpus of verified, synth…
Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuning (arxiv.org) On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories. This approach has achieved strong empirical gains and can often surpass conventional off-policy distillation with su…
SAPE: Sandwich Adapters for Parameter Efficiency in Large Language Model Fine-Tuning (arxiv.org) While Parameter-Efficient Fine-Tuning (PEFT) has substantially reduced the hardware cost of adapting Large Language Models (LLMs) by decreasing the number of trainable parameters, recent studies have sought to further improve PEFT through…
MoE Router-Guided Clustering for Heterogeneous Federated Instruction Tuning (arxiv.org) Federated instruction fine-tuning enables Large Language Models (LLMs) to adapt to decentralized, privacy-sensitive data without requiring data sharing. Recent Mixture-of-Experts (MoE) LLMs are particularly attractive for federated learnin…
PEFT-MuTS: A Multivariate Parameter-Efficient Fine-Tuning Framework for Remaining Useful Life Prediction based on Cross-domain Time Series Representation Model (arxiv.org) The application of data-driven remaining useful life (RUL) prediction has long been constrained by the availability of large amount of degradation data. Mainstream solutions such as domain adaptation and meta-learning still rely on large a…
Fine-Tuning Qwen3-27B for C-to-Rust Code Translation: A Three-Stage Curriculum of Pretraining, Debugging-Aware SFT, and Task-Specific SFT (arxiv.org) Translating C code into safe, idiomatic Rust is a longstanding software-engineering goal because it can eliminate entire classes of memory-safety vulnerabilities while preserving the functional behavior of legacy systems. Large language mo…
Towards Efficient Multimodal and Multilingual Opinion Extraction for STI: A QLoRA-Based Fine-Tuning Approach (arxiv.org) Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) requires concise core opinions from large information streams.
Follow the Norm: Accounting for Fine-Tuning and Prompt Effects on Model Rationales (arxiv.org) Normative datasets are often used to train and align AI systems, but the norms they contain can function as action-guiding patterns rather than neutral moral knowledge. We propose treating the AI system as a proxy actor and test whether da…
Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows (arxiv.org) We propose CVaR-penalized Generative Particle Algorithm (CVaR-GPA), a robust, tail-agnostic algorithm for fine-tuning generative models to learn heavy-tailed distributions and capture extreme events, requiring no prior knowledge or estimat…
Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models (arxiv.org) Optical character recognition (OCR) and multilingual scene-text understanding remain challenging for multimodal large language models (MLLMs), particularly in real-world images containing small or degraded text, cluttered layouts, occlusio…
Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport (arxiv.org) Supervised fine-tuning (SFT) is a standard approach for adapting LLMs to a target distribution, but in settings such as personalization, where each author requires separate weight access, optimization, storage, and retraining, its costs be…
COLORA: Efficient Fine-Tuning for Convolutional Models with a Study Case on Optical Coherence Tomography Image Classification (arxiv.org) We introduce \textbf{CoLoRA} (Convolutional Low-Rank Adaptation), a parameter-efficient fine-tuning method for convolutional neural networks (CNNs). CoLoRA extends LoRA to convolutional layers by decomposing kernel updates into lightweight…
Backdoor Decontamination Dynamics in LLM Agents (arxiv.org) Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it…
Critic-Free Pretraining for Efficient Online Reinforcement Learning Fine-Tuning (arxiv.org) Offline-to-online (O2O) reinforcement learning aims to leverage policies pretrained on static datasets while improving them through online interaction. However, directly reusing an offline-trained critic can hinder online fine-tuning: as t…
Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge (arxiv.org) Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answer…
Enhancing Automated Essay Scoring With Three Techniques: Two-Stage Fine-Tuning, Score Alignment, and Self-Training (arxiv.org) Automated Essay Scoring (AES) plays a crucial role in education by providing scalable and efficient assessment tools. However, in real-world settings, the extreme scarcity of labeled data severely limits the development and practical adopt…
myMediWhisper: Construction of Burmese Medical Speech Corpus and Whisper Fine-Tuning for Clinical Dialogue ASR (arxiv.org) Although Whisper models benefit from large-scale multilingual pre-training, their performance on Burmese medical speech remains limited. This work presents a Burmese medical speech recognition framework built on a high-quality 28-hour corp…
REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs (arxiv.org) We present the REAP system for the AKBC Shared Task 2026 on constructing knowledge bases from language models in a closed-book setting, subject to a budget of at most 32B parameters and no model fine-tuning. Our system combines structured…
Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies (arxiv.org) Deploying large language models (LLMs) for decision support in emergency departments (EDs) faces two major challenges: privacy risks of transmitting patient data to closed-source commercial LLMs and the lack of systematic evaluation of fin…
Early Data Exposure Improves Robustness to Subsequent Fine-Tuning (arxiv.org) How can we train models whose post-trained capabilities survive subsequent fine-tuning? Rather than focusing on downstream interventions to mitigate forgetting of upstream capabilities, we study how upstream training choices - that is, the…
Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training (arxiv.org) Evolution Strategies (ES) have emerged as a scalable gradient-free alternative to reinforcement learning based LLM fine-tuning, but it remains unclear whether comparable task performance implies comparable solutions in parameter space. We…
Mechanistic Interpretability-Guided Selective Fine-Tuning of Vision-Language Models for Centimeter-Level Flood Depth Estimation (arxiv.org) Urban flooding poses an escalating threat to transportation infrastructure, yet no operational system provides real-time, street-level flood-depth estimates at centimeter resolution. This paper presents three vision-language models fine-tu…
Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training (arxiv.org) Reinforcement learning post-training unlocks complex reasoning in LLMs. Yet benchmark scores reveal only whether a model improved, not what changed inside it, nor how it splits finite capability across tasks.
ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling (arxiv.org) Large language model (LLM) fine-tuning at the edge adapts the model to scenario-specific data while preserving privacy. Although existing studies proposed pipeline parallelism to address the limited memory and computing resources of edge d…
Benchmarking Parameter-Efficient Fine-Tuning of Large Language Models for Low-Resource Tajik Text Generation with the Tajik Web Corpus (arxiv.org) We release the Tajik Web Corpus (319k docs, 1.11B chars) and benchmark generative LLMs on prompt continuation in Tajik, a low-resource Cyrillic-script language. Seventeen configurations across nine architectures are evaluated under three f…
Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning (arxiv.org) Supervised fine-tuning (SFT) on chain-of-thought data is an essential post-training step for reasoning language models. Standard machine learning intuition suggests that training with more unique training samples yields better generalizati…
Listwise Cross-Encoder Fine-Tuning vs. Agentic Instruction Tuning for LLM Rerankers: A Systematic Study in Medical Procedure Reranking (arxiv.org) Reranking medical procedures against patient queries is a critical component of health insurance information retrieval, complicated by a substantial lexical gap between patient language and clinical nomenclature. We present a systematic co…
NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning (arxiv.org) Existing parameter-efficient fine-tuning (PEFT) methods primarily fall into two categories: addition-based and selective in-situ adaptation. The former, such as LoRA, introduce additional modules to adapt the model to downstream tasks, off…
SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL (arxiv.org) Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task-specific fine-tuning. However, existing LLM-based systems remain unreliable, often generating SQL queries that are inv…
SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs (arxiv.org) Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Translation (GFSLT) remains a challenge.
Can We Optimize the Performance-Carbon Emission Break-Even Point?: The Quest for Greener LLMs (arxiv.org) The carbon footprint of any deployed Large Language Model (LLM) accumulates during inference, where repeated use of the model substantially exceeds the one-time cost of fine-tuning. Yet most efficiency interventions target either pre-train…
Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification (arxiv.org) Large language models (LLMs) are high-value assets that can be derived through redeployment, fine-tuning, quantization, or further alignment. Because deployed LLMs are commonly exposed only through query APIs, ownership verification must o…
STEMMA: An Adversarial Multi-Agent Framework for Evaluating Self-Identity Consistency in LLMs (arxiv.org) Knowledge Distillation is a widely adopted technique in the training and fine-tuning of large language models (LLMs) enabling transfer of structured information and functional behavior from a large teacher model to a smaller student model…
Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models (arxiv.org) Mixture-of-Experts (MoE) models decouple total parameters from per-token compute, but deployment still requires storing every expert. Recent theory shows that pruning experts with the smallest router-norm changes during fine-tuning can pre…
LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks (arxiv.org) Parameter-efficient fine-tuning enables the adaptation of vision foundation models to biomedical tasks under limited computational resources, but a single low-rank update can constrain all task-specific changes to one narrow parameter subs…
From Manuals to Maintenance: Fine-Tuning MedGemma for Multi-Modal Imaging System Support in Low-Resource Settings (arxiv.org) Imaging device downtime is a major barrier to healthcare delivery in low- and middle-income countries (LMICs), often driven by limited access to specialized biomedical engineering support. We present a multi-modality medical equipment main…
FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents (arxiv.org) Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts. However, even st…
HoloAegis: Frozen Representation, Topological Inference: Minimally Parametric Safety Manifolds for Zero-Shot LLM Guardrails (arxiv.org) Current LLM safety guardrails face a fundamental tension: fine-tuning distorts pre-trained representations while generative judges incur prohibitive inference costs. We challenge the prevailing paradigm by asking: can safety be achieved th…
Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning (arxiv.org) Compact AI systems make local language-model experimentation increasingly accessible, yet practical evidence for multi-node training on desktop-class accelerators remains limited. This report presents a proof-of-concept deployment of distr…
Judge a Book by its Cover: Investigating Multi-Modal LLMs for Multi-Page Handwritten Document Transcription (arxiv.org) Handwriting text recognition (HTR) remains a challenging task. Existing approaches require fine-tuning on labeled data, which is impractical to obtain for real-world problems, or rely on zero-shot tools such as OCR engines and multi-modal…
Operating Multi-Node Full Fine-Tuning on NVIDIA B300: A Field Report on Telemetry-Based Triage, Negative Results, and Operational Hardening (arxiv.org) We report operational experience full-fine-tuning a 32.76B-parameter dense model (Qwen3-32B) on 16 x NVIDIA B300 (two nodes, FSDP / ZeRO-3) -- among the first published field accounts on this accelerator. We claim no new algorithm.
Kastor: An efficient fine-tuning strategy for generative emulation of PDE simulations (arxiv.org) Machine learning offers a promising avenue to accelerate physical simulations by replacing computationally expensive traditional Partial Differential Equation (PDE) solvers with fast, differentiable surrogate models. However, standard auto…
Align-RAG: Alignment Is All You Need for TSFM In-Context Learning (arxiv.org) Retrieval-augmented forecasting promises to adapt frozen Time Series Foundation Models (TSFMs) to new domains without fine-tuning, but recent methods typically rely on learned fusion modules, i.e., trained adapters that merge retrieved exa…
Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning (arxiv.org) Quantized orthogonal fine-tuning (qoft) enables parameter-efficient adaptation of low-bit language models by learning structured activation rotations before frozen quantized weights. However, its task-specific updates remain constrained to…
Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning (arxiv.org) Multi-task supervised fine-tuning (SFT) often casts a heterogeneous data mixture as a single optimization problem, even though different tasks may reach their best generalization at different times. msft exposes this mismatch through task-…
When Do Corrective Features Help? An Agent for Corrective Feature Discovery on Black-Box Forecasters (arxiv.org) Frozen pretrained forecasters often fail in structured, recurring ways that are costly to repair through fine-tuning. We study corrective feature discovery: mining interpretable features of a frozen forecaster's residual to drive a lightwe…
Behavioral Canaries: Auditing Private Retrieved Context Usage in RL Fine-Tuning (arxiv.org) In agentic workflows, LLMs frequently process retrieved contexts that are legally protected from further training. However, auditors currently lack a reliable way to verify if a provider has violated the terms of service by incorporating t…
Training-Free Token-Level Steering for LLM Personalized Co-Writing (arxiv.org) While Large Language Models (LLMs) show great promise for personalization, they often lack specialized domain knowledge. Conventional solutions like fine-tuning struggle with high computational costs and rapid data updates, while Retrieval…
APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning (arxiv.org) Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithm…
Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging (arxiv.org) Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning. However, directly applying ES to billion-parameter LLMs is highly ineffective.
DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning (arxiv.org) Task-specific fine-tuning can improve the performance of large language models (LLMs) on downstream tasks. However, our study reveals that task-specific fine-tuning can also weaken the safety guardrails of aligned LLMs.
State2State: Environment-Derived Mid-Training for LLM Agents (arxiv.org) Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specifi…
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning (arxiv.org) Fine-tuning enables a source model to acquire desired capabilities and behaviors in a target domain while retaining much of its general-purpose competence. However, this adaptation process can also degrade alignment properties that were pr…
Geometry-Informed Parameter-Efficient Fine-Tuning of Pre-trained Molecular GNNs for Blood-Brain Barrier Permeability Prediction (arxiv.org) Blood-brain barrier permeability (BBBP) prediction is a critical screening task in central nervous system drug discovery, where candidate molecules must be assessed for whether they can cross, or should be prevented from crossing, the bloo…
Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation (arxiv.org) Reinforcement learning (RL) has emerged as a powerful paradigm for fine-tuning large-scale generative models, such as diffusion and flow models, to align with complex human preferences and user-specified tasks. A fundamental limitation rem…
Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning (arxiv.org) Released aligned large language models remain vulnerable to malicious downstream finetuning. Existing defenses are largely designed for the fine-tuning-as-a-service (FTaaS) paradigm or rely on downstream users to follow additional safety p…
HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Models (arxiv.org) Large Audio-Language Models (LALMs) excel at general speech understanding; however, adapting them to fine-grained tasks like Speech Emotion Recognition (SER) remains a significant bottleneck. Current Parameter-Efficient Fine-Tuning (PEFT)…
Structure-Aware Robust Fine-Tuning: Defending Vision-Language-Action Robots Against Physical Attention Hijacking (arxiv.org) Vision-Language-Action (VLA) policies promise general robotic manipulation, but their robustness against physical-world attacks remains fragile. In particular, we show that physically realizable adversarial patches can reliably induce fail…
Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training (arxiv.org) Reinforcement fine-tuning (RFT) is widely believed to inherently resist catastrophic forgetting in continual post-training of multimodal large language models. Under pronounced task distributional shifts, however, forgetting across represe…
Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment (arxiv.org) Unsupervised domain adaptation (UDA) aims to mitigate domain shift, where the distribution of labeled source data differs from that of unlabeled target data. Despite recent advances, existing methods often rely on fine-tuning large backbon…
Omega-S: A Functional Resilience Index for LLM Fine-Tuning (arxiv.org) Fine-tuning a large language model on new data degrades what it previously learned. We present Omega-S, a drop-in penalty computed from the weight matrix alone: it needs no previous-task data, no Fisher matrix and no stored copy of the old…
Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning (arxiv.org) Designing effective reward functions remains a major bottleneck in Reinforcement Learning (RL). Recent work uses large foundation Vision-Language Models (VLMs) as reward models, computing text-observation similarity to bypass manual reward…
Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models (arxiv.org) Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations. Existing aggregation-based DP-ZO methods reconstruct model updates at a fixed sc…
Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning (arxiv.org) As large language models (LLMs) continue to grow in capability, so do the risks of harmful misuse through fine-tuning. While most prior studies assume that attackers rely on supervised fine-tuning (SFT) for such misuse, we systematically d…
SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs (arxiv.org) Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) exhibit fundamentally different behaviors in enhancing multi-task reasoning for large language models (LLMs). Our preliminary experiments revealed a phenomenon: SFT suffers from…
Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures (arxiv.org) Parameter-Efficient Fine-tuned (PEFT) models are frequently downloaded from open repositories by practitioners. This widespread practice creates a significant attack surface, as malicious actors can publish backdoored models that induce sp…
FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecasting (arxiv.org) Time-series foundation models (TSFMs) such as Chronos have demonstrated strong forecasting capabilities across domains, yet adapting them to institutionally fragmented settings, where data cannot be centralized due to regulatory, competiti…
EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuning (arxiv.org) Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning, but standard LoRA produces a single deterministic model and does not directly support predictive uncertainty estimation. We introduce EulerLoRA, a stochastic extension of…
Two-Stage Bengali Sentiment Classification: Domain Adaptation Through Continual Learning and Parameter-Efficient Fine-Tuning (arxiv.org) Understanding sentiment in low-resource languages remains a key challenge for Natural Language Processing (NLP), particularly when domain-specific data is scarce. In this work, we present SentiBanglaBERT, a two-stage Bengali sentiment clas…
Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs (arxiv.org) Organizations increasingly adapt Large Language Models (LLMs) from public repositories such as HuggingFace to downstream tasks. Prior work shows that even fine-tuning on benign datasets can weaken safety alignment, raising a practical ques…
SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models (arxiv.org) Geospatial foundation models (GeoFMs), pretrained on large-scale geospatial data such as Earth observation (EO), climate, and weather data, have shown promising performance when fine-tuned on diverse downstream tasks. However, there are tw…
Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics (arxiv.org) Teaching a language model a skill it has not mastered is obstructed by three recurring difficulties: training data is scarce, ground-truth reasoning traces are usually unavailable, and models often exhibit an apparent ceiling beyond which…
Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints (arxiv.org) The proliferation of customized Large Language Models (LLMs) poses critical risks of Data Intellectual Property (Data IP) infringement via unauthorized fine-tuning on proprietary data. Existing audit techniques are limited, as they require…
MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning (arxiv.org) Parameter-efficient fine-tuning (PEFT), such as low-rank adaptation (LoRA), has recently been adopted in federated learning to reduce communication and computation costs. In this setup, users download a pretrained model from the server pri…
CRAFTS: Collaborative Role-Adaptive Fine-Tuning of LLM Agents for Chemical Process Simulation (arxiv.org) Constructing an executable chemical-process model remains manually intensive. Chemical engineers translate underspecified requests into coupled decisions about unit operations, thermodynamics, streams, specifications, degrees of freedom (D…
Personalizing Large Language Model Agents with Small Policy Models (arxiv.org) Large language model (LLM) agents can retrieve memory, call tools, ask clarifying questions, and vary response style, yet adapting these execution decisions to an individual user remains difficult. Fine-tuning a separate LLM is costly or i…
DeltaServe: Host-Agnostic Co-Serving of Inference and Fine-Tuning for LLMs (arxiv.org) LLM serving systems are provisioned for peak load to meet strict latency targets, leaving substantial GPU compute idle whenever traffic falls below peak. We present DeltaServe, a host-agnostic co-serving design that converts this idle infe…
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs (arxiv.org) Parameter-Efficient Fine-Tuning (PEFT) commonly adapts large language models using a single shared Low-Rank Adapter (LoRA). This shared optimization space often suffers from interference when adapting heterogeneous task sequences, leading…
CLIFT: Turning Gemini Robotics On-Device into Humanoid Specialists via Non-Invasive Closed-Loop Iterative Fine-Tuning (arxiv.org) While robot foundation models are growing increasingly capable, the strongest models are typically trained on proprietary data and remain closed-source, limiting downstream users' ability to adapt them to new tasks, embodiments, and deploy…
Federated Foundation Models Fine-Tuning with Heterogeneous Compressed Clients (arxiv.org) Federated learning of foundation models faces a fundamental resource-asymmetry challenge: the institutions holding the most valuable domain-specific data cannot host billion-parameter models. Existing heterogeneous federated approaches att…
Evaluating Federated Pre-Training: On the Reliability of Downstream Fine-Tuning and Intrinsic Evaluation (arxiv.org) Federated pre-training offers a way to train foundation models on private or distributed data without centralizing the underlying datasets. However, evaluating federated pre-training remains challenging because differences in client partic…
The Claude chat step that actually changes my pace (www.reddit.comhttps) Been building for a while now, and adding a Claude chat step between Figma and Claude Code actually changed my pace. Back-and-forth on a component or flow, right in your chat, before it ever hits code.
REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning (arxiv.org) Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks. While diffusion policies (DPs) show promise for lear…
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks (arxiv.org) Supervised fine-tuning (SFT) can equip large language models (LLMs) with domain knowledge for high-performance computing (HPC) tasks such as data race detection and benchmark question answering. However, knowledge alone does not guarantee…
Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control (arxiv.org) Multi-zone variable-air-volume control must balance thermal comfort, indoor air quality, and electricity use across several continuous actuators. Model predictive control and reinforcement learning are widely studied, but deployment typica…
VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction (arxiv.org) Standard masked-language-model fine-tuning applies a uniform masking probability across every token position, assuming reconstruction difficulty is position-agnostic. When the degradation process is characterised and concentrated at predic…
LLM2Vec-Gen: Generative Embeddings from Large Language Models (arxiv.org) Fine-tuning LLM-based text embedders via contrastive learning maps inputs and outputs into a new representational space, discarding the LLM's output semantics. We propose LLM2Vec-Gen, a self-supervised alternative that instead produces emb…
TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement (arxiv.org) Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint [1] (arXiv:2606.20553), demonstrates that a malicious parameter server can corrupt a PEF…
Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning? (arxiv.org) Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) this raises a natural question: given a pretrained policy, should the Q-function be pretra…
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models (arxiv.org) Alignment training, model organisms, and toy models are usually treated as separate research areas. But projects in all three frequently use supervised fine-tuning (SFT) to pursue the same underlying goals.
Constitutional Midtraining: Content Presence Drives Alignment Gains (arxiv.org) Post-training alignment is often shallow, eroding under fine-tuning. Whether midtraining interventions, cleanly isolated from post-training, can produce durable alignment remains untested.
Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution (arxiv.org) Multimodal large language models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual understanding within a unified transformer architecture. However, fine-tuning all parameters of these models for visual in…
Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning (arxiv.org) Reinforcement learning (RL) fine-tuning is widely used in language model training to improve model performance on a target task while limiting drift from a reference policy. A standard way to balance this trade-off is via a KL-regularized…
On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment (arxiv.org) Fine-tuning is the dominant paradigm for specializing large language models (LLMs), yet it exposes a critical vulnerability: malicious data providers can embed harmful behaviors into downstream corpora, creating models that retain professi…
MemSFT: Mitigating Alignment Tax with an External Parametric Memory (arxiv.org) Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantially degrade performance on general tasks. We propose MemSFT…
Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models (arxiv.org) Vision-language models (VLMs) are increasingly applied to medical visual question answering (Med-VQA), yet whether they can \emph{localize} the evidence behind their answers---a prerequisite for clinical auditability---is poorly characteri…
Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning (arxiv.org) Optimization over the Stiefel manifold plays a significant role in various machine learning tasks. Existing methods either use the retraction operators, requiring costly orthonormalization for large-scale matrices, or employ landing method…
Robustifying pathology foundation models via fine-tuning (arxiv.org) Pathology foundation models (FMs) produce powerful tile-level representations which remain sensitive to scanner and staining variability, undermining deployment across laboratories. We develop a novel fine-tuning recipe that improves the r…
Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning (arxiv.org) Recent work shows that fine-tuning language models on even a small amount of poisoned data can install targeted misbehavior, and ostensibly benign data can transmit hidden preferences that generalize broadly. Standard defenses, such as dat…
Shift-Aware Calibration for Fine-Tuned CLIP: Leveraging Image-Text Alignment (arxiv.org) Vision-language models (VLMs), such as CLIP, adapt effectively to downstream tasks through prompt tuning, but fine-tuning can misalign predictive confidence and accuracy, particularly on unseen classes. Existing VLM-specific calibration me…
The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning (arxiv.org) LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting. Since their discovery,…
DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning (arxiv.org) The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data. Existing dynamic data scheduling methods face critical limitations in industrial-scale pretraining and supe…
Layer-wise LoRA fine-tuning: a similarity metric approach (arxiv.org) Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI. In contrast, enhancing their predictive performance on downstream tasks typically involves adapting their knowledge throu…
Encoding Invisible Causation for Bridge Diagnostic Agents: Triple-Guided Retrieval-Augmented Fine-Tuning with QLoRA (arxiv.org) Bridge infrastructure deteriorates gradually, yet its root causes---salt intrusion, freezing, fatigue cracking, and others---remain invisible to the naked eye. Expert diagnosis relies on tacit knowledge built over years of practice.
Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision (arxiv.org) Speech Large Language Models (LLMs) that understand and follow instructions in many languages are useful for real-world interaction, but are difficult to train with supervised fine-tuning, requiring large, task-specific speech corpora. Whi…
MoE$^2$-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation (arxiv.org) Mixture-of-Experts (MoE) architectures have been widely adopted in large language models, yet parameter-efficient fine-tuning (PEFT) for MoE models remains underexplored. Existing PEFT methods for MoE either ignore router priors with unifo…
IFCLoRA: Topology-Aware Rank Allocation for Parameter-Efficient Fine-Tuning (arxiv.org) Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning method for large language models, but its performance depends strongly on how a fixed rank budget is distributed across Transformer modules. Existing adaptive-rank…
Benchmarking Fine-tuning and Retrieval Strategies for a Multimodal Language Model on the NRC Reactor Operator Licensing Examination (arxiv.org) The integration of large language models (LLMs) into the nuclear power industry requires outputs grounded in domain-specific knowledge. This study evaluates a 31-billion-parameter open-weight multimodal model (Gemma 4 31B-IT) on its capaci…
FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs (arxiv.org) Transformer-based models have enabled unprecedented capabilities across language, vision, and multimodal tasks. On-device fine-tuning of transformer models offers a privacy-preserving path to personalized AI, yet remains inefficient on mob…
RadioTrace: Transmitter-Aware Diffusion for Radio Map Estimation without Deployment-Time Fine-Tuning (arxiv.org) Radio map (RM) estimation aims to reconstruct the spatial distribution of wireless signal characteristics, such as received signal strength (RSS), from sparse measurements, a task that is critical for spectrum management, interference miti…
How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning (arxiv.org) A LoRA adapter is a few megabytes that almost everyone treats as a skill rather than a record of the data behind it. We put that assumption on a scale.
Nipping the Butterfly Effect in the Bud: Self-Output Fine-Tuning for Autoregressive Weather Prediction (arxiv.org) Long-horizon weather forecasting is a fundamental challenge in atmospheric science, for which autoregressive Deep Learning Weather Prediction (DLWP) has emerged as the primary paradigm. Although the autoregressive pipeline is highly scalab…
Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs (arxiv.org) Large language models (LLMs) achieve strong generation and reasoning performance, but the Transformer architecture incurs high inference cost. Existing acceleration methods often rely on task-specific fine-tuning or training from scratch,…
Naver-News-KO: A Korean News Summarization Dataset for Open-Source Fine-Tuning of Summarization Models (arxiv.org) We release Naver-News-KO, a Korean news summarization dataset of 27,400 (document, summary) pairs collected from Naver News over a ten-day window in July 2022 across two categories (Economy and IT/Science; 77/23 split), with train/validati…
Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc (arxiv.org) Large language models excel at code generation for mainstream programming languages but struggle with rare, domain-specific languages such as MiniZinc, a constraint modeling language for combinatorial problems. We investigate whether targe…
Opto-ViT-v2: Noise-Resilient On-Chip Fine-Tuning for Photonic Near-Sensor Vision Transformer Accelerators (arxiv.org) Silicon-photonic (SiPh) accelerators have emerged as a promising platform for Vision Transformer (ViT) inference by performing matrix multiplications on microring-resonator (MRR) banks with high throughput and energy efficiency. Extending…
FormulaSPIN: Self-Play Fine-Tuning for Natural Language to Spreadsheet Formula Generation (arxiv.org) Spreadsheet applications are used by hundreds of millions worldwide, yet writing formulas remains a significant barrier. Existing approaches rely on static supervised data, which quickly saturates on limited annotations.
Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AI (arxiv.org) Cloud-based infrastructure has become the dominant platform for deploying large models, particularly large language models (LLMs). Fine-tuning and inference are increasingly delegated to cloud providers for simplified deployment and access…
Statistical Inference for Rank Allocation in Low-Rank Adaptation (arxiv.org) Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models. Since different modules and layers may contribute unequally to downstream adaptation, allocating rank resources under a f…
Reward-Aware Population Scaling of Evolutionary Strategies in LLM Fine-Tuning (arxiv.org) Using Evolutionary Strategies (ES) for fine-tuning large language models is attractive because it is memory-efficient, parallel, and compatible with black-box or discrete rewards. Yet its population-size conclusions conflict sharply: fine-…
When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play (arxiv.org) Supervised fine-tuning (SFT) is widely used to adapt large language models to downstream tasks, but its effect on behavioral diversity in sequential decision-making remains under-explored. We study this question in a controlled suite of de…
TD-DPO: Difference-Aware Preference Optimization for Mitigating Sycophancy in Clinical Autism Intervention Dialogue (arxiv.org) The sycophancy of large language models can increase the safety risk in intervention dialogue for autistic children. Supervised fine-tuning can somewhat reduce sycophancy, but relying solely on positive examples is often insufficient to id…
Reasoning Fine-Tuning Induces Persistent Latent Policy States (arxiv.org) Reasoning-specialized language models show large performance gains over base models, yet the internal changes responsible for improved multi-step reasoning remain poorly understood. It is unclear whether reasoning fine-tuning improves loca…
Parameter-Efficient Continual Fine-Tuning: A Survey (arxiv.org) The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance. However, these models inherit a fundamental limitation from traditional Machine Learning appr…
Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards (arxiv.org) Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine-tuning, while systematic rese…
Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model (arxiv.org) While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks, posing significant security risks. Existing defense metho…
Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA (arxiv.org) Large Language Models (LLMs) are increasingly fine-tuned for critical-domain Question-Answering (QA), yet choosing which small model to adapt, before paying the cost of adaptation, remains difficult. Fine-tuning can improve domain alignmen…
Federated Lightweight Fine-Tuning (arxiv.org) Federated fine-tuning is bottlenecked by communication: FedAvg and pseudo-gradient schemes transmit a payload that scales with the model, and gradient compression shrinks it by only a constant factor. We take a different lever.
Structured Synthetic Reasoning Data for Arithmetic Fine-Tuning of Small Language Models (arxiv.org) Small language models are attractive for local deployment, but they often struggle with multi-step arithmetic reasoning. We study whether structured synthetic reasoning data can improve this behaviour under consumer-hardware constraints.
AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning (arxiv.org) Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead. Parallel Split Learning (PSL) reduces client-side computation by…
Robust Assamese Speech Recognition through Controlled Fine-Tuning of Whisper Models (arxiv.org) Developing Automatic Speech Recognition (ASR) for morphologically rich, low-resource languages such as Assamese is challenging due to insufficient annotated speech data. The pretrained Whisper model performs poorly on Assamese speech recog…
MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning (arxiv.org) Real-world perception and decision making are inherently multimodal, integrating complementary signals across modalities. However, training multimodal models faces two main obstacles.
What Transfers Under Source Shift? Definitions, Examples, and Fine-Tuning for Climate Disclosure Classification (arxiv.org) Climate disclosure classification is a fundamental task for analysing corporate climate disclosures, yet such disclosures appear in many different sources -- annual reports, press releases, and earnings calls -- that differ in length, purp…
Schema-Constrained Document-Level Event Argument Extraction with Lightweight LLM Fine-Tuning (arxiv.org) Event Argument Extraction (EAE) converts documents into structured event records by identifying argument spans and assigning them schema-defined roles. Document-level EAE is challenging due to long-range dependencies between triggers and a…
SATQuest: A Verifier for Logical Reasoning Evaluation and Reinforcement Fine-Tuning of LLMs (arxiv.org) Large language models (LLMs) exhibit strong general reasoning, yet the community lacks controllable, scalable, and verifiable tools to analyze and improve these abilities. We present SATQuest, a verifier that generates diverse SAT-based re…
TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment (arxiv.org) Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment. In practice, service providers need to recover models' safety without re-r…
A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents (arxiv.org) Supervised fine-tuning (SFT) of open-weight LLMs on expert agent trajectories has emerged as a prominent approach to building capable code agents without reliance on proprietary models. A central yet underexplored question is how trajector…
Diversity-Oriented Fine-Tuning for Uncertainty-Based Hallucination Detection (arxiv.org) Existing hallucination detection methods are typically conducted at the inference stage, without making any modifications to the model itself. In this paper, we are interested in exploring fine-tuning strategies that enhance the detectabil…
Democratizing AI with Small Language Models: Structured Benchmarking and Parameter-Efficient Fine-Tuning for Local Deployment (arxiv.org) AI democratization is not primarily a question of matching frontier-scale generality; it is a question of whether capable models can be selected, audited, and specialized under hardware and governance constraints that ordinary institutions…
CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data (arxiv.org) Evaluations should do more than measure a models current performance. They should tell us what to fix for the next model iteration and provide a way to generate targeted post training data.
Full-data accuracy with fewer labels for training and fine-tuning machine-learning force fields (arxiv.org) Machine-learning force fields (MLFFs) are reliable only near their training distribution, making efficient construction of diverse training sets a major bottleneck for both train-from-scratch and foundation fine-tuning workflows. Active le…
A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Models (arxiv.org) We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chai…
Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion (arxiv.org) Large Language Models (LLMs) have revolutionized AI services, but a critical tension emerges: while personalization improves model performance, it consumes scarce computational resources that users must share. When should a user invest in…
Decoupled Alignment for Robust Plug-and-Play Adaptation (arxiv.org) We introduce a training-free safety enhancement method for aligning large language models (LLMs) without the need for supervised fine-tuning or reinforcement learning from human feedback. Our main idea is to provide a robust plug-and-play…
Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment (arxiv.org) Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge. Current approaches bifurcate into Supervised Fine-Tuning (SFT), whic…
Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect (arxiv.org) Can small language models detect and report on perturbations their own internal activations? We investigate this question through the lens of activation steering: injecting concept vectors into a model's residual stream and measuring wheth…
Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization (arxiv.org) Recent diffusion-based video generation models have enabled high-quality personalized video customization through both tuning-based pipelines, which fine-tune a video diffusion model, and reference-based pipelines such as image-to-video ge…
A Multi-Agent System for Autonomous, Fine-Tuning-Free Clinical Symptom Detection: Development and Validation Study (arxiv.org) Clinical notes contain many of the signs and symptoms that bring patients to care, yet this information rarely reaches structured fields. Existing extraction approaches either rely on context-insensitive rules that generate false positives…
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning (arxiv.org) Fine-tuning large language models (LLMs) for downstream tasks is an essential stage of modern AI deployment. Reinforcement learning (RL) has emerged as the dominant fine-tuning paradigm, underpinning many state-of-the-art LLMs.
Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control (arxiv.org) Buildings are expected to shift cooling loads in response to grid conditions. Thermal energy storage (TES) enables this shift, but scheduling it well requires planning hours ahead under storage constraints.
Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning (arxiv.org) Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: cat…
Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models (arxiv.org) The standard post-training recipe for large reasoning models, supervised fine-tuning followed by reinforcement learning (SFT-then-RL), may limit the benefits of the RL stage: while SFT imitates expert demonstrations, it often causes overco…
Transforming LLMs into Efficient Cross-Encoders via Knowledge Distillation for RAG Reranking (arxiv.org) Cross-encoders achieve high reranking accuracy in Retrieval-Augmented Generation (RAG) pipelines but impose quadratic inference costs that limit real-time deployment. We address this by fine-tuning LLaMA 3 (8B) as a drop-in reranker using…
On the Efficiency of LoRA Fine-Tuning for Vision-Language-Action Models in Industrial Robotic Manipulation (arxiv.org) Deploying billion-parameter Vision-Language-Action (VLA) models on industrial hardware requires fine-tuning to bridge the embodiment gap. Full Fine-Tuning (FFT) provides maximal plasticity but requires data centre-grade GPUs.
HyperSafe: Inference-Time Safety Recovery for Fine-Tuned Language Models (arxiv.org) Safety alignment in large language models can be fragile under fine-tuning, as even benign task adaptation may increase harmful compliance. Existing defenses mainly follow two directions: they either intervene during or after fine-tuning t…
SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptation (arxiv.org) Text-to-image diffusion models can emit copyrighted, unsafe, or private content. Safety alignment aims to suppress specific concepts, yet evaluations seldom test whether safety persists under benign downstream fine-tuning routinely applied…
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts (arxiv.org) Advanced reasoning in LLMs on challenging domains like mathematical reasoning can be tackled using verifiable rewards based reinforced fine-tuning (ReFT). In standard ReFT frameworks, a behavior model generates multiple completions with an…
World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning (arxiv.org) Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existi…
Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories (arxiv.org) Large Language Model (LLM) agents are commonly trained from expert trajectories using supervised fine-tuning (SFT), which treats multi-turn agent behavior as ordinary text imitation. This recipe is simple and low-cost, but it only learns t…
Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment (arxiv.org) Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed ``aligned'' can exhibit misaligned behavior after fine-tuning.
Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning (arxiv.org) Large language models (LLMs) remain expensive to fine-tune because full-parameter updates require substantial memory, compute, and per-task storage. We study whether saliency signals originally developed for pruning can be reused to choose…
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (arxiv.org) Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights. However, most exis…
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning (arxiv.org) Parameter-efficient fine-tuning adapts a large language model to one task cheaply, but across a task sequence LoRA-style methods keep stacking low-rank updates on the same frozen weight, so each new task tends to overwrite the previous one…
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning (arxiv.org) Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textit{\textbf{Knowing--Using Gap}}, character…
DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment (arxiv.org) Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-hori…
TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination (arxiv.org) Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updati…
Online Data Selection Is Implicit Alignment (arxiv.org) Supervised fine-tuning (SFT) is often treated as a capability-adaptation step, while alignment is attributed to later preference optimization or reinforcement learning. This separation is incomplete: when examples are scored and kept onlin…
SynthAVE: Scalable Synthetic Labeling for E-Commerce with LLM-Arena Validation (arxiv.org) Fine-tuning large language models (LLMs) for e-commerce attribute extraction requires labeled data representative across thousands of product types, attributes, and multiple languages. This combinatorial scale translates to millions of ann…
Fine-Tuning Integrity for Modern Neural Networks: Structured Drift Proofs via Norm, Rank, and Sparsity Certificates (arxiv.org) Fine-tuning is the dominant paradigm for adapting large machine learning models, yet current deployment pipelines provide no way to verify how a released model was updated. In particular, a model provider or auditor cannot check whether a…
Fine-Tuning Regimes Define Distinct Continual Learning Problems (arxiv.org) Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benchmarking CL methods, comparative evaluations typically keep the fine-tuning regime fixe…
Energy-Efficient GPU DVFS for Fine-Tuning of SLMs on Resource-constrained Embedded Devices (arxiv.org) Dynamic Voltage Frequency Scaling (DVFS) on resource-constrained embedded GPU platforms is essential for energy-efficient small language model (SLM) fine-tuning, as privacy- and personalization-driven adaptation increasingly requires local…
Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design (arxiv.org) Large language models (LLMs) can generate BPMN process models from natural-language descriptions, yet supervised fine-tuning (SFT) limits their output quality to the patterns present in the training data. Reinforcement learning (RL) can op…
Evaluating Fine-Tuning and Metrics for Neural Decompilation of Dart AOT Binaries (arxiv.org) Neural decompilation is increasingly studied as a code-generation problem, yet its evaluation methodology remains underdeveloped for modern languages. We present a systematic empirical study of fine-tuning effectiveness and metric validity…
AMT-APC: Automatic Piano Cover by Fine-Tuning an Automatic Music Transcription Model (arxiv.org) There have been several studies on automatically generating piano covers, and recent advancements in deep learning have enabled the creation of more sophisticated covers. However, existing automatic piano cover models still have room for i…
Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs (arxiv.org) Federated fine-tuning (FFT) has emerged as a privacy-preserving paradigm for collaboratively adapting large language models (LLMs). Built upon federated learning, FFT enables distributed agents to jointly refine a shared pretrained LLM by…
Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning (arxiv.org) Parameter-efficient fine-tuning still leaves a broad space of behavior-changing updates reachable, so a poisoned objective can be represented and optimized. We study an alternative: adaptation constrained to the subspace estimated from a t…
NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation (arxiv.org) Recent agentic approaches to LLM-based kernel generation have achieved impressive results on CUDA. For emerging AI accelerators such as AWS Trainium and Inferentia, automated kernel generation and optimization remain largely unaddressed.
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure (arxiv.org) Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fi…
OpenTinker: Separating Concerns in Agentic Reinforcement Learning (arxiv.org) We introduce \textsc{OpenTinker}, an open infrastructure for training large language model (LLM) agents with many LoRA-backed policies over shared execution resources. Modern agent workloads mix supervised fine-tuning (SFT), online reinfor…
Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing (arxiv.org) Large Language Models (LLMs) and high-dimensional perception networks increasingly rely on parameter-efficient fine-tuning (PEFT) to adapt to diverse operational contexts. However, standard methods like LoRA are structurally limited by a m…
Pretraining Curricula Enable Selective Fine-tuning (arxiv.org) Transformers follow implicit curricula whereby some tasks are learned before others. However, how explicit pretraining curricula influence learning, generalization, and the selectivity of fine-tuning is unclear.
A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training (arxiv.org) For LLM agents, supervised fine-tuning is not only about teacher labels' quality, but also about which interaction contexts those labels condition on. Pure behavioral cloning uses full teacher demonstrations, creating a mismatch between te…
MPSelectTune: Prompt-type Selection for Fine-tuning improves Concept Unlearning in LLMs (arxiv.org) LLMs can be conveniently adapted to a diverse set of tasks, e.g, prediction, question-answering tasks, etc, using appropriate prompts with few-shot examples. Biased or harmful concepts, e.g.
Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks (arxiv.org) We define Tiny Language Models (TLMs) as models below roughly 3B parameters that fit on mainstream consumer devices. We study how to adapt them for and use them on verifiable multiple-choice tasks.
A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning (arxiv.org) Adversarial robustness in Unsupervised Domain Adaptation (UDA) remains a significant challenge due to noisy pseudo labels and inherent distributional shifts between the clean source and adversarially perturbed target domains. Existing appr…
BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression (arxiv.org) Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations. We intr…
Probing Chemical Language Models: Effects of Pre-training and Fine-tuning (arxiv.org) Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic stu…
AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs (arxiv.org) Modern LLMs are increasingly accessed via black-box APIs, requiring users to transmit sensitive prompts, outputs, and fine-tuning data to external providers, creating a critical privacy risk at the API boundary. We introduce AlienLM, a dep…
Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation (arxiv.org) Large language models (LLMs) exhibit remarkable reasoning capabilities, but their task-specific fine-tuning is notoriously plagued by overconfidence, severely hindering trustworthy deployment. We propose Data-Adaptive Lower-Rank Adaptation…
MetaTT: A Global Tensor-Train Adapter for Parameter-Efficient Fine-Tuning (arxiv.org) We present MetaTT, a Tensor Train (TT) adapter framework for fine-tuning of pre-trained transformers. MetaTT enables flexible and parameter-efficient model adaptation by using a single shared TT to factorize transformer sub-modules.
kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail (arxiv.org) Large language models (LLMs) are increasingly deployed in domains requiring guardrails to detect unsafe, off-topic, or adversarial prompts. Existing guardrails predominately rely on fine-tuning to build classifiers, which often suffer from…
EPnG: Adaptive Expert Prune-and-Grow for Parameter-Efficient MoE Fine-tuning (arxiv.org) Mixture-of-Experts (MoE) models scale efficiently but remain costly to adapt due to redundant experts and uniform parameter allocation. Existing parameter-efficient fine-tuning (PEFT) methods such as LoRA ignore MoE routing dynamics, leadi…
Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space (arxiv.org) Model merging aims to combine existing single-task solutions into a multi-task solution without additional data-driven fine-tuning.~Most existing approaches achieve this using geometric properties of local solution spaces. However, such ge…
Atomic Task Graph: A Unified Framework for Agentic Planning and Execution (arxiv.org) LLM-based agents have shown strong potential for solving complex multi-step tasks, yet existing performance improvements often rely on either scaling to larger backbone models or task-specific fine-tuning. The former incurs substantial com…
REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations (arxiv.org) Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise. Standard practice aggregates labels via majority vote or simple av…
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices (arxiv.org) Federated fine-tuning provides a practical route to adapt large language models (LLMs) on edge devices without centralizing private data. However, in mobile deployments, the training wall-clock is often dominated by straggler-limited uplin…
When few labeled target data suffice: a theory of semi-supervised domain adaptation via fine-tuning from multiple adaptive starts (arxiv.org) Semi-supervised domain adaptation (SSDA) seeks to achieve accurate predictions in a target domain with limited labeled target data by exploiting abundant source and unlabeled target data. We study this problem under structural causal model…
A Theory of How Pretraining Shapes Inductive Bias in Fine-Tuning (arxiv.org) Pretraining and fine-tuning are central stages in modern machine learning systems. In practice, feature learning plays an important role across both stages: deep neural networks learn a broad range of useful features during pretraining and…
Fora: From Weight-Space to Function-Space Protection in Capability-Preserving Fine-Tuning (arxiv.org) Full fine-tuning adapts large language models to new tasks but can erode capabilities they already possess. Existing remedies protect through proxies such as parameter distances, importance penalties, output matching, or dominant singular…
An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU (arxiv.org) Fine-tuning Large Language Models (LLMs) has become essential for domain adaptation, but its memory-intensive property exceeds the capabilities of most GPUs. To address this challenge and democratize LLM fine-tuning, we present SlideFormer…
Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems (arxiv.org) We present a framework for fine-tuning flow-matching generative models to enforce physical constraints and solve inverse problems in scientific systems. Starting from a model trained on low-fidelity or observational data, we apply a differ…
Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models (arxiv.org) State-based fine-tuning has emerged as a compelling alternative to weight-based adaptation for transformers, updating lightweight controls into states rather than model weights, offering substantial memory savings while retaining parameter…
Seeing Through Multiple Views: Parameter-Efficient Fine-Tuning via Selective Neurons for Consistent Radiology Report Generation (arxiv.org) Recent years have seen substantial advances in radiology report generation (RRG), yet existing approaches predominantly adopt direct feature fusion when handling multi-view X-ray images. Such approaches overlook the potential clinical inco…
Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs (arxiv.org) Dockerless: Environment-Free Program Verifier for Coding Agents (arxiv.org) Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning (RL). Standard execution-based verification requires runnin…
Mechanistic Personality Analysis of LLMs Steering Personality via Latent Feature Interventions (arxiv.org) Large Language Models (LLMs) have demonstrated the ability to simulate human-like OCEAN personality traits in generated text. Previous efforts have focused on prompt engineering or fine-tuning to shape LLM personality.
Demonstration-Free Robotic Control via LLM Agents (arxiv.org) Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift. We inve…
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation (arxiv.org) Large Language Models (LLMs) demonstrate transformative potential, yet their reasoning remains inconsistent and unreliable. Reinforcement learning (RL)-based fine-tuning is a key mechanism for improvement, but its effectiveness is fundamen…
ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models (arxiv.org) Spatial reasoning remains a persistent challenge for many vision language models (VLMs), and improving it typically requires fine-tuning with substantial additional parameters. Our preliminary analysis reveals that rescaling activations in…
When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon (arxiv.org) Online imitation learning (IL), particularly on-policy distillation, has emerged as a strong LLM post-training approach, often outperforming offline supervised fine-tuning (SFT). Yet a principled understanding of when and why online intera…
Diffusion Fine-tuning with Rewarded Moment Matching Distillation (arxiv.org) Distillation and Reinforcement Learning (RL) fine-tuning are the primary pillars of diffusion post-training. While traditionally studied in isolation, the interaction between these phases remains poorly understood, and in particular how fi…
KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models (arxiv.org) Tabular foundation models have advanced deep learning for tabular data by delivering strong default performance across many small and medium tasks. Yet in niche domains, where data is scarce, high-dimensional, and shifted from the pretrain…
Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise (arxiv.org) Shuffle order can be a larger source of fine-tuning noise than a memoryless analysis predicts: fixed-clock optimizer memory makes local equal-multiset contrasts first order in the learning rate rather than second order, and the resulting o…
A Gravitational Interpretation of Fine-Tuning Reversion (arxiv.org) Fine-tuning on harmless data can partially undo behaviors acquired earlier in training. Safety can erode under benign post-alignment updates, unlearned capabilities can re-emerge, latent traits can transfer through apparently unrelated sup…
Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning (arxiv.org) While general-purpose Large Language Models (LLMs) applied to Geology often hallucinate when reasoning about subsurface structures and deep-time evolution, current AI in Earth sciences predominantly targets surface remote sensing and GIS.…
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL (arxiv.org) The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR). However, SFT introduces distributional d…
Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition (arxiv.org) Fine-tuning multilingual ASR models like Whisper for low-resource languages often improves read speech but degrades spontaneous audio performance. To diagnose this mismatch, we introduce Vividh-ASR, a complexity-stratified benchmark for Hi…
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation (arxiv.org) Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks.
Field Order Should Not Matter: Permutation-Invariant Embedding Model Fine-Tuning for Structured Metadata Retrieval (arxiv.org) We study retrieval over catalogs of structured metadata, where each record is a small schema whose fields answer different kinds of query. Embedding a record with a text encoder first serializes its fields into a string, which forces a cho…
Do We Still Need Fine Tuning? Turkish Sentiment Analysis in the Era of Large Language Model (arxiv.org) This study examines whether supervised fine-tuning remains necessary for Turkish sentiment analysis in the era of large language models. We compare classical machine learning methods, fine-tuned pretrained language models, and prompted lar…
Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks (arxiv.org) Would experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture? Large Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimiz…
Fine-Tuning General-Purpose Large Language Models for Agricultural Applications:A Reproducible Framework and Evaluation Protocol Based on Qwen3-8B (arxiv.org) General-purpose large language models (LLMs) have demonstrated strong abilities in opendomain question answering, information extraction, and text generation. Agricultural applications, however, are domain-specific, region-dependent, time-…
The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning (arxiv.org) Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task. However, prior work has shown that this increase in capability comes with a cost: it can increase a model's tendency to r…
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning (arxiv.org) Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models. When applied sequentially across tasks in Continual Learning (CL), the standard assumption is that each new task requir…
Causal Connections: Leveraging Multilingual Fine-Tuning for Financial QA@FinCausal 2026 (arxiv.org) This paper describes team HSA_CORAL's submission to the FinCausal 2026 shared task on extracting cause-effect relations from financial narratives via extractive question answering in English and Spanish. We compare three modeling families:…
MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation (arxiv.org) Unlike regular tokens derived from existing text corpora, special tokens are artificially created to annotate structured conversations during the fine-tuning process of Large Language Models (LLMs). Serving as metadata of training data, th…
Two-Stage Fine-Tuning for Protein Sequence Generation with Targeted Amino-Acid Composition (arxiv.org) Protein language models are standard priors for biological sequence generation, but steering them toward explicit distributional design targets remains largely unexplored. We study a constrained protein generation problem in which sequence…
Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuning (arxiv.org) Recent work has shown that fine-tuning large language models (LLMs) for social warmth degrades factual reliability and increases sycophancy. We investigate a related but distinct failure mode: warmth fine-tuning also weakens adversarial sa…
Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization (arxiv.org) Building general-purpose role-playing agents that faithfully portray any character from a natural-language profile remains challenging. The dominant paradigm -- supervised fine-tuning -- encourages behavioral mimicry without deep, human-li…
Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean (arxiv.org) Large pretrained text-to-speech (TTS) models sound almost human for well-resourced languages, but much worse for languages that are rare in their training data. We study this quality gap for Khmer and Korean using VoxCPM2, a 2.4B-parameter…
Inherited Circuits, Learned Semantics: How Fine-Tuning Creates Evasion Vulnerabilities Invisible to Standard Evaluation (arxiv.org) LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data. We show that this can miss vulnerabilities introduced by fine-tuning itself: models can learn token-l…
Localizing RL-Induced Tool Use to a Single Crosscoder Feature (arxiv.org) Fine-tuning through RL reshapes the internal representations of language models to enable agentic behaviors such as tool use, yet the mechanistic basis of these changes remains poorly understood. While RL substantially improves structured…
SSM Adapters via Hankel Reduced-order Modeling: Injection Site Determines Task Suitability in Long-Context Fine-Tuning (arxiv.org) While parameter-efficient fine-tuning (PEFT) typically targets attention projectors, its efficacy for tasks requiring sequential state accumulation remains under-explored. We examine if PEFT for such tasks can benefit from state space mode…
FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-Distillation (arxiv.org) Vision-Language-Action (VLA) models are often constrained by the imitation ceiling imposed by sub-optimal data. While Reinforcement Learning (RL) fine-tuning can surpass this limit, it is notoriously sample inefficient.
Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel (huggingface.co) Accelerating Transformers Fine-Tuning with NVIDIA NeMo AutoModel NVIDIA NeMo AutoModel is an open library part of the NVIDIA NeMo framework for building custom generative AI models at scale. NeMo AutoModel builds cleanly on top of v5, addi…
Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation (arxiv.org) Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs. To improve SFT data quality, we develop a unified bilevel fra…
Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models (arxiv.org) Prompt-based learning has emerged as a dominant paradigm in natural language processing. This study explores the impact of diverse pre-training objectives on the performance of encoder-decoder pre-trained language models across generation…
Oracle-RLAIF: An Improved Fine-Tuning Framework for Multi-modal Video Models using Reinforcement Learning from Ranking Feedback (arxiv.org) Recent advances in large video-language models (VLMs) rely on extensive fine-tuning techniques that strengthen alignment between textual and visual comprehension. Leading pipelines typically pair supervised fine-tuning (SFT) with reinforce…
On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs (arxiv.org) As increasingly large pre-trained models are released, deploying them on edge devices for privacy-preserving applications requires effective compression. Recent works combine quantization with the fine-tuning of high-precision LoRA adapter…
The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model (arxiv.org) Transformer-based models underpin modern natural language processing but incur rapidly growing computational and energy costs. As training scales in both model size and parallelism, accurately predicting energy consumption has become criti…
Attention-Spectrum Regularization for Replay-Free Continual Multimodal LLMs (arxiv.org) Multimodal large language models (MLLMs) are increasingly required to adapt to non-stationary streams of visual domains, question types, and user instructions, yet continual fine-tuning often causes severe forgetting of previously acquired…
Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning (arxiv.org) As large language models (LLMs) are increasingly deployed at the network edge to provide pervasive generative AI services, decentralized federated learning (DFL) provides a vital mechanism for privacy-preserving, domain-specific fine-tunin…
Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers (arxiv.org) Parameter-efficient fine-tuning (PEFT) has become a practical solution for adapting large pretrained vision transformers (ViTs) to downstream tasks while updating only a small subset of parameters. However, existing adapter-based methods p…
Translating Inference-Time Control to Radiology Vision-Language Models: Activation Steering for Pneumonia Classification on Chest X-rays (arxiv.org) Inference-time engineering can alter model behavior without fine-tuning. However, its utility for improving diagnostic performance in medical vision-language models (VLMs) remains unclear.
MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning (arxiv.org) Multi-modal Large Language Models (MLLMs) have demonstrated remarkable capabilities in semantic understanding and common sense reasoning, making them promising candidates for solving planning problems in autonomous driving. However, the ne…
Skin-Deep: A Geometric Diagnostic for Alignment Fragility in Large Language Model Representations (arxiv.org) Alignment tuning is meant to make harmful-request refusal robust, yet this safety behavior can be erased by a small set of benign fine-tuning examples. This is a deployment risk for open-weight models because a checkpoint can pass refusal…
Claude Opus 4.8 launched in May but says its training cutoff is Jan 2026. Am I understanding the cutoff vs launch gap correctly? (www.reddit.comhttps) Was debugging my TTS pipeline and doing some research on natural voice options, and Claude Opus 4.8 mentioned its training cutoff is January 2026. But the model launched on May 28, 2026.
Meta Flow Maps enable scalable reward alignment (arxiv.org) Controlling generative models is computationally expensive. This is because optimal alignment with a reward function--whether via inference-time steering or fine-tuning--requires estimating the value function.
Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates (arxiv.org) Low-rank adaptation (LoRA) makes it cheap to train many domain- and task-specific language model adapters, but whether two adapters can be merged is usually discovered only after both have been fully trained and evaluated. This late feedba…
AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models (arxiv.org) Post-training alignment of large language models often combines supervised fine-tuning (SFT) on expert demonstrations with reinforcement learning (RL) from preference or verifiable feedback. SFT provides a useful behavioral anchor but can…
Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software (arxiv.org) Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved. We present CWE-Trace, a framework for LLM vulnerability detection built from 834 manuall…
Cross-Dataset, Age, and Gender Generalization: A Comprehensive Analysis of Fine-Tuning Strategies for Low-Resource Children's ASR (arxiv.org) The challenge associated with recognizing dysarthric speech primarily arises from pronounced acoustic variability attributed to impaired articulatory precision. Past research has demonstrated improved recognition through the use of hybrid…
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices (arxiv.org) Fine-tuning of Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) on an end-user's data offers personalized experiences while keeping data private, but faces severe memory constraints on consumer hardware. Peak memory during fin…
Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer (arxiv.org) We study cross-lingual transfer by fine-tuning seven large language models (4B--671B parameters) on Arabic and evaluating zero-shot reading comprehension on Semitic languages and non-Semitic controls. Across dense and Mixture-of-Experts ar…
SWE-Future: Forecast-Conditioned Data Synthesis for Future-Oriented Software Engineering Agents (arxiv.org) Realistic coding-agent benchmarks often replay public GitHub issues and pull requests, making them vulnerable to overlap with model pretraining, fine-tuning, synthetic-data generation, or benchmark-driven model selection. Fully synthetic t…
ARIADNE: Agnostic Routing for Inference-time Adapter DyNamic sElection (arxiv.org) The increasing deployment of parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters. In this setting, inference-time queries often arrive without task lab…
Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Models (arxiv.org) While recent advances in machine learning have equipped Weather Foundation Models (WFMs) with substantial generalization capabilities across diverse downstream tasks, the escalating computational requirements associated with their expandin…
Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs (arxiv.org) Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data. We argue this not only risks reinforcing exposure to sensitive data, but also fundamentally contra…
Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning (arxiv.org) Pre-trained materials foundation models, or machine learning interatomic potentials, leverage general physicochemical knowledge to effectively approximate potential energy surfaces. However, they often require domain-specific calibration d…
Beyond LoRA: Can you beat the most popular fine-tuning technique? (huggingface.co) Beyond LoRA: Can you beat the most popular fine-tuning technique? When you plan to fine-tune a model in a parameter-efficient way, think beyond LoRA If you want to fine-tune an open model on your own data, you are probably interested in so…
X-REFINE: XAI-based RElevance input-Filtering and archItecture fiNe-tuning for channel Estimation (arxiv.org) AI-native architectures are vital for 6G wireless communications. The black-box nature and high complexity of deep learning models employed in critical applications, such as channel estimation, limit their practical deployment.
Gradual Fine-Tuning for Flow Matching Models (arxiv.org) Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or computational constraints. While recent work has produced significant advances, particularly in the area of reward-based fine…
Fine-tuning LLMs for Passive Depression Severity Estimation from AI Mental Health Dialogue (arxiv.org) Depression is the leading cause of disability worldwide, and early detection of symptom change is essential for timely intervention. Validated instruments such as the Patient Health Questionnaire-9 (PHQ-9) support symptom monitoring at sca…
Improving low-resource ASR using bilingual fine-tuning with language identification: a cross-linguistic evaluation (arxiv.org) This study explores how bilingual fine-tuning affects automatic speech recognition (ASR) in low-resource languages. We evaluate this method across nine linguistically and geographically diverse language pairs, covering a range of language…
RepSelect: Robust LLM Unlearning via Representation Selectivity (arxiv.org) Making large language models (LLMs) deeply forget specific knowledge and values without sacrificing general capabilities remains a central challenge in unlearning. However, current methods are easily reversed by fine-tuning or few-shot pro…
TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins (arxiv.org) Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and naïve runs can even degrade model performance. This raises a practical que…
A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction (arxiv.org) The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense. However, the theoretical limits of pre-hoc performance prediction remai…
Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steering (arxiv.org) Open-weight video diffusion models can generate photorealistic unsafe content, from violence to misinformation, yet existing defenses either require expensive safety fine-tuning that degrades general capability, or apply external filters t…
Could we use latent representations as internal safety checks during generation? (www.reddit.com via reddit) Reading Anthropic's work on emotion-like representations got me thinking. If we can identify latent representations for concepts such as fear, despair, etc., could similar methods be used to identify representations associated with malic…
Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes (arxiv.org) When pretrained VLA policies are fine-tuned through online RL, each rollout episode produces only a single binary outcome (success or failure), yet the actor update requires per-transition supervision. Existing approaches commonly reduce t…
Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning (arxiv.org) We study the task of CVE-conditioned exploit generation, where a model drafts proof-of-concept (PoC) exploits given software vulnerability context. We adopt a data-centric approach, constructing a high-quality dataset via multi-stage prepr…
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning (arxiv.org) Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data. A major obstacle in large-scale fine-tuning is the…
Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning (arxiv.org) Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities. This paper presents a systematic compar…
Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents (arxiv.org) Graphical user interface (GUI) agents powered by multimodal large language models (MLLMs) have shown greater promise for human-interaction. However, due to the high fine-tuning cost, users often rely on open-source GUI agents or APIs offer…
SDFLoRA: Selective Decoupled Federated LoRA for Privacy-preserving Fine-tuning with Heterogeneous Clients (arxiv.org) Federated learning (FL) for large language models (LLMs) has attracted increasing attention as a privacy-preserving approach for adapting models over distributed data, where parameter-efficient methods such as Low-Rank Adaptation (LoRA) ar…
Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning (arxiv.org) Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks. In practice, SFT on a full dataset is computationally expensive and sometimes suffers from overfitting or bias amplificati…
Fine-Tuning a 7B Advisor on Free-Tier GPUs: An Adapter-Handoff Recipe and a Synthetic-Data Reliability Caution (arxiv.org) Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on. We report two th…
SPARK: Security Knowledge Priming and Representation-Guided Knowledge Activation for LLM-based Secure Code Generation (arxiv.org) Large language models routinely generate code with exploitable security flaws. Prior literature attributes this limitation to a lack of security expertise, steering current defense mechanisms toward heavy fine-tuning or external knowledge…
LiteOdyssey: A Lightweight Reasoning AI Agent for Interpretable Rare-Disease Diagnosis (arxiv.org) Most medical AI systems improve by scaling additional machinery: more fine-tuning data, more agents, and/or larger retrieval databases. In rare-disease diagnosis, however, such scaling can produce systems that are difficult to deploy, audi…
Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning (arxiv.org) Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases, enabling non-expert users to access data intuitively. While recent advances in large language models (LLMs) have shown promise…
NeST: Neuron Selective Tuning for LLM Safety (arxiv.org) Safety alignment is essential for the responsible deployment of Large Language Models (LLMs). Yet, existing approaches often rely on heavyweight fine-tuning that is costly to update, audit, and maintain across model families.
Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback (arxiv.org) We study multi-domain LLM training in which two models, each stronger in a different domain, co-evolve by tutoring each other through on-policy feedback. Unlike one-way distillation or single-model fine-tuning, our goal is mutual Pareto im…
3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding (arxiv.org) Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its potential in 3D scene understanding remains under-explored.…
Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost (arxiv.org) Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune. Standard fine-tuning paradigms, including Reinforcement Learni…
Dense Coordinate-List Fine-Tuning Induces a Controllable Interference Surface in Vision-Language Models (arxiv.org) Fine-tuning vision-language models to emit dense coordinate lists improves visual grounding but also changes how models serialize, repeat, and terminate structured outputs. We study this behavior as a generation and control surface.
PolyAlign: Conditional Human-Distribution Alignment (arxiv.org) Post-training methods such as supervised fine-tuning (SFT) and preference optimization typically align language models toward a single global assistant behavior. While effective for improving average helpfulness, this can suppress the natu…
Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study (arxiv.org) We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique. Our experimental results demonstrate that DPO simplifies the training pipeline, improves computatio…
Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization (arxiv.org) Large Language Models such as GPT-4o and GPT-5 achieve strong zero-shot performance on biomedical claim verification, but cost and opacity limit scalable use. We fine-tune three small LLMs: Phi-3-mini (3.8B), Qwen2.5-3B, and Mistral-7B, vi…
MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection (arxiv.org) Detecting mental health disorders from Arabic social media text remains challenging due to dialectal variation, informal language, limited high-quality annotated resources, and severe class imbalance. While English mental health natural la…
Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning (arxiv.org) Retrieval-augmented generation (RAG) has become a standard mechanism for grounding language models in external knowledge, yet conventional retrieval based on lexical or semantic similarity is poorly suited for complex reasoning tasks: a se…
Fantastic Scientific Agents and How to Build Them: AgentBuild for Rietveld Refinement (arxiv.org) As scientific workflows shift from deterministic executables to LLM-based agents, the development practices on offer, such as fine-tuning, reinforcement learning, and prompt-and-go, bury the scientist's judgment. We propose treating agent…
MobileFineTuner: A Mobile-Native Framework for On-Device LLM Fine-Tuning in Real-World Embedded AI Applications (arxiv.org) Large language models (LLMs) are moving from cloud-centric services toward on-device embedded AI, where models interact with private, longitudinal signals sensed from users and their physical environments. Mobile phones are a natural platf…
AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin (arxiv.org) Fine-tuning large language models (LLMs) improves performance but introduces critical safety vulnerabilities: even minimal harmful data can severely compromise safety measures. We observe that perturbations orthogonal to the alignment dire…
Steering the Noise: Turning Random Perturbations into Effective Descent for Memory-Efficient LLM Fine-Tuning (arxiv.org) Fine-tuning large language models (LLMs) achieves strong performance but is often limited by the memory overhead of backpropagation. Zeroth-order (ZO) optimization avoids this overhead by estimating gradients through forward passes alone,…
ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing (arxiv.org) Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both m…
Harness In-Context Operator Learning with Chain of Operators (arxiv.org) Neural operators approximate mappings between function spaces, but often generalize poorly to other operators and usually require fine-tuning or retraining. In-Context Operator Networks (ICON) addresses this issue by prompting the model wi…
Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning (arxiv.org) Vision-Language-Action (VLA) models have demonstrated remarkable zero-shot generalization in robotic manipulation, yet the vast majority of pre-trained pipelines remain strictly confined to low-DoF parallel grippers. Adapting these rich se…
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training (arxiv.org) There are two main Parameter-Efficient Fine-Tuning (PEFT) techniques for Large Language Models (LLMs). While Low-Rank Adaptation (LoRA) introduces additional weights between the LLM layers, Soft Prompting introduces additional fine-tuning-…
Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It (arxiv.org) The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring (arxiv.org) A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design (arxiv.org) Supervised fine-tuning (SFT) typically maximizes the likelihood of every token in a demonstrated trajectory. However, an observed token can be non-unique, noisy, or misaligned with the model prior.
Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning (arxiv.org) Parameter-efficient fine-tuning (PEFT) aims to adapt pretrained models with a small trainable parameter subset, however, most existing methods choose this subset from fixed architectural heuristics rather than using dynamic, task-aware cri…
Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning (arxiv.org) Fine-tuning safety aligned large language models (LLMs) on downstream data improves adaptation but may erode learned safety behavior. Existing methods use fixed safety examples, global constraints, or one-sided task filtering.
Supervised Fine-tuning with Synthetic Rationale Data Hurts Real-World Disease Prediction (arxiv.org) Supervised fine-tuning with synthetic rationale data is widely assumed to improve language model performance on clinical prediction tasks by teaching models not just what to predict but why. We test this assumption on five-year Alzheimer's…
Domain-Adapted Small Language Models with Hybrid Post-Processing: Achieving Cost-Efficient, Low-Latency Multi-Label Structured Prediction via LoRA Fine-Tuning on Scarce Data (arxiv.org) Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space (arxiv.org) Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning Updates (arxiv.org) AutoTail-BSFGM: Class-Balance-Aware Fine-Tuning for Chinese Scholarly Text Classification (arxiv.org) PriFT: Prior-Support Guided Supervised Fine-Tuning (arxiv.org) Self-Mined Hardness for Safety Fine-Tuning (arxiv.org) Data Synthesis and Parameter-Efficient Fine-Tuning for Low-Resource NMT: A Case Study on Q'eqchi' Mayan (arxiv.org) FiberTune: Preserving Action-Fiber Visual Residuals in Vision-Language-Action Fine-Tuning (arxiv.org) Ego-Pi: VLA Fine-Tuning for Ego-Centric Human and Robot Data (arxiv.org) Single-Cell Cross-Modal Transfer by Adversarial Fine-Tuning of Foundation Models (arxiv.org) Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER) (arxiv.org) A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers (arxiv.org) Phantom transitions in language model fine-tuning (arxiv.org) How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions (arxiv.org) Financial transaction processing requires extracting structured merchant information from noisy, abbreviated bank transaction strings at scale. Our current production system, a LoRA-fine-tuned LLaMA 3.1-8B, achieves 96.95% F1 on this task,…
↯ Llama↯ Fine Tuning↯ Llama 3.1↯ Llama 3.1↯ Llama 3.1↯ Llama 3.1↯ Llama 3.1↯ Llama 3.1llamafine-tuning
SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows (arxiv.org) RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning (arxiv.org) Fine-Tuning and Serving Gemma 4 31B on Google Cloud TPU: A Technical Comparison with GPU Baselines (arxiv.org) The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning (arxiv.org) Deploying Small Language Models (SLMs) on edge devices requires efficient fine-tuning strategies that adapt models to new tasks without degrading their general capabilities. In this study, we benchmark five sub-1B models (135M-1B) on mathe…
SafeGene: Reusable Adapters for Transferable Safety Alignment (arxiv.org) Open-weight LLMs are increasingly fine-tuned into customized assistants, but downstream fine-tuning can weaken safety alignment and make models more vulnerable to malicious prompts, even when the training data is not intentionally harmful.…
Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning (arxiv.org) (Mis)generalization of Helpful-only Fine-tuning (arxiv.org) ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Models (arxiv.org) Emotion-Aware Image Generation from Korean Diary Text via LLM-based Prompt Translation and LoRA Fine-Tuning (arxiv.org) T2I models cannot effectively capture sentiment from various types of text, including diaries, as they primarily focus on visual object-related patterns rather than contextual emotional understanding. This paper proposes an emotion-aware t…
Multilingual Fine-Tuning via Localized Gradient Conflict Resolution (arxiv.org) The rapid evolution of Large Language Models (LLMs) has established cross-lingual versatility as a defining feature of modern systems. However, fine-tuning these models frequently induces negative interference across languages.
Fine-Tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for Robot Video Generation (huggingface.co) Fine-Tuning NVIDIA Cosmos Predict 2.5 with LoRA/DoRA for Robot Video Generation Motivation NVIDIA Cosmos Predict 2.5 is a large-scale world model capable of generating physically plausible videos conditioned on text, images, or video clips…
I drew the entire AI stack on one page... and it's mostly not models. (www.reddit.com) Most "AI progress" talk lives on one layer: models. Bigger model, smaller model, new benchmark, repeat.
Realistically, what is the best use of consumer hardware for AI? (www.reddit.com) I want to move past the "democratization" slogans. What is the most practical contribution consumer-grade hardware can make to the ecosystem right now?
MedQA: Fine-Tuning a Clinical AI on AMD ROCm — No CUDA Required (huggingface.co) MedQA: Fine-Tuning a Clinical AI on AMD ROCm — No CUDA Required The Idea Medical question answering is one of those tasks where the stakes are genuinely high. A model that confidently picks the wrong answer on a clinical MCQ isn't just wro…
Three lessons from fine-tuning a 5B code assistant — bad outputs from 5% → 0% (www.reddit.com) Spent a week doing LoRA fine-tuning on Gemma 4 E2B (gemma-4-e2b-it, ~5.1B total params, ~2B active in the text decoder) for a narrow Python code-generation task. Setup: Model: Gemma 4 E2B, bf16, language_model only (vision + audio towers f…
Hardware choice (www.reddit.com) We want to set up the following: A Local LLM environment for AI development, used by multiple software developers Infrastructure for training Vision AI models Capabilities for AI model fine-tuning I’m currently struggling to decide between…
We open-sourced Chaperone-Thinking-LQ-1.0 — a 4-bit GPTQ + QLoRA fine-tuned DeepSeek-R1-32B that hits 84% on MedQA in ~20GB (www.reddit.com) Hey everyone, We just open-sourced our reasoning model, Chaperone-Thinking-LQ-1.0, on Hugging Face. It's built on DeepSeek-R1-Distill-Qwen-32B but goes well beyond a simple quantization — here's what we actually did: The pipeline: 4-bit GP…
An Alignment Experiment: Native LLM vs. Custom Engine on Classical Naming. The statistical inertia is real. (www.reddit.com) [Project] I benchmarked my custom 2nd-order optimizer against AdamW across 1M, 5M, and 10M parameters. Here are the raw test results and scaling laws. (www.reddit.com) New Claude Opus 4.7 tell dropped (www.reddit.com) DGX Spark users: What's the easiest way to do multi-node vLLM clustering with a browser UI and training? (www.reddit.com) Hey r/LocalLLaMA, I've been running a small 4-node DGX Spark cluster on a 400µT fabric switch and got frustrated with the usual raw Ray/vLLM scripts and EXO basically ignoring pure NVIDIA paths. I started from the solid foundation in [eugr…
Curiosity about Chatterbox's architecture led me to fine-tune it for 8 Indian languages by LoRA, using 1.4% params (www.reddit.com) TL;DR: Fine-tuned Chatterbox-Multilingual for Telugu, Kannada, Bengali, Tamil, Malayalam, Marathi, Gujarati, and Hindi using LoRA adapters + tokenizer extension. Only 7.8M / 544M parameters trained.
I open-sourced media-tsunami — a tool that extracts your brand voice into a CLAUDE.md any LLM can load (www.reddit.com) Your brand voice is probably a PDF nobody reads, or it's trapped in one founder's head, or it's scattered across a thousand ChatGPT histories. I wanted to treat it like code instead — a file you can version, share, diff, and plug into any…
20x Faster TRL Fine-tuning with RapidFire AI (huggingface.co) (LoRA) Fine-Tuning FLUX.1-dev on Consumer Hardware (huggingface.co) Building smarter maps with GPT-4o vision fine-tuning (openai.com) Argilla 2.4: Easily Build Fine-Tuning and Evaluation Datasets on the Hub — No Code Required (huggingface.co) Introducing vision to the fine-tuning API (openai.com) Fine-tuning LLMs to 1.58bit: extreme quantization made easy (huggingface.co) Fine-tuning GPT-4o webinar (openai.com) LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning? (huggingface.co) Fine-tuning Florence-2 - Microsoft's Cutting-edge Vision Language Models (huggingface.co) Introducing improvements to the fine-tuning API and expanding our custom models program (openai.com) Fine-Tuning Gemma Models in Hugging Face (huggingface.co) Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL (huggingface.co) Fine-tuning Llama 2 70B using PyTorch FSDP (huggingface.co) OpenAI partners with Scale to provide support for enterprises fine-tuning models (openai.com) GPT-3.5 Turbo fine-tuning and API updates (openai.com) Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU (huggingface.co) Fine-tuning GPT-3 to scale video creation (openai.com) Accelerating PyTorch distributed fine-tuning with Intel technologies (huggingface.co)