Hey folks, this is for those who appreciate experimentation on open-sourced AI models. We fine-tuned open-sourced SMLs (3B and 7B parameters) with SFT + DPO against commercial models like GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.6, Google Do…
#dpo
15 items
SFT + DPO on open-sourced SLMs (www.reddit.com) Free hands-on lab: build a ReAct agent 3 ways (create_agent, raw LangGraph with tool-call budget, NVIDIA NAT YAML) (www.reddit.com) Probe-Detected Grokking in Multi-Probe DPO (openinterp.org via hn) Probe-Detected Grokking in Multi-Probe DPO Orthogonal Learning Beyond Task-Specific Detectors in Qwen3.6-27B Probe-Detected Grokking in Multi-Probe DPO: Orthogonal Learning Beyond Task-Specific Detectors Abstract We report a phase-transiti…
Karpathy's autoresearch, 50 DPO experiments, 300 human judges (huggingface.co via hn) When does autoresearch need a human? Autonomous research agents are everywhere in AI research workflows now.
I tried a selective training method for hallucination — beats DPO and SFT with ~10% data (www.reddit.com) github link : genji970/hallucination-mitigation-via-contrastive-sampling-method: Selective contrastive post-training for hallucination mitigation in LLMs — improves factuality with ~10% data. ## Experimental Results ### (a) DPO vs.
Overcoming State Inertia: Minimally Invasive Temporal Alignment for Evolving Contexts (arxiv.org) Long-context dialogue systems suffer from state inertia, where models over-attend to history and fail to adapt to evolving intents. We demonstrate that standard alignment methods like DPO and even recent long-context optimization technique…
DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory (and its Loss' Convexity is Dispensable) (arxiv.org) Repeated post-training is not Self-improving: Diagnosing Scientific Amnesia in Continual DPO Pipelines (arxiv.org) Industrial LLM teams often ship behavior updates by repeatedly DPO-training a base model on sequences of related preference-data campaigns. The dominant failure mode in this regime is not always classical catastrophic forgetting: a pipelin…
Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study (arxiv.org) We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique. Our experimental results demonstrate that DPO simplifies the training pipeline, improves computatio…
AMD R9700 vs GB10 (www.reddit.com via reddit) I have a budget of 5K, and want to buy some gpus my requirement is 48gb+ vram, because I finetune small language model, perform DPO, in general tinkering/ development is my usecase. if you where in my shoe which among these would you get,…
Fine-tuned Qwen2.5-7B to 96% of Claude Haiku on a domain-specific task using ~$3 of API calls and zero human labelers (www.reddit.comhttps) Built a decision-reasoning engine (Orlog) and wanted to fine-tune a local model for it instead of paying per-call forever. The method (DV-DPO): Run a 3-voice council on each question, produce a synthesis Cross-examine: losing voices challe…
DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment (arxiv.org) P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization (arxiv.org) Ultra-ml-intern: huggingface/ml-intern's workflow as a Claude Code plugin (www.reddit.com) huggingface/ml-intern is HF's autonomous ML engineer — reads papers, audits datasets, ships SFT/DPO/LoRA/GRPO runs to HF Jobs. it's a standalone python harness with its own agent loop calling the Claude API.
Fine-tune Llama 2 with DPO (huggingface.co)