Qwen 2.5

1. Fine-tuned Qwen2.5-7B to 96% of Claude Haiku on a domain-specific task using ~$3 of API calls and zero human labelers (www.reddit.comhttps)

   14w dpohaiku

   Built a decision-reasoning engine (Orlog) and wanted to fine-tune a local model for it instead of paying per-call forever. The method (DV-DPO): Run a 3-voice council on each question, produce a synthesis Cross-examine: losing voices challe…

2. Has anyone tried running retrieval inside the model, not before it? (www.reddit.com via reddit)

   14w llama

   Been messing with a bolt-on refiner block for small models. Insert a small trainable transformer layer at the midpoint of a frozen base model, loop it 2-4 times over the hidden states.

← all threads