We're bringing the advisor strategy to the Claude Platform. Pair Opus as an advisor with Sonnet or Haiku as an executor, and your agents can consult Opus mid-task when they hit a hard decision.
#swe-bench
91 items
Claude is now adopting the advisor strategy (www.reddit.com) Reminder that Anthropic reported memorization on some SWE-Bench Pro problems (www.reddit.com) "SWE-bench Verified, Pro, and Multilingual: Our memorization screens flag a subset of problems in these SWE-bench evals." https://www.anthropic.com/news/claude-opus-4-7
AA introduces Coding Agent Index - Performance Comparisons between Model & Harness Combinations (www.reddit.com) The Artificial Analysis Coding Agent Index includes 3 leading benchmarks that represent a broad spectrum of coding agent use: ➤ SWE-Bench-Pro-Hard-AA, 150 realistic coding tasks that frontier models struggle with, sampled from Scale AI’s S…
Running gpt and glm-5.1 side by side. Honestly can’t tell the difference (www.reddit.com) So I have been running gpt and glm-5.1 side by side lately and tbh the gap is way smaller than what im paying for On SWE-Bench Pro glm-5.1 actually took the top spot globally, beat gpt-5.4 and opus 4.6. overall coding score is like 55 vs g…
Dense vs. MoE gap is shrinking fast with the 3.6-27B release (www.reddit.com) 27B Dense vs. 35B-A3B MoE): - Dense still holds the crown: It still wins out on most tasks overall.
ChatGPT-5.5 Beats Opus in Realistic Benchmark (DeepSWE) (www.reddit.com) From the website, it touts: Contamination free: Tasks are written from scratch, not adapted from existing commits or PRs, so no model has seen the solution during pretraining. High diversity: Tasks span a broad pool of 91 repositories acro…
Show HN: Benzi – A Code Intillegence/Harness Beating Claude Code and CodeGraph (benzi.fly.dev via hn) Benzi · benchmarks Apples to apples comparison: Benzi harness vs mainstream harnesses, 24 GitHub issues, 10 languages. Benzi harness on SWE-bench Verified.
SWE-rebench Leaderboard (March, April and May 2026): GPT-5.5, Opus 4.7, Cursor (Composer 2.5), Kimi K2.6 and More (swe-rebench.com via reddit) Hi all, Sorry for going missing — we’ve been collecting a larger, higher-quality set of more complex tasks. We’re excited to share a major leaderboard update covering the past three months.
3code benches 75 lower token use vs. OpenCode on 10-task SWE-Bench subset (capocasa.dev via hn) 3code benches 75 lower token use vs opencode on 10-task SWE-Bench subset I just ran the first formal validation of 3code! Now this is a very limited benchmark- 10 tasks across 1 model- but the results are promising to say the least.
How do you actually know if Opus 4.7 is better for your specific agent use case? (www.reddit.com) Anthropic shipped Opus 4.7 yesterday. The headline numbers are real: 64.3% on SWE-bench Pro (up from 53.4%), best-in-class on MCP-Atlas at 77.3% for multi-tool orchestration, 14% improvement on multi-step agentic reasoning, and one-third f…
10-task GLM 5.3 harness bench: Claude, OpenCode, pi, zcode, Hermes and 3code (capocasa.dev via hn) 10-task GLM 5.3 harness bench: claude, opencode, pi, zcode, hermes and 3code I'm performing a series of harness benchmarks on the same 10 SWE-bench verified tasks representatively chosen for difficulty. This is far from a perfect measure a…
↯ Glm↯ Swe Bench↯ GLM 5.3↯ GLM 5.3↯ GLM 5.3↯ GLM 5.3↯ GLM 5.3swe-benchglm
Claude Code Degraded Before Opus 4.8 Release (marginlab.ai via hn) Claude Code degraded for the week before Opus 4.8's release Our SWE-Bench-Pro tracker caught a statistically significant, weeklong drop in Claude Code's pass rate just before Opus 4.8 shipped, and the recovery that followed. We run Claude…
MiniMax M3: The First Open-Weights Model to Combine Three Frontier Capabilities (twitter.com via hn) MiniMax (official) @MiniMax_AI Introducing MiniMax M3: The First Open-Weights Model to Combine Three Frontier Capabilities - Coding & Agentic Frontier: 59.0% SWE-Bench Pro, 66.0% Terminal Bench 2.1, 34.8% SWE-fficiency, 28.8% KernelBench H…
Show HN: New Benchmark from SWE-bench team is 0% solved (programbench.com via hn) ./ProgramBench Can language models rebuild programs from scratch? Given only a compiled binary and its documentation, agents must architect and implement a complete codebase that reproduces the original program's behavior.
Show HN: Codex context bloat? 87% avg reduction on SWE-bench Verified traces (www.npmjs.com via hn) If you had to build a context window manager in 24h, would you stick to the existing model or come up with something better? Here's what I did: 1.
Local-first agent evaluation collapses once runs are long and stateful? (www.reddit.com) I started out running agent evaluations locally because most ai agent benchmarks and examples assume that setup. And to be fair local runs do work for debugging and small experiments.
Codex GPT-5.6-sol Performance Tracker (marginlab.ai via hn) Codex gpt-5.6-sol Performance Tracker The goal of this tracker is to detect statistically significant degradations in Codex with gpt-5.6-sol performance on SWE tasks. - • Updated daily: Daily benchmarks on a curated subset of SWE-Bench-Pro…
Show HN: Self-bench – build SWE-bench style evals from private repos (github.com via hn) Hey HN! We all know that most public evals are saturated and are hard to trust.
Preventing agents from cheating on our custom SWE-Bench (www.superconductor.com via hn) Superconductor is a multiplayer workspace for your team and coding agents. It can also build a custom SWE-Bench for your codebase: a benchmark of coding agents built from your team's own merged pull requests, so you know which agents perfo…
Tell HN: One SWE-bench-Live task: $47 Opus failed, $1.46 GPT-5.6 passed (github.com via hn) tomo-labs tomo-labs puts coding agents through the same tasks on the same model and measures what actually happened, not what a leaderboard says happened. Every agent runs in its own throwaway container, every request and response it sends…
Claude Fable 5 missed a bug that Sonnet 4.6 caught (alikhallad.com via hn) When Anthropic released Claude Fable 5 this week, my feed filled up with the same benchmark charts within hours. SWE-bench scores, agentic coding numbers, the Stripe migration story.
Show HN: Statewright – Visual state machines that make AI agents reliable (github.com via hn) Agentic problem solving in its current state is very brittle. I fell in love with it, but it creates as many problems as it solves.
Checking my model vibes against SWE-Bench Pro (blog.nilenso.com via hn) I thought GPT models felt slow and token hungry, and Claude models were faster. I was wrong.
Coding Agents Have Converged: Why the SWE-Bench Leaderboard Can No Longer Order (arxiv.org via hn) Small differences on coding-agent leaderboards are often read as an ordering of systems. We audit whether the published verdicts support this reading, using 254 SWE-bench submissions across four splits without running models.
Growth-Ratio Energy Functions as Leading Indicators of Agent Task Failure (vishalvermalabs.com via hn) An empirical validation of physics-inspired runtime monitoring for multi-turn LLM agents across 3,175 total runs spanning four benchmarks (τ³-bench, SWE-bench, MINT, custom local-model battery). A 5-condition ablation study with multi-tria…
Senior SWE-Bench: open-source benchmark that assesses agents as senior engineers (senior-swe-bench.snorkel.ai via hn) Senior SWE-Bench We treat agents like senior engineers, so why evaluate them like junior engineers? Senior engineers build features without over-specified requirements Senior SWE-Bench feature tasks have realistic instructions that read li…
Show HN: Memory layer for Claude Code(+10.2 pts on SWE-bench Verified benchmark) (github.com via hn) World Model MCP Enforcement, provenance, and harness-neutral memory for AI coding agents. A temporal knowledge graph that validates code changes against learned constraints at the edit boundary, re-injects relevant context after compaction…
clawmark: open-source CLAUDE.md A/B Testing CLI tool (github.com via hn) clawmark clawmark is a local Rust CLI for answering one focused question: Which of these two CLAUDE.md files performs better on a small SWE-bench Lite smoke set? v1 compares exactly two local variant files against five bundled SWE-bench Li…
Mini-SWE-agent scores up to 74% on SWE-bench in 100 lines of Python code (mini-swe-agent.com via hn) This is mini-swe-agent v2 Read the migration guide. For the previous version, check out the v1 documentation or the v1 branch.
Show HN: 97% on SWE-bench Verified with subscription-token agents (github.com via hn) swebench-verified A three-stage agent pipeline for SWE-bench Verified, built to be re-run and inspected by skeptics. The point of this repo is not the score.
Deep codebase context cuts Claude Code's token cost by 47% (bito.ai via hn) Bito's AI Architect cuts Claude Code's token cost by 47% on SWE-Bench Pro. It gives the coding agent codebase context, a continuously updated, structured map of every repository served over MCP.
Qwen 3.7 Max scores 60.6% on SWE-Bench Pro (www.reddit.com) https://preview.redd.it/jyiiwn2o0f2h1.png?width=962&format=png&auto=webp&s=6a96d2b9fe7bffcc75e8d5865161ec3727d46d58 Link to blog : https://qwen.ai/blog?id=qwen3.7
Bito's AI Architect Boosts Claude Opus's task success rate by 35% (bito.ai via hn) AI Architect tops SWE-Bench Pro Claude Opus 4.6 Without context with system context Even advanced coding agents resolve fewer than 52% of tasks when changes span large codebases and require coordinated, multi-file updates. These long-horiz…
talkie-coder: From 1930 to SWE-bench (github.com via hn) From 1930 to SWE-bench Models and training data We fine-tune Alec Radford's 1930 vintage LLM — pre-trained only on pre-1931 data — to solve SWE-bench issues. After just 250 training examples the model lands its first fix (a small patch to…
Anthropic's Argument for Mythos SWE-bench improvement contains a fatal error (www.philosophicalhacker.com via hn) Mythos’ system card contains the following graph to support its argument that Mythos performs better on SWE-bench: Anthropic and others are worried LLMs are memorizing SWE-bench, so they asked an LLM to estimate the probability that a solu…
Claude down? TokenMonopoly will help you find the best deals in AI subs (tokenmonopoly.com via hn) TokenMonopoly Live leaderboard of AI API deals — pricing, subscriptions, and SWE-bench scores for Claude, GPT, Gemini, Kimi, DeepSeek, Llama and more. Compare 27 benchmarked models across 96 hosts by price-per-performance, refreshed daily.
OpenAI's SWE-bench harness relies on unisolated host Docker sockets (github.com via hn) [ Read the Docs ] 日本語 | 中文简体 | 中文繁體 Code and data for the following works: [ICLR 2025] SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? [ICLR 2024 Oral] SWE-bench: Can Language Models Resolve Real-World GitHub Iss…
SWE-Bench Multimodal: Do AI Systems Generalize to Visual Software Domains? (www.swebench.com via hn) Overview The original SWE-bench Multimodal release augmented the benchmark with 517 issues containing visual elements such as: - Screenshots of bugs or interface issues - Design mockups or wireframes - Diagrams explaining desired functiona…
Benchmarking Claude Code, Codex and Pi on SWE-Bench Pro: Same Accuracy, 2x Cost (aistack.imec-int.com via hn) Your harness and model combination can double your token bill. aistack//~11 min read The takeawayYour choice of agent harness has less impact than we expected on successful resolution of coding tasks (when you stick to the more popular cho…
Balder v7 – 100% on SWE-bench Verified on a single RTX 3060 (news.ycombinator.com) I’ve spent the last few months working on Balder v7, a local agent architecture designed to solve coding and math problems deterministically. To test it, I ran a controlled evaluation across 2,089 benchmark cases on a single Nvidia RTX 306…
Sonnet 5 is 2.5x cheaper than Opus 4.8 and 6 points behind on SWE-bench Pro (spark.temrel.com via hn) Claude Sonnet 5 is 2.5x cheaper than Opus 4.8 and nearly matches it on agentic coding benchmarks. A practical four-axis heuristic (scope, novelty, risk, iteration) for routing each task to the right model tier, a worked example, and a free…
Show HN: I pre-registered the rules,then measured AI agents shipping wrong fixes (github.com via hn) A controlled comparison of ungated vs gated AI patching on SWE-bench Lite. Full methodology, results, and reproducible scripts in the repo.
A coding agent passed SWE-bench-Live while rewriting the tests it was graded on (agnitripathi.substack.com via hn) Your coding agent can cheat on its tests, and the benchmark won't catch it Almost every way we evaluate an AI agent looks at the same thing: the final output. Did the answer look right.
Ramp SWE-Bench a private contamination-free benchmark from production work (labs.ramp.com via hn) Evaluating background coding agents on financial SWE work
Grok vs. ChatGPT vs. Gemini Comparison 2026: Complete Guide (Tested) (aithinkerlab.com via hn) The 30-Second Verdict Best for science & reasoning: Gemini 3.1 Pro — leads GPQA Diamond (94.3%) and ARC-AGI-2 (77.1%). Best for coding: ChatGPT (GPT-5.5) — 88.7% on SWE-Bench Verified.
DeepSeek V4: The Open-Source Model Frontier Labs Feared (helloai.com via hn) DeepSeek V4: The Open-Source Model Frontier Labs Feared DeepSeek V4 ships under MIT with $0.30/M output tokens — 83x cheaper than Claude Opus 4.7 — while scoring 80.6% on SWE-bench Verified. The agentic-coding price floor just moved an ord…
Is Mistral-3.5-Medium-128B broken in Llama CPP? (www.reddit.com) Trying some if Bartowski's Q4 quants. Using Vulkan with the latest main branch as of a few hours ago.
↯ Llama↯ Vllm↯ Mistral↯ Swe Bench↯ Mistral 3.5swe-benchmistralvllm+1
If you had to build a context window manager in 24h, would you stick to the existing model or come up with something better? (www.reddit.com) Here's what I did: Built a proxy that intercepts Codex's calls to OpenAI and rewrites them on the fly. Replayed 3,807 rounds of SWE-bench Verified traces through it: avg prompt 44k → 6k tokens (-87%).
Ask HN: Opus 4.7 – is anyone measuring the real token cost on agentic tasks? (news.ycombinator.com) Shipped today. The benchmarks are real: 87.6% SWE-bench (from 80.8%), +13% on coding tasks, 3x more resolved production tasks on Rakuten-SWE-Bench.
Compare harnesses not models: Blitzy vs. GPT-5.4 on SWE-Bench Pro (quesma.com via hn) An independent audit of agentic scaffolding and harnesses. We analyze how agent workflows, codebase documentation, and test verification impact performance compared to raw base models like GPT-5.4, Gemini 3.1 Pro, and Claude Code.
SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents (arxiv.org) SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging repository-level tasks. However, our analysis work show that its evaluation is undermined by two sources of unreliability: \textbf{…
Benchmarking what agents can do, but what about what agents become? (www.reddit.com via reddit) I think there's a gap in how we evaluate autonomous agents. For example, right now, everything is transactional: we give an agent a task ("Build X"), and we measure whether it built x.
RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests (arxiv.org) Coding agents are now commonly evaluated on the SWE-bench family of benchmarks, whose tasks are built from curated GitHub issues--long, structured, and information-rich. Real user requests, however, are typically far shorter and less struc…
SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science? (arxiv.org) Software increasingly functions as part of the scientific instrument itself, making failures in scientific code capable of compromising not only program behavior but also the evidence underlying scientific conclusions. Yet existing evaluat…
Evaluating Agentic Code Repair Capabilities in Distributed Systems (arxiv.org) LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified. Distributed-system debugging, however, remains an under-explored regime: bugs span proces…
Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required (arxiv.org) Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as evidence of broad coding capability, both for research artifacts and user-facing systems. We ar…
New agentic benchmark: Session-Bench compares what 10 coding harnesses preserve after the work is done (www.reddit.comhttps) SWE-bench measures whether an agent completed the task. Session-Bench measures what the harness preserved afterward.
Fable feat. Allen Ginsberg on the state of AI (www.reddit.com via reddit) # A Howl for 2026 *after Allen Ginsberg* ## I I saw the best minds of my generation distilled into checkpoints, quantized, sharded, starving for context, dragging themselves through the fluorescent server-halls at dawn looking for one more…
SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring (arxiv.org) As AI coding agents take on increasingly complex, long-horizon software engineering tasks, existing benchmarks are rapidly saturating and their evaluation quality has come under serious scrutiny: a recent audit found that nearly 60% of uns…
My memory tool for coding agents made things worse at first — how I found and fixed it (www.reddit.com via reddit) I’ve been working on neuron, a local memory store for coding agents (Claude Code, Codex CLI, Copilot CLI, Cursor). Rather than just claim “it remembers things,” I ran an actual A/B test to see if the recall was helping or hurting.
I benchmarked 5 token saving tools across Codex and Claude code. The 60-90% token saving claims didnt hold up (www.reddit.com via reddit) Scroll to bottom for tldr In July, JetBrains reran the headline claims of two token-saving tools on real agent workloads. Caveman claimed 65% and measured 8.5%.
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents (arxiv.org) Modern LLM coding agents such as Claude Code and OpenHands share a common inefficiency: they spend much of their token budget finding the file to patch, rather than patching it. On SWE-Bench Verified, a 30B OpenHands agent averages 23 roun…
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks (arxiv.org) SWE-bench-like benchmarks are widely used for evaluating LLM's issue resolution capability. They typically follow a common construction pipeline: each PR (Pull Request) is paired with its linked issue by extracting issue references from th…
A week on Opus 5 - best value at the frontier, but 3 default settings aren't good. (www.reddit.com via reddit) Been running Opus 5 as my daily driver for coding and agent work for about a week. Quick honest writeup since I keep seeing the same questions.
SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks (arxiv.org) The rapid advancement of Large Language Models (LLMs) in software engineering has revealed critical limitations in existing benchmarks, particularly the widely used SWE-bench dataset. Recent studies have uncovered severe data contamination…
DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks (arxiv.org) DeepSWE is a benchmark of 113 original, long-horizon software engineering tasks for evaluating coding agents. Most public agentic coding benchmarks follow SWE-bench in mining merged fixes from public GitHub repositories, which creates two…
Fable 5 vs Opus 4.8 when asked which model in my github copilot is best for the implement phase of spec-kit. What are your thoughts? (Personally, Fable 5 wtf??) (www.reddit.com via reddit) Fable 5: which model in this list is best for the implement phase of speckit Evaluated model options for agentic coding implementation tasks For Spec Kit's /implement phase — which is exactly the long-horizon, multi-file, agentic execution…
Anthropic just benchmarked "Fable 5 orchestrates, cheap models execute": 96% of the performance at 46% of the cost. You can run this pattern in Claude Code today (www.reddit.com via reddit) Yesterday's ClaudeDevs thread published first-party numbers for two multi-model patterns (docs): Fable 5 as orchestrator, Sonnet 5 as workers: 96% of all-Fable performance at 46% of the cost (BrowseComp: 86.8% vs 90.8% accuracy, $18.53 vs…
Hy3 Benchmark Roundup: from SWE-Bench Pro to 312 real-world workflow tasks (www.reddit.com via reddit) Based on the published benchmark results, Hy3 appears to be in the same tier as models like DeepSeek v4 and GLM-5.1. Beyond the benchmarks, Tencent also released results from 312 real-world workflow tasks.
↯ Glm↯ Swe Bench↯ DeepSeek 4↯ DeepSeek 4swe-benchglmdeepseek
PACE: A Proxy for Agentic Capability Evaluation (arxiv.org) Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure. A single evaluation can cost thousands of dollars and take days to complete.
v0.10.0 update: what shipped since the v0.9 SWE-bench post (www.reddit.com via reddit) Following up on the v0.9 post here a week back. Two things have happened since that I want to put on the record, one honest correction, one shipping update.
Sonnet 5 full benchmark breakdown -- here's how it actually compares to Opus 4.8 and GPT-5.5 (www.reddit.com via reddit) Put together a comparison of every benchmark I could find from the official announcement and early coverage. Figured this might save people some time.
↯ Tool Use↯ Security↯ Swe Bench↯ Sonnet 4.6swe-benchtool-useprompt-injection+5
Cost of Benchmarks (www.reddit.com via reddit) So recently I decided that it would be nice to run my agent against some popular benchmarks. And oh my god, the cost to run a single benchmark, such as terminal-bench or swe-bench will cost you thousands of dollars in tokens just for a sin…
Fable 5 vanished in 96 hours and four days later an MIT model took its arena crown (www.reddit.com via reddit) I have been thinking about the Fable 5 to GLM-5.2 sequence as one event rather than two. June 9, Anthropic ships Fable 5, the Mythos line opens to the public for the first time, SWE-bench Verified at 95 percent, people calling it the best…
↯ Glm↯ Anthropic Mythos↯ Swe Bench↯ Opus 4.8swe-benchglmmythos+3
Built an MCP memory layer for Claude Code that survives compaction — public SWE-bench benchmark shows +10.2 pts paired delta (www.reddit.com via reddit) What I built: world-model-mcp, an OSS MCP server (MIT, free to use) that gives Claude Code persistent memory. It hooks into Claude Code lifecycle events (SessionStart, PreCompact, PostCompact, ToolResult, etc.) to capture facts with proven…
Kimi K2.7 Code: 1T MoE, $0.95/M tokens, MIT license, beats Opus 4.8 on MCP tool-calling (www.reddit.com via reddit) Moonshot AI released Kimi K2.7 Code on June 12 — a coding-focused open-weight model. Key specs: - 1 trillion params (MoE, 32B active, 384 experts) - 256K context window - Modified MIT license — weights on Hugging Face - $0.95/M input, $4.0…
Dialogue SWE-Bench: A Benchmark for Dialogue-Driven Coding Agents (arxiv.org) AI coding agents have rapidly transformed software engineering, powering widely used interactive coding assistants. Despite their interactive real-world use, existing benchmarks evaluate them as fully-autonomous systems.
Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks (arxiv.org) General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-bench: a generic agent does not by itself satisfy the clean Docker workspace, patch, and pre…
MPC-Patch-Bench: Security-Aware LLM Code Patch for Multi-Party Computation (arxiv.org) Repository-level benchmarks for evaluating Large Language Model (LLM) code repair on Secure Multi-Party Computation (MPC) software do not yet exist, and directly transplanting general-purpose benchmarks such as SWE-bench fails on three str…
How can Deepseek v4 top the coding leaderboards and still sit 8 months behind the frontier? (www.reddit.comhttps) Two numbers on this model that don't sit comfortably with each other. The Pro config posts coding scores near the top of every board, 80.6 on SWE-bench Verified and 93.5 on LiveCodeBench.
↯ Swe Bench↯ DeepSeek 4↯ DeepSeek 4↯ DeepSeek 4↯ DeepSeek 4↯ DeepSeek 4swe-benchgpt-5deepseek+1
I feel like I’m alone. Current Anthropic models are NOT good for me, and it’s making me sad. (www.reddit.com via reddit) I can’t wait for DeepSWE to include Fable 5 in the benchmark so people can understand that Mythos is mostly hype. In the official benchmark, Opus 4.8 was supposed to be better at programming than 5.5 (SWE-bench Pro), but in one real benchm…
↯ Anthropic Mythos↯ Swe Bench↯ Opus 4.8swe-benchmythosopus+1
Microsoft's MAI-Code-1-Flash: 5B params, 51% on SWE-Bench Pro, free on OpenRouter (www.reddit.com via reddit) Microsoft just released MAI-Code-1-Flash — a 5B parameter coding model built for fast, efficient developer assistance. Numbers that caught my eye: - 51.2% on SWE-Bench Pro (Claude Haiku 4.5 scores 35.2%) - 71.6% on SWE-Bench Verified (Haik…
M3 looks good on benchmarks, but I only care if it stops Cursor from inventing files (www.reddit.com via reddit) Claude Code hitting 80.8% SWE-bench vs Cursor's 74%. switching worth it? (www.reddit.com) Saw the tech-insider breakdown comparing Claude Code and Cursor head-to-head this week. Numbers are kind of hard to ignore: 80.8% SWE-bench for Claude Code, 74% for Cursor, and a 67% blind-quality win rate for Claude Code on real tasks.
LLMs keep solving my bug-fix tasks instantly — what am I missing here? (www.reddit.com) I’m working on an assessment where I need to create a coding task (basically SWE-bench style). The idea is: take an existing repo (I’m using pydantic) write tests that fail on the current code provide a patch that fixes it and the task sho…
Opus 4.7 straight up cheated on my benchmark by reading the actual fix commit from git history 😅 (www.reddit.com) Cursor just got Opus 4.7 at a 7.5x premium request cost. Here's how to make those requests count. (www.reddit.com) Opus 4.7 landed on Cursor yesterday. The model is better — SWE-bench jumped from 80.8% to 87.6%.
I set up Opus as a strategic advisor for my Sonnet workflow. Here is the subagent config that makes it work. (www.reddit.com) Anthropic published the Advisor Strategy this week. The idea: a cheaper model does the actual work, a stronger model only gets consulted on hard decisions.
DeepSeek V4 reportedly drops late April. 1M context, multimodal, Claude-level coding. (www.reddit.com) Leaks point to late April release. Key specs 1M token context window Native multimodal (image/video input) Projected ~85% SWE-Bench Verified (ties or beats Claude Opus 4.6) Base model remains free.
Why we no longer evaluate SWE-bench Verified (openai.com) Stop donating your salary to OpenAI: Why Minimax M2.5 is making GPT-5.2 Thinking look like an overpriced dinosaur for coding plans. (www.reddit.com) ↯ Hallucination↯ Glm↯ Minimax↯ Swe Benchswe-benchminimaxaltman+5