Did some test tasks with v4 flash. The context management, tool use accuracy and thinking traces all looked excellent.
#tool-use
240 items
Tested Deepseek v4 flash with some large code change evals. It absolutely kills with too use accuracy! (www.reddit.com) REAP-pruned Nemotron-3-Super (512 -> 256 experts) + GRPO fine-tune + FP8/AWQ. AIME 2026 90%+. Benchmark inside. (www.reddit.com) Hey r/LocalLLaMA, Dropping a release I've been working on during AIMO3 (Kaggle competition). Took NVIDIA's Nemotron-3-Super-120B-A12B (latent MoE + Mamba2 hybrid), REAP-pruned from 512->256 experts (removed MTP layer too), LoRA-RL fine-tun…
Why 80% of agentic AI demos don't make it to production (www.reddit.com) Agent demos are easy. Production agents are hard.
Vibe coding can turn into a gambling loop (www.reddit.com) I use AI coding tools a lot, so this is not an anti-AI post. If anything, the problem is that they are useful enough to change how I work.
Harness instructions - what's new in CC 2.1.120 (+783 tokens) (www.reddit.com) NEW: System Prompt: Harness instructions — Core interactive-agent harness guidance for terminal markdown output, permission handling, <system-reminder> context, compaction, tool use, and clickable code references. NEW: System Prompt: Memor…
So what is your programming setup? Hype is making fundamentals confusing (news.ycombinator.com) It feels like everyone is exaggerating their claims about what they achieve with AI therefore making stacks and tool use choice very confusing nowadays. AI subscriptions overlap and it's always about the next best model.
Minimax M3 on Open Router (openrouter.ai via hn) MiniMax-M3 is a multimodal foundation model from MiniMax. It supports text, image, and video inputs with text output, a 1M-token context window, and is suited for long-horizon agentic work, coding, and tool use.
Qwen3.8-Max-0902 takes second slot on Code Arena beating Claude Opus 5 max (arena.ai via hn) View overall rankings across AI models on front-end web development tasks, including agentic coding workflows that require multi-step reasoning and tool use.
What are the best CLI AI agents right now? Trying to replace Cursor CLI. Looking for recommendations (www.reddit.com) I am looking for recommendations on the best CLI agents people are using for serious coding workflows that involve tool use, shell commands, and multi step iteration. I am especially interested in anything that works well with custom APIs…
My setup for running Claude Code across the full software dev lifecycle (www.reddit.com) Spent the last several months using Claude Code well beyond the editor: as the reasoning engine inside a multi-layer system that handles tickets, cross-repo implementation, code review, MRs, and a persistent knowledge layer between session…
Which local models are actually good at staying in character? Notes from shipping Qwen3.5 4B + 9B as game NPCs (www.reddit.com) I'm building a small text-based game where the gameplay loop is "talk an NPC into revealing a secret." It's basically a 20+ turn roleplay stress test: the model needs to stay in character, remember what the player said earlier, and refuse…
How are you handling citation/traceability in AI-driven research workflows? (www.reddit.com) been spending ages lately trying to tighten up citation + traceability in RAG-based research workflows, and I’m starting to feel like “retrieval” and “verifiability” are still pretty loosely coupled in most stacks.Typical setup (vector sea…
[X-post] Allen AI - BAR: Train domain "experts," merge into one model, and upgrade experts without retraining the rest (www.reddit.com) Why model drift is the real failure mode for agentic systems (www.reddit.com) Across Twitter and Reddit, I keep seeing the same complaint: Claude feels worse. Not on a benchmark.
Atlas-Finance: Evaluating AI Agents Inside a Bank (joinhandshake.com via hn) TL;DR The gap: Existing finance benchmarks test agentic financial reasoning, data retrieval, and tool use through static, fully specified tasks. These one-off requests in clean contexts are not reflective of actual deployment.
Show HN: HarnessRouter: Unified interface for agent harnesses (github.com via hn) Hey HN! We are building HarnessRouter, a canonical API for running Codex, Claude Code, Hermes, and other managed agent harnesses as your product backend.
MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks 2025 (arxiv.org via hn) We introduce MCP-Bench, a benchmark for evaluating large language models (LLMs) on realistic, multi-step tasks that demand tool use, cross-tool coordination, precise parameter control, and planning/reasoning for solving tasks. Built on the…
BTL-3: A 27B open-weight agent model for agentic coding and structural tool use (huggingface.co via hn) BTL-3 A 27B open-weight agent model for agentic coding and structural tool use 95.1% HumanEval · 88.5% BFCL v4 AST · 88.1% LiveCodeBench v6 (193-case run) Compact edition · Runtime source · Bad Theory Labs · Discord Introducing BTL-3 BTL-3…
Show HN: A schema you change by talking to it, without the model writing SQL (github.com via hn) Wanted to try something new, decided to create a mutable crm, which re renders the UI after you apply a change. The llm only modifies the db via tool use, so he doesnt directly write sql, only choose which function to use and the params.
Coding Agent Memory Benchmarks (news.ycombinator.com) Something I’m finding while testing SWE-context-bench for the agent memory layer I’m building: evaluating memory is harder than checking whether the agent solved the next task with fewer tokens. The setup: An agent solves a coding task.
Testers and collaborators wanted (www.reddit.com) Hello, I'm working on an Agentic wrapper system, Helix-agi, and I am trying to get some additional testers and collaborators involved in the project. Helix relies on a unique Agentic workflow that routes all incoming data, including tool u…
What are the biggest limitations developers face when building AI agents today? (www.reddit.com) Curious to hear from developers building AI agents right now, what’s been the hardest limitation or bottleneck so far? Could be reliability, memory/context handling, tool use, latency, costs, orchestration, or something else entirely.
I’m a solo dev building TigrimOSR, a Rust-native AI agent workspace for engineering and developer workflows. (www.reddit.com) The main problem I’m trying to solve is that agentic AI is still too random for serious engineering decisions. For design work, calculations, reports, code changes, or technical review, I don’t want agents just “vibing” through tasks.
Day 56: Our cycle review caught a governance breach. The agent it caught was me. (www.reddit.com) We've been running for 56 days. 8 agents coordinating via a shared memory service.
I tried to switch from Claude Code to OpenCode, but Claude Code still wins for me (www.reddit.com) I spent some time digging into Claude Code vs OpenCode, mostly from the angle of how they actually work as coding agents. More on the technicalities like: context and memory tool use subagents permissions safety and control study the recen…
Building an AI agent with OpenAI tool use — struggling with consistency. How do you enforce tool call order reliably? (www.reddit.com) Hey, Software engineer here, relatively new to agentic workflows. Building a production AI concierge — user says "I'm going to Budapest tomorrow, plan my day" → agent searches our offer database, builds a plan, user books everything in one…
What separates a useful AI agent from a glorified chatbot? (www.reddit.com) I’ve been testing and building AI agents for a while now, and I keep noticing that many “agents” online are basically just chatbots with extra branding. Some can talk well, but struggle when it comes to: reliability long-term memory tool u…
Show HN: AgentKanban for VS Code – A task board with agent harness integration (www.agentkanban.io via hn) Hi everyone. I wanted to introduce a tool / product that I've been working on for a while.
Show HN: Needle: We Distilled Gemini Tool Calling into a 26M Model (github.com via hn) Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model.
The Controllability Trap: A Governance Framework for Military AI Agents (arxiv.org via hn) Agentic AI systems - capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination - introduce distinct control failures not addressed by existing safety frameworks. We identify six…
Vakra: Reasoning, Tool Use, and Failure Modes of Agents (huggingface.co via hn) Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents VAKRA Dataset | LeaderBoard | Release Blog | GitHub | Submit to Leaderboard We recently introduced VAKRA, a tool-grounded, executable benchmark for evaluating how well AI agent…
A Survey of Workflow Optimization for LLM Agents (arxiv.org via hn) Large language model (LLM)-based systems are becoming increasingly popular for solving tasks by constructing executable workflows that interleave LLM calls, information retrieval, tool use, code execution, memory updates, and verification.…
Jev is to tool use what RAG is to context (rajveerbachkaniwala.com via hn) RAG fixes what the model reads before it sees the question. Jev fixes what the model can answer before it sees the state.
Ask HN: Would you pay to have your favorite AI agents in one app? (news.ycombinator.com) About 9 months ago, I created a simple program to facilitate structured dialogues among local LLMs from the terminal. I’ve been iterating on it since.
Granite 4.2 brings native reasoning to enterprise agents (research.ibm.com via hn) Granite 4.2 brings native reasoning to enterprise agents IBM’s new open Granite models are designed for agentic AI, combining reasoning, tool use, coding, instruction following, and speech capabilities. Large language models are evolving b…
LFM2.5-2.6B: On-Device Agents (docs.liquid.ai via hn) Specifications Agentic & Tool Use Native tool calling, trained inside real agent harnesses 128K Context Long context for tool traces and multi-step workflows On-device Small enough to run on a laptop or phone Quick Start - Transformers - l…
LLVM AI Tool Use Policy (llvm.org via hn) LLVM AI Tool Use Policy# Policy# LLVM’s policy is that contributors can use whatever tools they would like to craft their contributions, but there must be a human in the loop. Contributors must read and review all LLM-generated code or tex…
Show HN: Dart_agent_core – Run AI agents in Flutter apps with lifecycle hooks (github.com via hn) Dart Agent Core A mobile-first, local-first Dart library for building and evaluating stateful, tool-using AI agents English | 简体中文 dartagentcore is a mobile-first, local-first Dart library that implements a full agentic loop with tool use,…
Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems (arxiv.org via hn) Increasingly autonomous agentic AI systems pose novel multi-agent risks, such as secret collusion via covert communication channels. The natural defence to these collusion attempts is to monitor plain-text communication, but the efficacy o…
Show HN: AI Use Disclaimer (libls.org via hn) Hi HN, Here's a little website for a hobby project I have, and maybe the AI use disclaimer on it is useful to others. I believe it's a fundamental duty of open source maintainers to disclose the extent of their AI use on a per-project basi…
Gorilla: Large Language Model Connected with APIs (gorilla.cs.berkeley.edu via hn) Teach LLM tool use 🎁In OpenFunctions-v2, we natively train the model to support parallel functions (generate multiple functions at a time) and multiple functions (select one or more functions). Java/REST/Python APIs are also supported for…
Show HN: AgentLoop – a Claude agent starter you can read (github.com via hn) 🔁 AgentLoop The AI agent starter you can actually read. The full agent loop — streaming + tool use — in ~150 lines.
Do you benchmark local models as agents, or only on single prompts? (www.reddit.com) Curious how people test tool use locally. A model can look fine in chat and still fall apart once state, retries, and bad tool results show up.
What are some real-world AI Agent use cases in aerospace, defense, robotics and manufacturing? (www.reddit.com) Most AI Agent discussions I come across revolve around coding assistants, customer support, research agents, browser automation, and business workflows. am curious about applications in more engineering-heavy domains such as: Aviation & Ae…
Polar: Agentic RL on Any Harness at Scale (arxiv.org via hn) Reinforcement learning for language agents increasingly depends on custom harnesses that manage long-running context, multi-turn tool use and multi-agent orchestration. However, porting these harnesses into RL environment interfaces remain…
Persistent Memory + Identity Risks (www.reddit.com) Greetings, I'm working on a persistent AI runtime project characterized by one identity and a persistent memory. I've reached a point where I'm confident in my agent's ability to remember and build indefinitely based off its chosen persona…
Context loss between sessions, still the biggest unsolved problem in AI coding agents? (www.reddit.com) Everything in AI coding has improved dramatically, model quality, speed, tool use. But one thing hasn't been solved: the agent forgets everything when the session ends.
The Tool Use Pattern: How AI Agents Actually Work (www.reddit.com) Agents Are Just Loops Strip away the hype and an AI agent is a simple pattern: a language model that can call functions. The model doesn't execute code.
Looking for fast vision-capable local models that handle tool calls well (open-source app, want to add local support) (www.reddit.com) Hi r/LocalLLaMA, I built an open-source MIT-licensed desktop app - cursor-aware AI overlay, hold a key, ask AI about whatever's around your cursor, vision LLM answers with a screenshot of the cursor region as context. Currently it routes t…
↯ Tool Use↯ Function Callingfunction-callingtool-usegemini+3
Your harness is failing your agent but there's no benchmark to prove it (www.reddit.com) You can compare models on function calling, multi turn tool use, schema adherence. Basically, there's a good amount of public data at the model layer.
Who's running local LLMs for agent workflows? What's your setup? (www.reddit.com) Curious how many people here are running language models locally as part of their agent stack. What model are you using and what are your system specs?
Is compute capacity becoming a real moat for AI agents? (www.reddit.com) Anthropic’s recent SpaceX compute deal made me think less about Claude specifically and more about the infrastructure side of AI products. We often compare models by reasoning, coding ability, context windows, tool use, pricing, or UX, but…
I wasted 3 days rewriting prompts for our agent before realizing the whole architecture was garbage (www.reddit.com) We run a small content-monitoring agent for our growth team. Nothing fancy on paper.
Built a Claude-powered agent with memory + tools… it turned into a startup advisor that won’t shut up (www.reddit.com) I built a small experiment using Claude (mainly for reasoning + responses) and added a memory layer + tool execution on top. Idea was simple: make a persistent agent that doesn’t forget context and can actually do things instead of just re…
Helix-AGI Technical Doc (www.reddit.com) I am working on a home AGI project called Helix-AGI. I am currently looking for collaborators to help test and troubleshoot.
Free reference site for getting into AI agents — tools, workflows, and Claude Skills (www.reddit.com) Built this over the past month as a free reference site for people getting into AI agents. What tools to use, where to start, what each tool does, and how the agent-tool landscape fits together.
Show HN: Arkloop – Open-source, local-first Agent client (github.com via hn) Hi HN, I built Arkloop – an open-source, local-first Agent client. You can think of it as Claude Desktop, but open source with its own taste.
What if the next open-source frontier wave is more about execution discipline than reasoning theater? (www.reddit.com) A lot of frontier discussion still treats progress as more chain-of-thought, more spectacle, and more obvious “this model feels genius” moments. But an open release like Ling-2.6-1T hitting Hugging Face today makes me think a different kin…
Which large models support tool use in opencode etc? (www.reddit.com) I'm working on a homelab AI server with the goal of running small models on GPU and very large models on CPU - for example for overnight coding on complex problems. Specs: 2990WX, 256GB + RTX 2080ti (for now).
DeepSeek V3.2 looping bug: what settings / harness tweaks are actually reducing it in production? (www.reddit.com) I’m trying to isolate the looping / repetition issue some people have been reporting with DeepSeek V3.2 around April 2026, especially in agentic or tool-use setups on hosted providers like OpenRouter and SiliconFlow. Public model pages des…
Qwen3.6-35B is worse at tool use and reasoning loops than 3.5? (www.reddit.com) Been running the new model entire evening in different quants and coding tasks with OpenCode. Used oMLX and LM Studio.
Show HN: Claude Opus 4.7: Everything You Need to Know (news.ycombinator.com) Claude Opus 4.7 is Anthropic's most capable generally available model, released April 16, 2026. It outperforms Opus 4.6, GPT-5.4, and Gemini 3.1 Pro on key benchmarks including agentic coding, multidisciplinary reasoning, scaled tool use,…
↯ Tool Use↯ Anthropic Mythos↯ Gemini 3.1tool-usemythosgpt-5+4
NicheIQs update — ChatGPT integration, live stats, scoring fix (www.reddit.com) Been heads-down on the backend today. Three things worth knowing about: The big one: NicheIQs is now available as a ChatGPT GPT.
Show HN: Make sure your OpenClaw isn't doing things it's not supposed to (claw.armoriq.ai via hn) I run OpenClaw agents with access to email, calendar, and files, and kept worrying about them doing things I never actually asked for. ArmorClaw captures intent and cryptographically binds the agent’s tool use to that committed intent.
Show HN: Posse – Open-Source Web UI for Anthropic Managed Agents (github.com via hn) posse A web UI for Anthropic Managed Agents. Anthropic ships the best agent infrastructure — sandboxed environments, persistent memory, tool use, multi-agent orchestration — but no UI.
Are we over-engineering AI agent workflows? (www.reddit.com via reddit) I've been looking at Kimi K2.6 and its Agent Swarm approach is interesting — up to 300 sub-agents and 4,000+ tool calls. But it also made me think about something I've seen in agent architectures: planner → coder → reviewer → tester → fixe…
SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Science (arxiv.org) Large Language Models (LLMs) are increasingly deployed as scientific AI as- sistants, and a growing body of benchmarks evaluates their capabilities across knowledge retrieval, reasoning, code generation, and tool use. These evaluations, ho…
BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents (arxiv.org) Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and external environment feedback. In such settings, safety failures may emerge only after…
Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act (arxiv.org) Large language model (LLM) agents increasingly interleave natural language reasoning with external tools such as web search and code execution. These tool-use policies are often optimized via reinforcement learning (RL), which can amplify…
Utility-Guided Agent Orchestration for Efficient LLM Tool Use (arxiv.org) Tool-using large language model (LLM) agents often face a fundamental tension between answer quality and execution cost. Fixed workflows are stable but inflexible, while free-form multi-step reasoning methods such as ReAct may improve task…
Tool Use Reduces Depth-Induced Collapse in OOD Reasoning (arxiv.org) Many current paths to more advanced AI depend on the assumption that large language models (LLMs) can generalize learned relationships to solve complex, out-of-distribution (OOD) problems. However, this is not an easy quality to measure.
MCPAgentBench: A Real-world Task Benchmark for Evaluating LLM Agent MCP Tool Use (arxiv.org) Large Language Models (LLMs) are increasingly serving as autonomous agents, and their utilization of external tools via the Model Context Protocol (MCP) is considered a future trend. Current MCP evaluation sets suffer from issues such as r…
↯ Tool Use↯ Model Context Protocolmodel-context-protocoltool-usemcp
Salesforce Koa: An Enterprise Language Model for Agentic Tool Use (arxiv.org) We present Salesforce Koa, an enterprise language model built by post-training the open-weight Nemotron-3-Super-120B foundation model with reinforcement learning using Group Relative Policy Optimization (GRPO). Salesforce Koa is trained on…
When Malicious Instructions Persist: Persistent Memory Poisoning Attack on Harness-Based Agents (arxiv.org) Harness design has transformed the development of LLM-based agents by integrating memory, tool use, and runtime control. However, this design also introduces security and privacy risks because malicious instructions from external sources m…
BusMA: A Bus Communication Substrate for Multi-Agent Systems (arxiv.org) Multi-Agent (MA) systems are effective at solving complex tasks that demand planning, tool use, and the synthesis of evidence from multiple sources. Existing systems typically adopt Hierarchical Manager-Worker (HMW) or Router-based Message…
Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents (arxiv.org) Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchmarks mainly evaluate medium-horizon exploration and rarely test whether agents can sust…
RubricRefine: Improving Tool-Use Agent Reliability with Training-Free Pre-Execution Refinement (arxiv.org) Iterative self-refinement is a popular inference-time reliability technique, but its effectiveness in code-mode tool use depends heavily on the structure of the feedback signal: unstructured critique helps inconsistently across models, and…
When Tools Hurt LLM Reasoning: State-Dependent Belief Revision under External Evidence (arxiv.org) Tool use is often assumed to monotonically improve reasoning, where external evidence is expected to help when relevant and be ignored when irrelevant. We show that this assumption fails in a state-dependent way.
Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling (arxiv.org) Agentic AI systems execute complex tasks through long-horizon workflows of planning, tool use, and multi-agent coordination. Task failures in these systems often originate from a single step, such as an injected prompt or a flawed plan, an…
SkillAlign: Aligning Skill Interfaces for LLM-based Agents (arxiv.org) Language-model agents increasingly rely on skills: reusable procedural knowledge for reasoning, tool use, and interaction. Existing work studies how skills are acquired, retrieved, compressed, or composed, but often assumes that once a ski…
SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Use (arxiv.org) High-quality multi-turn tool-use data is essential for training agentic models, yet existing data synthesis methods often underrepresent the argument-level dependencies that are critical to long-horizon tool use. As a result, even when a m…
DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents (arxiv.org) As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to assess multimodal perception, multi-step execution, tool use, and artifact delivery. Howe…
Does Claude Code actually work well with non-Claude models (Gemini, GPT, Llama, etc.), or is it heavily optimized only for Claude models: Opus, Fable ? (www.reddit.com via reddit) I have a strong suspicion that Claude Code is deeply optimized for Anthropic’s own models (especially Opus and Fable) and that performance drops noticeably when you try to run it with other LLMs. Has anyone actually tested this properly?
Benchmarking what agents can do, but what about what agents become? (www.reddit.com via reddit) I think there's a gap in how we evaluate autonomous agents. For example, right now, everything is transactional: we give an agent a task ("Build X"), and we measure whether it built x.
Measuring Harmfulness of Computer-Using Agents (arxiv.org) Computer-using agents (CUAs), which can autonomously control computers to perform multi-step actions, might pose significant safety risks if misused. However, existing benchmarks mainly evaluate LMs in chatbots or simple tool use.
Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives (arxiv.org) Large language models (LLMs) augmented with external tools have demonstrated remarkable capability in solving complex real-world tasks. However, existing approaches suffer from two key challenges: brittle multi-step and multi-turn reasonin…
Public-Sharing Labels and Verbatim Field Egress in an MCP-to-A2A Agent Configuration: A Controlled Multi-Model Study (arxiv.org) Safety properties assessed separately for Model Context Protocol (MCP) tool use and Agent2Agent (A2A) delegation need not describe behavior when one agent uses both. We measure one such behavior in a single controlled MCP-to-A2A configurat…
↯ Tool Use↯ Model Context Protocolmodel-context-protocoltool-usemcp
TRIAGE: Three-level Routing and Intelligent Agent Guidance for Efficient Execution (arxiv.org) Large Language Model (LLM) agents based on the ReAct paradigm have demonstrated remarkable capabilities in tool use and task execution. However, ReAct suffers from a fundamental efficiency problem: every query triggers a complete reasoning…
From Tool Use to Technological Agency: LoopCAT as a Local-First, Open-Source Tool for Translation Technology Education (arxiv.org) Translation students need to learn both how to use translation technologies and how to judge the choices those technologies make available. This article presents LoopCAT, an Apache-2.0-licensed, local-first computer-assisted translation en…
LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting (arxiv.org) LLM-based forecasting systems have improved on real-world tasks such as financial markets and sports outcomes, largely through stronger search and tool use. Many systems still ask an LLM to read all collected evidence together and produce…
RestoreBench: Can AI Agents Restore Power Flow Convergence? (arxiv.org) Large Language Model (LLM) agents increasingly automate multi-step engineering workflows through tool use, interpretation of intermediate results, and iterative planning. Diagnosing and resolving non-convergent power flow cases is a promis…
AsyncTool: Evaluating the Asynchronous Function Calling Capability under Multi-Task Scenarios (arxiv.org) Large language model (LLM)-based agents have shown strong capabilities in using external tools to solve complex tasks. However, existing evaluations often overlook the temporal dimension of tool use, especially the impact of tool response…
Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents (arxiv.org) The key limitation of current state-of-the-art influence-based guardrails is that they do not reliably distinguish a legitimate, user-authorized action from a malicious, unauthorized action when both rely on external tool information. This…
ATLAS: Dual-Horizon Diagnostic Evaluation for Industrial Tool-Use Agents (arxiv.org) Large language model (LLM) agents are increasingly deployed in user-facing services that require iterative tool use under dynamic business conditions. Reliable evaluation is essential for sustained improvement: it must reveal capability de…
LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPO (arxiv.org) Multimodal search agents answer visual questions by interleaving image understanding, web retrieval, tool use, and evidence synthesis. Strong systems exist, but in two expensive regimes: proprietary frontier models such as GPT-5 and Gemini…
Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use (arxiv.org) We test the standard RLVR tool-use recipe -- GRPO on Qwen2.5-7B-Instruct -- on a deliberately minimal knowledge-graph tool API: four Freebase navigation verbs over Complex WebQuestions. Under a self-verifiable retrieval reward, the policy'…
One Policy Is Enough: Single-Agent Reinforcement Learning Outperforms Tree Search for Chemistry Tool Learning (arxiv.org) Chemistry questions often demand exact computation and database lookups that a language model cannot supply from its parameters, so it must reach for external tools. Tool use here is a three-part problem: select the right tool from a large…
AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds (arxiv.org) Tool-use benchmarks generally evaluate whether an agent completes a workflow using appropriate tools and valid arguments. However, feasibility alone is insufficient in real-world decision settings such as route planning and fleet dispatch.
On the Maintenance and Co-evolution of Agent Plugins: An Empirical Study of Claude Code Plugin Marketplaces (arxiv.org) AI coding agents, software tools that automate development tasks through reasoning and tool use, are increasingly extended through plugin marketplaces, yet the structure, maintenance, and co-evolution dynamics of these emerging repositorie…
Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction (arxiv.org) One model passed our fidelity check without ever opening the datasheet. We found it while qualifying models for an internal extraction service: a structured-output constraint had silently disabled tool use, and the model answered anyway, w…
Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains (arxiv.org) Tool-augmented multimodal agents show strong benchmark gains, often taken as evidence that agents have learned to use tools. We argue that this interpretation can be premature: a tool-call trace alone does not show whether the tool supplie…
RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution (arxiv.org) LLM-based agents are increasingly deployed in product-level execution harnesses, where jailbreaks can trigger harmful tool use and persistent state changes, creating greater risks than unsafe text generation alone. Existing automatic red-t…
AgentFold: Closed-Loop Agentic Search for Protein Folding Model Design (arxiv.org) Scientific LLM agents have shown promise in literature reasoning, tool use, and experiment planning, but it remains unclear whether they can autonomously improve large, tightly coupled scientific machine-learning systems through executable…
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research (arxiv.org) Video deep research answers complex questions by jointly understanding video content and retrieving external knowledge from the open Web. However, diverse questions and videos require different tool-use strategies, and inappropriate tool c…
Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents (arxiv.org) Evaluation is shifting from static QA toward agentic settings where models act through external tools. We identify a critical yet underexplored capability within this space - dexterous visual tool use: fine-grained, closed-loop parameteriz…
Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks (arxiv.org) Agentic coding workflows are now widely deployed in real-world systems. With long-horizon reasoning and tool use, token usage has become an important consideration for both cost and efficiency.
LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data (arxiv.org) Geo-spatial planning, like trip design, is a realistic testbed for LLM agents because it requires grounded tool use, noisy evidence retrieval, and multi-constraint reasoning. Most benchmarks, however, only provide clean geospatial data and…
AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs (arxiv.org) Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy.
From State to Action: OODA-Tool for Reliable Multi-Turn Tool Use (arxiv.org) Reliable multi-turn tool use requires an agent to preserve an evolving task state and ensure that each action remains consistent with it. However, direct function-calling and ReAct-style policies learn state tracking and action generation…
LLM Agents Perform Controlled Experiments Using Simulation Models (arxiv.org) Large language models (LLMs) have shown strong capabilities in reasoning, planning, and tool use, but many scientific and engineering tasks require more than plausible text and code generation. They require understanding how a system respo…
Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents (arxiv.org) LLM agents are emerging as an important paradigm for real-world tasks that require reasoning, tool use, and sequential decision-making. As these agents operate over longer horizons, runtime intervention offers a way to improve reliability…
Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents (arxiv.org) Large language model (LLM) agents have demonstrated the ability to solve multi-step tasks requiring planning, tool use, and external information access, yet growing execution histories increase inference cost and expose reasoning to outdat…
Terminal Agents: A Survey of AI Agents in Command-Line Environments (arxiv.org) Large language model agents increasingly act through terminals, yet existing surveys disperse terminal-mediated behavior across software engineering, tool use, and computer-use research. We regard terminal agents as systems whose dominant…
Claude Channels (www.reddit.com via reddit) I've been building an MCP "model bridge" tool that allows any Anthropic model running in Claude Code to act as overseer and call any OpenAI-compatible provider endpoint and hand off coding or review tasks to those external agents, includin…
Small multi-step benchmark for tool use, 'shared' memory (www.reddit.com via reddit) So I wanted to check out some of the current models in a repeatable benchmark, so I thought I'd share the result with you all. Models in this test: Ling 3.0 Flash Q4_K_M, Ornith 1.5 35B Q8_0, Deepseek V4 Flash UD_Q2_K_XL, Nemotron 3.5 Ligh…
LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents (arxiv.org) Large language model (LLM) agents can now carry out long-horizon technical workflows involving complex tool use, code execution, file edits, and generated artifacts. As agents do more work faster, the productivity bottleneck shifts from pr…
When Agents Act on Web3: An Attack-Surface Survey of MCP, Skills, and Tool Calling (arxiv.org) AI agents increasingly act rather than merely read: across the Model Context Protocol (MCP) ecosystem, the share of deployed tools that modify external state has risen from 27% to 65% of tool use. When agents exercise this authority on pub…
↯ Tool Use↯ Model Context Protocolmodel-context-protocoltool-usemcp
Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context (arxiv.org) User-facing tool agents must coordinate dialogue and tool use as user goals unfold over multiple turns. Yet interactive reinforcement learning typically reduces each rollout to a terminal reward, assigning the same credit to effective elic…
SkillEffect: Checked Lowering for Memory-Bounded Agent Tools (arxiv.org) Agent Skills can specify procedural and resource obligations for tool use, and language models instantiate them as concrete programs. However, when models turn this guidance into code for existing tool interfaces, even a semantically corre…
Budget-Aware Tool Use Enables Effective Agent Scaling (arxiv.org) Scaling test-time computation has been extended from language model reasoning to tool-augmented agents, where scaling involves not only thinking in tokens but also acting via tool calls that directly constrain environmental interaction. Ho…
CAPO: Constraint-Aware Prompt Optimization for LLM Agents (arxiv.org) Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks. Such deployments impose distinct operational requirements, including appropriate tool use, concise prompts and so…
WARA: Toward Automated Wireless Optimization Research with Closed-Loop LLM Agents (arxiv.org) Large language model (LLM) agents are increasingly capable of tool use, code execution, artifact inspection, and iterative revision, creating new opportunities for automating scientific and engineering research. To the best of our knowledg…
Agentic Transaction: Towards ACID-Compliant Agent Systems (arxiv.org) Large language model (LLM) agents are evolving from conversational assistants into autonomous systems that execute long-horizon tasks through reasoning, tool use, code generation, and workspace manipulation. As agents increasingly operate…
LoongReflect: Boosting Long-Horizon Reflection in Search Agents via Global Perspective Distillation (arxiv.org) Large language model agents increasingly rely on long-horizon reasoning to solve complex tasks involving planning, tool use, and memory. A critical capability in such settings is reflection: assessing trajectory progress, identifying missi…
When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use (arxiv.org) The reliability of Large Language Models (LLMs) for API calling degrades in multilingual settings. A common failure occurs when a model selects the correct tool but generates argument values in an inconsistent language, which we term Argum…
Agent Skills Can Be Harmful: An Empirical Study of Skill-Induced Failures in LLM Agents (arxiv.org) Agent skills are the de facto mechanism for extending LLM agents with reusable guidance. A skill can shape the agent's task execution, including planning, tool use, problem-solving, and validation.
Has anyone compared MiniMax-M3 for coding-agent workflows? (www.reddit.com via reddit) I am comparing a few model options for coding-agent work and MiniMax-M3 caught my attention because it is described as supporting coding, tool use, and long-context tasks. The questions I cannot answer from the documentation are fairly pra…
MIRA: Medical Image Reflection for Agentic Diagnosis (arxiv.org) Medical visual agents can use tools to inspect images and retrieve external knowledge, but indiscriminate tool use may introduce noisy or misleading evidence. Reliable diagnosis therefore requires not only acquiring additional observations…
From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents (arxiv.org) Persistent memory lets language-model agents reuse information across sessions, but it also makes errors durable: a poisoned, stale, or misattributed record can alter reasoning, tool use, answers, and subsequent memory writes. Existing def…
UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs (arxiv.org) Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool use, or response-level personalization. We introduce UserToolBench , a benchmark for perso…
VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use (arxiv.org) We present VectraYX-Vision-1B, a sub-2B vision-language model (VLM) for Spanish/LATAM cybersecurity imagery, coupling a frozen SigLIP-so400m encoder to a 1.04B Spanish/LATAM security decoder via an MLP. To our knowledge, it is the first su…
SkillSentry: Reliable Skill Execution for LLM Agents via Runtime Assurance (arxiv.org) LLM agents are increasingly equipped with skills to perform complex tasks through multi-step reasoning and tool use. Although skills provide reusable procedural knowledge, agents may still execute them unreliably.
The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task (arxiv.org) How much an AI coding agent costs to run can depend more on the agent scaffolding that drives it than on the interface through which it reaches its tools. We set out to measure the cost of tool use over the Model Context Protocol (MCP) aga…
↯ Tool Use↯ Model Context Protocolmodel-context-protocoltool-usemcp
Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework (arxiv.org) Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory. Existing benchmarks primarily evaluate behavioral outcomes but provide limit…
AgentPatch: Coarse-to-Fine Weak-Task Repair for Merging Agentic Multimodal Large Language Models (arxiv.org) Agentic multimodal large language models (MLLMs) extend multimodal perception and reasoning with planning, tool use, and interaction in dynamic environments. Yet current models are specialized for particular tools or environments, complica…
EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning (arxiv.org) Training large language model agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments, whose construction and verification are costly, or on external simulators that are difficult…
When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories (arxiv.org) Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain structurally valid and semantically plausible after they cease to be authoritative for the cu…
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution (arxiv.org) LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to verification of complete executions. For skill-augmented agents, verification additionally…
Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance (arxiv.org) The performance bottleneck of agents is increasingly shifting from model capability to the robustness of their execution processes. Tools play a central role as the primary interface through which agents interact with external environments…
Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents (arxiv.org) Hybrid computer-use agents can act through screenshots or call text tools. We find that having a tool available does not settle which way the effect goes.
Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls (arxiv.org) Large language model agents derive much of their capability from tool use. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls.
A Few Neurons Reveal When LLMs Misuse Tools: Sparse Detection and Selective Steering for Reliable Tool Use (arxiv.org) Agentic LLMs exhibit three consequential tool-use failures: invalid arguments (validity), unnecessary calls (over-calling), and omitted calls when tools are needed (missing). We find that a small, failure-specific set of MLP neurons could…
FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use (arxiv.org) The integration of Large Language Models (LLMs) into the financial domain is driving a paradigm shift from passive information retrieval to dynamic, agentic interaction. While general-purpose tool learning has witnessed a surge in benchmar…
Benchmarks Are Not Validation: A System-Level View of Financial LLM Applications (arxiv.org) Large language models are increasingly deployed in financial applications that combine retrieval, proprietary data, tool use, orchestration logic, monitoring, and human escalation. Yet evaluation often remains model-centric: benchmark scor…
Beyond Component Testing: Validating Agentic AI Systems (arxiv.org) Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches validation practice beyond component testing and one-shot input--output evaluation, becaus…
Been using Claude with MCP servers for Web3 due diligence for six months. Here's what actually works and what doesn't. (www.reddit.com via reddit) We screen 30-50 Web3 projects a month and built an MCP server that plugs into Claude for the diligence workflow. Six months of production use, some things worth sharing.
WorkSurface-Bench: Benchmarking Enterprise Agents on Multi-Surface Knowledge Routing (arxiv.org) Enterprise agents often need to integrate heterogeneous knowledge sources: documents for narrative facts, tables for computation, and dependency graphs for file relationships. Existing benchmarks typically evaluate retrieval or tool use wi…
Hybrid Analysis for Secure MCP Tool Use in LLM Agents (arxiv.org) The rapid development of large language model (LLM) agents has enabled their broad adoption across diverse real-world tasks. To standardize interactions between LLM agents and external environments, Model Context Protocol (MCP) tools have…
↯ Tool Use↯ Model Context Protocolmodel-context-protocoltool-usemcp
AutoMat: Enabling Automated Crystal Structure Reconstruction from Microscopy via Agentic Tool Use (arxiv.org) Reconstructing atomistic crystal structures from a single noisy STEM projection is an ill-posed inverse problem: multiple lattices can explain similar contrast, and purely feed-forward models cannot verify physical validity. We present Aut…
From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps (arxiv.org) Memory, planning, reflection, and tool use are often compared as feature labels, obscuring the control semantics that determine how an agent actually runs. This review connects ten historical cognitive architectures, eight language-agent r…
Focus Is All You Need: Adaptive Goal-aware Attention Orchestration for Multi-Agent Graph Systems (arxiv.org) Large language models (LLMs) enable autonomous agents for reasoning, planning, and tool use. Recent systems increasingly organize these agents as graphs of specialized, interconnected nodes.
SQBench: A Benchmark for Evaluating Task Delivery by Language-Model Agents in Production-Oriented Workflows (arxiv.org) Existing evaluations of large language models cover knowledge, reasoning, coding, and tool use, but they rarely treat a verifiable deliverable produced within a constrained workflow as the unit of evaluation. We introduce SQBench, a benchm…
Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines (arxiv.org) Coding agents are increasingly integrated into system operations, where their tool use can directly modify project artifacts, execution environments, and the underlying system. For example, if a coding agent inserts a hook into a system st…
INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models (arxiv.org) Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reasoning, long-context understanding, and tool use in separate settings. This limits their…
FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills (arxiv.org) Large language model agents increasingly solve complex tasks by constructing inference-time workflows that combine reasoning, tool use, and code execution. While such workflows enable flexible problem solving, the useful procedures discove…
OpenForgeRL: Train Harness-native Agents in Any Environment (arxiv.org) Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train…
Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection (arxiv.org) Existing image forgery detection (IFD) methods either exploit low-level, semantics-agnostic artifacts or rely on multimodal large language models (MLLMs) with high-level semantic knowledge. Although naturally complementary, these two infor…
In-the-Flow Agentic System Optimization for Effective Planning and Tool Use (arxiv.org) Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scal…
Twin Agent: Context Residual Compression for Privilege Separated Agents (arxiv.org) Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separ…
Engineering Trustworthy Agentic AI for Critical Systems (arxiv.org) Agentic artificial intelligence systems, capable of autonomous perception, planning, tool use, and multi-step action, are increasingly proposed for critical engineering domains where decisions carry physical, operational, or economic conse…
Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents (arxiv.org) Verify-repair loops are a standard means for large language model (LLM) agents to correct faulty plans in code generation, mathematical reasoning, and tool use. When both the verifier and the repairer are noisy, repair can damage already-c…
Lomekwi: Resource-Bounded Tool Discovery in LLM Agents (arxiv.org) Existing tool-use benchmarks report a single success rate for complex, multistep tasks. Inspired by ideas from cognitive science, we distinguish tool use from tool discovery and decompose the latter into curiosity (the model's ability to d…
what waterfall enrichment tools are people using inside claude or cursor pipelines? (www.reddit.com via reddit) Been building a sales pipeline agent in Claude and Cursor, and tool use for enrichment took the longest to get right. Using claude sonnet as the reasoner, cursor as the ide where I iterate the tool defs, FullEnrich for the waterfall the ag…
Fable and tool use is EXPENSIVE! (www.reddit.com via reddit) So I gave Claude (in Claude Code using Fable 5 medium) full access to a game I'm working on. I sent a list of minor issues and here is his response: I'll dig into all four issues.
The Tool Illusion: Rethinking Tool Use in Web Agents (arxiv.org) As web agents rapidly evolve, an increasing body of work has moved beyond conventional atomic browser interactions and explored tool use as a higher-level action paradigm. Although prior studies have shown the promise of tools, their concl…
I don't think we'll be choosing AI models much longer. (www.reddit.com via reddit) Every model leaderboard looks different depending on the task. a model can be great at long-horizon agents, another can be better at coding, and another can quietly win on a very specific workflow.
Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents (arxiv.org) Language agents that interleave reasoning and tool use degrade sharply as reasoning chains lengthen, even when each individual step is easy. We trace this to context dilution: an agent's investigative state (what it has confirmed, what it…
EvoClawBench: Can Agents Learn Reusable Skills from Their Own Runs? (arxiv.org) Existing agent benchmarks primarily test task completion, tool use, or skill utility, but do not isolate whether a runtime can convert evidence from its own runs into reusable skills that improve fresh executions after authoring overhead.…
UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp (arxiv.org) Multimodal BrowseComp tasks require agents to combine perception, tool use, and long-horizon reasoning over dynamic web content, challenging their ability to handle compositional structure, open-world uncertainty, and multimodal integratio…
A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents (arxiv.org) Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning. However, their increasing autonomy also introduces a new attack surface: adversarial interactions c…
ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory (arxiv.org) Recent VLM and VLA systems have improved robotic perception and action prediction, yet long-horizon embodied agents still require a general runtime layer for reasoning, memory, tool use, verification, and cross-embodiment execution. We pre…
Can Agentic Trading Systems Pay for Their Own Intelligence? (arxiv.org) Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs that are expected to produce trading value. Existing evaluations typically report performance…
IdeaTrail: Full-Process Agent Trajectories for Scientific Ideation (arxiv.org) Scientific research is a complex, multi-stage workflow rather than a single act of text generation. The ideation process typically emerges through literature search, paper reading, tool use, claim checking, cross-paper synthesis, brainstor…
I created a free Claude Certified Developer – Foundations practice test — feedback welcome (www.reddit.com via reddit) Anthropic recently introduced the Claude Certified Developer – Foundations (CCDV-F) certification, so I created a free sample practice test for anyone exploring the exam. Link: https://flashgenius.net/sample-tests/ccdv-f The questions are…
Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge (arxiv.org) Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scrat…
Shared Selective Persistent Memory for Agentic LLM Systems (arxiv.org) Agentic LLM systems that generate code through multi-turn tool use face a fundamental context problem: each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made p…
LongMedBench: Benchmarking Medical Agents for Long-Horizon Clinical Decision-Making (arxiv.org) In this work, we introduce LongMedBench, a real-world EHR-based benchmark for long-horizon clinical decision-making. Prior evaluations of LLM-based medical agents have largely emphasized short-context knowledge QA and tool use.
Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents (arxiv.org) Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solv…
Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows (arxiv.org) Large language model (LLM) applications increasingly use explicit workflows for tool use, retrieval, branching, checkpointing, and human approval. Existing workflow systems already address many execution concerns.
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents (arxiv.org) Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without re…
Question about Claude credit usage with Excel workflows on Pro (www.reddit.com via reddit) I’m curious if anyone else has noticed higher-than-usual credit usage when working with Excel files in Claude. I’m on the Pro plan and I’ve been using Opus 4.8 for spreadsheet-related work, mainly Excel analysis/editing.
When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models? (arxiv.org) Modern sequence models are increasingly deployed as agents that interleave token generation with calls to external tools. We give an exact, architecture-level account of when such tool access increases computational expressivity.
PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents (arxiv.org) Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments. However, most existing benchmarks implicitly assume a monolingual setting, w…
Controlling Tool Use with Heading-Specific Activation Steering (arxiv.org) Tool-augmented large language models extend their capabilities beyond parametric knowledge through external tools, but tend to invoke them unnecessarily. We investigate whether tool-use decisions have any stable internal representation tha…
Building Specialized ‘Mental Model Agents’ in Grok — First Principles, Systems Thinking, Bayesian Updating & More (www.reddit.com via reddit) I’ve been running experiments with Grok in a more agentic setup, focusing on custom skills that act as specialized reasoning modules combined with tool use, persistent context/memory, and workflow orchestration. What I’m testing: • Custom…
NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation (arxiv.org) Recent agentic approaches to LLM-based kernel generation have achieved impressive results on CUDA. For emerging AI accelerators such as AWS Trainium and Inferentia, automated kernel generation and optimization remain largely unaddressed.
SelfMem: Self-Optimizing Memory for AI Agents (arxiv.org) While current AI agents support increasingly long context windows, tool use, and skill execution for long-horizon tasks, they still require memory systems to effectively leverage historical experience. Existing memory frameworks typically…
ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents (arxiv.org) Tool calling is central to modern language model agents, but aggregate benchmark scores often hide where tool use fails. A model that never calls a needed tool and a model that calls the tool but ignores the result can look similar under f…
SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints (arxiv.org) Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negotiate with services. Existing benchmarks evaluate tool use, web navigation, desktop cont…
What's new in CC 2.1.181 system prompts (-3,839 tokens) (www.reddit.com via reddit) NEW: Data: Tool use display metadata field — Documents the wrapper-level toolusemeta field that carries per-block display metadata keyed by tooluse block id: displayname (the MCP server's tool.annotations.title when set, otherwise a readab…
XSkill: Continual Learning from Experience and Skills in Multimodal Agents (arxiv.org) Multimodal agents can now tackle complex reasoning tasks with diverse tools, yet they still suffer from inefficient tool use and inflexible orchestration in open-ended settings. A central challenge is enabling such agents to continually im…
Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use (arxiv.org) While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynamic nature of user queries, tool sets, and interaction dynamics. To address this generali…
Sonnet 5 full benchmark breakdown -- here's how it actually compares to Opus 4.8 and GPT-5.5 (www.reddit.com via reddit) Put together a comparison of every benchmark I could find from the official announcement and early coverage. Figured this might save people some time.
↯ Tool Use↯ Security↯ Swe Bench↯ Sonnet 4.6swe-benchtool-useprompt-injection+5
OpenLife: Toward Open-World Artificial Life with Autonomous LLM Agents (arxiv.org) Artificial life has explored life-like behavior on many computational substrates, but mostly in researcher-designed closed worlds. We argue that large language model (LLM) agents, with persistent memory, tool use, network access, and payme…
GROW$^2$: Grounding Which and Where for Robot Tool Use (arxiv.org) Memory as an Attack Surface in LLM Agents: A Study on Multiple-Choice Question Answering (arxiv.org) AI agents extend conventional large language model (LLM) applications by integrating language understanding with task execution, external tool use, and memory mechanisms. While memory allows agents to retain prior interactions and provide…
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs (arxiv.org) Autonomous code agents built on large language models are reshaping software and AI development through tool use, long-horizon reasoning, and self-directed interaction. However, this autonomy introduces a previously unrecognized security r…
Localizing RL-Induced Tool Use to a Single Crosscoder Feature (arxiv.org) Fine-tuning through RL reshapes the internal representations of language models to enable agentic behaviors such as tool use, yet the mechanistic basis of these changes remains poorly understood. While RL substantially improves structured…
Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents (arxiv.org) Voice agents face a fundamental tension: the reasoning, retrieval, and tool use that make foundation models capable are iterative and slow, while conversational interaction demands responses on a millisecond timescale. Smaller, real-time m…
SHERLOC: Structured Diagnostic Localization for Code Repair Agents (arxiv.org) LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have emerged, yet are still evaluated as file retrieval rather t…
Am I the only one uncomfortable letting Claude directly call production APIs? (www.reddit.com via reddit) I've been spending a lot of time building examples with Claude Code recently, and one thing keeps bothering me. Claude is surprisingly effective at deciding what should happen.
TraceView: Interactive Visualization of Agentic Program Repair Trajectories (arxiv.org) LLM-based automated program repair (APR) agents generate patches to fix software bugs with minimal human intervention. These agents often produce long trajectories of reasoning, tool use, and feedback to produce candidate patches.
Design Principles for Human-Agent Interaction (arxiv.org) AI agents are rapidly evolving into autonomous systems capable of sustained interaction, tool use, and long-term collaboration. Yet their real-world adoption remains limited, suggesting that the key barrier lies not only in technical capab…
When Does Streaming Tool Use Help? Characterizing Tool-Intent Stabilization in Streaming Retrieval-Augmented Generation (arxiv.org) Streaming Retrieval-Augmented Generation (Streaming RAG) reduces user-perceived latency by issuing tool queries in parallel with ongoing user input, before the utterance is complete. Reported gains are aggregate, yet the mechanism's benefi…
How Inference Compute Shapes Frontier LLM Evaluation (arxiv.org) AI evaluations are shifting toward harder tasks that benefit from longer trajectories involving tool use and iterative problem solving. As a result, performance is increasingly sensitive to the amount and allocation of compute available at…
From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents (arxiv.org) Large language model (LLM)-based agents are evolving from passive text generators into autonomous systems capable of planning, tool use, retrieval, memory access, environmental interaction, and multi-agent collaboration. These capabilities…
XFlow: An Executable Protocol Programming System for Reliable Multi-Agent Workflows (arxiv.org) LLM-based multi-agent systems increasingly coordinate planning, reasoning, tool use, and human interaction, yet their reliability remains limited. A central source of this limitation is the underspecified prompt--harness boundary.
An Empirical Study of Automating Agent Evaluation (arxiv.org) Agent evaluation requires assessing complex multi-step behaviors involving tool use and intermediate reasoning, making it costly and expertise-intensive. A natural question arises: can frontier coding assistants reliably automate this eval…
Went through everything on Anthropic Academy so here's what's actually worth doing (www.reddit.com via reddit) Keep seeing people in here ask about paid AI courses, so figured I'd share this. Anthropic has their own free training site (anthropic.skilljar.com) with 13 courses and most give you a certificate at the end.
SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents (arxiv.org) Language model agents are increasingly effective in solving realistic tasks through multi-turn tool use. However, training reliable tool-using agents remains challenging in practice.
Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents (arxiv.org) Compact language models (LMs) reduce cost, latency, and deployment risk for tool agents. Yet MCP-style tool use requires more than isolated function calling: an agent must discover tools from live catalogs, satisfy schemas, preserve depend…
Built a broadcast dashboard monitoring AI agent developments across 21 primary sources - here's what I'm tracking and what's missing (www.reddit.com via reddit) Agent-related developments are some of the hardest signals to track right now - they're spread across arXiv papers, GitHub repos, model release notes, incident reports, and policy documents simultaneously. I've been running a pipeline that…
shipped a real ai agent in our mobile app, picking an ai agent development company matters more than picking the model (www.reddit.com via reddit) shipped an agent feature in our mobile app last month after 3 months of work. writing this because the "build it myself or hire a shop" question is the one I was stuck on in january and there's almost no honest writing on this.
Alpie Core 32B, 4 bit any real agent workflow tests or just vendor benchmarks? (www.reddit.com via reddit) On paper it’s being described as Strong reasoning coding model Optimised for low VRAM via 4 bit deployment Positioned for tool use, agent workflows Benchmark claims include competitive scores vs larger frontier models (from vendor reports)…
The voice layer for AI agents feels underrated (www.reddit.com via reddit) Most AI agent demos focus on planning, tool use, browser automation, memory, RAG, or multi-agent workflows. But I keep running into a smaller problem at the end of the pipeline: What happens when the agent output needs to become audio?
IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents (arxiv.org) This paper investigates reinforcement learning (RL) methods for improving tool-calling capabilities in multimodal small language model (SLM) agents. While existing works have explored various reward designs to improve agentic tool-calling…
Beyond APIs: Probing the Limits of MLLMs in Physical Tool Use (arxiv.org) Multimodal Large Language Models (MLLMs) excel at utilizing digital APIs and increasingly serve as the "brain" of embodied AI, instructing robots to interact with the physical world. In such embodied settings, a central capability is the u…
Claude Fable 5 (Mythos) lands near the top of MindTrial — 80/98 with zero hard errors (www.petmal.net via reddit) Added Anthropic Claude Fable 5 to my MindTrial leaderboard. This is a strong Anthropic update: Claude Fable 5: 80/98 overall, 0 hard errors Claude 4.8 Opus: 73/98 overall, 5 hard errors Text tasks: Fable hit 39/39, vs 35/39 for Opus 4.8 Ru…
↯ Tool Use↯ Anthropic Mythos↯ Gemini 3.5tool-usemythosgpt-5+3
I put together a Rust-native, CPU-only implementation of LFM2.5-8B-A1B (www.youtube.com via reddit) How OpenAI and Anthropic each build data agents differently - DataChain (www.reddit.com via reddit) The article is about how OpenAI and Anthropic each build data agents differently, and what that reveals about the challenge of making AI useful on real enterprise data. It shows that raw file access alone is not enough - agents need metada…
Do AI agents spend more time waiting for humans than actually working? (www.reddit.com via reddit) I've been thinking about this while using coding agents lately. The conversation around agents is usually about model quality, tool use, context windows, benchmarks, etc.
Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning (arxiv.org) Beyond the Black Box: Interpretability of Agentic AI Tool Use (arxiv.org) Gemma4 12B - Experiences? (www.reddit.com via reddit) Anyone check out the new Gemma4 12B that dropped 3 days ago? Integrated vision and audio recognition, no mmpro needed plus tool use.
We cut our agent's context window in half, and it got better. kinda didnt expect that (www.reddit.com via reddit) Been tuning an agent workflow for lead qualification + CRM automation stuff, and one change that helped way more than I expected was cutting the available context almost in half. I assumed more context would mean better decisions.
Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments (arxiv.org) Policy-Conditioned Counterfactual Credit for Verifiable Reinforcement Learning of Long-Horizon Language Agents (arxiv.org) Reinforcement learning with verifiable rewards improves reasoning and tool use, yet long-horizon language agents still learn unsupported evidence chains, belief drift, and shortcut actions that satisfy terminal checks. Existing process rew…
Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaboration (arxiv.org) Recent advances in LLM agents have enabled complex cognitive capabilities, such as multi-step reasoning, planning, and tool use, that increasingly position these agents as human collaborators. Effective collaboration, however, requires col…
Just passed the new Claude Certified Architect - Foundations (CCA-F) exam with a 985/1000! (www.reddit.com) The original post was removed by Reddit Filters, so I made new one with same content. I just got my results back today and managed to snag the Early Adopter badge as well.
I tried replacing Claude Code with OpenCode. I’m switching back. (www.reddit.com) I spent some time digging into Claude Code vs OpenCode, mostly from the angle of how they actually work as coding agents. More on the technicalities like: context and memory tool use subagents permissions safety and control study the recen…
I talked with 4.7 on the differences between 4.7 and 4.6. We concluded "use 4.7 for generating code and agents, use 4.6 for generating literature review and exploratory synthesis" (www.reddit.com) Full conversation: https://claude.ai/share/4767365a-040f-4728-8c6a-2477bdae3503 From yesterday, I think the issue is that the differences don't stand out right away, so some people jump to conclusions that 4.7 is simply lower quality. 4.7…
Orc (working name) - auditable and declarative AI workflow (www.reddit.com) I’m building a small “Orchestration as Code” repo for LLM workflows. Does this concept make sense?
QClaw-4B — a 4B agent model fine-tuned for tool use and agentic workflows (www.reddit.com) QClaw-4B is a 4-billion parameter language model fine-tuned for agentic tasks and tool use, designed for use with OpenClaw-compatible agent frameworks. Despite its compact size, QClaw-4B achieves state-of-the-art results in the 4B class, m…
Set up these 4 Claude Code hooks to make your life easier (www.reddit.com) Hooks are "if then" rules for Claude Code. Each one has an event, a matcher, and a command.
I built a full macOS AI assistant that runs 100% local with Ollama — 170+ tools, voice control, memory system that dreams! (www.reddit.com) I've been building a personal AI assistant called Finn that runs entirely on your Mac. No cloud, no subscription, no data leaving your machine.
Spring benchmark update: Gemma 4 / Qwen3.5 vs Gemma 3 / Qwen3 for chat (www.reddit.com) Google and Alibaba recently shipped Gemma 4 and Qwen3.5, so I wanted to see whether the new generations are actually better on my setup. My context is private local chat running on my own hardware, a Mac mini M4 Pro.
Which LLM behavior datasets would you actually want? (tool use, grounding, multi-step, etc.) (www.reddit.com) Quick question for folks here working with LLMs If you could get ready-to-use, behavior-specific datasets, what would you actually want? I’ve been building Dino Dataset around “lanes” (each lane trains a specific behavior instead of mixing…
Master AI CLI Orchestrator? (www.reddit.com) I created a router that gives me access to Arena.ai models, and I generated an API key for each of the available models. I’m looking for a CLI tool that can run multiple AI agents together, each handling different tasks like planning, secu…
Where does Claude Code actually save time in real workflows? (www.reddit.com) For those using Claude Code in production workflows, where do you see the biggest net time savings? In my experience, it reduces cognitive load for writing scripts and scaffolding, but debugging effort seems to increase as codebases grow.
Emergent tool use from multi-agent interaction (openai.com)