#tool-calling
84 items
Hot take: the biggest bottleneck in AI agents right now isn't models, frameworks, or even cost. It's that nobody knows how to properly evaluate if their agent is actually working (www.reddit.com) We are finally there: Qwen3.6-27B + agentic search; 95.7% SimpleQA on a single 3090, fully local (www.reddit.com) LDR maintainer here. Thanks to the strong support of r/LocalLLaMA community LDR got very far.
AI agent roadmap for developers who can code but have never built an agent (www.reddit.com) Go LLM SDK for streaming, tool-calling AI backends (plus frontend React lib) (github.com via hn) Call language models, stream responses, execute tools, and serve AI-powered endpoints from Go. Use the SDK on its own or pair it with an AI SDK React frontend.
Show HN: Local Coding Agent with LLMs to Delegate Tool Calls to Small AI Models (github.com via hn) Open Agent Tools Coder Open Agent Tools (oats) enables small-to-large self-hosted ai models to use local source code when running tool-calling agentic workloads. We actively data mine 20,970+ (2+ TB) popular github repos using large and sm…
SpaceXAI: Grok 4.6 (openrouter.ai via hn) Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).
Benchmarked Needle 26M vs Qwen3-0.6B on CPU function calling, 50 queries across 5 difficulty tiers. The 23x smaller model wins on accuracy and is 4.4x faster. (www.reddit.com) Ran a head-to-head on two open-weight models for tool-calling on a 4-core CPU, no GPU, no cherry-picking. Wanted to see if the small specialist (Needle, 26M, distilled from Gemini 3.1 for function calls) actually holds up against a small g…
↯ Function Calling↯ Gemini 3.1function-callingtool-callinggemini
Computer use is 45x more expensive than a structured API call (www.reddit.com) Hi r/AI_Agents, I recently did a benchmark on computer use agents vs api calls as part of a feature launch for my company. I wanted to share the benchmark here since it seems relevant to this sub: See, most teams default to computer use ag…
how do you design an ai agent to handle heavy data processing and large files? (www.reddit.com) looking for architectural patterns on handling data gravity in production agent pipelines. every tutorial I've found assumes light text payloads or short tool-calling loops, but once your agents have to actually interact with massive sourc…
AI Support Agents & Workflows Worth Exploring in 2026 (www.reddit.com) Been exploring how AI agents are slowly changing customer support workflows, especially for smaller teams trying to scale without adding headcount. Some interesting tools/workflows worth checking out: • SparrowDesk’s Zoona: AI support agen…
What we learned trying to fine-tune a small tool-calling model from production traces (and what not to do) (www.reddit.com) TL;DR: We wanted a small, fast model for multi-turn tool-calling. Training on clean, curated data worked brilliantly (1.7B student beating a 744B teacher).
Ask HN: How realistic is a modern-day era NLU (news.ycombinator.com) I'm looking for people who are a bit more familiar with neural networks than me (I follow kaparthy's tutorial once, that's about it). Lets say I like using LLMs for quality/complex tool-calling to e.g.
Tool-Calling Is Not a Guarantee, and Most Agents Are Betting That It Is (medium.com via hn) could not extract summary
Finetuning a Reasoning LLM with Supervised or Reinforcement Learning? (discuss.huggingface.co via hn) Hello, I have a task to fine-tune small LLMs on annotated conversational data. The dataset contains not only the final answers, but also reasoning traces and tool-calling decisions (i.e., when the model should think and when it should cal…
Show HN: Kitchen Rush, Overcooked inspired LLM tool calling benchmark (github.com via hn) An agent tool-calling benchmark where latency matters as much as intelligence. Why this exists Most tool-calling benchmarks (BFCL, τ-bench, ToolSandbox, AppWorld) check whether a model makes the right calls — and the world politely waits w…
Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM (deemwar-products.github.io via hn) mochallamaA local, tool-calling LLM inside your JVM The only in-process, tool-calling local LLM for the JVM — Spring-first, OpenAI-compatible, llama.cpp-backed via Project Panama FFM. No JNI, no daemon, no native-install dance.
Show HN: Prism Coder – Qwen3.5-14B fine-tuned for MCP tool-routing decisions (github.com via hn) 🧠 Prism Coder 🌐 Read in your language: 🇬🇧 English · 🇪🇸 Español · 🇫🇷 Français · 🇵🇹 Português · 🇷🇴 Română · 🇺🇦 Українська · 🇷🇺 Русский · 🇩🇪 Deutsch · 🇯🇵 日本語 · 🇰🇷 한국어 · 🇨🇳 中文 · 🇸🇦 العربية Persistent memory + tool-calling intelligence for AI a…
The Oats Protocol – Open Agent Tools for Local Coding Agents (news.ycombinator.com) Recently I was using functiongemma and watched it load and run local source code as a tool call without any training/tuning. A couple days later I got Qwen35 in Open-WebUI to use the "native" tool-calling.
Is Haiku good for building a chatbot with MCP tools ? (www.reddit.com) Hi, We’re experimenting with building a chatbot that handles consumer interactions. The agent currently has access to about 5–8 tools, and we’re exploring different models to find the right balance of speed, cost, and tool-calling reliabil…
I tried to get my AI agent to schedule a meeting over email. The failure mode revealed a problem almost nobody in the agent space is talking about. (www.reddit.com) I've been building an AI agent that operates across SMS, email, WhatsApp, and Slack — and the hardest problem I've run into isn't tool-calling or reasoning. It's what happens when the agent interacts with multiple people who have different…
What we learned building a data agent that talks to 4 database types simultaneously (DAB benchmark) (www.reddit.com) UC Berkeley published DataAgentBench (DAB) in March — 54 queries across PostgreSQL, MongoDB, SQLite, and DuckDB. Best score so far is 54.3% (PromptQL + Gemini).
Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines (arxiv.org via hn) The Model Context Protocol (MCP) enables LLMs to invoke external tools, but every tool interaction exposes the model to attacker-controlled text through multiple input channels (tool descriptions, tool results, sampling messages) that shar…
↯ Model Context Protocoltool-callingmodel-context-protocolmcp
Show HN: MaskShift – a maximalist coding agent with zero NPM dependencies (github.com via hn) MaskShift is a local-first coding agent harness. Features: - Works with models that have no tool-calling API.
Toolcall-doctor – Shrink broken LLM tool-call reproducers (github.com via hn) toolcall-doctor Stop manually shrinking broken LLM tool calls. Got a huge request that reproduces a weird tool-calling bug?
Ask HN: What eval harness holds up in practice, and what is still missing? (news.ycombinator.com) There’s no shortage of eval frameworks, but once you move past simple single-turn prompt checks into multi-step, tool-calling agents, most of them start to feel either too heavy or too brittle for CI. Testing intermediate trajectories, ret…
Show HN: Tracelint – a linter for AI agent traces, no LLM judge (github.com via hn) tracelint A linter for agent runs — it reads the execution trace of a tool-calling agent (what it actually did) and flags structural bugs deterministically, with the exact evidence and a CI exit code. It runs after the run, on the trace —…
Orbit, an Open-Source Toolkit for Retrieval-Based Inference (github.com via hn) # ORBIT ### Open Retrieval-Based Inference Toolkit A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers. Quick Start…
Probelock – lockfile for LLM tool calling (github.com via hn) probelock A capability lockfile for local models. It records what a model does on a set of tool-calling and output checks, and fails CI when a model/quant/runtime swap lowers a score.
How does Google Antigravity IDE actually work internally? (www.reddit.com) Hey everyone, I’ve been exploring Google Antigravity recently, and I’m really curious about its internal architecture and engineering design. From the demos, it seems much more advanced than a normal AI coding assistant — almost like an au…
ReAct tool-calling issue: Orchestration model computes internally instead of using tools (www.reddit.com) Built a local ReAct-style calculator agent with 6 tools: add subtract multiply divide modulo etc. The setup is: orchestrator agent dynamic tool selection ReAct loop tools exposed as functions Problem: Even when the user asks multi-step ari…
Training a 22MB prompt injection classifier (www.stackone.com via hn) Training a 22MB Prompt Injection Classifier Table of Contents When we started building Defender (our prompt injection guard for MCP tool-calling agents), the constraint was simple and unforgiving: ship inline inside a TypeScript Lambda, st…
Needle-rs – AI Function calling in the browser, 258 KB WASM (needle-rs.pages.dev via hn) AI TOOL CALLING · WASM · NO_STD Below is a 26M-parameter tool-calling transformer running entirely in this tab — no server, no API key, no data leaving your device. The model is Needle by Cactus Compute; needle-rs is the pure-Rust runtime…
sAI2.m6s (www.reddit.com) Hey everyone, I'm designing a powerful, autonomous AI chatbot(agent) , fully private, using a Python backend (for the core intelligence and tool-calling loops) and a Flutter frontend for a cross-platform UI. Since this moves past a basic…
I built an OSS CLI to catch regressions when migrating between LLMs (www.reddit.com) I’ve been working on EvalShift, an open-source Python CLI for testing whether moving from one LLM/model version to another introduces regressions. The use case is simple: You have prompts, agents, or tool-calling workflows that work well o…
Built a practical voice-first AI tool for ADHD/executive dysfunction — one-tap brain dump → structured reminders & tasks (not a full autonomous agent) (www.reddit.com) Not a full autonomous agent in the Auto-GPT / LangChain sense, but I built something that uses AI in a very practical, daily way for executive dysfunction / ADHD brains. SAVI is a one-tap voice capture tool.
Show HN: Mlx-code – I built a "backyard shed" AI coding agent for Mac (github.com via hn) mlx-code A lightweight coding agent for Mac, built on Apple's MLX framework. Fast local inference, built-in prompt caching, robust tool-calling.
Help setting up Chrome MCP for Hermes Agent (www.reddit.com) Hi everyone, I'm trying to set up Chrome MCP (Model Context Protocol) for Hermes Agent and need some guidance. **Background:** - Hermes Agent (by NousResearch) has self-learning features - I want to integrate Chrome browser automation via…
↯ Model Context Protocoltool-callingmodel-context-protocolmcp
Title: Is it just me, or is the "Multi-Agent Swarm" the new "Over-Engineered Spreadsheet"? (www.reddit.com) We’re four months into 2026 and every demo I see features "15 agents working together to write a blog post." In my experience, the more agents you add, the higher the "Cognitive Tax." You get more hallucinations, more token cost, and more…
Defender – Local prompt injection detection for AI agents (no API calls) (www.npmjs.com via hn) Prompt injection defense framework for AI tool-calling Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and neutralizes prompt injection attacks hidden in t…
↯ Security↯ Function Callingfunction-callingtool-callingprompt-injection+2
SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale (arxiv.org) Limited controlled evidence exists on how training data, adaptation method, and model scale jointly affect tool-calling performance in language-model agents. We evaluate supervised fine-tuning (SFT) with LoRA, reinforcement learning (RL) v…
Built with Claude Code: Free scenario practice simulator & 2026 certification roadmap for Claude CCDV-F (www.reddit.com via reddit) I built CertArena, an interactive exam simulator and certification roadmap tool designed specifically for developers preparing for Anthropic's Claude Certified Developer Foundations (CCDV-F) and Cloud AI benchmarks. What was built CertAren…
↯ Model Context Protocoltool-callingmodel-context-protocolmcp+2
Substrate-Portable Execution for Production LLM Workflows (arxiv.org) Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often coupled to one runtime. We encountered this portability problem in Rufus, a conversational…
Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe (arxiv.org) Data-sovereignty regulations increasingly require public institutions to deploy open-source, on-premise LLM agents that chain multiple tool-calls across live government APIs. However, open-source models consistently underperform in this mu…
Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling (arxiv.org) Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy.
CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents (arxiv.org) Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and mu…
The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs (arxiv.org) The hidden states of large language models (LLMs) are known to capture rich information relating to model knowledge and behavior that can be hard to extract from examination of input and output alone. As LLM-based systems increasingly inte…
AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling (arxiv.org) LLM judges are widely used to evaluate agentic tool-calling systems, yet their reliability on structured, dependency-driven workflows remains largely unexamined. We present AgentJudgeBench, the first benchmark to systematically study LLM-a…
Benchmarking AI Agents for Hardware Design Automation via MCP Tool Calling (arxiv.org) We ask whether AI agents powered by locally deployed large language models can reliably automate expert-defined hardware design workflows in an industry-realistic tool-calling setting. In these environments, engineers issue repetitive, dep…
ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents (arxiv.org) Large language models (LLMs) rely on tool calling as a fundamental agent capability, enabling them to invoke external systems and complete tasks beyond text generation. However, clean end-to-end (E2E) success cannot identify where a tool-u…
Noise Floor Audit for Agent Benchmarks (arxiv.org) We audit measurement variability for 3 native tool-calling endpoints across 2 providers on the official BFCL multiple and parallel categories, using matched AST grading. At temperature 0, reruns are nearly deterministic across Groq endpoin…
LayerStoRm: Run frontier-scale MoE LLMs on a handful of consumer GPUs by streaming experts over PCIe. (www.reddit.com via reddit) I am open sourcing an early release of LayerStoRm, a MoE LLM serving engine for limited VRAM multi-GPU systems, taking advantage of RAM and parallel PCI-E transfers. Currently tailored for my system with 2 x RTX5090 + 2 x RTX5080 and 512GB…
I run the same jobs through both Claude Code and Codex every day and cross-check the outputs. Five things that surprised me (www.reddit.com via reddit) For the past few months I've had Claude Code and Codex running the same recurring jobs on one always-on server, with outputs compared against each other. Not a benchmark, real production work, every day.
DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents (arxiv.org) Equipping Large Language Models (LLMs) with multi-turn tool-calling capabilities is essential for building autonomous agents. However, progress is fundamentally limited by the reliance on full-length trajectory imitation.
Cleared all 4 Anthropic Claude Certifications — AMA on the experience (www.reddit.com via reddit) Hey everyone, I recently decided to work my way through the Anthropic certification track and managed to clear all four: • Claude Certified Associate – Foundations (CCAO-F) • Claude Certified Developer – Foundations (CCDV-F) • Claude Certi…
Schema-Agnostic Graph Reasoning Agent for Hybrid Knowledge Graphs (arxiv.org) Tool-calling LLM agents navigate unfamiliar codebases with a handful of generic primitives for listing, reading and searching files (ls, cat, grep). A knowledge graph admits the same interface: listing neighbours, reading node content and…
Mandato: Protocol-Level Enforcement of Digitally Signed Mandates on AI Agent Actions with Cryptographically Chained Audit Trails (arxiv.org) AI agents increasingly act on external systems through standardized tool-calling protocols such as the Model Context Protocol (MCP), yet no infrastructure layer constrains their actions to what a principal has verifiably authorized: author…
↯ Model Context Protocoltool-callingmodel-context-protocolmcp
On Effectiveness and Efficiency of Agentic Tool-calling and RL Training (arxiv.org) Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge. This paper studies tool-calling along two complementary axes: effectiveness, i.e., how this capa…
TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories (arxiv.org) As large language models (LLMs) evolve from static chatbots into autonomous agents, the primary vulnerability surface shifts from final outputs to intermediate execution traces. While safety guardrails are well-benchmarked for natural lang…
FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks (arxiv.org) Recent studies demonstrate that tool-calling capability enables large language models (LLMs) to interact with external environments for long-horizon financial tasks. While existing benchmarks have begun evaluating financial tool calling, t…
When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories (arxiv.org) Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain structurally valid and semantically plausible after they cease to be authoritative for the cu…
Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents (arxiv.org) Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most.
Wait, Claude can draw? (www.reddit.com via reddit) Inspired by MineBench and this reddit post, I got curious about what other non-text things LLMs could make. That turned into Pixel Art Lab, a local tool that lets a model iteratively create pixel art.
MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agents (arxiv.org) We introduce MM-ToolSandBox, a benchmark and evaluation framework for visually grounded tool-calling agents. The framework provides a stateful execution environment spanning 500+ tools across 16 application domains, supporting multi-image,…
SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning (arxiv.org) Agentic multimodal large language models (MLLMs) (e.g., OpenAI o3 and Gemini Agentic Vision) achieve remarkable reasoning capabilities through iterative visual tool invocation. However, the cascaded perception, reasoning, and tool-calling…
Claude hallucinated its own internal tools, freaked out, and accused me of a prompt injection attack 💀 (www.reddit.comhttps) Ran into a fascinating UI/pipeline bug today while pasting standard text from a job board into Claude. As you can see in the screenshot, the backend text compaction or tool-calling layer leaked its own JSON definitions (referencing Apify/N…
Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning (arxiv.org) Multi-turn tool-using agents must coordinate long-horizon tool sequences while tracking dialogue state and policy constraints. Existing approaches often separate inference-time orchestration from parameter-level learning, leaving tool sele…
LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents (arxiv.org) Policy-adherent tool-calling agents in customer-service domains must maintain task states across turns while calling tools and obeying domain policies. Task states consist of relevant facts, identifiers, constraints, and conditions observe…
Building independent LLM drift detection - sharing the methodology, looking for feedback on the approach (www.reddit.com via reddit) Disclosed upfront: I run [Tickerr dot ai], an independent external monitor for AI APIs. Today it tracks latency, TTFT, uptime, and error rates across major models.
Kimi K2.7 Code: 1T MoE, $0.95/M tokens, MIT license, beats Opus 4.8 on MCP tool-calling (www.reddit.com via reddit) Moonshot AI released Kimi K2.7 Code on June 12 — a coding-focused open-weight model. Key specs: - 1 trillion params (MoE, 32B active, 384 experts) - 256K context window - Modified MIT license — weights on Hugging Face - $0.95/M input, $4.0…
Can LLM Agents Infer World Models? Evidence from Agentic Automata Learning (arxiv.org) We propose agentic automata learning to evaluate the extent to which tool-calling LLM agents can uncover hidden environments through interaction. In our setup, an agent should uncover a hidden deterministic finite automaton (DFA) by intera…
CacheRL:Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward (arxiv.org) We present CacheRL, a system for training small agent foundation models that achieves 92 percent process accuracy on multi-step tool-calling tasks, approaching GPT-5's 94 percent while requiring 100 times less compute. Our approach address…
JudgeOS V5.7 / EBH — The Governance Firewall Above AI, Robots, Agents, and Autonomous Workflows (www.reddit.com via reddit) Below is the whole-system tree map showing how JudgeOS V5.7 / EBH connects the locked core, Universal Adapter, domain adapters, capability registry, evidence trust, exact-action ALLOW binding, receipt/replay layer, SDK, dashboard, and exec…
Has anyone tried turning Gemini Web + Playwright into an OpenAI-compatible API to skip paid API purchases? (www.reddit.com via reddit) I've always wondered if there was a way to avoid expensive API costs while still building AI-powered applications. That got me thinking: what if, instead of using the official API, I simply automated a web browser to interact with Gemini d…
IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents (arxiv.org) This paper investigates reinforcement learning (RL) methods for improving tool-calling capabilities in multimodal small language model (SLM) agents. While existing works have explored various reward designs to improve agentic tool-calling…
I built a visual, local-first AI agent platform - no Docker, no terminal, double-click installer (v0.3.5, open source) (www.reddit.comhttps) I built a free, visual, local-first AI agent platform because every option was either cloud-locked or required living in Docker or using a terminal. You've probably seen OpenClaw and Odysseus recently.
ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents (arxiv.org) Adapting LLM agents to domain-specific tool calling remains notably brittle under evolving interfaces. Prompt and schema engineering is easy to deploy but often fragile under distribution shift and strict parsers, while continual parameter…
T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arxiv.org) Recent advances in reasoning and tool-calling capabilities of large language models (LLMs) have enabled increasingly capable agentic systems. However, existing benchmarks remain limited in task complexity, realism, and domain diversity, an…
Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning (arxiv.org) Large language model (LLM)-based agents often make suboptimal tool-use decisions, including unsupported tool invocation and hallucinated direct responses, which may accumulate errors throughout multi-step interactions. Existing approaches…
we really all are going to make it, aren't we? 2x3090 setup. (www.reddit.com) i'm blown away. i saw someone made a post the other day about "club-3090" and after having sonnet patch some fixes into it, specifically a sse-session drop bug and a bug with tool-calling, it's fair to say that even "budget" setups like my…
Should I buy Claude Pro as a BTech student — especially for the agentic/coding side? Honest takes wanted (www.reddit.com) https://preview.redd.it/l23rgf5z4qyg1.png?width=1402&format=png&auto=webp&s=73a7a278ca50527c9605488141d7e5ea48089a85 Hey everyone, I'm a BTech (AI/ML) student considering Claude Pro ($20/month) but want to separate the real value from the…
I built AI agents that play Pokemon Showdown autonomously using free LLM APIs via tool-calling (www.reddit.com) I've built a system where models like Llama 3, Qwen, and Gemma play Pokémon Showdown battles autonomously. Instead of simple prompt-response, they analyze the full battle state every turn (type matchups, HP, weather, field conditions, reve…
llm 0.32a1 (simonwillison.net) 29th April 2026 - Fixed a bug in 0.32a0 where tool-calling conversations were not correctly reinflated from SQLite. #1426 Recent articles - LLM 0.32a0 is a major backwards-compatible refactor - 29th April 2026 - Tracking the history of the…
TPS wasn't enough, tool-calling pass rate decided the winner in my Qwen 7B runs (www.reddit.com) I kept running into the same problem: TPS and TTFT tell you which config is fast, and perplexity is helpful only as a rough quality signal. None of them reliably tell you how the model will behave after changing quant, ctx size, kv_cache,…
Small models fail at tool selection - but it's not what I expected (www.reddit.com) Been running small models (1.5B-4B) with tool-calling agents. They consistently failed at selecting the right tool from 80+ options.