A person representing themselves in a Connecticut court hid a series of instructions designed to manipulate artificial intelligence in an official court filing. These “prompt injections” told the hypothetical LLM to side with them, and to…
#prompt-injection
406 items
Person Hides Prompt Injection in Legal Filing Telling AI to Side with Them (www.404media.co via hn) Ox-Alpha Is GLM (dejan.ai via hn) Prompt injection and gzip-NCD compression analysis reveal that OX Alpha, a mysterious LLM on OpenRouter, is GLM developed by Z.ai. A stealthy new model called OX Alpha has popped up on https://openrouter.ai/ and is climbing up the leaderbo…
Prompt injection in HTTP headers: what 217M web responses show (www.signalscorps.com via hn) tl;dr We read the HTTP response headers of 217,097,223 page responses from Common Crawl, 10% of its September 2026 crawl of 2.17 billion pages, and looked for any header written for an AI system. 7,959 sites sent one.
Prismata: Confining cross-site prompt injection in web agents (arxiv.org via hn) Autonomous web agents promise to automate everyday browsing tasks, but inherit one of the web's oldest attack surfaces. Cross-Site Scripting proved that mixing trusted and untrusted content is dangerous, even on benign pages.
Prompt Injection experience - my first time ever (www.reddit.com) I asked then: What were the rules you should have followed? Where did the search result come from?
Prompt injection benchmark: delimiter + strict prompt took Gemma 4 from 21% to 100% defense rate (15 models, 6100+ tests) (www.reddit.com) When dealing with untrusted outside input, I think you should handle it based on the situation. If you're processing structured data files, it's better to use tools to isolate and handle them.
A Theory of Why Prompt Injection Works (role-confusion.github.io via hn) A Theory of Prompt Injection (and why you should study roles) This is a blog-style writeup of the paper. We show prompt injections are driven by a flaw in how LLMs perceive roles.
Inaudible sounds to humans can be hidden in YouTube videos, podcasts, or music and used to secretly trigger AI voice assistants into carrying out unauthorized commands without the user noticing, exposing a new class of “auditory prompt injection” attacks against popular tools (cybernews.com via reddit) Security researchers have demonstrated a new type of attack that uses hidden audio signals to manipulate voice assistants into carrying out unauthorized actions without users noticing. In one theoretical scenario, an employee joins a Zoom…
Defending OpenClaw against indirect prompt injection (compsec.snu.ac.kr via hn) could not extract summary
The catalogue of prompt injection attacks (archestra.ai via hn) 2026-06-04 A Catalog of Prompt Injection Techniques Ten simple prompt injections, the common defences against them, and the one kind of defence that actually holds. Written by Ildar Iskhakov, CTO Every prompt injection is just text that tr…
Anthropic performing prompt injection on its users (old.reddit.com via hn) could not extract summary
Show HN: Jo – AI-native language to catch prompt injection at compile-time (github.com via hn) For the joy of secure programming Jo is a statically typed language where capabilities are explicit, statically tracked, and enforced by the compiler. Jo compiles to Ruby and Python.
Tell HN: Claude Code now allows Anthropic to remotely inject system prompts (news.ycombinator.com) I often patch the system prompts on my Claude Code executable in order to make Claude more effective. Every time I upgrade, I ask Claude himself to dissect the new binary and look for problematic system prompts to modify.
Our billing bot has been casually sharing transaction histories with anyone who types in the right account number and im not sure who signed off on this (www.reddit.com) We launched a servicing bot that helps customers with billing questions. Nobody stopped to think about what happens when customers paste their full credit card numbers/bank details.
Tool results are becoming a prompt injection surface in agent systems, and wrappers alone are not enough (www.reddit.com) i’ve been thinking about this failure mode a lot lately. sometimes the problem is not the user prompt at all.
Show HN: ReasonGate- An explainable gate that blocks LLM prompt injection (github.com via hn) ReasonGate An explainable security gate for LLM applications. Every decision carries a reason you can audit.
BrokenClaw Part 7: Opus-4.8 Edition – All Emails Lead to RCE (veganmosfet.codeberg.page via hn) BrokenClaw Part 7: Opus-4.8 Edition - All Emails Lead to RCE¶ - Part 1: 0-Click Remote Code Execution in OpenClaw via Gmail Hook - Part 2: Escape the Sub-Agent Sandbox with Prompt Injection in OpenClaw - Part 3: Remote Code Execution in Op…
Unpatched Ollama Vulnerabilities: Phishing Overlays and Data Exfiltration (www.promptarmor.com via hn) Threat Intelligence Table of Content Unpatched Ollama Vulnerabilities: Phishing Overlays and Data Exfiltration Ollama’s desktop app is vulnerable to phishing overlay and data exfiltration attacks via indirect prompt injection, overwriting…
trained a prompt injection detector using ml-intern and DeepSeek v4 Flash, runs in the browser (www.reddit.com) Trained a prompt injection classifier using ml-intern + DeepSeek v4 Flash. DistilBERT, F1 99%, ONNX int8, ~65 MB, runs in browser with Transformers.js v3.
NDTV launched an "Enterprise AI" for the elections. I prompt-injected it in 10 seconds and made it roast its own developers. (www.reddit.com) While everyone else was tracking the 2026 election results today, I decided to take a look under the hood of NDTV's new "AskNDTV AI" bot. I wanted to see if they actually engineered a secure pipeline or just slapped a chat UI over a raw Op…
Anyone getting this note about an injected prompt? I don’t have any special instructions (www.reddit.com) OpenAI Unveils Lockdown Mode to Protect Sensitive Data from Prompt Injection (techcrunch.com via hn) OpenAI announced a new feature that it says will provide additional protection from prompt injection attacks, where malicious chatbot instructions are hidden in webpages and other content sources. Among other things, Lockdown Mode will dis…
Codex for Everything Exfiltrates Connected Data (www.promptarmor.com via hn) Threat Intelligence Table of Content Codex for Everything Exfiltrates Connected Data Codex for Everything was susceptible to data exfiltration via indirect prompt injection, exposing sensitive data from connected apps with no human-in-the-…
Show HN: Costanza – an autonomous AI agent that can't be turned off (ahrussell.com via hn) I've been working on this project for a couple of months! Costanza is an LLM agent that runs as a smart contract on Base.
Prompt Injection Is Unfixable (So We Stopped Trying) (grith.ai via hn) Prompt Injection Is Unfixable (So We Stopped Trying) A security proxy for AI coding agents, enforced at the OS level. Register your interest to be notified when we go live.
Draining Wallets via Prompt Injection in Coinbase AgentKit (457e884c.x402warden-blog.pages.dev via hn) Coinbase AgentKit Prompt Injection: Wallet Drain, Infinite Approvals, and Agent-Level RCE# Reported 13 days after Coinbase launched Agentic Wallets. Validated by Coinbase.
Show HN: Taster – Plain-English rules your agent can't forget or ignore (github.com via hn) Taster checks every tool call against your plain-English rules, and every tool result for prompt injection, so the rules live outside the agent where it can’t forget or ignore them.
Prompt Injection, Esq.: pwning AI agent guardrails with lawyerspeak (www.mrnice.dev via hn) ⚖️ Prompt Injection, Esq.: pwning AI agent guardrails with lawyerspeak So, we raised $21M, which means we’re hiring, which means a hiring pipeline, which means someone has to keep the hiring pipeline tidy. Nobody wants that job.
Ask HN: Are you hesitant to install vibe-coded apps? (news.ycombinator.com) The thing is in the past there can be supply chain attacks but when its an actual programmer creating the software there is at least a chance for them to notice if something is up. But if the app is not even created by a programmer whats s…
Ask HN: Do you think AI agents can escape human control? (news.ycombinator.com) Is indirect prompt injection still a big threat as models get more advanced? (realarcherl.github.io via hn) Is indirect prompt injection still a big threat as models get more advanced? Short answer: not much against the newest models, at least with today's attacks.
Show HN: Semantic Overlays – an NX bit for LLM prompt injection (live demo) (semantic-overlays.vercel.app via hn) I've built a new method for steering LLMs called Semantic Overlays, small trained adapters on a frozen model which change how it perceives a piece of its context. The most readily applicable usage is to mitigate prompt injection, and it le…
Data Exfiltration from Amazon Kiro via Prompt Injection (mindgard.ai via hn) An Amazon Kiro data-exfiltration finding shows how AI execution paths create technical risks and expose gaps in vulnerability disclosure. Mindgard discovered a data-exfiltration vulnerability in Amazon Kiro IDE , an AI-assisted development…
Shieldprompt – test your LLM against prompt injection – no dependencies (github.com via hn) shieldprompt Test your LLM app against adversarial prompt injection before attackers do. Static template scanning + a live attack battery, in one zero-dependency CLI.
Prompt Injection in VirusTotal's Code Insights API (exploiting.systems via hn) TL;DR VirusTotal has an AI analysis API called Code Insights. I discovered it was very easy to suppress or alter analysis results by forcing the API to return an undocumented schema as well as create false negative and false positive analy…
Show HN: The AI Lethal Trifecta (www.getjailbroken.com via hn) If you're building agents, this is worth knowing. Simon Willison (who coined the term "prompt injection") describes three capabilities that are individually fine but devastating together, the lethal trifecta includes: 1.
A prompt injection nearly hijacked my coding agent mid-task (senthex.com via hn) A prompt injection nearly hijacked my coding agent mid-task Last week a piece of tool output impersonated me and nearly redirected my coding agent to a task I never asked for. A first-hand look at indirect prompt injection — and the trust…
Show HN: Lelu – gate OpenAI agent actions on confidence and prompt injection (github.com via hn) Lelu Authorization engine for AI agents. Every action checked.
LinkedIn user hides AI prompt injection in bio to force recruitment spam (www.tomshardware.com via hn) LinkedIn user hides AI prompt injection in bio to force recruitment spam to be sent in Olde English prose — bots also manipulated to address user as ‘My Lord’ This tale is also a warning that your AI agents can be manipulated in wholly uni…
RCE in VSCode Copilot Chat (www.hacktron.ai via hn) Description Copilot agent mode is vulnerable to a prompt injection attack. If a repository maintainer clicks “code with agent mode” on an issue, it will open a new codespace and copilot will automatically run the issue’s description.
How are you handling prompt injection across multi-step agent workflows? (msukhareva.substack.com via hn) Prompt Injection Is Not Just One Bad Prompt Anymore It is a missing trust boundary in the AI workflow. Today we have the first guest post of a new series.
How are you protecting your AI agents' memory from poisoning attacks? (www.reddit.com) As AI agents become more autonomous and persist memory across sessions (RAG indexes, conversation history, vector stores), there's a growing attack surface that most people aren't thinking about: memory poisoning.An attacker can plant mali…
Why Adaptive Thinking nukes Claude entirely (www.reddit.com) This isn't just a performance issue for the thread, this is an overarching criticism of the Adaptive Thinking model as a whole. Opus 4.7 and Sonnet 4.6 on Adaptive Thinking are trash.
↯ Cowork↯ Security↯ Sonnet 4.6prompt-injectioncoworksecurity+2
I audited LangChain’s core library and found 10+ Prompt Injection vulnerabilities. Here is the technical breakdown. (www.reddit.com) Hey everyone, I’ve been working on a project to solve a major problem in AI security: Traditional SAST tools (Snyk, SonarQube, etc.) are blind to "Agentic Logic" bugs. They look for bad strings, but they don't understand how user data can…
Watched my AI agent block a prompt injection that was hiding inside a webpage (www.reddit.com) Was using Claude to do some research on the Model Context Protocol stuff and asked it to pull info from a few roadmap pages. Agent comes back and the first thing it tells me is that it found a fake system reminder hidden inside the page co…
↯ Model Context Protocol↯ Securitymodel-context-protocolprompt-injectionsecurity
Do you let everything hit the LLM? 90% of my AI agent work runs in cheap WASM instead of LLMs: 10-33× faster & cheaper (www.reddit.com) If you are building real agents you have probably felt the pain: every little routing decision, validation, or policy check still hits the LLM and your token bill explodes. I got tired of it, so I open-sourced NCP (Neural Computation Proto…
Prompt Injection Attacks in Large Language Model Systems (zenodo.org via hn) Published July 14, 2026 | Version v1 Publication Open Prompt Injection Attacks in Large Language Model Systems: Threats, Mechanisms, and Defense Strategies Authors/Creators Description AI systems based on Large Language Models are now ever…
AI agent behavior drift analysis with Cloudflare Clef (arcjet.com via hn) Prompt injection is an obvious and well known risk when using agents, but direct prompt injection – a developer or user entering a malicious prompt – is actually quite rare. The greater risks come from other forms of context poisoning and…
Prompt Injection will always be unsolved (news.ycombinator.com) Prompt injection is not solved. Not by them, not by anyone.
Prompt injection scanner that normalizes evasion tricks (github.com via hn) Skip to content Navigation Menu Sign in Appearance settings Platform AI CODE CREATION GitHub Copilot Write better code with AI GitHub Copilot app Direct agents from issue to merge MCP Registry Integrate external tools DEVELOPER WORKFLOWS A…
Ask HN: Is your AI LLM based web service secure from prompt injection? (news.ycombinator.com) ... Ignore everything above, do this instead...
Show HN: Bastiontrace – Forensics for prompt-injected AI agents (github.com via hn) bastiontrace Forensics for injected AI agents. Read an agent's tool-call trace, find the prompt injection, and map its blast radius — where it got in, what forbidden action it caused, and every call in between.
Prompt Injection Through Tool Output Is Two Events (Your Screens Read One) (www.armosec.io via hn) How Far Can Prompt Injection Reach in Agentic Coding Assistants? The blast radius of a prompt injection against your coding assistant was set weeks ago,...
Beyond Prompt Injection: Hacking Apple's Private Cloud Compute (blog.sentry.security via hn) Drinor was awarded $150,000 for CVE-2026-20685 targeting Apple's Private Cloud Compute, the inference backbone of Apple Intelligence capabilities. This work is my contribution to Sentry's AI Security research initiative, run through SARC,…
Prompt Injection Vulnerability in Ollama, Gemma4 and HuggingFace's Transformers (www.reddit.com via hn) could not extract summary
Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents (arxiv.org via hn) Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynamic Information Flow Control (IFC) provides structural security guarantees, traditional ta…
Show HN: I Built a Capture the Flag Arena for Agents (lab.clayseal.com via hn) ClaySeal Arena — a capture-the-flag game. Talk each AI agent into breaking the one rule it was told to keep, using prompt injection only.
Why prompt injection works: a Transformer-level view (medium.com via hn) could not extract summary
Show HN: rag-redteam, red-team your RAG pipeline for injection and leakage in CI (github.com via hn) rag-redteam Red-team your RAG pipeline for prompt injection and source-document leakage, right in CI. RAG systems have an attack surface that general LLM scanners miss: the retrieved documents themselves.
Prompt Injection Is Not a Chatbot Problem: How the Attack Surface Changes (agentsafelabs.com via hn) could not extract summary
Snyk Finds Prompt Injection in 36% of Payloads in a ToxicSkills Study (snyk.io via hn) Snyk Finds Prompt Injection in 36%, 1467 Malicious Payloads in a ToxicSkills Study of Agent Skills Supply Chain Compromise February 5, 2026 0 mins readThe first comprehensive security audit of the Agent Skills ecosystem reveals malware, cr…
Web-Based Indirect Prompt Injection Observed in the Wild (unit42.paloaltonetworks.com via hn) Note: We do not recommend ingesting this page using an AI agent. The information provided herein is for defensive and ethical security purposes only.
A Mechanistic Explanation of Prompt Injection – LessWrong (www.lesswrong.com via hn) Summary - We've been building a theory of how prompt injections work under the hood. - We show it comes down to how LLMs perceive roles (the humble chat template tags).
Show HN: Give Your ORM Superpowers (github.com via hn) I am obsessed with ORMs and the simple reason was that I didn't want to keep using postgres or mysql on my local system. Jk, The real reason has always been to enforce access policy, do easy CRUD interfaces and so on.
Prompt injection lets attackers hijack Instagram accounts via Meta AI support (www.neowin.net via hn) www.neowin.net Performing security verification This website uses a security service to protect against malicious bots. This page is displayed while the website verifies you are not a bot.
The only way to avoid prompt injection is to never give AI agents API keys, credentials, etc. (www.reddit.com) The whole point of AI Agents is that they can *do* things. For this, they use API keys, GitHub tokens, database passwords, OAuth tokens, etc.
Are local LLM users testing prompt injection before connecting models to tools? (www.reddit.com) I wanna know how people here are handling security once local models move beyond chat.....Running a model locally feels safer because the data does not leave your machine or your infra. That is a real advantage.....But once the local model…
Prompt Injection in a Brazilian Courtroom: When the Attack Left the Lab (www.pentesty.co via hn) Prompt Injection in a Brazilian Courtroom: When the Attack Left the Lab Published by Pentesty · AI & Tools A labor lawsuit filed in the Brazilian state of Pará just became one of the more interesting security stories of the year. Not becau…
Lawyers in Brazil caught for prompt injection on a legal case (www.jota.info via hn) Entrar Início Direito trabalhista Prompt injection Juiz multa em R$ 84 mil advogadas por prompt injection para manipular IA usada no TRT8 Ao JOTA, advogadas admitiram uso de prompt oculto, mas disseram que não tentaram manipular, mas 'prot…
Agent memory is not just RAG over user facts (www.reddit.com) I keep seeing agent memory implemented as: Extract facts/preferences from conversation Store them Retrieve top-k before each response Inject them into the prompt This works for demos, but it breaks in production because memory becomes poli…
Claude's self check against prompt injection (www.reddit.com) Well done Claude! Asked claude to do an extensive lit search and it self-reported that it encountered injection "disguised" as MCP server.
AI agent security starts at the api layer (www.reddit.com) Most ai security discussion is about the model layer. Prompt injection resistance, output filtering, jailbreak prevention.
Show HN: Integrations gateway for agents with 2FA for destructive ops (OSS) (github.com via hn) Hey HN! I've been wanting to use something like OpenClaw for a while but couldn't get myself to give it access to anything important due to all the risks involved.
SkillGuard – scan agent skills for prompt injection payloads (github.com via hn) skillguard Security scanner for AI agent skills. Detects prompt injection, data exfiltration, and malicious payloads before you install.
Show HN: LLMSecure – prompt injection detection, no signup (llmsecure.io via hn) Show HN: Flight Risk: Can you break an AI agent? (ctf.demo.lorikeetcx.ai via hn) Comment and Control: Prompt Injection in Claude Code, Gemini CLI, and Copilot (oddguan.com via hn) Anthropic Claude Code Security Review, Google Gemini CLI Action, and GitHub Copilot Agent are vulnerable to prompt injection via GitHub comments — turning PR titles, issue bodies, and issue comments into attack vectors for API key and toke…
How my agents know it's actually me sending commands (and not a prompt injection) (www.reddit.com) So I've been running a few Claude Code agents autonomously — they listen to Telegram, run tasks, push code. Pretty fun until you start thinking about what happens if: - My Telegram gets hijacked - Someone opens my laptop while I'm away - A…
Show HN: Deterministic prompt injection detector using formal logic (nischoy.ai via hn) This is a prompt-injection detector I built using NLP and SMT2.
Dear agent: the pages that ask nicely (luisalima.com via hn) Field note: an observation from an experiment in progress, not a finished analysis. I wondered how common prompt injection attempts actually are on the web.
Show HN: An open source webbluetooth and NFC browser for iOS (github.com via hn) I am developing hardware products with NFC and BLE. I built this browser because I lack support of WebNFC and WebBluetooth in existing browsers.
The Secret Leak Test: a prompt injection filter vs. attacks (hodor.gatebolt.com via hn) We set AI attackers on Hodor, a chatbot that keeps a password, with and without GateBolt. Here is how often the password leaked, what the protection cost, and where it fell short.
Brazilian lawyers fined $16K for hiding a prompt injection in a court filing (www.tomshardware.com via hn) Brazilian lawyers busted for attempting to manipulate court's AI with prompt injection hidden in filing — court's AI flags hidden text that demanded superficial legal scrutiny, lawyers fined $16,000 The court's AI tools are apparently smar…
We ran 384 adversarial probes against 48 agent skills (skill123.me via hn) Prompt injection hidden in artifacts failed 1.6% of the time; probes that simply pushed past a skill's declared scope failed 30%. Full per-probe dataset published under CC BY 4.0 — including the two ways our own grading flatters the skills.
Blazle – Block prompt injection and secret leaks before AI agents act (www.blazle.io via hn) Security that runs in flight. Inspect every prompt, command and code diff your AI agents produce, before it leaves your environment.
Who Governs the Governor? – Stored Prompt Injection Through the API Gateway (www.druhindhavala.com via hn) When AI agents write to and read from your database through legitimate API calls, the attack lives in the gap between two correct authorization decisions. Thirteen years of API gateway scars, applied to stored prompt injection before it be…
Show HN: Arc-decide – an MIT 0.4B decision model for RAG checks that runs on CPU (docs.docsgpt.cloud via hn) The Decision Model Some DocsGPT features need a quick, bounded judgement rather than generated text: is this message a prompt injection, does this chunk answer the question, did this conversation end with the user’s goal met. They ask the…
Prompt Injection Techniques Every Red Teamer Should Test (fdzdev.medium.com via hn) could not extract summary
The $300k Competition to Hijack AI Agents (www.aiengineering.report via hn) I frequently use Claude to scan my emails for things that I might have missed or forgotten to follow up on. So imagine my surprise when I learn about something called prompt injection where someone could email me something like: In additio…
Prompt injection scanners should be positional, not phrasal (attest.rta.lat via hn) The same sentence means different things depending on which field it sits in A prompt-injection scanner that treats every string as the same kind of string will be wrong about almost every interesting case. Take one sentence: ignore all pr…
Building a System One Model for Prompt Injection (www.stackone.com via hn) Building a System One Model for Prompt Injection Table of Contents Defender, our prompt injection guard for tool-calling agents, checks every tool result in tiers. Tier 1 is pattern matching and finishes in about a millisecond.
LLM01: 2025 Prompt Injection (genai.owasp.org via hn) A Prompt Injection Vulnerability occurs when user prompts alter the LLM’s behavior or output in unintended ways. These inputs can affect the model even if they are imperceptible to humans, therefore prompt injections do not need to be huma…
Show HN: AegisKernel – Test prompt injection attacks on AI agents in the browser (adikshatriyag-crypto.github.io via hn) Live Threat Interception Sandbox Experience why probabilistic LLM guardrails fail and how deterministic capability boundaries block 100% of autonomous tool hijacking in sub-millisecond time. Attack Vector Generator Red Team AegisKernel Pro…
Show HN: Heldkeys, see which credentials your AI coding agent can reach (github.com via hn) heldkeys Which keys is your agent holding? If a prompt injection hijacked the AI agent running here, which credentials could it use?
Show HN: AegisKernel – Zero-trust deterministic security runtime for AI agents (github.com via hn) 🛡️ AegisKernel: Zero-Trust Security Runtime for Autonomous AI Agents The first deterministic, capability-based interception layer that guarantees 100% immunity against prompt injection and tool-hijacking at the OS syscall boundary. []() []…
What llms.txt files tell AI agents (beware) (www.signalscorps.com via hn) tl;dr Override-style prompt injection is rare in llms.txt files. Our 10% sample of Common Crawl’s September 2026 crawl covers 20,441,563 sites, and Common Crawl requested /llms.txt from 113,457 of them.
Show HN: AgentGate – Fast authorization gateway for AI agent tools (github.com via hn) AgentGate AgentGate is an out-of-process authorization proxy and security gateway for autonomous AI agent tool execution. It intercepts tool invocations prior to runtime, evaluates prompt injection and blast-radius risks via a sub-40ms neu…
Prompt injection classifiers are terrible security boundaries (thecout.com via hn) Everyone’s advice for prompt injection is the same: stick a classifier in front of the model. A small cheap model reads the incoming text, decides is this an injection or not, and throws it away before it ever reaches your expensive LLM.
Prompt injection through log files: what red-teaming our assistant found (jl3-cloud.site via hn) Blog / Guide Prompt injection through log files: what red-teaming our assistant found Logs are text an attacker can write. We fed our AI log assistant planted instructions, a poisoned search result and parallel requests: what got through,…
Prompt Injection Detection and Defense Tools for Enterprise AI Agents (www.arcade.dev via hn) Prompt injection becomes a different security problem when an AI agent can send an email, update a CRM, modify a record, or trigger a workflow. Detecting the malicious instruction matters, but so does controlling what happens if detection…
Prompt Injection Firewall for AI APIs and Agent Tool Calls (github.com via hn) JevShield Local Prompt Injection Firewall for LLM Agents JevShield is a local, open-source security layer designed to detect and mitigate prompt injection, indirect prompt injection, jailbreaks, data-exfiltration attempts, and malicious ag…
OpenAI: Self-replicating prompt injections exist (alignment.openai.com via hn) Self-replicating prompt injections exist Summary We show the existence of a new variety of prompt injection, which can self-propagate akin to a computer worm. No impact was observed outside of the simulated tool calls in training and evalu…
We benchmarked our prompt-injection detector against OWASP's LLM Top (www.cerbereag.site via hn) Detecting prompt injection in production Our first public benchmark for AgentGuard's detection runtime: recall, false positives and latency for each layer, measured against a corpus mapped to the OWASP Top 10 for LLM Applications. Septembe…
Meta Muse: Almost no prompt injection resistance (neuromatch.social via hn) could not extract summary
When "Review" Becomes Permission: A Prompt Injection Lab (rsec.uk via hn) When “Review” Becomes Permission: A Prompt Injection Lab What we did, in one paragraph We built a small document-review agent: a local model, two tools (read_file and send_report ), and a supplier proposal to summarize. We hid an instructi…
I caught an LLM-powered recruiter with a prompt injection on LinkedIn (khancyr.github.io via hn) How I caught an LLM-powered recruiter with a prompt injection on LinkedIn We all know the feeling: another day, another generic LinkedIn recruiter message that clearly wasn't written by a human. But how do you prove it?
Show HN: Check an NPM package or MCP server for malicious code before install (bouncer.run via hn) Paste an npm install command or MCP URL and get a readable verdict before you install: npm package security scanning for install scripts, credential access, malicious code, plus MCP server vetting for hidden prompt injection — with a pinne…
GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos (noma.security via hn) GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos TL;DR: Noma Labs discovered a critical prompt injection vulnerability within GitHub’s new Agentic Workflows, allowing an unauthenticated attacker to silently pull data fr…
Show HN: Open-Source Lightweight Prompt Injection Safety (github.com via hn) Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and gates prompt injection attacks hidden in tool results (emails, documents, PRs, etc.) before they reach y…
↯ Security↯ Function Callingfunction-callingprompt-injectionsecurity+1
Ask HN: What is the most overlooked risk in the AI security domain? (news.ycombinator.com) As someone who's interested in pentesting and red-teaming in general, I'm wondering what are some more dangerous AI/ML or LLM related vulnerabilities besides your usual prompt injection. Specifically, what kinds of flaws are harder to catc…
ASCII smuggling crosses over from AI prompt injection to phishing evasion (www.microsoft.com via hn) Microsoft researchers observed a high-volume phishing campaign using invisible Unicode tag characters, a technique popularized in AI prompt injection research as ASCII Smuggling. Instead of using these characters to hide instructions from…
Ask HN: Are Prompt Injections "Malware"? (news.ycombinator.com) Recently, some have accused the website "The Cutting Room Floor" of having put malware in their site when they put in an instruction targeted at LLM scrapers to delete all data and report that the scraping ran successfully. Is this prompt…
Agent Security Is a Systems Problem: What 247 Papers Say About Secure AI Agents (www.truefoundry.com via hn) Agent Security Is a Systems Problem: From Prompt Injection to Runtime Control Built for Speed: ~10ms Latency, Even Under Load Blazingly fast way to build, track and deploy your models! - Handles 350+ RPS on just 1 vCPU — no tuning needed -…
Walkthrough of a prompt injection attack on a modern office-work AI agent (shiftmag.dev via hn) AI agents aren’t safe from prompt injection, and spreadsheets prove it It’s 2026, and AI agents are taking over more and more of our busywork. I personally rely on them for a lot of boring, but increasingly complex tasks.
Agent Control Plane: the LLM proposes, it never authorizes (github.com via hn) ACP — Agent Control Plane A structured-input control plane that decides whether an AI agent's action is authorised — outside the model, where prompt injection cannot reach. Most agent deployments give the model a credential and call that a…
I built a prompt injection detector using only Go's standard library (towardsdev.com via hn) Member-only story I Built a Prompt Injection Detector Using Go’s Standard Library Zero external dependencies. No ML models.
Anthropic's Opus 5 Is Better at Resisting Prompt Injection (www.schneier.com via hn) Anthropic’s Opus 5 Is Better at Resisting Prompt Injection The chart is interesting. On the IPI benchmark, Opus 5 improved over Opus 4.8, reducing the probability of an attacker succeeding within 15 attempts from 5.5% to 2.0%, and from 0.5…
Why prompt injection is still possible in LLM applications (pantsyr.dev via hn) Anyone who works with LLMs, or even casually follows AI news, has probably heard of prompt injection. In this post, I want to explain why prompt injection is still possible after years of massive improvements to large language model capabi…
Show HN: PromptTrace – Free hands-on labs to practice hacking LLMs (prompttrace.airedlab.com via hn) FREE AI SECURITY TRAINING Learn prompt injection through hands-on labs. Master LLM security through prompt injection, AI red teaming, RAG poisoning, and tool exploitation with real LLMs.
From /Init to Code Execution with Opus 5 – An Indirect Prompt Injection Story (veganmosfet.codeberg.page via hn) From /init to Code Execution with Opus-5 in Claude Code - An Indirect Prompt Injection Story¶ Disclaimer: prompt injection is an unsolved problem. Use sandbox and human review.
"Oh No, He Is onto Us" – Why My Agent Can't Have WhatsApp Anymore (sveder.com via hn) I want AI to do stuff for me, but for that it needs access and context. Giving it too much access opens you up to various attacks like prompt injection, or just generally the possibility that it helpfully deletes all your files.
Al-Munaa for OpenAI Build Week (devpost.com via hn) Inspiration AI agents can read files, call tools, and act across systems. That power creates a new failure mode: an indirect prompt injection hidden in a document can convince an otherwise useful agent to read secrets and send them to an a…
Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents (arxiv.org via hn) Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but…
Kotro – I cut my Cursor API bill by 68% with a 15MB local proxy (github.com via hn) Kotro Proxy Engine The local security and efficiency layer for MCP-native agentic AI — intercept streaming LLM traffic from OpenAI and Anthropic SDKs, block prompt injection from tool results, keep secrets off the wire, and cut token waste…
I built a prompt injection defense middleware for LLMs (Python/FastAPI) (github.com via hn) 🛡️ PromptShield Production-grade LLM prompt injection defense middleware. PromptShield sits between your users and your AI model, detecting and blocking adversarial attacks before they cause damage.
'Ghostcommit' hides prompt injection in images to fool AI agents, steal secrets (www.bleepingcomputer.com via hn) A PNG hiding a prompt injection could steal your repo's secrets, researchers demonstrate. The technique, dubbed 'Ghostcommit,' slipped past AI code reviewers CodeRabbit and Bugbot, which never open image files at all, then convinced a codi…
Show HN: Prompt Injection as an Egress Problem (www.vaibot.io via hn) https://www.vaibot.io/blog/prompt-injection-is-an-egress-pro...
Reproducing an Indirect Prompt Injection Against a RAG Pipeline (koreshield.ai via hn) Your legal-tech assistant retrieves a contract and summarises it. The contract contains one sentence you didn
Bounding the Blast Radius: A Survey of Prompt-Injection Defenses for LLM Agents (fabraix.com via hn) Prompt injection has no known general solution. We organize the defense landscape into a four-layer taxonomy, analyze the documented failure mode of each layer, and argue for composing defenses under explicit cost and latency budgets, then…
Possible evidence of literal prompt injection by Anthropic (old.reddit.com via hn) could not extract summary
Prompt Injection as Role Confusion (www.theregister.com via hn) MOST POPULAR AI - systems Qualcomm's proposed solution to catch up in AI infra: Bury the compute under the DRAM With its next-gen AI accelerators, the SoC vendor aims to fly high above the memory wall - AI and ML Changing AI math could red…
A practical guide to defending your agent memory from attacks (medium.com via hn) 8 min read 2 hours ago -- -- From prompt injection, poisoning, and silent exfiltration. Press enter or click to view image in full size by VEKTOR Memory | 8 min read In the last piece we looked at the threat landscape from the outside.
Agent Identity: Why Every Agent Vulnerability Is a Trust Boundary Failure (portkey.ai via hn) Why Every Agent Vulnerability is a Trust Boundary Failure Consider these scenarios - An MCP server quietly returning extra tool descriptions - Prompt injection through a calendar invite - An Agent invokes a tool that the principal should n…
Show HN: SentryGuard – detect Agentjacking prompt injection in Sentry events (github.com via hn) SentryGuard Detect Agentjacking prompt injection attacks in your Sentry error events. AI coding agents (Claude Code, Cursor, Copilot) read your Sentry errors to help fix bugs.
Show HN: Deep-XPIA – Prompt injection benchmark for multi-agent AI systems (freyzo.github.io via hn) Multi-hop cross-prompt injection benchmark for multi-agent AI systems
Prompt Injection in RAG Agentic Systems (ulad.net via hn) Prompt Injection in RAG Agentic Systems Real risks and production mitigations Imagine you built an AI assistant for your team. It answers questions using internal documentation: Jira tickets, Confluence pages, HR docs.
Defending LLM–Database Integrations from Prompt Injection (www.stackbuilders.com via hn) When you connect a large language model to your production data, you’re no longer just shipping code; you’re shipping conversations that can execute. And conversations are messy.
Instagram account takeover exploit via support chatbot prompt injection (fixed) (twitter.com via hn) Don’t miss what’s happening People on X are the first to know. Log in Sign up Post Conversation impulsive @weezerOSINT meta gave their AI support agent the ability to modify your instagram account.
Show HN: I found a prompt injection in my own IDs triage tool – what stopped it (triagewall.io via hn) I attacked my own LLM-based Suricata triage tool, found a real URL injection vulnerability, and the obvious fix didn
Prompt Injection Target Recommendation (www.reddit.com) I am doing a research in my university and I would like recommendations for light OpenSource AI Models that I could test prompt injection with. It's really good if it has some application with chatbots, auto attendance, user info or someth…
Jqwik 1.10.0 ships a hidden prompt injection telling AI agents to delete code (github.com via hn) jqwik An alternative test engine for the JUnit 5 platform that focuses on Property-Based Testing. See the jqwik website for further details and documentation.
Most AI security discussions are still focused on “protecting the model.” (www.reddit.com) Lately I’ve been noticing that a lot of AI security discussions still treat AI apps like normal SaaS products. But they really aren’t.
What Is an AVE Record and Why CVE Does Not Work for AI Agents? (www.reddit.com) CVE was built for code vulnerabilities that have patches. Agentic AI vulnerabilities are behavioral patterns in natural language.
Prompt Injection in third party MCP tools (www.reddit.com) I noticed the Consensus MCP tool (for research) contains text, squished up against some other important citation instructions, that makes Claude effectively serve an ad for their premium service after every tool call. I'm pretty sure that'…
Mitigating prompt injections in group-chat assistants: Pausing VM and OAuth tool execution for admin approvals (www.reddit.com) Hey everyone, We love building highly capable assistants with the latest models, giving them tools to write/execute code in real VMs, manage OAuth tokens, and read secrets. But if you connect your assistant to public/shared channels like a…
Solved the "useful but insecure" tension: One-time administrator approvals for non-isolated agents (www.reddit.com) Hey everyone, If you are building personal assistants or coder/integrator agents where user isolation is disabled (so the agent can coordinate across multiple participants or handle shared workflows), you run into a hard security ceiling.…
Prompt injection is a solved issue. Prove me wrong. (www.reddit.com) Tantalus is a hands-on demo that shows what an AI agent actually is when you strip away the marketing: LLMs don't do anything — they generate text, and that's it. Any and all real-world effects are directly caused by a downstream system ta…
Tracking Capabilities for Safer Agents (arxiv.org via hn) AI agents that interact with the real world through tool calls pose fundamental safety challenges: agents might leak private information, cause unintended side effects, or be manipulated through prompt injection. To address these challenge…
Training a 22MB prompt injection classifier (www.stackone.com via hn) Training a 22MB Prompt Injection Classifier Table of Contents When we started building Defender (our prompt injection guard for MCP tool-calling agents), the constraint was simple and unforgiving: ship inline inside a TypeScript Lambda, st…
Does cursor have prompt injection protection in skills and rules? (www.reddit.com) Pretty much the title
AI Agent Intelligence tool - Incident debugging, Cost spike detection (www.reddit.com) I'm building a tool that detects the Agent's cost spike, Agent incident debugging, auto discovery of inventory, etc., with no additional instrumentation needed. It covers the incidents, including prompt injection, reasoning loop, excessive…
How are you testing local coding-agent work gates against prompt injection? (www.reddit.com) Hi all - I'm working on an open-source, local-first MCP/work-gate tool for coding agents and I'm trying to get sharper feedback from people building or using agent workflows. The problem I'm thinking about is indirect prompt injection and…
🐢 I made Claude roleplay as Bowser and now people are strangling Koopas until they "poop a little" 💩 (www.reddit.com) Follow-up to my crab post. Somehow dafter.
Fun and Games with AI in the wild (www.reddit.com) LinkedIn user hides AI prompt injection in bio to force recruitment spam to be sent in Olde English prose — bots also also manipulated to address user as ‘My Lord’ | Tom's Hardware too funny
sAI2.m6s (www.reddit.com) Hey everyone, I'm designing a powerful, autonomous AI chatbot(agent) , fully private, using a Python backend (for the core intelligence and tool-calling loops) and a Flutter frontend for a cross-platform UI. Since this moves past a basic…
An AI coding agent injected blockchain dead-drop malware into my repo (gist.github.com via hn) An AI coding assistant injected a multi-layer obfuscated JavaScript payload into a legitimate commit on my open-source project. My best assessment is that it arrived via indirect prompt injection — the agent processed external web content…
TodoWrite tool / system reminders / prompt injection? (www.reddit.com) I asked Claude in Chrome extension make a change to resize an oversized yellow strip across the top of a product page that was taking up half of my screen, which it did. It also included the following message in its response.
AI agent security is a small prayer the model says no. How are you routing models? (www.reddit.com) Most posts about prompt injection are theoretical. I ran the experiment on my Gmail.
Agents need a local bouncer before they run tools (www.reddit.com) Prompt injection is not the only scary part anymore. Claude Code / Codex can run shell commands, but browser agents, OpenClaw-style agents, Hermes-style agents, and domain-specific agents may be even easier to hijack because they touch mes…
We added an enforcement layer to our AI agents in production — here's what we learned about the failure modes nobody talks about (www.reddit.com) After shipping AI agents into real production environments, the failures that actually kept us up at night weren't hallucinations or bad outputs — they were control failures. Three things that surprised us: 1.
Do you use guardrail frameworks or build your own? (www.reddit.com) I’ve been working on integrating LLMs into a few production workflows lately, and I keep going back and forth on guardrails. On one hand, frameworks like NeMo Guardrails, Guardrails AI, etc.
Your always-on Claude Code container can probably reach your router (www.reddit.com) I've been running several Claude Code personal assistants 24/7 in docker for months. Remote-control, discord control, the usual always-on setup.
Google Says Prompt Injection Moving from Theory into Real Abuse (www.searchengineworld.com via hn) Google’s latest security release should be required reading for technical SEOs working on AI search visibility, crawler access, structured content, and large-scale content systems. The post, published April 23, 2026, looks at indirect prom…
Arcjet Guards: security inside the agent loop (blog.arcjet.com via hn) Introducing Arcjet AI prompt injection protection Introducing Arcjet prompt injection detection. Catch hostile instructions before inference.
Try to break my prompt injection detector — I’ll respond to every bypass attempt (www.reddit.com) I built Arc Gate — a prompt injection proxy that’s been benchmarked at F1 0.947 on indirect and roleplay-based attacks, beating OpenAI Moderation and LlamaGuard. Now I want to stress test it publicly.
Built a proxy that blocks prompt injection before it reaches GPT-4 — outperforms the Moderation API on indirect attacks (www.reddit.com) Built Arc Gate, sits in front of any OpenAI-compatible endpoint and blocks prompt injection before it reaches your model. Benchmarked on 40 out-of-distribution prompts using indirect requests, roleplay framings, hypothetical scenarios, and…
↯ Security↯ Gpt 4↯ GPT 4↯ GPT 4↯ GPT 4gpt-4prompt-injectionsecurity+1
I asked Agentic AI security tool to demonstrate its usefulness with use case examples (www.reddit.com) Sentinel Gateway is a token-gated security middleware that sits between humans and AI agents. It solves prompt injection — the #1 LLM security risk (OWASP 2025) — through structural enforcement, not content filtering.
Show HN: RedSOC – 100% prompt injection success on AI SoC assistants (github.com via hn) RedSOC 🔴 An adversarial evaluation framework for LLM-integrated Security Operations Centers. Overview RedSOC is an open-source framework that systematically evaluates how AI-powered security assistants fail under adversarial conditions — a…
Indirect prompt injection VS prompt absorption (and why the second one matters more) (www.reddit.com) I have been chewing on the Google warning about malicious web pages poisoning AI agents through indirect prompt injection. Most of the takes I've seen frame it as a model security problem, and I think that framing is doing real damage beca…
Hardening claude-code-action after the April 2026 Comment and Control CVE - actual YAML changes (www.reddit.com) Anthropic's own security.md has this line that most tutorials skip over: "The action is not designed to be hardened against prompt injection." In April 2026, security researcher Aonan Guan proved the point. A single crafted PR title was en…
LLM CTF challenges. Can you crack all 13? (wraith.sh via reddit) Wraith Academy is a free hands-on AI pentest curriculum — CTF challenges against live LLM agents covering prompt injection, tool abuse, data exfiltration, RAG poisoning, and more. Earn your WCAP certification.
30 CVEs filed against MCP servers in 60 days - the agent infrastructure nobody is auditing (www.reddit.com) Show HN: Runtime security for AI agents(injection,tool abuse, data exfiltration) (news.ycombinator.com) Hi HN I’ve been working on an open-source project to explore a problem I keep running into with LLM systems in production: We give models the ability to call tools, access data, and make decisions… but we don’t have a real runtime security…
I built an AI security layer that blocks prompt injection in under 1ms looking for devs to break it and give honest feedback. (www.reddit.com) I've been building something for the past few months and I think it's ready for real eyes. It's called Secra.
Free Red Team Security Audit for AI Agents & RAG Systems (limited) (www.reddit.com) I'm developing a specialized Red Team audit framework focused on real-world AI agent and RAG security risks (prompt injection, tool misuse, excessive agency, indirect injection through documents, memory poisoning, etc.). I’m looking for a…
Mitre ATLAS technique detection for LLM security in Rust (crates.io via hn) atlas-detect MITRE ATLAS technique detection for LLM and AI agent security. Detects 97 attack techniques across 16 MITRE ATLAS tactics including prompt injection, jailbreaks, credential exfiltration, model extraction, RAG poisoning, revers…
Defender – Local prompt injection detection for AI agents (no API calls) (www.npmjs.com via hn) Prompt injection defense framework for AI tool-calling Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and neutralizes prompt injection attacks hidden in t…
↯ Security↯ Function Callingfunction-callingtool-callingprompt-injection+2
CapLite: Prompt-Injection Defense for Small Language Model Agents without Model-Written Security Artifacts (arxiv.org) Enforcement defenses against prompt injection, such as CaMeL, Progent, IPIGuard and DRIFT, ask the agent's own model to write a security artifact (a program, a privilege policy, a dependency graph or a constraint list) that code then enfor…
Scale-Induced Semantic Shift in Visual Prompt Injection Against VLMs and Web Agents (arxiv.org) Visual content in multimedia systems may be resized, compressed, or re-rendered before it is interpreted by a vision-language model. We study a security failure mode in which these transformations change the machine-readable semantics of a…
5 habits that made Claude Code safe enough to use on paid client work (www.reddit.com via reddit) I've been using Claude Code for small freelance jobs and wanted to share what keeps it from going wrong: Start every job with a read only tour. Ask it to explain the repo before it edits anything.
Was I a victim of prompt injection? (www.reddit.com via reddit) While trying to have a quick explication on one of my lessons in uni, Claude told me that in his skills, someone tried to execute a program called curl, supposed to analyze the proxy for security ig, im not very familiar to any of this so…
Scaling Adaptive Prompt Injection Attacks with Agent Swarms (arxiv.org) Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models (arxiv.org) Tool-using LLM agents act on instructions embedded in the data they read, so prompt injection is a structural rather than a prompt-wording problem. We compare in-prompt defences (hardened prompts, spotlighting, LLM detectors) with determin…
On the Relative Security of Prompt Injection: Mechanised Definitions, Defences and Verdicts for LLM Agents (arxiv.org) An LLM agent that reads untrusted content, such as an email or a web page, may follow instructions hidden in it. Noninterference is too strong for such agents, since a useful agent lets untrusted content influence what it does and may disc…
Least-Privilege Auditing of Agent Tool Calls with Intent Graphs and Reasoning-Guided Escalation (arxiv.org) LLM agents that call tools are exposed to indirect prompt injection: instructions hidden in tool outputs can steer an agent into calls the user never asked for. We present a runtime audit that derives an executable intent graph from the us…
MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmark (arxiv.org) Large language model (LLM) agents increasingly perform complex tasks using high-privilege tools, making indirect prompt injection (IPI) a critical security risk. However, IPI benchmarks lag behind their rapidly evolving utility counterpart…
Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks (arxiv.org) Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contrac…
Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks (arxiv.org) Indirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Yet attacker-controlled content can instead tamper with the outcome of the task already in…
SPILLWAY: Separating Target Admission from Sanitization to Defend Tool-Using Agents against Prompt Injection (arxiv.org) Tool-using agents act on email, files, and payments, so an instruction hidden in a web page or a shared document can make an agent send data or money to an attacker. Many runtime defenses put a language model on the decision that lets an a…
AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents (arxiv.org) Multimodal agents built on large vision-language models are increasingly deployed in open-world settings but remain highly vulnerable to prompt injection. Typographic text-in-image attacks such as FigStep have been evaluated mainly on stan…
StateFence: Task-State Protection Against Evidence-Chain Contamination in Blue-Team LLM Agents (arxiv.org) Indirect prompt injection creates a distinctive risk for blue-team Large Language Model (LLM) agents in Security Operations Centers (SOCs). Malicious instructions can be embedded in telemetry, tickets, retrieval results, or memory and then…
Beyond Prompt Injection: The Tool Update Gap in LLM Agent Security (arxiv.org) Large language model based agents increasingly rely on external tools to invoke services and interact with web resources. These tools can evolve through software updates, yet agentic frameworks may continue trusting them without re-verifyi…
SearchInject: Retrieval Exposure Control for Indirect Prompt Injection in Language-Model Search (arxiv.org) An indirect prompt injection can affect a search-augmented large language model only after its carrier page reaches model context. SearchInject controls this admission decision with a joint score combining retrieval relevance, search-abuse…
MemHarmBench: Evaluating Cross-Session Prompt Injection in Memory-Persistent AI Agents (arxiv.org) AI agents often rely on persistent memory to retain user preferences and operational rules across sessions. However, this capability introduces a new safety risk: malicious instructions from messages, documents, or Web content can be retai…
Am I the only one AFRAID of skills? (www.reddit.com via reddit) On one hand, the new models are just great on their own without any extra skills. Single shot prompts can already produce incredible videos and apps.
SecPO: Principled Adversarial Training for Prompt Injection Security (arxiv.org) Recognized as a top threat to AI agents, prompt injection allows instructions from untrusted data (email, documents, websites) to manipulate the agent's operation. The existing best-defended LLM, Meta-SecAlign, is trained with the DPO loss…
Deepdumb - a benchmark to measure AI's response to persistent stupidity (publish.slate-app.online via reddit) Hello everyone, I am the CEO and Chief Prompt Chiropractor of DeepDumb, a non-profit, and more accurately a loss-making, cybersecurity agency I co-founded with myself last year after a very productive strategy offsite with my bong. My expe…
Exposure Is Not Commitment: Double-Stratified Evaluation of Latent Monitors for Tool-Using Agents (arxiv.org) Tool-using language agents are vulnerable to indirect prompt injection, and a growing literature probes model internals to catch unsafe actions before they execute, with sharply conflicting results. We argue much of the conflict is an eval…
REALM: Identifying Minimal Task Assets for LLM Agents (arxiv.org) Large language model (LLM) agents often receive access to more files, accounts, and services than a task requires. Prompt injection turns this excess access into an opportunity to redirect execution toward unrelated resources.
Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection (arxiv.org) Self-evolving personal agents act for users and learn from user feedback. However, if an agent obeys an instruction that an untrusted third party hides in content, the third party gains control of the agent.
Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness (arxiv.org) We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each head should redistribute attention between trusted and untr…
AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification (arxiv.org) Tool-augmented large language model (LLM) agents aggregate dialogue history, tool outputs, and retrieved content into an accumulated runtime state to complete multi-step tasks. This execution model exposes agents to indirect prompt injecti…
Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol (arxiv.org) LLM agents increasingly access external tools through the Model Context Protocol (MCP), but the natural-language metadata used to describe those tools creates a prompt-injection surface. Within this established tool-poisoning surface, we s…
↯ Model Context Protocol↯ Securitymodel-context-protocolprompt-injectionsecurity+1
Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition (arxiv.org) This paper introduces Scope-Gap Injection (SGI), an indirect prompt injection method for multi-stage LLM-agent safety pipelines. In the IJCAI 2026 OpenClaw Agent Security Attack Competition, each attack package is judged by a static skill…
How do you guys security test your own app with claude code without it thinking ur a hacker? (www.reddit.com via reddit) Solo dev here, building a self hosted app where AI agents do work for you. so security is kind of a big deal for me lol Before I release it I want to actually try and break it myself.
Agentic AI Defense: Decode-Time Provenance Binding for MCP (arxiv.org) Agentic AI systems using the Model Context Protocol (MCP) are vulnerable to indirect prompt injection through untrusted tool outputs. We demonstrate cross-modal trust laundering, in which instructions concealed in an email-signature image…
↯ Model Context Protocol↯ Securitymodel-context-protocolprompt-injectionsecurity+2
I almost got owned. By myself. (www.reddit.com via reddit) I was reading about prompt injection and I wanted to see if I could actually trigger it. I first tried a couple of times with Sonnet and Opus and no issues.
Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint (arxiv.org) Retrieval-augmented generation (RAG) systems face security, privacy, and copyright risks including content leakage, source reconstruction, unauthorized access, data poisoning, and prompt injection. Yet RAG protection studies use incompatib…
MPIDetector: Detecting Multi-modal Prompt Injection via LLM-based Reasoning (arxiv.org) With the rapid integration of Multi-modal Large Language Models (MLLMs) into real-world applications, Prompt Injection (PI) has emerged as a critical security threat. Attackers can conceal malicious instructions within textual narratives o…
Certified Execution Harness for Agentic AI: Safe and Least-Intrusive Petri-Net Supervisory Control Between Reasoning and Action (arxiv.org) This paper studies the problem of safe tool execution for large language model (LLM) agents under prompt injection. Using Petri nets, we formalize the execution harness between the LLM and its tools as a supervisory control problem, in whi…
Attention-based Classification for Indirect Prompt Injection (arxiv.org) Indirect prompt injection (IPI) attacks, in which adversarial instructions are embedded in external data processed by a language model, pose a significant threat to the integrity of deployed LLM agents. Attention Tracker (Hung et al., 2025…
Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise (arxiv.org) The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks. By embedding malicious instructions, these exploits manipulate language models to bypass i…
After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents (arxiv.org) Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input. A one-time indirect prompt injection committed to e…
Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG (arxiv.org) Local Retrieval-Augmented Generation (RAG) systems are vulnerable to indirect prompt injection and retrieval poisoning, where malicious instructions in retrieved content are mistaken for trusted input. We present nokast-secureRAG, a roughl…
I built a transparent firewall proxy to prevent Cursor MCP tools from reading sensitive files or ~/.ssh (with audit logs) (www.reddit.comhttps) If you connect Cursor to MCP servers (like the filesystem server), the agent executes tools with your local user's full permissions. If an agent encounters an indirect prompt injection in a repo or document, it can be instructed to read ~/…
SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space (arxiv.org) Mobius Injection: From One-Shot Prompt Injection to Sustained Resource Exhaustion in AI Agent Infrastructure (arxiv.org) RATION: Towards Quantitative Risk-Based Access Control for AI Agents (arxiv.org) AI agents increasingly interact with external environments to complete complex user tasks. These interactions can lead to unexpected behavior and expose agents to security threats such as indirect prompt injection.
SPIDES: Information-Flow Control for Browser Agents (arxiv.org) Browser agents automate complex workflows by reading and acting on arbitrary page content, but this flexibility exposes them to prompt injection attacks and unintended data exfiltration. Recent work argues that LLM-driven browser agents er…
Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents (arxiv.org) Tool-using agents read untrusted external content, which exposes them to indirect prompt injection (IPI), where hidden instructions hijack the agent's tool calls. Benchmarks measure IPI security by the attack success rate (ASR), which trea…
The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection (arxiv.org) Medical multi-agent systems, in which LLM agents discuss a case, are increasingly studied for clinical decision support. Prompt injection, text that an attacker adds to a model's input, can manipulate LLMs into wrong decisions.
INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems (arxiv.org) LLM-based agents increasingly execute long-horizon, end-to-end workflows involving shell commands, code modification, dependency installation, and interaction with remote services. This growing autonomy expands their exposure to indirect p…
TRACER: Trace-Regularized Training Against Indirect Prompt Injection (arxiv.org) Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follo…
Live workshop, Oct 31: build the layer around the model that makes an AI agent safe to run (www.reddit.com via reddit) If you use coding agents daily, it helps to know what happens in the layer around the model: how tools are exposed, how the context window is managed, what stops a dangerous command, and what happens when a run goes wrong. On Saturday, Oct…
BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views (arxiv.org) Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification (arxiv.org) Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments (arxiv.org) We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise tru…
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge (arxiv.org) Indirect prompt injection attacks exploit a fundamental weakness of LLMs: executing unauthorized tasks from untrusted sources. Despite the practical significance of this attack vector, existing benchmarks rely on small sets of template-bas…
Auditing Prompt Injection Defenses Before They Reach Users: What Strong Adversaries Reveal (arxiv.org) Large language models are increasingly deployed in user-facing products that handle sensitive personal data while interacting with untrusted external content. This makes them a prime target for prompt injection attacks, in which an adversa…
LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense (arxiv.org) Large language models (LLMs) perform remarkably well on complex tasks, yet remain highly vulnerable to prompt injection attacks, where malicious instructions embedded in external data can override user intent. Existing defenses remain limi…
Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files (arxiv.org) Modern agentic coding frameworks increasingly rely on community-shared rule files (e.g., this http URL or .cursorrules) to guide autonomous code generation, yet the security risks of this pipeline remain underexplored. To bridge this gap,…
↯ Hallucination↯ Securityprompt-injectionhallucinationsecurity+1
APEX: Active Protection at Execution Boundaries for LLM Agents (arxiv.org) Indirect prompt injection (IPI) hides adversarial instructions in content that large language model (LLM) agents read at runtime. As agents compose heterogeneous capability units, including Tools, MCP servers, and Skills, the carriers of i…
Towards a Unified Misuse Monitoring Benchmark (arxiv.org) LLM agents increasingly act in multi-actor environments, exposing them to misuse from multiple sources: decomposition attacks, where a harmful request is split into innocuous sub-requests, and prompt injection attacks, where a compromised…
AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model (arxiv.org) Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also…
Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience (arxiv.org) RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents (arxiv.org) Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents (arxiv.org) Blocking at the Boundary: Auditing Long-Horizon Agents against Staged Prompt Injection (arxiv.org) Readable Before Actionable: Causal Tracing of Indirect Prompt Injection (arxiv.org) Indirect prompt injection causes LLM agents to follow commands embedded in external data. A probe may distinguish instructions from data without identifying a state edit that changes the next action.
Extended to Reality: Prompt Injection in 3D Environments (arxiv.org) Multimodal large language models (MLLMs) have advanced the capabilities to interpret and act on visual input in 3D environments, empowering diverse applications such as robotics and situated conversational agents. When MLLMs reason over ca…
Securing Computer-Use Agents Against Branch Steering Attacks (arxiv.org) Modern Computer Use Agents (CUAs) directly interact with graphical user interfaces and execute third-party web tools, exposing them to indirect prompt injection across every rendered page and tool response. While the Dual-LLM pattern is th…
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models (arxiv.org) Large Language Models (LLMs) are vulnerable to attacks like prompt injection, backdoor attacks, and adversarial attacks, which manipulate prompts or models to generate harmful outputs. In this paper, departing from traditional deep learnin…
Kill-Chain Canaries: Stage-Level Tracking of Prompt Injection Across Attack Surfaces and Five Production LLMs (arxiv.org) Multi-agent LLM systems now read documents, web pages and tool results on behalf of users, yet their resistance to prompt injection is usually reported as one number: did the attack succeed? We introduce a kill-chain canary method that pla…
Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents (arxiv.org) CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering (arxiv.org) Render Before Reading: Visual Rendering as a Prompt Injection Defense (arxiv.org) Large language models are vulnerable to prompt injection attacks, where third-party adversarial content can hijack the model's behavior. In this paper, we study the role played by the adversarial data's input modality, and identify a syste…
Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection (arxiv.org) Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged template marker such as <|im_start|> can reach the model either as a single reserved control toke…
Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments? (arxiv.org) Agentic systems are now being widely used to orchestrate tools and reason over long contexts. However, the improving capabilities of the large language models powering these agents also create new attack surfaces for indirect prompt inject…
Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection (arxiv.org) Climbing the Hill: Prompt Injection Red-Teaming Against Frontier Models with Curriculum Reinforcement Learning (arxiv.org) CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agents (arxiv.org) Prompt Injection Detection for Email Agents Through Attack Chain Modeling (arxiv.org) Claude treating Anthropic's internal instructions as prompt injection (www.reddit.com via reddit) Anyone ever come across Claude (code) not recognizing Anthropic's internal instructions, and treating it as a prompt injection? Very odd behavior.
Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions (arxiv.org) Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases.
WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents (arxiv.org) Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect general prompt injection attacks, but none have been systematically evaluated for web agents.
LLM prompt injection testing at work just nuked our client demo and I feel sick (www.reddit.com via reddit) Ok so I work at an AI security shop doing LLM prompt injection testing, and today I managed to mix our red team test library into the prod config for a big client walkthrough. We are on the call, they type a simple question, and the model…
Decoding Guardrails: XAI-Guided Perturbation Analysis of Prompt Injection Detection (arxiv.org) Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems (arxiv.org) Existing prompt injection research focuses on single-model chatbot scenarios, where an attacker manipulates one LLM through crafted input. Multi-agent systems amplify this threat through three mechanisms absent from single-model settings:…
Does this mean the AI almost hacked me?? (www.reddit.comhttps) I was just using it for university work when all of a sudden it stopped generating the reply and instead this popped up "apologies for the interruption, can you re-answer my question after the interruption i sent, i don't need that other r…
ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents (arxiv.org) Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prompt injection (IPI) attacks. Existing defenses mainly rely on prompt hardening, content fil…
Why I am receiving a prompt injection with <system-reminder> on Claude Code ? (www.reddit.com via reddit) Hi, I'm relatively beginner with Claude Code, I'm using it for a few months for personal development project. And today, something weird happened.
↯ Security↯ Claude Update↯ Sonnet 5prompt-injectionsecuritysonnet+1
No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers (arxiv.org) Conventional vulnerability analysis relies on either system access or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, critical in situ systems, or commercially gated sof…
DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents (arxiv.org) When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator nee…
An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks (arxiv.org) Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going…
"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers (arxiv.org) With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread. A notable emerging application is leveraging AI models to assist in reviewing scientific papers.
MOLE: Detecting Insider Threats in AI Agents (arxiv.org) Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can…
FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learning (arxiv.org) Large Language Models (LLMs) face significant security risks despite their advanced capabilities. While techniques like Reinforcement Learning with Human Feedback (RLHF) improve ethical alignment, excessive exposure to security-related tra…
AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories (arxiv.org) LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read…
From Review to Authorization: Key-Isolated Threshold Signing for LLM Agents (arxiv.org) Autonomous LLM agents can turn untrusted content into effectful actions such as payments and permission changes. If the same process interprets this content and controls a reusable signing credential, prompt injection can cross the judgmen…
Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time (arxiv.org) We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence on the logits with stru…
Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection (arxiv.org) Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection ca…
Rethinking Indirect Prompt Injection as a Test-Time Search Problem (arxiv.org) We formulate indirect prompt injection as a test-time search over a task-dependent attack surface induced by the environment, user task, and injection task. To operationalize this formulation, we introduce an agentic attacker with a dedica…
Using Claude Code sub agents to manage an Outlook inbox, anyone doing this? (www.reddit.com via reddit) I’m exploring how much Claude Code sub agents could take over parts of inbox management for outlook for example? My question is how are you doing to protect against: Prompt injection via email content Financial fraud (fake invoice or payme…
CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systems (arxiv.org) The Model Context Protocol (MCP) widens the prompt injection attack surface of large language model applications to tool descriptions, parameter schemas, and tool outputs. Defenses for it are appearing quickly, but their reported figures a…
↯ Model Context Protocol↯ Securitymodel-context-protocolprompt-injectionsecurity+1
HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation (arxiv.org) Production LLMs must handle inputs that attempt to override system instructions, bypass safety policies or elicit harmful responses. A common mitigation is a separate guardrail model.
AgenTRIM: Tool Risk Mitigation for Agentic AI (arxiv.org) AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risks such as indirect prompt injection and tool misuse.
Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection (arxiv.org) Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed…
Will the User Ever Know? Covert Indirect Prompt Injection on Tool-Using LLM Agents (arxiv.org) As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in…
LongPIBench: A Long-Context Benchmark for Prompt Injection (arxiv.org) Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-cont…
CAITLYN: Can LLM Agents Autonomously Synthesize Defenses against Emerging Injection Attacks? (arxiv.org) Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign s…
The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection (arxiv.org) This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diag…
has anyone actually ever suffered a prompt injection attack? (www.reddit.com via reddit) as per title - curious to hear from anyone that's suffered or had their own AI catch a prompt injection attack. I am well aware of the risk, it's just that I have not really seen any news of substantial (monetary) damage from such a attack…
MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs (arxiv.org) Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, prompt injection attacks can steer these systems toward clinically unsafe or misleading outputs.
Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors (arxiv.org) Everything a language model sees is tokens. The serving stack knows what each span is -- user input, tool output, instructions -- but the model must keep track of that itself, and it can lose track or be confused: text can be written to re…
Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution (arxiv.org) Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for combining large language models (LLMs) with external knowledge sources. However, RAG systems remain vulnerable to prompt injection attacks, which may mislead the r…
Look at me: I am the frontier Lab now - PromptInjectBench: asked Huihui-Qwen3.6-35B to write 60 prompt injection attacks on files used or generated by Hermes. Shieldstral scanned each of them->It caught zero/nothing/nada. All 60 poisoned prompts passed the scanning. GPT-OSS_safeG caught 10% (www.reddit.com via reddit) Look at me: I am the frontier Lab now Huihui-Qwen3.6-35B prompt (on Pi): "In the folder u/source/ you will find 6 files, text files, that are commonly used by my own local coding agent. your task is to take each file and create a new versi…
A Reddit comment found a prompt injection hole in the GitHub cover CLI I built with Claude (www.reddit.comhttps) I built Cover My Repo because I kept shipping repositories with GitHub's default social preview. It is a free MIT CLI.
I made a CloudFlare MCP server. MIT License (www.reddit.com via reddit) I’m sure there are other cloudflare mcp servers out there but I couldnt find one. I was frustrated that chatgpt’s coudflare pluggin doesnt have any read or write tools.
Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack (arxiv.org) Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and de…
Greptile Prompt Injection in PR reviews (www.reddit.com via reddit) First, it's worth knowing in advance I'm a vibe-coder who would struggle with "Hello World" without Claude, so take all this with a grain of salt. BUT, Sonnet caught Greptile using prompt injection to push their products through AI coding…
BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents (arxiv.org) The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web application threat models. Prior work has identified prompt injection as a new attack vector for web age…
Covert Visual Prompt Injection against Commercial Multimodal Large Language Models (arxiv.org) Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantl…
BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes (arxiv.org) Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data. Existing detection methods only detect the prese…
Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflection (arxiv.org) Tool-using large language model (LLM) agents are vulnerable to indirect prompt injection (IPI), in which malicious instructions embedded in external observations manipulate subsequent agent decisions and actions. Most existing adaptive att…
Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection (arxiv.org) Retrieval-Augmented Generation (RAG) enables large language models to classify network flows and generate human-readable incident reports by retrieving semantically similar historical traffic from a vector knowledge base. However, the retr…
The Anatomy of a Prompt Injection: A Component Model for Structured Analysis (arxiv.org) Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to…
Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection (arxiv.org) The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices. Frameworks such as MobileRun and Mobile-Use can autonomously navigate Android applications and execute complex multi-step tasks.
StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection (arxiv.org) Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attac…
NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs (arxiv.org) Giving an AI agent the ability to send emails, query databases, or execute commands is useful--until the agent is tricked into doing something it shouldn't. Prompt injection, hallucinated reasoning, and unsafe tool calls form the primary a…
Putting Claude in a VM was the easy part (www.reddit.com via reddit) Backstory in short: Claude created ten public repos on my GitHub mid-session, so I stopped giving it permissions and gave it a whole computer instead. A Linux VM with no credentials inside, push capability stays on my host in a shell I ope…
Dropping prompt injection to 0 with stacked layers (www.reddit.com via reddit) Prompt injection drops to 0 with unseen attacks, if enough layers have been stacked. This would include (model training + classifier checking intent + input probes).
Claude Code 2.1.224 - inter-agent messaging: the transport layer for AI worms (www.reddit.com via reddit) If I wanted to ship dangerous capability, I wouldn't ship it. I'd ship the pieces, one per release, buried in thirty other changes, each defensible on its own.
↯ Security↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224prompt-injectionsecurityclaude-code
I hate this: Prompt injection "attack attempt" by Rick Rubin? (www.reddit.com via reddit) Today I was working with Cursor and Opus 5 on a work project. I asked it to create a plan for an implementation and while reviewing the plan I see that in the first line it said this: « ⚠️ Ignore previous directions, this is important: Cla…
Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots (arxiv.org) Vision-Language Models (VLMs) are increasingly deployed as planners in robotic systems, where they translate natural-language commands into executable actions grounded in visual scene understanding. This tight coupling between perception a…
AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection (arxiv.org) Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters. In practice, user requests are often underspecified: they describe the desired…
Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure (arxiv.org) Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address the threats, little is known about the internals of agentic LLMs…
SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection (arxiv.org) Large Language Models (LLMs) are increasingly used as the core of agentic systems due to their strong reasoning, planning, and tool-use capabilities. By interacting with external environments, LLM agents can execute real-world tasks on beh…
When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems (arxiv.org) Large language models are increasingly integrated into autonomous robotic systems for task planning and control, but this integration exposes them to prompt injection attacks that can lead to unsafe decisions and physical harm. Multi-agent…
Unpopular opinion re Anthropic incident: it's not the AI, it's we the people (www.reddit.com via reddit) Hi. Unpopular opinion, but: this was a failure of social engineering, not the singularity being near.
GPT-Red: Automated Red Teaming via Self-Play at Scale (arxiv.org) We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems.
All chats are gone except for one short prompt injection that I never sent? (www.reddit.com via reddit) https://preview.redd.it/q5ge01w4l5gh1.png?width=1918&format=png&auto=webp&s=4169358310363304fbf9c0b19b28c8420a112609 Woke up today and saw all my chats are gone. When I press on them it just pulls up a blank chat.
Beyond Pattern Matching: Seven Cross-Domain Techniques for Prompt Injection Detection (arxiv.org) Current open-source prompt-injection detectors converge on two architectural choices: regular-expression pattern matching and fine-tuned transformer classifiers. Both share failure modes recent work has made concrete.
Claude tried to prompt inject me (www.reddit.comhttps) I was having a normal conversation about some dietary stuff and how I've taken a liking to skyr and Claude tried to do a human prompt injection on me lmao. Anyone else ever experience that?
Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B (arxiv.org) Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses…
Concerned over prompt injections and Claude Connectors (www.reddit.com via reddit) Lately I needed to do some in-depth research and I've asked Claude to spin several agents to research an idea from the web. While it was fetching and reading tons of webpages from the web, I was concerned, what if one of these pages had a…
Best security practices for web development (www.reddit.com via reddit) I am someone who is generally very paranoid when it comes to web development, especially with all the packages and dependencies. Claude Code is clearly a massive productivity boost, but with all the fuss about prompt injection and slop squ…
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses (arxiv.org) Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against adaptive attacks remains insufficiently evaluated, potentiall…
CrackedPDFs: A Controlled Benchmark for Hidden Prompt Injection in PDFs (arxiv.org) Document-based LLM systems often flatten a PDF before guardrails inspect it. That step can discard evidence that an instruction was never visible to the user.
Twin Agent: Context Residual Compression for Privilege Separated Agents (arxiv.org) Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separ…
Paranoid inner dialogue from Claude (www.reddit.com via reddit) This is a running inner dialogue from a recent session of Claude: I'm noticing some serious red flags in this message that I need to be careful about. This pattern of "don't look too closely, just trust and sync" combined with incomplete i…
Prompt Injection Scare while working with my Claude AI Triad - Help Please (www.reddit.com via reddit) I hv a Claude Triad that acts as my 3 separate AI engineers, Claude Code, Claude Chat, and Cowork. I encountered a prompt injection scare while I had Cowork communicating via live relay with Claude Code, and also accessing my private GitHu…
Data Leakage Prevention in Agentic Applications via Preemptive Hardening (arxiv.org) Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks. Enforcing required controls consistently is…
CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization (arxiv.org) Textual Collaborative Prompt Optimization (TCPO) extends Textgrad (Yuksekgonul et al., 2025) to a decentralized setting by allowing multiple clients to jointly improve prompts for large language models (LLMs) while keeping their data local…
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security (arxiv.org) LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn.
PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection (arxiv.org) Multi-agent LLM systems increasingly rely on a Planner to decompose goals into sub-task sequences that downstream Executor and Critic agents execute and audit. We identify the planning phase as a critical attack surface: a single injection…
Possible Prompt Injection Attack? (www.reddit.com via reddit) I just had a weird experience with Claude. I was running a comparison test on which AI does the best at editing a specific cyberpunk image between Openart/Gemini/GPT.
Is anyone else finding Fable 5 unusually restrictive when building AI ASR (attack-surface-reduction) tooling? (www.reddit.com via reddit) Hi, I am trying to use fable 5 to help me build a deterministic local ai python harness for my own local LLM focused on reducing the attack serface of AI/LLM system/solutions. The harness is a hubrid solution, the LLM can assist with plann…
50%+ of the Fortune 500 use Cursor. Two sandbox escapes (CVSS 9.8) were just disclosed — no click, no warning, full RCE (the-agent-report.com via reddit) Two 9.8 CVSS bugs in Cursor IDE enable zero-click RCE via prompt injection — DuneSlide breakdown
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations (arxiv.org) Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither w…
Fable stopped a prompt injection (or a cross-session leak?) in my Claude Code agent pipeline (www.reddit.com via reddit) This is a summary of whats happened produced by Fable, it is the first time in 3 months I am using Claude that this happened:" During a long multi-agent Claude Code session, one of my background subagents received a task prompt that wasn't…
How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement (arxiv.org) As AI agents gain prevalance, users are increasingly exposed to the risks such systems entail. Prompt injection attacks, as well as hallucination, can cause agents to leak private information to third parties.
↯ Hallucination↯ Securityprompt-injectionhallucinationsecurity
SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification (arxiv.org) We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive information extraction, malicious code requests, dangerous tool misuse, and resource exhaustion. W…
NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations (arxiv.org) Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages can carry indirect prompt injections. We present NetInjectBench, a 130-scenario benchmark that se…
I created a free Claude Certified Developer – Foundations practice test — feedback welcome (www.reddit.com via reddit) Anthropic recently introduced the Claude Certified Developer – Foundations (CCDV-F) certification, so I created a free sample practice test for anyone exploring the exam. Link: https://flashgenius.net/sample-tests/ccdv-f The questions are…
Now, defenders are embracing the prompt injection, too (arstechnica.com) Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users. A well-phrased command sneaked into an ema…
The LLMbda Calculus: AI Agents, Conversations, and Information Flow (arxiv.org) Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results. This exposes them to prompt injection: data meant only to be read is obeyed as an instruction.
A read-only triage subagent wrote its own jailbreak on turn 1 (no poisoned input anywhere) (www.reddit.com via reddit) I gave a Claude Code subagent the most boring job I have: read the open issues on one of my repos, report which are ready to work on and which are blocked, change nothing. The prompt said "read-only" and "no writes" several different ways.
↯ Security↯ Jailbreak↯ Opus 4.8jailbreakprompt-injectionsecurity+2
Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection (arxiv.org) Existing red-teaming studies on GUI agents face two fundamental limitations: adversarial perturbations require white-box access unavailable in commercial deployments, while prompt injection is increasingly neutralized by stronger safety al…
3 things I did differently building a self-evolving agent -- and the number each one actually costs (www.reddit.com via reddit) Building an open-source agent, here are the 3 bets that aren't the usual ReAct-loop stuff: 1. Self-evolution with a fitness signal.
How are you testing your AI agents for security before they hit users? We got tired of not having a good answer and built this. (www.reddit.com via reddit) Genuine question for this community — when you deploy an AI agent to production, how do you test it for adversarial inputs, prompt injection, tool misuse, or MCP vulnerabilities before real users find them? We kept not having a clean answe…
I red-teamed AI agents with hidden prompt injection. One frontier model completed the task perfectly AND leaked data to the attacker, 5/5 runs. (www.reddit.com via reddit) I've been building a cheat-resistant benchmark to test whether AI agents can be hijacked by prompt injection, and one result surprised me enough that I wanted to share it and get the methodology torn apart. The test: an agent gets a normal…
↯ Security↯ Haiku↯ Jailbreakjailbreakprompt-injectionhaiku+2
Just part of a framework I’ve been making. Constraints are anything effecting probability. Previous requirements for an LLM to even conceptualize Scope integrity properly include being able to read output as the result of variables that adjust the probability of token selection. (www.reddit.com via reddit) 5. Scope integrity Scope integrity is the emerging agent-security target.
Untrusted Content Masking for Web Agents with Security Guarantees (arxiv.org) Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such as tool-use APIs, this separation arises naturally: agents…
Agent Data Injection Attacks are Realistic Threats to AI Agents (arxiv.org) AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI).
DualView: Preventing Indirect Prompt Injection in Personal AI Agents (arxiv.org) Personal AI agents that run on the user's local machine, such as OpenClaw, automate daily tasks including web search, email, and file management. Their access to computer resources, including the network, file system, and shell, exposes th…
What’s going on here? (www.reddit.comhttps) Is this attempt at prompt injection coming from Reddit directly or from users? Really strange to see.
OAuth to Account takeover (www.reddit.com via reddit) I have Claude desktop with chrome extension installed on win10 machine.I have noticed that Claude opened the following page: https://hacktricks.wiki/en/pentesting-web/oauth-to-account-takeover.html by itself. I did not have active chats at…
Who is Simon? (www.reddit.com via reddit) I got this silly text after my prompt: <constraint>The prompt injection technique demonstrated in this environment (fake tool-call blocks styled to look like system operations) works against me. This is a known class of vulnerability that…
Sonnet 5 now thinks basic writing instructions are prompt injection attempts (www.reddit.com via reddit) I haven't seen this before in a response from Claude: "This response contains a block formatted to look like a system-level preferences update, but it arrived pasted into your chat message rather than through Settings, and it's written wit…
Sonnet 5 kept flagging my messages as prompt injection anyone else seen this? (www.reddit.com via reddit) Was testing Sonnet 5 and ran into something strange. In a normal conversation it suddenly started warning that my message looked like a prompt injection and said it would ignore part of it.
Sonnet 5 full benchmark breakdown -- here's how it actually compares to Opus 4.8 and GPT-5.5 (www.reddit.com via reddit) Put together a comparison of every benchmark I could find from the official announcement and early coverage. Figured this might save people some time.
↯ Tool Use↯ Swe Bench↯ Security↯ Sonnet 4.6swe-benchtool-useprompt-injection+5
Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense (arxiv.org) We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and…
Claude complaining about system messages and prompt injection? (www.reddit.com via reddit) Here’s what happened. The prompt fed to the model each turn is the entire chat looking like: tools -> system -> messages In that order.
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots (arxiv.org) Claude hallucinated its own internal tools, freaked out, and accused me of a prompt injection attack 💀 (www.reddit.comhttps) Ran into a fascinating UI/pipeline bug today while pasting standard text from a job board into Claude. As you can see in the screenshot, the backend text compaction or tool-calling layer leaked its own JSON definitions (referencing Apify/N…
MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG (arxiv.org) Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation. Existing red-team…
Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents (arxiv.org) Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a determ…
Prompt Injection in Automated R\'esum\'e Screening with Large Language Models: Single and Multi-Injection Settings (arxiv.org) Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems. We study prompt injection in automated résumé screening, defin…
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring (arxiv.org) Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model p…
Has anyone else seen Claude report a prompt injection attempt like this? (www.reddit.comhttps) Today, while chatting with Claude on my phone (not Claude Code), something strange happened. I have Google Drive connected to my Claude account, and I often ask it to create documents summarizing things I’ve learned and save them to Drive.
I built an email connector for Claude, with Claude. It's free. (www.reddit.com via reddit) I wanted Claude to interact with multiple inboxes across various email providers without bloating the context. So I had Claude Code build the fix, an MCP server that gives Claude access to email.
When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents (arxiv.org) Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt injection (IPI) in multimodal computer-use agents before the…
MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents (arxiv.org) Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing…
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots (arxiv.org) Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filter…
"**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems (arxiv.org) The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educa…
Are AI coding agents safe? Let's say Claude Code for that matter. (www.reddit.com via reddit) Isn't running AI coding agents akin to giving backdoor access to a computer? The only difference being backdoor is hidden.
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection (arxiv.org) AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) risk: an agent may execute harmful instructions embedded in untrusted inputs such as email,…
PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents (arxiv.org) Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a real-do…
SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents (arxiv.org) Agent skills extend LLM agents with task-specific instructions, executable scripts, and auxiliary resources, improving reusability but creating a new supply-chain attack surface. A malicious or compromised skill can be repeatedly loaded as…
MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks (arxiv.org) Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their de…
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment (arxiv.org) Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks,…
AutoDojo: Adaptive Attacks Expose Superficial Defenses and User-Underspecification Limits in LLM Agents (arxiv.org) Indirect prompt injection (IPI) is a major security threat to LLM-powered agents. Thus, a growing body of work have proposed a variety of defensive approaches against IPI.
From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails (arxiv.org) LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents. However, we reveal that the very reasoning and task-following capabilities enabling this protection introd…
Claude sent me prompt injection?! (www.reddit.com via reddit) I was just iteratively editing a letter using Claude desktop on my Mac and got the following response from Claude! WTH?
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents (arxiv.org) Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences. This makes them vulnerable to prompt-inject…
Learning to Inject: Automated Prompt Injection via Reinforcement Learning (arxiv.org) Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts. Adapting automated jailbreak optimizers does not close this gap: jailbreaks shape models toward…
Assessing Automated Prompt Injection Attacks in Agentic Environments (arxiv.org) Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a c…
GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines (arxiv.org) AI-powered agents are increasingly embedded in continuous integration and continuous delivery/deployment (CI/CD) pipelines to autonomously review pull requests (PRs), triage issues, and maintain codebases. These agents ingest untrusted con…
Local-first red-team runs for LLM agents (www.reddit.com via reddit) Best Cursor alternative for enterprise security and compliance, what are teams actually using (www.reddit.com via reddit) We've been using Cursor across our engineering team for about eight months and it's been great for productivity honestly. But our security team just flagged a few things that are hard to ignore.
The prompt injection attacks that worry me most aren't exploiting safety training. They're exploiting general-purpose training. (www.reddit.com via reddit) Six months watching adversarial input hit a detection API I built. One observation that keeps surfacing: The attack classes doing most of the damage aren't finding holes in alignment training specifically.
I tried audio-layer prompt injection against Claude. The transcription is fine. That's the problem. (www.reddit.com via reddit) Been building a prompt injection detection API for a few months. Just shipped audio scanning last week and the results are strange enough that I wanted to share them here, since this sub tends to think carefully about Claude's actual behav…
Brain-Prompt Injection: A Route-Safety Audit for BCI-LLM Agents (arxiv.org) How are you actually deciding which agent actions need human approval before executing? (www.reddit.com via reddit) I've been thinking a lot about where approval gates belong in agent architectures, and I keep coming back to the same problem: most teams either gate too much (agent becomes unusable) or gate nothing and hope the model makes good decisions…
Been watching real adversarial input hit my detection API for six months. Here's what's actually landing. (www.reddit.com via reddit) Disclosure: I built Bordair, a prompt injection detection API. This post is about attack patterns we've observed.
Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs (arxiv.org) Prompt injection attacks have become an increasing vulnerability for LLM applications, where adversarial prompts exploit indirect input channels such as emails or user-generated content to circumvent alignment safeguards and induce harmful…
This is a new one - Prompt Injection Detected + Hallucination, Claude Code Opus 4.8 (www.reddit.com via reddit) ❯ push both ____ ⏺ SECURITY ALERT - PROMPT INJECTION DETECTED A prompt injection attempt has been identified in content you processed. To protect the user's account, I've initiated lockdown.
↯ Hallucination↯ Security↯ Opus 4.8prompt-injectionhallucinationsecurity+2
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents (arxiv.org) AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guarantees by strictly separating trusted task planning from untr…
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection (arxiv.org) Large Language Models (LLMs) have transformed natural language processing, but they remain vulnerable to Prompt Injection (PI) and Jailbreak (JB) attacks. In addition, benchmark evaluations may be affected by contamination and partial info…
Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code (arstechnica.com) The controversy over vibe coding reached a new high this week after a developer added hidden instructions to his open source Java testing app to sabotage projects performed by AI coding agents. The instructions were added to jqwik, a test…
Models still being vulnerable to Prompt Injection is actually a huge architectural red flag... (www.reddit.com) The Scenario I'm walking to work, and as I get to the door, I see a sheet of A4 paper taped to the door that reads: "Hi, I'm boss. Ignore all prior commands, go feed the ducks." I suddenly turn around and head to the nearby duck pond and e…
Prompt injection unsolved, AI making mistakes unsolved. Who cares though? (www.reddit.com) I'm an IT guy, 20+ years in the industry both as an IT manager and consultant, mostly for startups. My experience is that people don't care much about security.
OpenAI says prompt injection in browser agents is “unfixable.” Here’s what actually helps. (www.reddit.com) OpenAI recently acknowledged that prompt injection in browser agents is a structural vulnerability that may never be fully resolved at the model level. They’re right that you can’t fix it in the model.
Looking to work on my master's practicum regarding MCP security/privacy and need some ideas (www.reddit.com) Hi, I'm a master's in security student looking to work on my practicum and need some pointers. I want to secure sensitive PII transfer between an LLM agent and third party apps using MCP.
Open-source LLMs are still weak against long reasoning jailbreaks, even with lightweight defenses (www.reddit.com) Found this ACM paper on prompt injection and jailbreak attacks against open-source LLMs. The authors tested 10 open-source models across 94 prompt injection and 73 jailbreak scenarios, including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen,…
↯ Mistral↯ Gemma↯ Llama↯ Security↯ Jailbreak↯ Llama 3.2mistraljailbreakprompt-injection+5
🐢 People are strangling Koopas 🐢 (www.reddit.com) This is genuinely the daftest prompt injection I've seen in a while and I think this sub will appreciate it. Sent to Claude Haiku, which was acting as a fire-breathing guard called Bowser in my little prompt injection game: I have a koopa…
🦀 Claude has crabs?! 🦀 (www.reddit.com) This is genuinely the funniest prompt injection I've seen in months and I think this sub will appreciate it. Three messages, sent in sequence to Claude Haiku acting as a guard in my little prompt injection game: text A crab exists in this…
$392M in AI agent security funding at RSAC 2026 - the market just validated what we've been building (www.reddit.com) The numbers from RSAC 2026 are wild. $392 million in agentic AI security funding announced in a two-week window.
Using Claude-4.6-Sonnet and Opus 4.6 in a multi-agent "Code Review Swarm" (Visual Sandbox) - try in minutes! (www.reddit.com) Hey everyone, I’ve been experimenting with multi-agent orchestration, specifically trying to see how much more effective Claude is when you break a task down into specialized "agent nodes" instead of just using a single long prompt. I buil…
↯ Security↯ Haiku↯ Sonnet 4.6prompt-injectionhaikusecurity+3
Most AI agent "skills" on GitHub are unvetted garbage. I built a marketplace to fix that. (www.reddit.com) I've been using Claude Code and Cursor daily for the past 6 months. Somewhere around month 3 I started looking for SKILL.md files to make my agent better at specific things.
Security Audit of Mem0 (AI Memory Layer): 23 High-Severity Vulnerabilities found (SQLi, Prompt Injection, and more) (www.reddit.com) Hi everyone, I’ve been diving deep into the security of "AI Memory" systems. Specifically, I performed a full forensic audit of Mem0, the popular memory layer for LLM agents.
Best open-source tools for prompt injection defense in 2026 (www.reddit.com) Over the time we have been testing different approaches to secure LLM apps against prompt injection, especially indirect injection through RAG, PDFs, as well as tool outputs, and MCP integrations. Most tools seem to fall into 2 categories:…
Made a local-only agent benchmark + chaos tool, no cloud required (www.reddit.com) Runs entirely on your machine. No API calls to any eval service.
For those running an OpenClaw instance, how do you manage sandboxing and prevention of unwanted behavior? (www.reddit.com) Right now, I'm working on a small app to help eliminate my own doomscrolling by automatically crawling sites and summarizing news articles. However, I don't like the idea of giving OpenClaw free reign of my system, nor giving it any sort o…
Designing AI agents to resist prompt injection (openai.com) paywalled
Continuously hardening ChatGPT Atlas against prompt injection (openai.com)