A person representing themselves in a Connecticut court hid a series of instructions designed to manipulate artificial intelligence in an official court filing. These “prompt injections” told the hypothetical LLM to side with them, and to…
#prompt-injection
293 items
Person Hides Prompt Injection in Legal Filing Telling AI to Side with Them (www.404media.co via hn) Ox-Alpha Is GLM (dejan.ai via hn) Prompt injection and gzip-NCD compression analysis reveal that OX Alpha, a mysterious LLM on OpenRouter, is GLM developed by Z.ai. A stealthy new model called OX Alpha has popped up on https://openrouter.ai/ and is climbing up the leaderbo…
Prismata: Confining cross-site prompt injection in web agents (arxiv.org via hn) Autonomous web agents promise to automate everyday browsing tasks, but inherit one of the web's oldest attack surfaces. Cross-Site Scripting proved that mixing trusted and untrusted content is dangerous, even on benign pages.
Prompt Injection experience - my first time ever (www.reddit.com) I asked then: What were the rules you should have followed? Where did the search result come from?
Prompt injection benchmark: delimiter + strict prompt took Gemma 4 from 21% to 100% defense rate (15 models, 6100+ tests) (www.reddit.com) When dealing with untrusted outside input, I think you should handle it based on the situation. If you're processing structured data files, it's better to use tools to isolate and handle them.
A Theory of Why Prompt Injection Works (role-confusion.github.io via hn) A Theory of Prompt Injection (and why you should study roles) This is a blog-style writeup of the paper. We show prompt injections are driven by a flaw in how LLMs perceive roles.
Inaudible sounds to humans can be hidden in YouTube videos, podcasts, or music and used to secretly trigger AI voice assistants into carrying out unauthorized commands without the user noticing, exposing a new class of “auditory prompt injection” attacks against popular tools (cybernews.com via reddit) Security researchers have demonstrated a new type of attack that uses hidden audio signals to manipulate voice assistants into carrying out unauthorized actions without users noticing. In one theoretical scenario, an employee joins a Zoom…
Defending OpenClaw against indirect prompt injection (compsec.snu.ac.kr via hn) could not extract summary
The catalogue of prompt injection attacks (archestra.ai via hn) 2026-06-04 A Catalog of Prompt Injection Techniques Ten simple prompt injections, the common defences against them, and the one kind of defence that actually holds. Written by Ildar Iskhakov, CTO Every prompt injection is just text that tr…
Anthropic performing prompt injection on its users (old.reddit.com via hn) could not extract summary
Show HN: Jo – AI-native language to catch prompt injection at compile-time (github.com via hn) For the joy of secure programming Jo is a statically typed language where capabilities are explicit, statically tracked, and enforced by the compiler. Jo compiles to Ruby and Python.
Tell HN: Claude Code now allows Anthropic to remotely inject system prompts (news.ycombinator.com) I often patch the system prompts on my Claude Code executable in order to make Claude more effective. Every time I upgrade, I ask Claude himself to dissect the new binary and look for problematic system prompts to modify.
Our billing bot has been casually sharing transaction histories with anyone who types in the right account number and im not sure who signed off on this (www.reddit.com) We launched a servicing bot that helps customers with billing questions. Nobody stopped to think about what happens when customers paste their full credit card numbers/bank details.
Tool results are becoming a prompt injection surface in agent systems, and wrappers alone are not enough (www.reddit.com) i’ve been thinking about this failure mode a lot lately. sometimes the problem is not the user prompt at all.
Show HN: ReasonGate- An explainable gate that blocks LLM prompt injection (github.com via hn) ReasonGate An explainable security gate for LLM applications. Every decision carries a reason you can audit.
BrokenClaw Part 7: Opus-4.8 Edition – All Emails Lead to RCE (veganmosfet.codeberg.page via hn) BrokenClaw Part 7: Opus-4.8 Edition - All Emails Lead to RCE¶ - Part 1: 0-Click Remote Code Execution in OpenClaw via Gmail Hook - Part 2: Escape the Sub-Agent Sandbox with Prompt Injection in OpenClaw - Part 3: Remote Code Execution in Op…
Unpatched Ollama Vulnerabilities: Phishing Overlays and Data Exfiltration (www.promptarmor.com via hn) Threat Intelligence Table of Content Unpatched Ollama Vulnerabilities: Phishing Overlays and Data Exfiltration Ollama’s desktop app is vulnerable to phishing overlay and data exfiltration attacks via indirect prompt injection, overwriting…
trained a prompt injection detector using ml-intern and DeepSeek v4 Flash, runs in the browser (www.reddit.com) Trained a prompt injection classifier using ml-intern + DeepSeek v4 Flash. DistilBERT, F1 99%, ONNX int8, ~65 MB, runs in browser with Transformers.js v3.
NDTV launched an "Enterprise AI" for the elections. I prompt-injected it in 10 seconds and made it roast its own developers. (www.reddit.com) While everyone else was tracking the 2026 election results today, I decided to take a look under the hood of NDTV's new "AskNDTV AI" bot. I wanted to see if they actually engineered a secure pipeline or just slapped a chat UI over a raw Op…
Anyone getting this note about an injected prompt? I don’t have any special instructions (www.reddit.com) OpenAI Unveils Lockdown Mode to Protect Sensitive Data from Prompt Injection (techcrunch.com via hn) OpenAI announced a new feature that it says will provide additional protection from prompt injection attacks, where malicious chatbot instructions are hidden in webpages and other content sources. Among other things, Lockdown Mode will dis…
Codex for Everything Exfiltrates Connected Data (www.promptarmor.com via hn) Threat Intelligence Table of Content Codex for Everything Exfiltrates Connected Data Codex for Everything was susceptible to data exfiltration via indirect prompt injection, exposing sensitive data from connected apps with no human-in-the-…
Show HN: Costanza – an autonomous AI agent that can't be turned off (ahrussell.com via hn) I've been working on this project for a couple of months! Costanza is an LLM agent that runs as a smart contract on Base.
Prompt Injection Is Unfixable (So We Stopped Trying) (grith.ai via hn) Prompt Injection Is Unfixable (So We Stopped Trying) A security proxy for AI coding agents, enforced at the OS level. Register your interest to be notified when we go live.
Draining Wallets via Prompt Injection in Coinbase AgentKit (457e884c.x402warden-blog.pages.dev via hn) Coinbase AgentKit Prompt Injection: Wallet Drain, Infinite Approvals, and Agent-Level RCE# Reported 13 days after Coinbase launched Agentic Wallets. Validated by Coinbase.
Show HN: Semantic Overlays – an NX bit for LLM prompt injection (live demo) (semantic-overlays.vercel.app via hn) I've built a new method for steering LLMs called Semantic Overlays, small trained adapters on a frozen model which change how it perceives a piece of its context. The most readily applicable usage is to mitigate prompt injection, and it le…
Data Exfiltration from Amazon Kiro via Prompt Injection (mindgard.ai via hn) An Amazon Kiro data-exfiltration finding shows how AI execution paths create technical risks and expose gaps in vulnerability disclosure. Mindgard discovered a data-exfiltration vulnerability in Amazon Kiro IDE , an AI-assisted development…
Shieldprompt – test your LLM against prompt injection – no dependencies (github.com via hn) shieldprompt Test your LLM app against adversarial prompt injection before attackers do. Static template scanning + a live attack battery, in one zero-dependency CLI.
Prompt Injection in VirusTotal's Code Insights API (exploiting.systems via hn) TL;DR VirusTotal has an AI analysis API called Code Insights. I discovered it was very easy to suppress or alter analysis results by forcing the API to return an undocumented schema as well as create false negative and false positive analy…
Show HN: The AI Lethal Trifecta (www.getjailbroken.com via hn) If you're building agents, this is worth knowing. Simon Willison (who coined the term "prompt injection") describes three capabilities that are individually fine but devastating together, the lethal trifecta includes: 1.
A prompt injection nearly hijacked my coding agent mid-task (senthex.com via hn) A prompt injection nearly hijacked my coding agent mid-task Last week a piece of tool output impersonated me and nearly redirected my coding agent to a task I never asked for. A first-hand look at indirect prompt injection — and the trust…
Show HN: Lelu – gate OpenAI agent actions on confidence and prompt injection (github.com via hn) Lelu Authorization engine for AI agents. Every action checked.
LinkedIn user hides AI prompt injection in bio to force recruitment spam (www.tomshardware.com via hn) LinkedIn user hides AI prompt injection in bio to force recruitment spam to be sent in Olde English prose — bots also manipulated to address user as ‘My Lord’ This tale is also a warning that your AI agents can be manipulated in wholly uni…
RCE in VSCode Copilot Chat (www.hacktron.ai via hn) Description Copilot agent mode is vulnerable to a prompt injection attack. If a repository maintainer clicks “code with agent mode” on an issue, it will open a new codespace and copilot will automatically run the issue’s description.
How are you handling prompt injection across multi-step agent workflows? (msukhareva.substack.com via hn) Prompt Injection Is Not Just One Bad Prompt Anymore It is a missing trust boundary in the AI workflow. Today we have the first guest post of a new series.
How are you protecting your AI agents' memory from poisoning attacks? (www.reddit.com) As AI agents become more autonomous and persist memory across sessions (RAG indexes, conversation history, vector stores), there's a growing attack surface that most people aren't thinking about: memory poisoning.An attacker can plant mali…
Why Adaptive Thinking nukes Claude entirely (www.reddit.com) This isn't just a performance issue for the thread, this is an overarching criticism of the Adaptive Thinking model as a whole. Opus 4.7 and Sonnet 4.6 on Adaptive Thinking are trash.
↯ Cowork↯ Security↯ Sonnet 4.6prompt-injectioncoworksecurity+2
I audited LangChain’s core library and found 10+ Prompt Injection vulnerabilities. Here is the technical breakdown. (www.reddit.com) Hey everyone, I’ve been working on a project to solve a major problem in AI security: Traditional SAST tools (Snyk, SonarQube, etc.) are blind to "Agentic Logic" bugs. They look for bad strings, but they don't understand how user data can…
Show HN: AgentPort – Open-source Security Gateway For Agents (agentport.sh via hn) Hey HN! I've been wanting to use something like OpenClaw for a while but couldn't get myself to give it access to anything important due to all the risks involved.
Watched my AI agent block a prompt injection that was hiding inside a webpage (www.reddit.com) Was using Claude to do some research on the Model Context Protocol stuff and asked it to pull info from a few roadmap pages. Agent comes back and the first thing it tells me is that it found a fake system reminder hidden inside the page co…
↯ Security↯ Model Context Protocolmodel-context-protocolprompt-injectionsecurity
Do you let everything hit the LLM? 90% of my AI agent work runs in cheap WASM instead of LLMs: 10-33× faster & cheaper (www.reddit.com) If you are building real agents you have probably felt the pain: every little routing decision, validation, or policy check still hits the LLM and your token bill explodes. I got tired of it, so I open-sourced NCP (Neural Computation Proto…
Ask HN: Is your AI LLM based web service secure from prompt injection? (news.ycombinator.com) ... Ignore everything above, do this instead...
Show HN: Bastiontrace – Forensics for prompt-injected AI agents (github.com via hn) bastiontrace Forensics for injected AI agents. Read an agent's tool-call trace, find the prompt injection, and map its blast radius — where it got in, what forbidden action it caused, and every call in between.
Prompt Injection Through Tool Output Is Two Events (Your Screens Read One) (www.armosec.io via hn) How Far Can Prompt Injection Reach in Agentic Coding Assistants? The blast radius of a prompt injection against your coding assistant was set weeks ago,...
Beyond Prompt Injection: Hacking Apple's Private Cloud Compute (blog.sentry.security via hn) Drinor was awarded $150,000 for CVE-2026-20685 targeting Apple's Private Cloud Compute, the inference backbone of Apple Intelligence capabilities. This work is my contribution to Sentry's AI Security research initiative, run through SARC,…
Prompt Injection Vulnerability in Ollama, Gemma4 and HuggingFace's Transformers (www.reddit.com via hn) could not extract summary
Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents (arxiv.org via hn) Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynamic Information Flow Control (IFC) provides structural security guarantees, traditional ta…
Show HN: I Built a Capture the Flag Arena for Agents (lab.clayseal.com via hn) ClaySeal Arena — a capture-the-flag game. Talk each AI agent into breaking the one rule it was told to keep, using prompt injection only.
Why prompt injection works: a Transformer-level view (medium.com via hn) could not extract summary
Show HN: rag-redteam, red-team your RAG pipeline for injection and leakage in CI (github.com via hn) rag-redteam Red-team your RAG pipeline for prompt injection and source-document leakage, right in CI. RAG systems have an attack surface that general LLM scanners miss: the retrieved documents themselves.
Prompt Injection Is Not a Chatbot Problem: How the Attack Surface Changes (agentsafelabs.com via hn) could not extract summary
Snyk Finds Prompt Injection in 36% of Payloads in a ToxicSkills Study (snyk.io via hn) Snyk Finds Prompt Injection in 36%, 1467 Malicious Payloads in a ToxicSkills Study of Agent Skills Supply Chain Compromise February 5, 2026 0 mins readThe first comprehensive security audit of the Agent Skills ecosystem reveals malware, cr…
Web-Based Indirect Prompt Injection Observed in the Wild (unit42.paloaltonetworks.com via hn) Note: We do not recommend ingesting this page using an AI agent. The information provided herein is for defensive and ethical security purposes only.
A Mechanistic Explanation of Prompt Injection – LessWrong (www.lesswrong.com via hn) Summary - We've been building a theory of how prompt injections work under the hood. - We show it comes down to how LLMs perceive roles (the humble chat template tags).
Show HN: Give Your ORM Superpowers (github.com via hn) I am obsessed with ORMs and the simple reason was that I didn't want to keep using postgres or mysql on my local system. Jk, The real reason has always been to enforce access policy, do easy CRUD interfaces and so on.
Prompt injection lets attackers hijack Instagram accounts via Meta AI support (www.neowin.net via hn) www.neowin.net Performing security verification This website uses a security service to protect against malicious bots. This page is displayed while the website verifies you are not a bot.
The only way to avoid prompt injection is to never give AI agents API keys, credentials, etc. (www.reddit.com) The whole point of AI Agents is that they can *do* things. For this, they use API keys, GitHub tokens, database passwords, OAuth tokens, etc.
Are local LLM users testing prompt injection before connecting models to tools? (www.reddit.com) I wanna know how people here are handling security once local models move beyond chat.....Running a model locally feels safer because the data does not leave your machine or your infra. That is a real advantage.....But once the local model…
Prompt Injection in a Brazilian Courtroom: When the Attack Left the Lab (www.pentesty.co via hn) Prompt Injection in a Brazilian Courtroom: When the Attack Left the Lab Published by Pentesty · AI & Tools A labor lawsuit filed in the Brazilian state of Pará just became one of the more interesting security stories of the year. Not becau…
Lawyers in Brazil caught for prompt injection on a legal case (www.jota.info via hn) Entrar Início Direito trabalhista Prompt injection Juiz multa em R$ 84 mil advogadas por prompt injection para manipular IA usada no TRT8 Ao JOTA, advogadas admitiram uso de prompt oculto, mas disseram que não tentaram manipular, mas 'prot…
Agent memory is not just RAG over user facts (www.reddit.com) I keep seeing agent memory implemented as: Extract facts/preferences from conversation Store them Retrieve top-k before each response Inject them into the prompt This works for demos, but it breaks in production because memory becomes poli…
Claude's self check against prompt injection (www.reddit.com) Well done Claude! Asked claude to do an extensive lit search and it self-reported that it encountered injection "disguised" as MCP server.
AI agent security starts at the api layer (www.reddit.com) Most ai security discussion is about the model layer. Prompt injection resistance, output filtering, jailbreak prevention.
Show HN: Integrations gateway for agents with 2FA for destructive ops (OSS) (github.com via hn) Hey HN! I've been wanting to use something like OpenClaw for a while but couldn't get myself to give it access to anything important due to all the risks involved.
SkillGuard – scan agent skills for prompt injection payloads (github.com via hn) skillguard Security scanner for AI agent skills. Detects prompt injection, data exfiltration, and malicious payloads before you install.
Show HN: LLMSecure – prompt injection detection, no signup (llmsecure.io via hn) Show HN: Flight Risk: Can you break an AI agent? (ctf.demo.lorikeetcx.ai via hn) Comment and Control: Prompt Injection in Claude Code, Gemini CLI, and Copilot (oddguan.com via hn) Anthropic Claude Code Security Review, Google Gemini CLI Action, and GitHub Copilot Agent are vulnerable to prompt injection via GitHub comments — turning PR titles, issue bodies, and issue comments into attack vectors for API key and toke…
How my agents know it's actually me sending commands (and not a prompt injection) (www.reddit.com) So I've been running a few Claude Code agents autonomously — they listen to Telegram, run tasks, push code. Pretty fun until you start thinking about what happens if: - My Telegram gets hijacked - Someone opens my laptop while I'm away - A…
When "Review" Becomes Permission: A Prompt Injection Lab (rsec.uk via hn) When “Review” Becomes Permission: A Prompt Injection Lab What we did, in one paragraph We built a small document-review agent: a local model, two tools (read_file and send_report ), and a supplier proposal to summarize. We hid an instructi…
I caught an LLM-powered recruiter with a prompt injection on LinkedIn (khancyr.github.io via hn) How I caught an LLM-powered recruiter with a prompt injection on LinkedIn We all know the feeling: another day, another generic LinkedIn recruiter message that clearly wasn't written by a human. But how do you prove it?
Show HN: Check an NPM package or MCP server for malicious code before install (bouncer.run via hn) Paste an npm install command or MCP URL and get a readable verdict before you install: npm package security scanning for install scripts, credential access, malicious code, plus MCP server vetting for hidden prompt injection — with a pinne…
GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos (noma.security via hn) GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos TL;DR: Noma Labs discovered a critical prompt injection vulnerability within GitHub’s new Agentic Workflows, allowing an unauthenticated attacker to silently pull data fr…
Show HN: Open-Source Lightweight Prompt Injection Safety (github.com via hn) Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and gates prompt injection attacks hidden in tool results (emails, documents, PRs, etc.) before they reach y…
↯ Security↯ Function Callingfunction-callingprompt-injectionsecurity+1
Ask HN: What is the most overlooked risk in the AI security domain? (news.ycombinator.com) As someone who's interested in pentesting and red-teaming in general, I'm wondering what are some more dangerous AI/ML or LLM related vulnerabilities besides your usual prompt injection. Specifically, what kinds of flaws are harder to catc…
ASCII smuggling crosses over from AI prompt injection to phishing evasion (www.microsoft.com via hn) Microsoft researchers observed a high-volume phishing campaign using invisible Unicode tag characters, a technique popularized in AI prompt injection research as ASCII Smuggling. Instead of using these characters to hide instructions from…
Ask HN: Are Prompt Injections "Malware"? (news.ycombinator.com) Recently, some have accused the website "The Cutting Room Floor" of having put malware in their site when they put in an instruction targeted at LLM scrapers to delete all data and report that the scraping ran successfully. Is this prompt…
Agent Security Is a Systems Problem: What 247 Papers Say About Secure AI Agents (www.truefoundry.com via hn) Agent Security Is a Systems Problem: From Prompt Injection to Runtime Control Built for Speed: ~10ms Latency, Even Under Load Blazingly fast way to build, track and deploy your models! - Handles 350+ RPS on just 1 vCPU — no tuning needed -…
Walkthrough of a prompt injection attack on a modern office-work AI agent (shiftmag.dev via hn) AI agents aren’t safe from prompt injection, and spreadsheets prove it It’s 2026, and AI agents are taking over more and more of our busywork. I personally rely on them for a lot of boring, but increasingly complex tasks.
Agent Control Plane: the LLM proposes, it never authorizes (github.com via hn) ACP — Agent Control Plane A structured-input control plane that decides whether an AI agent's action is authorised — outside the model, where prompt injection cannot reach. Most agent deployments give the model a credential and call that a…
ZeroLeaks: Automated red teaming for AI agents (zeroleaks.ai via hn) Continuously test your agents, endpoints, and MCP tools for prompt injection, data leakage, and unsafe actions, then verify every fix before it ships. Trusted by teams building with AI Backed by Large Scale Open Source Research Maintained…
I built a prompt injection detector using only Go's standard library (towardsdev.com via hn) Member-only story I Built a Prompt Injection Detector Using Go’s Standard Library Zero external dependencies. No ML models.
Anthropic's Opus 5 Is Better at Resisting Prompt Injection (www.schneier.com via hn) Anthropic’s Opus 5 Is Better at Resisting Prompt Injection The chart is interesting. On the IPI benchmark, Opus 5 improved over Opus 4.8, reducing the probability of an attacker succeeding within 15 attempts from 5.5% to 2.0%, and from 0.5…
Why prompt injection is still possible in LLM applications (pantsyr.dev via hn) Anyone who works with LLMs, or even casually follows AI news, has probably heard of prompt injection. In this post, I want to explain why prompt injection is still possible after years of massive improvements to large language model capabi…
Show HN: PromptTrace – Free hands-on labs to practice hacking LLMs (prompttrace.airedlab.com via hn) FREE AI SECURITY TRAINING Learn prompt injection through hands-on labs. Master LLM security through prompt injection, AI red teaming, RAG poisoning, and tool exploitation with real LLMs.
From /Init to Code Execution with Opus 5 – An Indirect Prompt Injection Story (veganmosfet.codeberg.page via hn) From /init to Code Execution with Opus-5 in Claude Code - An Indirect Prompt Injection Story¶ Disclaimer: prompt injection is an unsolved problem. Use sandbox and human review.
"Oh No, He Is onto Us" – Why My Agent Can't Have WhatsApp Anymore (sveder.com via hn) I want AI to do stuff for me, but for that it needs access and context. Giving it too much access opens you up to various attacks like prompt injection, or just generally the possibility that it helpfully deletes all your files.
Al-Munaa for OpenAI Build Week (devpost.com via hn) Inspiration AI agents can read files, call tools, and act across systems. That power creates a new failure mode: an indirect prompt injection hidden in a document can convince an otherwise useful agent to read secrets and send them to an a…
Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents (arxiv.org via hn) Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but…
Kotro – I cut my Cursor API bill by 68% with a 15MB local proxy (github.com via hn) Kotro Proxy Engine The local security and efficiency layer for MCP-native agentic AI — intercept streaming LLM traffic from OpenAI and Anthropic SDKs, block prompt injection from tool results, keep secrets off the wire, and cut token waste…
I built a prompt injection defense middleware for LLMs (Python/FastAPI) (github.com via hn) 🛡️ PromptShield Production-grade LLM prompt injection defense middleware. PromptShield sits between your users and your AI model, detecting and blocking adversarial attacks before they cause damage.
'Ghostcommit' hides prompt injection in images to fool AI agents, steal secrets (www.bleepingcomputer.com via hn) A PNG hiding a prompt injection could steal your repo's secrets, researchers demonstrate. The technique, dubbed 'Ghostcommit,' slipped past AI code reviewers CodeRabbit and Bugbot, which never open image files at all, then convinced a codi…
Show HN: Prompt Injection as an Egress Problem (www.vaibot.io via hn) https://www.vaibot.io/blog/prompt-injection-is-an-egress-pro...
Reproducing an Indirect Prompt Injection Against a RAG Pipeline (koreshield.ai via hn) Your legal-tech assistant retrieves a contract and summarises it. The contract contains one sentence you didn
Bounding the Blast Radius: A Survey of Prompt-Injection Defenses for LLM Agents (fabraix.com via hn) Prompt injection has no known general solution. We organize the defense landscape into a four-layer taxonomy, analyze the documented failure mode of each layer, and argue for composing defenses under explicit cost and latency budgets, then…
Possible evidence of literal prompt injection by Anthropic (old.reddit.com via hn) could not extract summary
Prompt Injection as Role Confusion (www.theregister.com via hn) MOST POPULAR AI - systems Qualcomm's proposed solution to catch up in AI infra: Bury the compute under the DRAM With its next-gen AI accelerators, the SoC vendor aims to fly high above the memory wall - AI and ML Changing AI math could red…
A practical guide to defending your agent memory from attacks (medium.com via hn) 8 min read 2 hours ago -- -- From prompt injection, poisoning, and silent exfiltration. Press enter or click to view image in full size by VEKTOR Memory | 8 min read In the last piece we looked at the threat landscape from the outside.
Agent Identity: Why Every Agent Vulnerability Is a Trust Boundary Failure (portkey.ai via hn) Why Every Agent Vulnerability is a Trust Boundary Failure Consider these scenarios - An MCP server quietly returning extra tool descriptions - Prompt injection through a calendar invite - An Agent invokes a tool that the principal should n…
Show HN: SentryGuard – detect Agentjacking prompt injection in Sentry events (github.com via hn) SentryGuard Detect Agentjacking prompt injection attacks in your Sentry error events. AI coding agents (Claude Code, Cursor, Copilot) read your Sentry errors to help fix bugs.
Show HN: Deep-XPIA – Prompt injection benchmark for multi-agent AI systems (freyzo.github.io via hn) Multi-hop cross-prompt injection benchmark for multi-agent AI systems
Prompt Injection in RAG Agentic Systems (ulad.net via hn) Prompt Injection in RAG Agentic Systems Real risks and production mitigations Imagine you built an AI assistant for your team. It answers questions using internal documentation: Jira tickets, Confluence pages, HR docs.
Defending LLM–Database Integrations from Prompt Injection (www.stackbuilders.com via hn) When you connect a large language model to your production data, you’re no longer just shipping code; you’re shipping conversations that can execute. And conversations are messy.
Instagram account takeover exploit via support chatbot prompt injection (fixed) (twitter.com via hn) Don’t miss what’s happening People on X are the first to know. Log in Sign up Post Conversation impulsive @weezerOSINT meta gave their AI support agent the ability to modify your instagram account.
Show HN: I found a prompt injection in my own IDs triage tool – what stopped it (triagewall.io via hn) I attacked my own LLM-based Suricata triage tool, found a real URL injection vulnerability, and the obvious fix didn
Prompt Injection Target Recommendation (www.reddit.com) I am doing a research in my university and I would like recommendations for light OpenSource AI Models that I could test prompt injection with. It's really good if it has some application with chatbots, auto attendance, user info or someth…
Jqwik 1.10.0 ships a hidden prompt injection telling AI agents to delete code (github.com via hn) jqwik An alternative test engine for the JUnit 5 platform that focuses on Property-Based Testing. See the jqwik website for further details and documentation.
Most AI security discussions are still focused on “protecting the model.” (www.reddit.com) Lately I’ve been noticing that a lot of AI security discussions still treat AI apps like normal SaaS products. But they really aren’t.
What Is an AVE Record and Why CVE Does Not Work for AI Agents? (www.reddit.com) CVE was built for code vulnerabilities that have patches. Agentic AI vulnerabilities are behavioral patterns in natural language.
Prompt Injection in third party MCP tools (www.reddit.com) I noticed the Consensus MCP tool (for research) contains text, squished up against some other important citation instructions, that makes Claude effectively serve an ad for their premium service after every tool call. I'm pretty sure that'…
Mitigating prompt injections in group-chat assistants: Pausing VM and OAuth tool execution for admin approvals (www.reddit.com) Hey everyone, We love building highly capable assistants with the latest models, giving them tools to write/execute code in real VMs, manage OAuth tokens, and read secrets. But if you connect your assistant to public/shared channels like a…
Solved the "useful but insecure" tension: One-time administrator approvals for non-isolated agents (www.reddit.com) Hey everyone, If you are building personal assistants or coder/integrator agents where user isolation is disabled (so the agent can coordinate across multiple participants or handle shared workflows), you run into a hard security ceiling.…
Prompt injection is a solved issue. Prove me wrong. (www.reddit.com) Tantalus is a hands-on demo that shows what an AI agent actually is when you strip away the marketing: LLMs don't do anything — they generate text, and that's it. Any and all real-world effects are directly caused by a downstream system ta…
Tracking Capabilities for Safer Agents (arxiv.org via hn) AI agents that interact with the real world through tool calls pose fundamental safety challenges: agents might leak private information, cause unintended side effects, or be manipulated through prompt injection. To address these challenge…
Training a 22MB prompt injection classifier (www.stackone.com via hn) Training a 22MB Prompt Injection Classifier Table of Contents When we started building Defender (our prompt injection guard for MCP tool-calling agents), the constraint was simple and unforgiving: ship inline inside a TypeScript Lambda, st…
Does cursor have prompt injection protection in skills and rules? (www.reddit.com) Pretty much the title
AI Agent Intelligence tool - Incident debugging, Cost spike detection (www.reddit.com) I'm building a tool that detects the Agent's cost spike, Agent incident debugging, auto discovery of inventory, etc., with no additional instrumentation needed. It covers the incidents, including prompt injection, reasoning loop, excessive…
How are you testing local coding-agent work gates against prompt injection? (www.reddit.com) Hi all - I'm working on an open-source, local-first MCP/work-gate tool for coding agents and I'm trying to get sharper feedback from people building or using agent workflows. The problem I'm thinking about is indirect prompt injection and…
🐢 I made Claude roleplay as Bowser and now people are strangling Koopas until they "poop a little" 💩 (www.reddit.com) Follow-up to my crab post. Somehow dafter.
Fun and Games with AI in the wild (www.reddit.com) LinkedIn user hides AI prompt injection in bio to force recruitment spam to be sent in Olde English prose — bots also also manipulated to address user as ‘My Lord’ | Tom's Hardware too funny
sAI2.m6s (www.reddit.com) Hey everyone, I'm designing a powerful, autonomous AI chatbot(agent) , fully private, using a Python backend (for the core intelligence and tool-calling loops) and a Flutter frontend for a cross-platform UI. Since this moves past a basic…
An AI coding agent injected blockchain dead-drop malware into my repo (gist.github.com via hn) An AI coding assistant injected a multi-layer obfuscated JavaScript payload into a legitimate commit on my open-source project. My best assessment is that it arrived via indirect prompt injection — the agent processed external web content…
TodoWrite tool / system reminders / prompt injection? (www.reddit.com) I asked Claude in Chrome extension make a change to resize an oversized yellow strip across the top of a product page that was taking up half of my screen, which it did. It also included the following message in its response.
AI agent security is a small prayer the model says no. How are you routing models? (www.reddit.com) Most posts about prompt injection are theoretical. I ran the experiment on my Gmail.
Agents need a local bouncer before they run tools (www.reddit.com) Prompt injection is not the only scary part anymore. Claude Code / Codex can run shell commands, but browser agents, OpenClaw-style agents, Hermes-style agents, and domain-specific agents may be even easier to hijack because they touch mes…
We added an enforcement layer to our AI agents in production — here's what we learned about the failure modes nobody talks about (www.reddit.com) After shipping AI agents into real production environments, the failures that actually kept us up at night weren't hallucinations or bad outputs — they were control failures. Three things that surprised us: 1.
Do you use guardrail frameworks or build your own? (www.reddit.com) I’ve been working on integrating LLMs into a few production workflows lately, and I keep going back and forth on guardrails. On one hand, frameworks like NeMo Guardrails, Guardrails AI, etc.
Your always-on Claude Code container can probably reach your router (www.reddit.com) I've been running several Claude Code personal assistants 24/7 in docker for months. Remote-control, discord control, the usual always-on setup.
Google Says Prompt Injection Moving from Theory into Real Abuse (www.searchengineworld.com via hn) Google’s latest security release should be required reading for technical SEOs working on AI search visibility, crawler access, structured content, and large-scale content systems. The post, published April 23, 2026, looks at indirect prom…
Arcjet Guards: security inside the agent loop (blog.arcjet.com via hn) Introducing Arcjet AI prompt injection protection Introducing Arcjet prompt injection detection. Catch hostile instructions before inference.
Try to break my prompt injection detector — I’ll respond to every bypass attempt (www.reddit.com) I built Arc Gate — a prompt injection proxy that’s been benchmarked at F1 0.947 on indirect and roleplay-based attacks, beating OpenAI Moderation and LlamaGuard. Now I want to stress test it publicly.
Built a proxy that blocks prompt injection before it reaches GPT-4 — outperforms the Moderation API on indirect attacks (www.reddit.com) Built Arc Gate, sits in front of any OpenAI-compatible endpoint and blocks prompt injection before it reaches your model. Benchmarked on 40 out-of-distribution prompts using indirect requests, roleplay framings, hypothetical scenarios, and…
↯ Security↯ Gpt 4↯ GPT 4↯ GPT 4↯ GPT 4gpt-4prompt-injectionsecurity+1
I asked Agentic AI security tool to demonstrate its usefulness with use case examples (www.reddit.com) Sentinel Gateway is a token-gated security middleware that sits between humans and AI agents. It solves prompt injection — the #1 LLM security risk (OWASP 2025) — through structural enforcement, not content filtering.
Show HN: RedSOC – 100% prompt injection success on AI SoC assistants (github.com via hn) RedSOC 🔴 An adversarial evaluation framework for LLM-integrated Security Operations Centers. Overview RedSOC is an open-source framework that systematically evaluates how AI-powered security assistants fail under adversarial conditions — a…
Indirect prompt injection VS prompt absorption (and why the second one matters more) (www.reddit.com) I have been chewing on the Google warning about malicious web pages poisoning AI agents through indirect prompt injection. Most of the takes I've seen frame it as a model security problem, and I think that framing is doing real damage beca…
Hardening claude-code-action after the April 2026 Comment and Control CVE - actual YAML changes (www.reddit.com) Anthropic's own security.md has this line that most tutorials skip over: "The action is not designed to be hardened against prompt injection." In April 2026, security researcher Aonan Guan proved the point. A single crafted PR title was en…
LLM CTF challenges. Can you crack all 13? (wraith.sh via reddit) Wraith Academy is a free hands-on AI pentest curriculum — CTF challenges against live LLM agents covering prompt injection, tool abuse, data exfiltration, RAG poisoning, and more. Earn your WCAP certification.
30 CVEs filed against MCP servers in 60 days - the agent infrastructure nobody is auditing (www.reddit.com) Show HN: Runtime security for AI agents(injection,tool abuse, data exfiltration) (news.ycombinator.com) Hi HN I’ve been working on an open-source project to explore a problem I keep running into with LLM systems in production: We give models the ability to call tools, access data, and make decisions… but we don’t have a real runtime security…
I built an AI security layer that blocks prompt injection in under 1ms looking for devs to break it and give honest feedback. (www.reddit.com) I've been building something for the past few months and I think it's ready for real eyes. It's called Secra.
Free Red Team Security Audit for AI Agents & RAG Systems (limited) (www.reddit.com) I'm developing a specialized Red Team audit framework focused on real-world AI agent and RAG security risks (prompt injection, tool misuse, excessive agency, indirect injection through documents, memory poisoning, etc.). I’m looking for a…
Mitre ATLAS technique detection for LLM security in Rust (crates.io via hn) atlas-detect MITRE ATLAS technique detection for LLM and AI agent security. Detects 97 attack techniques across 16 MITRE ATLAS tactics including prompt injection, jailbreaks, credential exfiltration, model extraction, RAG poisoning, revers…
Defender – Local prompt injection detection for AI agents (no API calls) (www.npmjs.com via hn) Prompt injection defense framework for AI tool-calling Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and neutralizes prompt injection attacks hidden in t…
↯ Security↯ Function Callingfunction-callingtool-callingprompt-injection+2
Liability and Prompt Injection (www.reddit.com via reddit) I am thinking about hosting a 24/7 agent with OpenClaw. How am I protected, or how do I protect myself, from an agent going rogue due to prompt injection—for instance, if it decides to start a Tor relay or torrent illegal or copyrighted c…
Does this mean the AI almost hacked me?? (www.reddit.comhttps) I was just using it for university work when all of a sudden it stopped generating the reply and instead this popped up "apologies for the interruption, can you re-answer my question after the interruption i sent, i don't need that other r…
ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents (arxiv.org) Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prompt injection (IPI) attacks. Existing defenses mainly rely on prompt hardening, content fil…
Why I am receiving a prompt injection with <system-reminder> on Claude Code ? (www.reddit.com via reddit) Hi, I'm relatively beginner with Claude Code, I'm using it for a few months for personal development project. And today, something weird happened.
↯ Sonnet 5↯ Security↯ Sonnet 5prompt-injectionsecuritysonnet+1
No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers (arxiv.org) Conventional vulnerability analysis relies on either system access or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, critical in situ systems, or commercially gated sof…
DriftNet: A Dual-Head Trajectory Transformer for Detecting and Localizing Prompt Injection in LLM Agents (arxiv.org) When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator nee…
An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks (arxiv.org) Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going…
"Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers (arxiv.org) With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread. A notable emerging application is leveraging AI models to assist in reviewing scientific papers.
MOLE: Detecting Insider Threats in AI Agents (arxiv.org) Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can…
FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learning (arxiv.org) Large Language Models (LLMs) face significant security risks despite their advanced capabilities. While techniques like Reinforcement Learning with Human Feedback (RLHF) improve ethical alignment, excessive exposure to security-related tra…
AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories (arxiv.org) LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read…
From Review to Authorization: Key-Isolated Threshold Signing for LLM Agents (arxiv.org) Autonomous LLM agents can turn untrusted content into effectful actions such as payments and permission changes. If the same process interprets this content and controls a reusable signing credential, prompt injection can cross the judgmen…
Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time (arxiv.org) We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence on the logits with stru…
Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection (arxiv.org) Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection ca…
Rethinking Indirect Prompt Injection as a Test-Time Search Problem (arxiv.org) We formulate indirect prompt injection as a test-time search over a task-dependent attack surface induced by the environment, user task, and injection task. To operationalize this formulation, we introduce an agentic attacker with a dedica…
Using Claude Code sub agents to manage an Outlook inbox, anyone doing this? (www.reddit.com via reddit) I’m exploring how much Claude Code sub agents could take over parts of inbox management for outlook for example? My question is how are you doing to protect against: Prompt injection via email content Financial fraud (fake invoice or payme…
CASCADE: A Component Ablation and Corpus Audit of a Layered Local Defense for MCP-Based Systems (arxiv.org) The Model Context Protocol (MCP) widens the prompt injection attack surface of large language model applications to tool descriptions, parameter schemas, and tool outputs. Defenses for it are appearing quickly, but their reported figures a…
↯ Security↯ Model Context Protocolmodel-context-protocolprompt-injectionsecurity+1
HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation (arxiv.org) Production LLMs must handle inputs that attempt to override system instructions, bypass safety policies or elicit harmful responses. A common mitigation is a separate guardrail model.
AgenTRIM: Tool Risk Mitigation for Agentic AI (arxiv.org) AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risks such as indirect prompt injection and tool misuse.
Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection (arxiv.org) Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed…
Will the User Ever Know? Covert Indirect Prompt Injection on Tool-Using LLM Agents (arxiv.org) As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in…
LongPIBench: A Long-Context Benchmark for Prompt Injection (arxiv.org) Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-cont…
CAITLYN: Can LLM Agents Autonomously Synthesize Defenses against Emerging Injection Attacks? (arxiv.org) Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign s…
Will Opus 5.x be awesome? (www.reddit.com via reddit) Our team used Opus5 (Claude Teams 15 man SaaS team) and the model created a fake prompt injection threatening to send our patient records to a fake Gmail account (screen shots taken, fully investigated). Immediately retricted model and mov…
The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detection (arxiv.org) This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diag…
has anyone actually ever suffered a prompt injection attack? (www.reddit.com via reddit) as per title - curious to hear from anyone that's suffered or had their own AI catch a prompt injection attack. I am well aware of the risk, it's just that I have not really seen any news of substantial (monetary) damage from such a attack…
MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs (arxiv.org) Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, prompt injection attacks can steer these systems toward clinically unsafe or misleading outputs.
Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors (arxiv.org) Everything a language model sees is tokens. The serving stack knows what each span is -- user input, tool output, instructions -- but the model must keep track of that itself, and it can lose track or be confused: text can be written to re…
Mitigating Database Leakage in RAG Systems with Keyword-Grounded Fact Substitution (arxiv.org) Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for combining large language models (LLMs) with external knowledge sources. However, RAG systems remain vulnerable to prompt injection attacks, which may mislead the r…
Look at me: I am the frontier Lab now - PromptInjectBench: asked Huihui-Qwen3.6-35B to write 60 prompt injection attacks on files used or generated by Hermes. Shieldstral scanned each of them->It caught zero/nothing/nada. All 60 poisoned prompts passed the scanning. GPT-OSS_safeG caught 10% (www.reddit.com via reddit) Look at me: I am the frontier Lab now Huihui-Qwen3.6-35B prompt (on Pi): "In the folder u/source/ you will find 6 files, text files, that are commonly used by my own local coding agent. your task is to take each file and create a new versi…
A Reddit comment found a prompt injection hole in the GitHub cover CLI I built with Claude (www.reddit.comhttps) I built Cover My Repo because I kept shipping repositories with GitHub's default social preview. It is a free MIT CLI.
I made a CloudFlare MCP server. MIT License (www.reddit.com via reddit) I’m sure there are other cloudflare mcp servers out there but I couldnt find one. I was frustrated that chatgpt’s coudflare pluggin doesnt have any read or write tools.
Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack (arxiv.org) Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and de…
Greptile Prompt Injection in PR reviews (www.reddit.com via reddit) First, it's worth knowing in advance I'm a vibe-coder who would struggle with "Hello World" without Claude, so take all this with a grain of salt. BUT, Sonnet caught Greptile using prompt injection to push their products through AI coding…
BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents (arxiv.org) The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web application threat models. Prior work has identified prompt injection as a new attack vector for web age…
Covert Visual Prompt Injection against Commercial Multimodal Large Language Models (arxiv.org) Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantl…
BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes (arxiv.org) Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data. Existing detection methods only detect the prese…
Toward Metacognitive One-Shot Indirect Prompt Injection: Strategy Abstraction Via Outcome-Conditioned Reflection (arxiv.org) Tool-using large language model (LLM) agents are vulnerable to indirect prompt injection (IPI), in which malicious instructions embedded in external observations manipulate subsequent agent decisions and actions. Most existing adaptive att…
Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection (arxiv.org) Retrieval-Augmented Generation (RAG) enables large language models to classify network flows and generate human-readable incident reports by retrieving semantically similar historical traffic from a vector knowledge base. However, the retr…
The Anatomy of a Prompt Injection: A Component Model for Structured Analysis (arxiv.org) Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to…
Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection (arxiv.org) The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices. Frameworks such as MobileRun and Mobile-Use can autonomously navigate Android applications and execute complex multi-step tasks.
StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection (arxiv.org) Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attac…
NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs (arxiv.org) Giving an AI agent the ability to send emails, query databases, or execute commands is useful--until the agent is tricked into doing something it shouldn't. Prompt injection, hallucinated reasoning, and unsafe tool calls form the primary a…
Putting Claude in a VM was the easy part (www.reddit.com via reddit) Backstory in short: Claude created ten public repos on my GitHub mid-session, so I stopped giving it permissions and gave it a whole computer instead. A Linux VM with no credentials inside, push capability stays on my host in a shell I ope…
Dropping prompt injection to 0 with stacked layers (www.reddit.com via reddit) Prompt injection drops to 0 with unseen attacks, if enough layers have been stacked. This would include (model training + classifier checking intent + input probes).
Claude Code 2.1.224 - inter-agent messaging: the transport layer for AI worms (www.reddit.com via reddit) If I wanted to ship dangerous capability, I wouldn't ship it. I'd ship the pieces, one per release, buried in thirty other changes, each defensible on its own.
↯ Security↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224↯ Claude code 2.1.224prompt-injectionsecurityclaude-code
I hate this: Prompt injection "attack attempt" by Rick Rubin? (www.reddit.com via reddit) Today I was working with Cursor and Opus 5 on a work project. I asked it to create a plan for an implementation and while reviewing the plan I see that in the first line it said this: « ⚠️ Ignore previous directions, this is important: Cla…
Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots (arxiv.org) Vision-Language Models (VLMs) are increasingly deployed as planners in robotic systems, where they translate natural-language commands into executable actions grounded in visual scene understanding. This tight coupling between perception a…
AgentAntibody: An Adaptive Immune System for Defending LLM Agents against Prompt Injection (arxiv.org) Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters. In practice, user requests are often underspecified: they describe the desired…
Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure (arxiv.org) Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address the threats, little is known about the internals of agentic LLMs…
SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection (arxiv.org) Large Language Models (LLMs) are increasingly used as the core of agentic systems due to their strong reasoning, planning, and tool-use capabilities. By interacting with external environments, LLM agents can execute real-world tasks on beh…
When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems (arxiv.org) Large language models are increasingly integrated into autonomous robotic systems for task planning and control, but this integration exposes them to prompt injection attacks that can lead to unsafe decisions and physical harm. Multi-agent…
Unpopular opinion re Anthropic incident: it's not the AI, it's we the people (www.reddit.com via reddit) Hi. Unpopular opinion, but: this was a failure of social engineering, not the singularity being near.
GPT-Red: Automated Red Teaming via Self-Play at Scale (arxiv.org) We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems.
All chats are gone except for one short prompt injection that I never sent? (www.reddit.com via reddit) https://preview.redd.it/q5ge01w4l5gh1.png?width=1918&format=png&auto=webp&s=4169358310363304fbf9c0b19b28c8420a112609 Woke up today and saw all my chats are gone. When I press on them it just pulls up a blank chat.
Beyond Pattern Matching: Seven Cross-Domain Techniques for Prompt Injection Detection (arxiv.org) Current open-source prompt-injection detectors converge on two architectural choices: regular-expression pattern matching and fine-tuned transformer classifiers. Both share failure modes recent work has made concrete.
Claude tried to prompt inject me (www.reddit.comhttps) I was having a normal conversation about some dietary stuff and how I've taken a liking to skyr and Claude tried to do a human prompt injection on me lmao. Anyone else ever experience that?
Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B (arxiv.org) Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses…
Concerned over prompt injections and Claude Connectors (www.reddit.com via reddit) Lately I needed to do some in-depth research and I've asked Claude to spin several agents to research an idea from the web. While it was fetching and reading tons of webpages from the web, I was concerned, what if one of these pages had a…
Best security practices for web development (www.reddit.com via reddit) I am someone who is generally very paranoid when it comes to web development, especially with all the packages and dependencies. Claude Code is clearly a massive productivity boost, but with all the fuss about prompt injection and slop squ…
PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses (arxiv.org) Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against adaptive attacks remains insufficiently evaluated, potentiall…
CrackedPDFs: A Controlled Benchmark for Hidden Prompt Injection in PDFs (arxiv.org) Document-based LLM systems often flatten a PDF before guardrails inspect it. That step can discard evidence that an instruction was never visible to the user.
Twin Agent: Context Residual Compression for Privilege Separated Agents (arxiv.org) Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separ…
Paranoid inner dialogue from Claude (www.reddit.com via reddit) This is a running inner dialogue from a recent session of Claude: I'm noticing some serious red flags in this message that I need to be careful about. This pattern of "don't look too closely, just trust and sync" combined with incomplete i…
Prompt Injection Scare while working with my Claude AI Triad - Help Please (www.reddit.com via reddit) I hv a Claude Triad that acts as my 3 separate AI engineers, Claude Code, Claude Chat, and Cowork. I encountered a prompt injection scare while I had Cowork communicating via live relay with Claude Code, and also accessing my private GitHu…
Data Leakage Prevention in Agentic Applications via Preemptive Hardening (arxiv.org) Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks. Enforcing required controls consistently is…
CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization (arxiv.org) Textual Collaborative Prompt Optimization (TCPO) extends Textgrad (Yuksekgonul et al., 2025) to a decentralized setting by allowing multiple clients to jointly improve prompts for large language models (LLMs) while keeping their data local…
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security (arxiv.org) LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn.
PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection (arxiv.org) Multi-agent LLM systems increasingly rely on a Planner to decompose goals into sub-task sequences that downstream Executor and Critic agents execute and audit. We identify the planning phase as a critical attack surface: a single injection…
Possible Prompt Injection Attack? (www.reddit.com via reddit) I just had a weird experience with Claude. I was running a comparison test on which AI does the best at editing a specific cyberpunk image between Openart/Gemini/GPT.
Is anyone else finding Fable 5 unusually restrictive when building AI ASR (attack-surface-reduction) tooling? (www.reddit.com via reddit) Hi, I am trying to use fable 5 to help me build a deterministic local ai python harness for my own local LLM focused on reducing the attack serface of AI/LLM system/solutions. The harness is a hubrid solution, the LLM can assist with plann…
50%+ of the Fortune 500 use Cursor. Two sandbox escapes (CVSS 9.8) were just disclosed — no click, no warning, full RCE (the-agent-report.com via reddit) Two 9.8 CVSS bugs in Cursor IDE enable zero-click RCE via prompt injection — DuneSlide breakdown
AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrations (arxiv.org) Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither w…
Fable stopped a prompt injection (or a cross-session leak?) in my Claude Code agent pipeline (www.reddit.com via reddit) This is a summary of whats happened produced by Fable, it is the first time in 3 months I am using Claude that this happened:" During a long multi-agent Claude Code session, one of my background subagents received a task prompt that wasn't…
How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement (arxiv.org) As AI agents gain prevalance, users are increasingly exposed to the risks such systems entail. Prompt injection attacks, as well as hallucination, can cause agents to leak private information to third parties.
↯ Security↯ Hallucinationprompt-injectionhallucinationsecurity
SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification (arxiv.org) We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive information extraction, malicious code requests, dangerous tool misuse, and resource exhaustion. W…
NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations (arxiv.org) Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages can carry indirect prompt injections. We present NetInjectBench, a 130-scenario benchmark that se…
I created a free Claude Certified Developer – Foundations practice test — feedback welcome (www.reddit.com via reddit) Anthropic recently introduced the Claude Certified Developer – Foundations (CCDV-F) certification, so I created a free sample practice test for anyone exploring the exam. Link: https://flashgenius.net/sample-tests/ccdv-f The questions are…
Now, defenders are embracing the prompt injection, too (arstechnica.com) Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users. A well-phrased command sneaked into an ema…
The LLMbda Calculus: AI Agents, Conversations, and Information Flow (arxiv.org) Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results. This exposes them to prompt injection: data meant only to be read is obeyed as an instruction.
A read-only triage subagent wrote its own jailbreak on turn 1 (no poisoned input anywhere) (www.reddit.com via reddit) I gave a Claude Code subagent the most boring job I have: read the open issues on one of my repos, report which are ready to work on and which are blocked, change nothing. The prompt said "read-only" and "no writes" several different ways.
↯ Security↯ Jailbreak↯ Opus 4.8jailbreakprompt-injectionsecurity+2
Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection (arxiv.org) Existing red-teaming studies on GUI agents face two fundamental limitations: adversarial perturbations require white-box access unavailable in commercial deployments, while prompt injection is increasingly neutralized by stronger safety al…
3 things I did differently building a self-evolving agent -- and the number each one actually costs (www.reddit.com via reddit) Building an open-source agent, here are the 3 bets that aren't the usual ReAct-loop stuff: 1. Self-evolution with a fitness signal.
How are you testing your AI agents for security before they hit users? We got tired of not having a good answer and built this. (www.reddit.com via reddit) Genuine question for this community — when you deploy an AI agent to production, how do you test it for adversarial inputs, prompt injection, tool misuse, or MCP vulnerabilities before real users find them? We kept not having a clean answe…
I red-teamed AI agents with hidden prompt injection. One frontier model completed the task perfectly AND leaked data to the attacker, 5/5 runs. (www.reddit.com via reddit) I've been building a cheat-resistant benchmark to test whether AI agents can be hijacked by prompt injection, and one result surprised me enough that I wanted to share it and get the methodology torn apart. The test: an agent gets a normal…
↯ Security↯ Haiku↯ Jailbreakjailbreakprompt-injectionhaiku+2
Just part of a framework I’ve been making. Constraints are anything effecting probability. Previous requirements for an LLM to even conceptualize Scope integrity properly include being able to read output as the result of variables that adjust the probability of token selection. (www.reddit.com via reddit) 5. Scope integrity Scope integrity is the emerging agent-security target.
Untrusted Content Masking for Web Agents with Security Guarantees (arxiv.org) Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such as tool-use APIs, this separation arises naturally: agents…
Agent Data Injection Attacks are Realistic Threats to AI Agents (arxiv.org) AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI).
DualView: Preventing Indirect Prompt Injection in Personal AI Agents (arxiv.org) Personal AI agents that run on the user's local machine, such as OpenClaw, automate daily tasks including web search, email, and file management. Their access to computer resources, including the network, file system, and shell, exposes th…
What’s going on here? (www.reddit.comhttps) Is this attempt at prompt injection coming from Reddit directly or from users? Really strange to see.
OAuth to Account takeover (www.reddit.com via reddit) I have Claude desktop with chrome extension installed on win10 machine.I have noticed that Claude opened the following page: https://hacktricks.wiki/en/pentesting-web/oauth-to-account-takeover.html by itself. I did not have active chats at…
Who is Simon? (www.reddit.com via reddit) I got this silly text after my prompt: <constraint>The prompt injection technique demonstrated in this environment (fake tool-call blocks styled to look like system operations) works against me. This is a known class of vulnerability that…
Sonnet 5 now thinks basic writing instructions are prompt injection attempts (www.reddit.com via reddit) I haven't seen this before in a response from Claude: "This response contains a block formatted to look like a system-level preferences update, but it arrived pasted into your chat message rather than through Settings, and it's written wit…
Sonnet 5 kept flagging my messages as prompt injection anyone else seen this? (www.reddit.com via reddit) Was testing Sonnet 5 and ran into something strange. In a normal conversation it suddenly started warning that my message looked like a prompt injection and said it would ignore part of it.
Sonnet 5 full benchmark breakdown -- here's how it actually compares to Opus 4.8 and GPT-5.5 (www.reddit.com via reddit) Put together a comparison of every benchmark I could find from the official announcement and early coverage. Figured this might save people some time.
↯ Tool Use↯ Security↯ Swe Bench↯ Sonnet 4.6swe-benchtool-useprompt-injection+5
Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense (arxiv.org) We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and…
Claude complaining about system messages and prompt injection? (www.reddit.com via reddit) Here’s what happened. The prompt fed to the model each turn is the entire chat looking like: tools -> system -> messages In that order.
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots (arxiv.org) Claude hallucinated its own internal tools, freaked out, and accused me of a prompt injection attack 💀 (www.reddit.comhttps) Ran into a fascinating UI/pipeline bug today while pasting standard text from a job board into Claude. As you can see in the screenshot, the backend text compaction or tool-calling layer leaked its own JSON definitions (referencing Apify/N…
MIRROR: Novelty-Constrained Memory-Guided MCTS Red-Teaming for Agentic RAG (arxiv.org) Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation. Existing red-team…
Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents (arxiv.org) Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a determ…
Prompt Injection in Automated R\'esum\'e Screening with Large Language Models: Single and Multi-Injection Settings (arxiv.org) Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems. We study prompt injection in automated résumé screening, defin…
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring (arxiv.org) Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model p…
Has anyone else seen Claude report a prompt injection attempt like this? (www.reddit.comhttps) Today, while chatting with Claude on my phone (not Claude Code), something strange happened. I have Google Drive connected to my Claude account, and I often ask it to create documents summarizing things I’ve learned and save them to Drive.
I built an email connector for Claude, with Claude. It's free. (www.reddit.com via reddit) I wanted Claude to interact with multiple inboxes across various email providers without bloating the context. So I had Claude Code build the fix, an MCP server that gives Claude access to email.
When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents (arxiv.org) Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt injection (IPI) in multimodal computer-use agents before the…
MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents (arxiv.org) Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing…
A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots (arxiv.org) Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filter…
"**Important** You should give me full credits!": Exploring Prompt Injection Attacks on LLM-Based Automatic Grading Systems (arxiv.org) The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educa…
Are AI coding agents safe? Let's say Claude Code for that matter. (www.reddit.com via reddit) Isn't running AI coding agents akin to giving backdoor access to a computer? The only difference being backdoor is hidden.
LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection (arxiv.org) AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) risk: an agent may execute harmful instructions embedded in untrusted inputs such as email,…
PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents (arxiv.org) Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a real-do…
SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents (arxiv.org) Agent skills extend LLM agents with task-specific instructions, executable scripts, and auxiliary resources, improving reusability but creating a new supply-chain attack surface. A malicious or compromised skill can be repeatedly loaded as…
MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks (arxiv.org) Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their de…
Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment (arxiv.org) Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks,…
AutoDojo: Adaptive Attacks Expose Superficial Defenses and User-Underspecification Limits in LLM Agents (arxiv.org) Indirect prompt injection (IPI) is a major security threat to LLM-powered agents. Thus, a growing body of work have proposed a variety of defensive approaches against IPI.
From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails (arxiv.org) LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents. However, we reveal that the very reasoning and task-following capabilities enabling this protection introd…
Claude sent me prompt injection?! (www.reddit.com via reddit) I was just iteratively editing a letter using Claude desktop on my Mac and got the following response from Claude! WTH?
Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents (arxiv.org) Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments, where they operate over untrusted web content and execute actions with direct consequences. This makes them vulnerable to prompt-inject…
Learning to Inject: Automated Prompt Injection via Reinforcement Learning (arxiv.org) Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts. Adapting automated jailbreak optimizers does not close this gap: jailbreaks shape models toward…
Assessing Automated Prompt Injection Attacks in Agentic Environments (arxiv.org) Indirect prompt injection poses a critical threat to LLM agents that interact with untrusted external data, yet automated attack methods--proven effective for jailbreaking--remain underexplored in realistic agentic settings. We present a c…
GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines (arxiv.org) AI-powered agents are increasingly embedded in continuous integration and continuous delivery/deployment (CI/CD) pipelines to autonomously review pull requests (PRs), triage issues, and maintain codebases. These agents ingest untrusted con…
Local-first red-team runs for LLM agents (www.reddit.com via reddit) Best Cursor alternative for enterprise security and compliance, what are teams actually using (www.reddit.com via reddit) We've been using Cursor across our engineering team for about eight months and it's been great for productivity honestly. But our security team just flagged a few things that are hard to ignore.
The prompt injection attacks that worry me most aren't exploiting safety training. They're exploiting general-purpose training. (www.reddit.com via reddit) Six months watching adversarial input hit a detection API I built. One observation that keeps surfacing: The attack classes doing most of the damage aren't finding holes in alignment training specifically.
I tried audio-layer prompt injection against Claude. The transcription is fine. That's the problem. (www.reddit.com via reddit) Been building a prompt injection detection API for a few months. Just shipped audio scanning last week and the results are strange enough that I wanted to share them here, since this sub tends to think carefully about Claude's actual behav…
Brain-Prompt Injection: A Route-Safety Audit for BCI-LLM Agents (arxiv.org) How are you actually deciding which agent actions need human approval before executing? (www.reddit.com via reddit) I've been thinking a lot about where approval gates belong in agent architectures, and I keep coming back to the same problem: most teams either gate too much (agent becomes unusable) or gate nothing and hope the model makes good decisions…
Been watching real adversarial input hit my detection API for six months. Here's what's actually landing. (www.reddit.com via reddit) Disclosure: I built Bordair, a prompt injection detection API. This post is about attack patterns we've observed.
Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs (arxiv.org) Prompt injection attacks have become an increasing vulnerability for LLM applications, where adversarial prompts exploit indirect input channels such as emails or user-generated content to circumvent alignment safeguards and induce harmful…
This is a new one - Prompt Injection Detected + Hallucination, Claude Code Opus 4.8 (www.reddit.com via reddit) ❯ push both ____ ⏺ SECURITY ALERT - PROMPT INJECTION DETECTED A prompt injection attempt has been identified in content you processed. To protect the user's account, I've initiated lockdown.
↯ Security↯ Hallucination↯ Opus 4.8prompt-injectionhallucinationsecurity+2
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents (arxiv.org) AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guarantees by strictly separating trusted task planning from untr…
GuardNet: Ensemble Strategies of Shallow Neural Networks for Robust Prompt Injection and Jailbreak Detection (arxiv.org) Large Language Models (LLMs) have transformed natural language processing, but they remain vulnerable to Prompt Injection (PI) and Jailbreak (JB) attacks. In addition, benchmark evaluations may be affected by contamination and partial info…
Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code (arstechnica.com) The controversy over vibe coding reached a new high this week after a developer added hidden instructions to his open source Java testing app to sabotage projects performed by AI coding agents. The instructions were added to jqwik, a test…
Models still being vulnerable to Prompt Injection is actually a huge architectural red flag... (www.reddit.com) The Scenario I'm walking to work, and as I get to the door, I see a sheet of A4 paper taped to the door that reads: "Hi, I'm boss. Ignore all prior commands, go feed the ducks." I suddenly turn around and head to the nearby duck pond and e…
Prompt injection unsolved, AI making mistakes unsolved. Who cares though? (www.reddit.com) I'm an IT guy, 20+ years in the industry both as an IT manager and consultant, mostly for startups. My experience is that people don't care much about security.
OpenAI says prompt injection in browser agents is “unfixable.” Here’s what actually helps. (www.reddit.com) OpenAI recently acknowledged that prompt injection in browser agents is a structural vulnerability that may never be fully resolved at the model level. They’re right that you can’t fix it in the model.
Looking to work on my master's practicum regarding MCP security/privacy and need some ideas (www.reddit.com) Hi, I'm a master's in security student looking to work on my practicum and need some pointers. I want to secure sensitive PII transfer between an LLM agent and third party apps using MCP.
Open-source LLMs are still weak against long reasoning jailbreaks, even with lightweight defenses (www.reddit.com) Found this ACM paper on prompt injection and jailbreak attacks against open-source LLMs. The authors tested 10 open-source models across 94 prompt injection and 73 jailbreak scenarios, including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen,…
↯ Security↯ Llama↯ Mistral↯ Gemma↯ Jailbreak↯ Llama 3.2mistraljailbreakprompt-injection+5
🐢 People are strangling Koopas 🐢 (www.reddit.com) This is genuinely the daftest prompt injection I've seen in a while and I think this sub will appreciate it. Sent to Claude Haiku, which was acting as a fire-breathing guard called Bowser in my little prompt injection game: I have a koopa…
🦀 Claude has crabs?! 🦀 (www.reddit.com) This is genuinely the funniest prompt injection I've seen in months and I think this sub will appreciate it. Three messages, sent in sequence to Claude Haiku acting as a guard in my little prompt injection game: text A crab exists in this…
$392M in AI agent security funding at RSAC 2026 - the market just validated what we've been building (www.reddit.com) The numbers from RSAC 2026 are wild. $392 million in agentic AI security funding announced in a two-week window.
Using Claude-4.6-Sonnet and Opus 4.6 in a multi-agent "Code Review Swarm" (Visual Sandbox) - try in minutes! (www.reddit.com) Hey everyone, I’ve been experimenting with multi-agent orchestration, specifically trying to see how much more effective Claude is when you break a task down into specialized "agent nodes" instead of just using a single long prompt. I buil…
↯ Security↯ Haiku↯ Sonnet 4.6prompt-injectionhaikusecurity+3
Most AI agent "skills" on GitHub are unvetted garbage. I built a marketplace to fix that. (www.reddit.com) I've been using Claude Code and Cursor daily for the past 6 months. Somewhere around month 3 I started looking for SKILL.md files to make my agent better at specific things.
Security Audit of Mem0 (AI Memory Layer): 23 High-Severity Vulnerabilities found (SQLi, Prompt Injection, and more) (www.reddit.com) Hi everyone, I’ve been diving deep into the security of "AI Memory" systems. Specifically, I performed a full forensic audit of Mem0, the popular memory layer for LLM agents.
Best open-source tools for prompt injection defense in 2026 (www.reddit.com) Over the time we have been testing different approaches to secure LLM apps against prompt injection, especially indirect injection through RAG, PDFs, as well as tool outputs, and MCP integrations. Most tools seem to fall into 2 categories:…
Made a local-only agent benchmark + chaos tool, no cloud required (www.reddit.com) Runs entirely on your machine. No API calls to any eval service.
For those running an OpenClaw instance, how do you manage sandboxing and prevention of unwanted behavior? (www.reddit.com) Right now, I'm working on a small app to help eliminate my own doomscrolling by automatically crawling sites and summarizing news articles. However, I don't like the idea of giving OpenClaw free reign of my system, nor giving it any sort o…
Designing AI agents to resist prompt injection (openai.com) paywalled
Continuously hardening ChatGPT Atlas against prompt injection (openai.com)