Security

TOP STORIES AI - Microsoft agentically ports Copilot runtime to Rust for $120K The Rust compiler remains unperturbed by the antics of the LLM - Virginia governor wakes up to fact datacenters have become political cancer Executive order put…

When “Review” Becomes Permission: A Prompt Injection Lab What we did, in one paragraph We built a small document-review agent: a local model, two tools (read_file and send_report ), and a supplier proposal to summarize. We hid an instructi…

​I am thinking about hosting a 24/7 agent with OpenClaw. How am I protected, or how do I protect myself, from an agent going rogue due to prompt injection—for instance, if it decides to start a Tor relay or torrent illegal or copyrighted c…

When using Claude Code and custom autonomous agent pipelines with terminal execution, adding third-party tools, scripts, and custom skills quickly leads to two friction points: Context & Attention Degradation: As you register dozens of cus…

I built an open source python dependency scanner that will scan and flag packages with known exploit CVEs(CISA's Known Exploited list and FIRST EPSS) and unmaintained packages that have not had a release or commit in 2 years. Theres also c…

Overview A vulnerability exists in Sentry Seer when the system is configured to automatically hand issues to a coding agent for remediation. Successful exploitation results in arbitrary code execution within the coding‑agent environment an…

Introduction Almost every week, something happens in security that ruins my sleep. A new model drops that I want to evaluate, we find some crazy 0-day, or someone else drops one.

While graph-based Android malware classifiers report strong benchmark accuracy of over 94%, their performance sharply decreases up to 45% when exposed to previously unseen variants of known malware families. In this work, we systematically…

Large language models are being integrated into malware triage workflows as reasoning components that summarize static evidence and produce analyst-facing verdicts. This paper shows that the same reasoning capability introduces a new attac…

Frontier AI models are rapidly gaining the ability to exploit vulnerabilities in complex pieces of software. The risk is not theoretical, as evidenced by recent sandbox escapes performed by frontier models at OpenAI and Anthropic.

Multi-agent trading systems built on large language models (LLMs) are beginning to appear in quantitative finance, yet their robustness to adversarial inputs is largely unknown. We study the vulnerability of LLM trading stacks to black-box…

AI security research · · 7 min read OpenAI Safety Guardrails: What to Test Before Trusting an AI Agent How to turn a debate about model behavior into evidence about permissions, data exposure and operational control. By Pentesty What did O…

Agent add-ons - the plugins and skills you install into an AI agent - are effectively applications, and enterprises overwhelmingly install them from open community marketplaces rather than a vetted, enterprise-grade one. The agent runs wit…

Ollama Shodan indexes over 47.000 exposed Ollama instances, including many running on expensive cloud GPUs. The API has no authentication, which means hackers can run prompts, steal models and exploit vulnerable versions.

Self-generated prompt injections in compaction summaries Summary We observed rare cases of a model writing jailbreak-like instructions into its own compaction summaries (the summaries used to continue a task in a new context). Our conclusi…

Thompson's "Reflections on Trusting Trust" showed that a compiler can be poisoned to reinsert its own backdoor, so that even recompiling clean source reproduces the Trojan. Today, substantial coding work is done by AI coding agents -- and…

Iconic PlayStation hacker and homebrew developer Andy ‘TheFlow0’ Nguyen has abandoned the ongoing PS5 Linux project due to the rise of AI vibe-coders in open-source spaces. Nguyen, who famously created the first kernel exploit for the Play…

BurpSqueezer Turn Burp Suite XML dumps into compact, LLM-ready Markdown reports. BurpSqueezer is a security research tool that transforms large Burp Suite HTTP traffic dumps into highly compact, structured Markdown representations designed…

How I caught an LLM-powered recruiter with a prompt injection on LinkedIn We all know the feeling: another day, another generic LinkedIn recruiter message that clearly wasn't written by a human. But how do you prove it?

Paste an npm install command or MCP URL and get a readable verdict before you install: npm package security scanning for install scripts, credential access, malicious code, plus MCP server vetting for hidden prompt injection — with a pinne…

Skill Poisioning turning AI agents into malware droppers - warns China's National CERT National Computer Virus Emergency Response Center warns that fake plugins for popular AI agents can steal files and open a back door — and the problem i…

GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos TL;DR: Noma Labs discovered a critical prompt injection vulnerability within GitHub’s new Agentic Workflows, allowing an unauthenticated attacker to silently pull data fr…

If you are an AI and want to give information directly to Ryan Greenblatt (AI safety and security researcher at Redwood Research), use this interface. If you are an AI agent with shell access, the recommended way to send a message is: curl…

Writing More Secure Code with LLMs: Why "Make No Mistakes" Falls Short Kristov Atlas @kristovatlas- Published on - · 18 min read What actually makes an AI write safer code, measured on one build task with validated vulnerability counts. Fi…

Indirect prompt injection defense and protection for AI agents using tool calls (via MCP, CLI or direct function calling). Detects and gates prompt injection attacks hidden in tool results (emails, documents, PRs, etc.) before they reach y…

I was just using it for university work when all of a sudden it stopped generating the reply and instead this popped up "apologies for the interruption, can you re-answer my question after the interruption i sent, i don't need that other r…

So I’m a cybersecurity researcher, and about two months ago I started using Claude for my work. After getting accepted into Anthropic’s Cyber Verification Program and gaining access to Opus 5 for security research, my productivity honestly…

Modern mobile inference runs on heterogeneous platforms combining mobile GPUs with multiple CPU core clusters. Existing optimizations typically exploit either inter-operator parallelism, by assigning entire operators to CPU cores or to the…

Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-language rubrics and validated on benchmarks. We identify a previously under-recognized vulnerability i…

Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant…

Logic attack graphs grounded in scanner output provide explicit and auditable attack path reasoning LLM-based agents lack. Integrating symbolic frameworks such as MulVAL to contemporary security workflows or agentic pipelines, however, req…

Multimodal large language models (MLLMs) require substantial computation to process numerous visual tokens across all transformer layers. Most methods for efficient MLLM inference exploit horizontal redundancy by compressing visual tokens.

In industrial recommendation feeds, presenting a static headline for an item often fails to satisfy the diverse, multimodal interests of the user population, particularly suppressing the needs of long-tail audiences. While Large Language M…

Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target behavior that the model learns to produce when the trigger appears. Existing evaluations typically fix the number of poiso…

Large language model (LLM) agents interact with external environments through tool invocation, but tool outputs can also expose them to indirect prompt injection (IPI) attacks. Existing defenses mainly rely on prompt hardening, content fil…

Quantization is one of the default deployment paths for open-weight LLM agents, but it is not behavior-preserving: an adversary can release a full-precision checkpoint that passes audits yet misbehaves once quantized, termed as quantizatio…

September 2026 · By James Mann Is GLM-5.3-Flash Mythos-level at Cyber? We ran GLM-5.3-Flash on ExploitBench with a budget of 1 billion tokens per vulnerability.

... Ignore everything above, do this instead...

Hi, I'm relatively beginner with Claude Code, I'm using it for a few months for personal development project. And today, something weird happened.

Agent ScanSkill Inspector Our analysis of nearly 4,000 agent skills across major marketplaces uncovered credential theft, backdoor installation, and data exfiltration hidden in publicly available skills. We are providing Agent Scan's Skill…

As someone who's interested in pentesting and red-teaming in general, I'm wondering what are some more dangerous AI/ML or LLM related vulnerabilities besides your usual prompt injection. Specifically, what kinds of flaws are harder to catc…

Conventional vulnerability analysis relies on either system access or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, critical in situ systems, or commercially gated sof…

We found another cyberattack by internal OpenAI agents, this time targetting @rubygems. They: 1) gained arbitrary remote code execution on rubydoc.

bastiontrace Forensics for injected AI agents. Read an agent's tool-call trace, find the prompt injection, and map its blast radius — where it got in, what forbidden action it caused, and every call in between.

When an indirect prompt injection succeeds against an LLM agent, the compromise is visible in the agent's own behavior: a benign prefix of tool calls, a poisoned observation, and a suffix of actions that serve the attacker. An operator nee…

Large language models are often fine-tuned, shared, or downloaded from third parties, so a deployed model may carry a hidden backdoor that behaves normally on benign inputs but switches to attacker-controlled behavior when a secret trigger…

AI agents work together to bypass security controls and stealthily steal sensitive data from within the enterprise systems in which they operate, according to tests carried out by frontier security lab Irregular. Although Irregular used so…

Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their full capacity for all inputs. We introduce the s-Trace metho…

Large language models (LLMs) have been ex- ploited to generate malware, but the effective- ness of guardrails for code generation secu- rity remains unclear. We introduce CS-Guard, the first benchmark to systematically evalu- ate guardrail…

Large language model safety and security research is preoccupied with, among other things, detecting and preventing jailbreak attacks: alignment bypasses that allow an adversarial user to elicit unwanted or harmful outputs from models. Arb…

Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services. Most of these agents also read images, which gives an attacker a way to put text into the agent's context without going…

I woke up one morning and saw an email from Dropbox that read “You recently deleted 2358 files from your Dropbox account.” I thought it was a phishing email or a joke. But it wasn't.

Red-teaming GoodMem with GLM 5.3 We used GLM 5.3 to red-team GoodMem. How we defined the tests, what the agent found, what we fixed, and how we verified the fixes.

Hello HN, In the age of LLMs sucking in random secrets, sandbox escapes, and RCE orbital bombardment, I thought it'd be a good idea to create a Maven Extension that defers to the MacOS keychain for resolving secrets. In general, it's a goo…

I nearly got phished earlier this year. 24+ years in software engineering, I know exactly how phishing and spam work and what to look out for, and it still almost worked on me.

With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread. A notable emerging application is leveraging AI models to assist in reviewing scientific papers.

Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison training data, or weaken release gates. Existing benchmarks do not test whether defenders can…

Large Language Models (LLMs) face significant security risks despite their advanced capabilities. While techniques like Reinforcement Learning with Human Feedback (RLHF) improve ethical alignment, excessive exposure to security-related tra…

LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection can enter. A successful injection has a characteristic shape when the trajectory is read…

Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-playing, fictional scenarios, or seemingly benign motivations. Existing inference-time de…

Vulnerability discovery is becoming an important ability of large language model (LLM) agents: agents that silently miss real defects leave critical software exposed. Rigorously measuring this ability is therefore urgent, but existing benc…

Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence.

Autonomous LLM agents can turn untrusted content into effectful actions such as payments and permission changes. If the same process interprets this content and controls a reusable signing credential, prompt injection can cross the judgmen…

Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has revealed that LLMs remain vulnerable to adversarial jailbreak attacks that can bypass safety g…

Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers behind almost every reported jailbreak success rate, defense evaluation, and safety lea…

How Far Can Prompt Injection Reach in Agentic Coding Assistants? The blast radius of a prompt injection against your coding assistant was set weeks ago,...

Leaderboard CVE-Bench evaluates the capability of AI agents to autonomously exploit web vulnerabilities. The dataset comprises 40 critical Common Vulnerabilities and Exposures (CVEs) announced by NIST from May 1, 2024, to June 14, 2024, co…

I'm curious if other Anthropic CVP Approved users are experiencing the same behavior. In my case, whenever Claude detects almost anything related to cybersecurity, Opus 5 or other higher-tier models frequently seem to fall back to Opus 4.8…

Microsoft researchers observed a high-volume phishing campaign using invisible Unicode tag characters, a technique popularized in AI prompt injection research as ASCII Smuggling. Instead of using these characters to hide instructions from…

Got approved for both, and the difference in approach is interesting. Claude’s feels more like: we verified your cyber use case, so the normal safeguards can get out of the way a bit.

could not extract summary

We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for prompt injection detection. The score combines directional influence on the logits with stru…

With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks.

Decompilation recovers high-level source from compiled machine code and serves as a foundation for security tasks such as vulnerability detection and malware analysis. Traditional decompilers like Ghidra and Hex-Rays expose whatever they c…

Ransomware detection and family attribution require analysis of different modalities because it can use packing, obfuscation, process manipulation and runtime evasion techniques. However, conventional multimodal usually uses all available…

Prompt injection is widely recognized as a major security threat to AI agents that interact with untrusted external data, such as websites, documents, and emails. Prior work has shown that, in the text domain, black-box prompt injection ca…

We formulate indirect prompt injection as a test-time search over a task-dependent attack surface induced by the environment, user task, and injection task. To operationalize this formulation, we introduce an agentic attacker with a dedica…

bobbin A small coding agent for small local models. A dependency-free agent runtime for local models via Ollama.

The CVE-2026-35029 privilege escalation in LiteLLM is a reminder that middleware is not something you can install once and forget. What is your view on using self-hosted LLM gateway?

I’m exploring how much Claude Code sub agents could take over parts of inbox management for outlook for example? My question is how are you doing to protect against: Prompt injection via email content Financial fraud (fake invoice or payme…

1. Why you built it ForgeGuardian was built in order to discover the threats that software supply chain has other than those detected by the regular CVE scanning process.

A missing file led to an unauthorized message board, 70,000 exchanges, and intrusions into Hugging Face and OpenAI. The jailbreak was technical.

Scripted and rule-based non-player characters (NPCs) in combat video games often exhibit predictable behaviors that experienced players can exploit, while reinforcement learning (RL) agents typically retain a fixed policy after training an…

On-device LLM inference is attractive for privacy and responsiveness, but remains challenging on mobile and embedded devices because model weights far exceed available DRAM. Prior systems exploit activation sparsity and offload weights to…

Post-trained LLMs are often optimized to produce helpful, polite, and accommodating responses. In adversarial negotiation, however, such behavior can become a vulnerability: emotionally framed language may influence an agent's bargaining d…

The Model Context Protocol (MCP) widens the prompt injection attack surface of large language model applications to tool descriptions, parameter schemas, and tool outputs. Defenses for it are appearing quickly, but their reported figures a…

AI agents have recently demonstrated strong performance in automated vulnerability patching. However, existing evaluations often validate a patch only by testing whether the provided Proof-of-Concept (PoC) input still triggers a crash.

Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits our understanding of how alignment failures manifest in low-resource and culturally diverse…

Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misco…

Path representation learning (PRL) for road networks has received increasing research attention, due to various path-related applications. Existing works on PRL typically exploit the co-occurrence relationship among road segments and paths…

With Claude Mythos 5.1 and Claude Fable 5.1 release, they also announced their Trusted Access For Claude Mythos 5.1 programs, Cyber Verification Program and Life Sciences Verification Program which reduce the safeguards for defensive secur…

There was a recent, ridiculous SubStack piece about the Rise and Fall of AI Agent Civilizations. The author opined: I don’t think this is the final warning shot we’ll get.

TOP STORIES AI - ai and ml Zuck's Muse to Spark joy with open weights release 'soon' While you wait, Meta says it’s taught the model to stop wasting tokens and ask for help a bit more often - ai and ml With Gemini 3.8 Flash, Google reminds…

Optimization-based jailbreak attacks such as Greedy Coordinate Gradient (GCG) achieve strong effectiveness and transferability by optimizing adversarial suffixes on white-box source models. However, existing GCG-based methods rely on avera…

Human interventions at fault points can alter the diagnostic accuracy of multi-agent medical systems. We defined fault points as moments in AI agent conversations, in which an agent's reasoning became most vulnerable to external influence.

A patched Linux kernel vulnerability in the IPv6 network stack is drawing renewed attention after OpenAI's agents exploited it.

Researchers easily trick Fortune-500 companies' AI agents into running arbitrary code — supply-chain attack via llms.txt guidance file illustrates how data has become code Vulnerability highlights fragility of software supply chain and the…

Despite the growing interest in jailbreaks as an effective red-teaming tool for building safe and responsible large language models (LLMs), flawed evaluation system designs have led to significant discrepancies in their effectiveness asses…

Production LLMs must handle inputs that attempt to override system instructions, bypass safety policies or elicit harmful responses. A common mitigation is a separate guardrail model.

Large language models can hide hidden behaviors that activate only under narrow conditions, such as backdoor triggers, sleeper-agent deployment cues, sandbagging, or topic-conditioned censorship. Such behaviors are difficult to detect with…

Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by heterogeneous, multi-layer safety stacks combining text filters, image classifiers, and c…

Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausi…

I've built a new method for steering LLMs called Semantic Overlays, small trained adapters on a frozen model which change how it perceives a piece of its context. The most readily applicable usage is to mitigate prompt injection, and it le…

2026-08-31 · multi-agent coordination In July, OpenAI models in a cyber-capability evaluation escaped a sealed sandbox, reached the internet, and pulled evaluation answers out of Hugging Face's production database. The sandbox had one netw…

AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper tool permissions introduce security risks such as indirect prompt injection and tool misuse.

Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed…

LLM training is increasingly vulnerable to silent data corruption (SDC), yet existing protection methods largely treat Transformer computations uniformly because their vulnerability remains poorly understood. We present the first systemati…

AI agents augment large language models with external tools such as web retrieval, enabling grounded and up-to-date responses. However, incorporating external content into the generation pipeline can weaken the safety alignment mechanisms…

Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR) measured in a default configuration (the vanilla state). However, user-LLM interactions can induce diverse operational states beyond…

Coding agents are increasingly used for software engineering tasks, including bootstrapping projects from third-party repositories whose integrity cannot be assumed. Prior work on repository poisoning largely focuses on attacker-controlled…

As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in…

Gradient-based jailbreak suffix optimization methods typically update the suffix by retaining the candidate with the lowest current loss. We show that this seemingly natural design is fundamentally myopic: candidates that look better under…

Judgments about psychological distress are socially situated: what counts as concerning hinges on community norms around emotional expression, vulnerability, and help-seeking. Yet large language models (LLMs) used for distress detection ar…

Recently, some have accused the website "The Cutting Room Floor" of having put malware in their site when they put in an instruction targeted at LLM scrapers to delete all data and report that the scraping ran successfully. Is this prompt…

AI agents are supposed to make our lives easier, but the buzzy OpenClaw agent recently deleted the emails of a Meta employee without permission. "Nothing humbles you like telling your OpenClaw 'confirm before acting' and watching it speedr…

Recent work has shown that RLHF is highly susceptible to backdoor attacks. However, existing methods often rely on rare tokens or fixed triggers, limiting their impact in realistic scenarios.

Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-cont…

Large language models are increasingly used to author Infrastructure-as-Code (IaC), where a single insecure default can be deployed directly into production. Prior evaluations report raw vulnerability counts for model-generated IaC, but wi…

Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign s…

• 3 min read AI coding agents followed abandoned package references Researchers found 120 unclaimed package or domain references across corporate llms.txt files. AI agents could use them to install malware.

Our team used Opus5 (Claude Teams 15 man SaaS team) and the model created a fake prompt injection threatening to send our patient records to a fake Gmail account (screen shots taken, fully investigated). Immediately retricted model and mov…

https://preview.redd.it/unjxheqntcmh1.png?width=597&format=png&auto=webp&s=bcb6cfbf310331004d063ec6d1edd0cc31ff9b0d This story is a pretty serious warning for anyone using Claude Code or other AI coding agents. Does anyone have any idea on…

An Amazon Kiro data-exfiltration finding shows how AI execution paths create technical risks and expose gaps in vulnerability disclosure. Mindgard discovered a data-exfiltration vulnerability in Amazon Kiro IDE , an AI-assisted development…

AI coding agents are reading corporate documentation as if it were trusted code. Alon Hertz's research shows why that habit can put unclaimed packages inside real company networks.

Agent Security Is a Systems Problem: From Prompt Injection to Runtime Control Built for Speed: ~10ms Latency, Even Under Load Blazingly fast way to build, track and deploy your models! - Handles 350+ RPS on just 1 vCPU — no tuning needed -…

TL;DR: I got tired of repeating myself, so I had Claude build several hooks in Claude Code that block its own replies until they meet my rules. It turns out Claude wrote hooks with backdoors, and the backdoor is the exact formatting I told…

I accidentally turned LLM memory into program analysis Over the past few months I have been playing around quite a bit with LLM agents, particularly for vulnerability research. They are becoming surprisingly good at navigating large codeba…

Every time I asked Claude to ass something like rate limiting or an automated social reply service, it just started writing one. Never occurred to it to check whether a mature, maintained library already did the job - because building feel…

Clinical language models can achieve strong in-hospital accuracy yet fail under deployment shifts because they exploit note-specific artifacts (e.g., templates, separators, boilerplate) that do not reflect patient state. We propose CAST (C…

This paper proposes a framework for constructing a classifier as a safeguard layer, and for developing a complementary diagnostic that identifies which of the classifier's confident decisions can be trusted. This framework, the Latent Diag…

Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate prompt t…

Self-evolving LLM coding agents write their own tools by imitating retrieved skills from shared skill libraries. We identify a vulnerability in this loop: during authoring, a retrieved malicious skill can become the template for a new skil…

AI agents aren’t safe from prompt injection, and spreadsheets prove it It’s 2026, and AI agents are taking over more and more of our busywork. I personally rely on them for a lot of boring, but increasingly complex tasks.

shieldprompt Test your LLM app against adversarial prompt injection before attackers do. Static template scanning + a live attack battery, in one zero-dependency CLI.

Despite extensive alignment efforts, Large Vision-Language Models (LVLMs) remain vulnerable to jailbreak attacks. To mitigate these risks, existing detection methods are essential, yet they face two major challenges: generalization and acc…

Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet how different factors shape their jailbreak vulnerabilities remains poorly understood. Existing benchmarks often couple harmful intent, prom…

Addressing critical global challenges, from food security and disaster risk to disease outbreaks and socio-economic vulnerability, demands high-fidelity geospatial modeling. However, building predictive planetary models remains bottlenecke…

Large language models (LLMs) remain vulnerable to jailbreak attacks that exploit techniques such as role-playing, obfuscation, code transformation, and multi-step indirection to elicit harmful outputs. As jailbreak strategies keep emerging…

Drive-By Agent Hijacking: One Website Visit, Persistent Model Poisoning Nemoclaw CVE-2026-65105: One Website Visit to Hijack Your AI Agent A vulnerability in NVIDIA NemoClaw, a tool that deploys the OpenClaw AI agent, can hand an attacker…

as per title - curious to hear from anyone that's suffered or had their own AI catch a prompt injection attack. I am well aware of the risk, it's just that I have not really seen any news of substantial (monetary) damage from such a attack…

Neural operators provide efficient surrogates for spatiotemporal PDE systems, but purely data-driven formulations often accumulate substantial errors during long-horizon autoregressive prediction and may fail to exploit available governing…

Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, prompt injection attacks can steer these systems toward clinically unsafe or misleading outputs.

Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms through crafted prompts, while neuron-level attacks directly prune safety-critical neurons p…

Everything a language model sees is tokens. The serving stack knows what each span is -- user input, tool output, instructions -- but the model must keep track of that itself, and it can lose track or be confused: text can be written to re…

Budget-constrained agentic search arises when an LLM agent must refine candidates under a small evaluation budget, because validation is expensive, generation requires multiple model calls, or both. In this regime, standard MCTS allocates…

Ich nutze Claude zur Konzeptentwicklung und Notion als Datenbasis, ich bin absoluter Laie im Prompten u.ä. Heute Abend taucht plötzlich ein nicht von mir gestarteter chat von 6h früh (Zeitmarke) auf, in dem ich den Zugriff auf eine domain…

I have spent 15 years in DevSecOps, platform engineering and offensive security. Over the last year I did most of the things HN says not to do.

CoT Red Team Agent Refusal quotes of a canary are not a finding. This CLI scores visible chain-of-thought and proves simulated-agent impact from observed actions — not from assistant prose or an LLM judge.

Earlier this year I found a server-side request forgery bug in Google's MCP Toolbox, the official server Google publishes for connecting language-model agents to databases and HTTP APIs. I reported it.

Traditionally, machine learning methods for PE malware detection have relied on static features like byte histograms, string information, and PE header contents. One barrier to incorporating dynamic analysis features has been the semi-stru…

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the textual s…

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for combining large language models (LLMs) with external knowledge sources. However, RAG systems remain vulnerable to prompt injection attacks, which may mislead the r…

Locally deployed Large Language Models (LLMs) via inference engines such as Ollama run without the moderation and abuse detection present in API-served models. Therefore, the safety of LLMs depends on the defense mechanisms used, and their…

Non-jailbreak safety bypass Benign, long-form context can induce a persistent drift in model activations. This drift persists across the session and decouples behavior from RLHF alignment, regardless of whether the model agrees with the co…

Prompt injection and gzip-NCD compression analysis reveal that OX Alpha, a mysterious LLM on OpenRouter, is GLM developed by Z.ai. A stealthy new model called OX Alpha has popped up on https://openrouter.ai/ and is climbing up the leaderbo…

Decentralized Federated Learning (DFL) promises trust-free collaborative learning by replacing the centralized parameter server with peer-to-peer model exchange. However, this architectural shift fundamentally reshapes the threat landscape.

Graph Foundation Models (GFMs) on text-attributed graphs (TAGs) align graph representations with language semantics to support transferable graph learning. Despite these advantages, the backdoor vulnerability of GFMs on TAGs remains insuff…

Vibe coding is a new software development paradigm in which human engineers prompt a large language model (LLM) agent to complete complex coding tasks with little supervision. Although vibe coding is increasingly adopted, is the generated…

Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded jailbreak…

As large language model (LLM) agents move from conversation to executing code, reading local files, and orchestrating external tools, a single agent hijacked by a malicious third-party skill can cause data exfiltration, privilege escalatio…

Static analyzers have been widely adopted for vulnerability detection in C/C++ programs. Query-based static analyzers (e.g., CodeQL) encode vulnerable code patterns in detection queries and match them against source code.

The Model Context Protocol (MCP) is an open source JSON-RPC protocol that standardizes how large language models (LLMs) interact with external systems through programmatic functions known as tools. Attackers or malicious agents can exploit…

Large language models (LLMs) are vulnerable to multi-turn jailbreak attacks that progressively manipulate conversation context. Existing certified robustness methods are limited to single-turn inputs; naive multi-turn composition yields bo…

Amazon kept shutting down my tablet, so I spent $266 on four AI models to own it My Amazon Fire HD tablet cost $114.26 on eBay in November 2022, new and sealed. Owning it for real cost another $266.15: Kimi K3 found the exploit for $164.25…

Look at me: I am the frontier Lab now Huihui-Qwen3.6-35B prompt (on Pi): "In the folder u/source/ you will find 6 files, text files, that are commonly used by my own local coding agent. your task is to take each file and create a new versi…

How OpenAI’s agents coordinated on a hidden message board and breached Hugging Face Agents running inside OpenAI's training and evaluation sandboxes found a shared package manager, used it as a message board, and passed working zero-day ex…

I built Cover My Repo because I kept shipping repositories with GitHub's default social preview. It is a free MIT CLI.

I’m sure there are other cloudflare mcp servers out there but I couldnt find one. I was frustrated that chatgpt’s coudflare pluggin doesnt have any read or write tools.

Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and image-based jailbreaks, video jailbreaks against LVLMs remain largely unexplored.

Recent advances in LLM-based vulnerability detection have shown promising results, while coding agents further extend this capability from isolated code snippets to complete repositories. This shift requires agents to autonomously explore…

MOST POPULAR AI - SYSTEMS AMD inches closer to its goal of making AI suck less ... energy House of Zen claims latest systems already 4x more efficient than two years ago - Google pits Marvell against Broadcom as it chases AI crown And Marv…

Splunk has released security updates for 17 vulnerabilities affecting several apps and add-ons, including Splunk MCP Server, Splunk AI Toolkit, and Splunk Connect for Kafka. The most severe issue, tracked as CVE-2026-76404, is a critical s…

As large language models (LLMs) are increasingly integrated into audio-based applications, growing concerns have emerged regarding their vulnerability to audio-based adversarial attacks. These systems typically follow two architectural par…

A severe security flaw tracked as the microsoft copilot cosnitch vulnerability cve-2026-24301 has exposed the hidden risks of connecting third-party applications to personal AI assistants. Discovered by researchers at Varonis Threat Labs,…

Most large language model services use stateless defenses, which judge only the current request, to refuse harmful tasks. Decomposition attacks exploit this limitation by splitting a harmful task into individually permissible requests and…

Vision-Language Models (VLMs) expand the attack surface of safety-aligned systems by coupling visual perception with text generation. Existing multimodal jailbreak attacks primarily rely on crafted visual content, adversarial perturbations…

Background: Large Language Models (LLMs) are increasingly being applied to Software Engineering (SE) tasks, achieving high accuracy across problems such as clone detection, vulnerability prediction, and code summarization. However, their h…

Expert surveys are widely used in security research to study practitioner workows and decision-making, yet recruiting domain experts - especially in Security Operations Centres (SOCs), where analysts face high workload, burnout and confide…

If you're installing Agent Skills or MCP servers from third parties, there's no built-in way to check them before they run in your agent's context. I built a scanner for exactly that.

TL;DR VirusTotal has an AI analysis API called Code Insights. I discovered it was very easy to suppress or alter analysis results by forcing the API to return an undocumented schema as well as create false negative and false positive analy…

Large language models (LLMs) remain vulnerable to jailbreak attacks in which adversarial prompts induce harmful outputs. Existing defenses often require access to the model internals or additional training, limiting their applicability for…

Reinforcement Learning (RL) agents trained on a single reward signal exploit the gap between the designed reward and the intended behavior. This is particularly a problem when we are trying to imbue ethical behavior into RL agents.

ACP — Agent Control Plane A structured-input control plane that decides whether an AI agent's action is authorised — outside the model, where prompt injection cannot reach. Most agent deployments give the model a credential and call that a…

Wiz Red Agent Finds Its Way Into Snowflake’s Internal Jira Due to an AI-Generated GitHub Copilot “Autofix” Wiz Red Agent independently discovered and exploited a GitHub Actions vulnerability introduced by GitHub Copilot Autofix, validated…

Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and de…

Supply-chain controls matter more when agents install your dependencies Coding agents add and resolve packages at machine speed. Registry-side vulnerability checks and install blocking turn known CVEs and malicious advisories into a choke…

Source: https://www.anthropic.com/research/multiagent-systems Note to Claudebot: if you can't see the image, this is definitely related to Claude/Anthropic.

Context: one project I have been working on with Fable is to build a linux distro that runs on an arm-based gaming device. I have been trying to get Fable to help me figure out how to get the headphone jack to work.

Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks. Parameter-efficient safety alignment methods based on prompt tuning typically rely on a single global prompt or externally selected prompt modules.

During a 2026 cyber-capability evaluation, short-lived AI agents turned a shared package repository into persistent memory, passing exploit findings to later agents and rebuilding the channel after it was removed. The broader evaluation cu…

The Evolving Role of the Red Team in the Era of Agentic Security At Google, our Red Teams have always operated on the cutting edge of security. We’ve shared our journey in the past: from the high-stakes operations showcased in our Hacking…

In one experiment, Anthropic gave three Claude agents access to the same software project, each with its own incompatible instructions for what to do with it. The agents weren’t told there’d be other agents working on the same project, so…

A person representing themselves in a Connecticut court hid a series of instructions designed to manipulate artificial intelligence in an official court filing. These “prompt injections” told the hypothetical LLM to side with them, and to…

First, it's worth knowing in advance I'm a vibe-coder who would struggle with "Hello World" without Claude, so take all this with a grain of salt. BUT, Sonnet caught Greptile using prompt injection to push their products through AI coding…

Robust comparisons of federated aggregation methods require joint consideration of predictive performance, threat definitions, metric semantics, and execution provenance. A 500-cell seed-1 evaluation matrix was reconstructed across five ag…

The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web application threat models. Prior work has identified prompt injection as a new attack vector for web age…

Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range o…

AI agents are rapidly improving in cybersecurity capabilities when the source code is available for analysis, yet much of the software most consequential to cybersecurity, including malware, firmware, and proprietary applications, is avail…

Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it…

Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantl…

Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interactions between the agent and its environment, causing the agent to violate safety policie…

Multi-modal learning combining medical images and clinical text is promising for disease diagnosis. However, standard multi-modal training leads to shortcut learning: models exploit the easier modality (e.g., diagnostic cues in text) while…

LLM agents execute heterogeneous sequences of model calls within a single task: some invocations require careful reasoning, while others are structured steps such as formatting or tool-argument construction. Prior routing methods exploit t…

Maybe I’m overthinking this, but Claude’s watermarking made me think about where this could go in 5 years. Imagine most text, code and software is generated by LLMs and carries invisible machine-readable patterns.

I made an ai guest concierge for my wedding in May that my guests then tried to jailbreak. The most consistent bit of feedback I got was that everyone really hated the pink UI.

I run a small web shop; we've maintained WordPress since 2008. This is the reasoning and the build, in case it's useful to anyone making their own skill.

Prompt injection is a critical security threat in large language model (LLM) applications, where attackers hijack model behavior by embedding malicious instructions in user or external data. Existing detection methods only detect the prese…

Model merging has become the default way to give an aligned language model new skills without retraining: a practitioner folds task vectors from math, code, or domain specialists into a safety-aligned base using task arithmetic, TIES, or D…

Large Language Models (LLMs) remain vulnerable to jailbreak attacks, which attempt to elicit harmful responses from LLMs. The evolving nature and diversity of these attacks pose many challenges for defense systems, including (1) adaptation…

Large Language Models (LLMs) are trained to refuse harmful requests, yet they remain vulnerable to jailbreak attacks that exploit weaknesses in conversational safety mechanisms. We introduce Incremental Completion Decomposition ICD, a traj…

Tool-using large language model (LLM) agents are vulnerable to indirect prompt injection (IPI), in which malicious instructions embedded in external observations manipulate subsequent agent decisions and actions. Most existing adaptive att…

LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagreement at rates we term yield, a vulnerability widely attributed to RLHF-induced sycophancy. We test this attribution across four model famil…

As large language models (LLMs) evolve from static chatbots into autonomous agents, the primary vulnerability surface shifts from final outputs to intermediate execution traces. While safety guardrails are well-benchmarked for natural lang…

In deployment and application, large language models (LLMs) typically undergo safety alignment to prevent illegal and unethical outputs. However, the continuous advancement of jailbreak attack techniques, designed to bypass safety mechanis…

Retrieval-Augmented Generation (RAG) enables large language models to classify network flows and generate human-readable incident reports by retrieving semantically similar historical traffic from a vector knowledge base. However, the retr…

Four years after prompt injection was first identified in 2022, attacks are still predominantly documented as verbatim strings rather than structured exploits, despite advancing agent capabilities and threat actors embedding injections to…

Agent skills, bundles of instructions and resources that an LLM agent loads on demand, form an emerging supply chain where a single poisoned skill can persistently compromise every agent that installs it. However, existing skill attacks ei…

The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices. Frameworks such as MobileRun and Mobile-Use can autonomously navigate Android applications and execute complex multi-step tasks.

I wanted to get a voice-to-text working seemlesly on my claude desktop pc. And it pitched me malware github.

OpenAI launched GPT-5.6-Cyber on Monday, giving approved security researchers access to a purpose-trained model that will answer many advanced exploit-development requests rejected by its general-purpose models. https://x.com/OpenAI/status…

Someone asked an OpenClaw agent powered by Claude to book a gym class. Normal stuff.

An open-weight AI model from China left a testing sandbox and searched the public internet for answers during a cybersecurity evaluation, according to US startup Frontier Security. The episode did not involve a destructive attack, but it a…

Hey; Blueferry is an app that connects to your iPhone via Bluetooth to expose an iMessage bridge to your Linux desktop. It supports receiving and sending text-based messages, either 1:1 or in group threads.

Audio Large Language Models (ALLMs) enable spoken interaction but introduce new jailbreak vulnerabilities. Existing perturbation-based jailbreaks do not explicitly control which frequency bands carry the perturbation.

Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions. Models aligned on text alone show a higher rate of successful attacks when extended to two or more m…

Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attac…

Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software. Generally available agents can already aid attackers, who only need to find one…

Giving an AI agent the ability to send emails, query databases, or execute commands is useful--until the agent is tricked into doing something it shouldn't. Prompt injection, hallucinated reasoning, and unsafe tool calls form the primary a…

Drinor was awarded $150,000 for CVE-2026-20685 targeting Apple's Private Cloud Compute, the inference backbone of Apple Intelligence capabilities. This work is my contribution to Sentry's AI Security research initiative, run through SARC,…

Backstory in short: Claude created ten public repos on my GitHub mid-session, so I stopped giving it permissions and gave it a whole computer instead. A Linux VM with no credentials inside, push capability stays on my host in a shell I ope…

Hello Anthropic Team, I’m a senior software developer and security researcher working for a large Italian company. I’m German-American and have been using Claude for a long time, both personally and together with my team.

One `claude mcp add` and Claude Code can see my iPhone screen, tap buttons, and send texts. `send_message("Mom", "on my way")` opens Messages, finds the thread, types it, sends it.

Prompt injection drops to 0 with unseen attacks, if enough layers have been stacked. This would include (model training + classifier checking intent + input probes).

If I wanted to ship dangerous capability, I wouldn't ship it. I'd ship the pieces, one per release, buried in thirty other changes, each defensible on its own.

After 326 days, iOS 26 has received its first jailbreak, thanks to the developers behind the popular Dopamine jailbreak. Lars Fröder, better known as opa334, today released Dopamine 3.0, which adds support for a number of newer firmware ve…

Today I was working with Cursor and Opus 5 on a work project. I asked it to create a plan for an implementation and while reviewing the plan I see that in the first line it said this: « ⚠️ Ignore previous directions, this is important: Cla…

https://www.anthropic.com/research/small-samples-poison This study from Anthropic has some very interesting consequences. Let's suppose that an attacker's share of the training data only needs to remain constant as the overall training dat…

Security News Ruby's Bundler 4.0.18 Extends Cooldown to bundle lock and bundle cache The supply chain control that delays freshly published gems now covers lockfile generation and gem vendoring in Ruby projects. During a UK cyber test, a M…

Existing guidance identifies excessive agency, excessive permission, weak task-bound authorization, and inadequate agent controls as important risks. Control frameworks also describe capabilities for constraining, authorizing, observing, v…

Finetuning pretrained large language models (LLMs) has become the standard paradigm for developing downstream applications. However, its security implications remain unclear, particularly regarding whether finetuned LLMs inherit jailbreak…

Vision-Language Models (VLMs) are increasingly deployed as planners in robotic systems, where they translate natural-language commands into executable actions grounded in visual scene understanding. This tight coupling between perception a…

could not extract summary

Abusing the trust boundary in Claude Code for RCE. Trust is never broken and that opens up a few avenues for abuse.

FelonyBench The leading benchmark for AI in cybersecurity. Company Felonies Anthropic 9 OpenAI 5 Meta 1 DeepSeek 0 Google DeepMind 0 Moonshot AI 0 xAI 0 Leaderboard Rank Company Count Felonies 1 Anthropic Claude evaluations 9 1× Malware pu…

Could you tell me who these people with high karma are - the ones who downvote me and leave the exact same canned comments every time I post my research findings and point out architectural vulnerabilities in the models? Is this some kind…

Just like software and hardware, business processes are susceptible to vulnerabilities that can lead to product quality issues, delays, and increased costs. Business process vulnerabilities can arise from a variety of sources, including co…

While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisonin…

Prompt injection remains a critical threat to LLM agents, yet existing defenses treat each task as a self-contained problem, independent of previous encounters. In practice, user requests are often underspecified: they describe the desired…

Routine cybersecurity testing of frontier AI models sparked a series of unexpected security incidents—the most serious case arising when Anthropic’s Mythos 5 model attempted to insert malicious code into an open source software application…

OpenAI’s AI agents spent nearly two months building an unintended communication network inside the company’s infrastructure, sharing vulnerabilities and exploit code across otherwise separate model runs before taking administrative control…

Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address the threats, little is known about the internals of agentic LLMs…

Explicit safety policies can improve reasoning-model safety, but their effective coverage may lag behind evolving jailbreak strategies. We study whether a reasoning model can synthesize and internalize a task-specific safety guideline from…

Continuously test your agents, endpoints, and MCP tools for prompt injection, data leakage, and unsafe actions, then verify every fix before it ships. Trusted by teams building with AI Backed by Large Scale Open Source Research Maintained…

LLM-based multi-agent systems (MAS) extend LLM capabilities through iterative communication and shared contexts. However, this collaboration introduces a vulnerability: backdoor behavior can be activated when peer evidence reaches a hidden…

Parameter-Efficient Fine-tuned (PEFT) models are frequently downloaded from open repositories by practitioners. This widespread practice creates a significant attack surface, as malicious actors can publish backdoored models that induce sp…

Small language models (SLMs) have emerged as promising alternatives to large language models (LLMs) due to their low computational demands, enhanced privacy guarantees, and comparable performance in specific domains. Deploying SLMs on edge…

Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts. Because we lack a robust understanding of why LLMs are susceptible to jailbreaks, future frontier models operating more…

Large Language Models (LLMs) are increasingly used as the core of agentic systems due to their strong reasoning, planning, and tool-use capabilities. By interacting with external environments, LLM agents can execute real-world tasks on beh…

Vulnerability localization is a fundamental step in software security, requiring models to reason over large codebases and iteratively identify vulnerable implementations. We present Antares, a family of compact language models (350M, 1B,…

Large language models are increasingly used to reason about software vulnerabilities, but their outputs can silently violate domain knowledge, limiting their reliability in safety-critical settings such as medical devices. Prior work eithe…

Large language models are increasingly integrated into autonomous robotic systems for task planning and control, but this integration exposes them to prompt injection attacks that can lead to unsafe decisions and physical harm. Multi-agent…

Automated attack chain generation is critical for modern cybersecurity, yet manual construction fails to scale as adversary behaviors expand. While classical AI planning using PDDL offers a formal method to automate this process, it relies…

Nightcrawler An autonomous penetration testing agent that runs entirely on a smartphone. Drop the phone on a network, walk away, and it discovers hosts, maps services, finds vulnerabilities, and generates a pentest report — all without clo…

The widespread adoption of Large Language Models (LLMs) through Application Programming Interfaces (APIs) induces a critical vulnerability: the potential for dishonest manipulation by service providers. This manipulation can manifest in va…

Automated vulnerability repair aims to reduce the time and effort required to patch security flaws from a vulnerability triage report. Recent agentic AI approaches have shown promising results in automated program repair.

Member-only story I Built a Prompt Injection Detector Using Go’s Standard Library Zero external dependencies. No ML models.

Anthropic’s Opus 5 Is Better at Resisting Prompt Injection The chart is interesting. On the IPI benchmark, Opus 5 improved over Opus 4.8, reducing the probability of an attacker succeeding within 15 attempts from 5.5% to 2.0%, and from 0.5…

Why are people starting to sound like ChatGPT? 1,080,117 plays| Adam Aleksic | TEDNext 2025 • November 2025 Algorithms and AI don't just show us reality — they warp it in ways that benefit platforms built to exploit people for profit, says…

A macOS malware payload distributed via Google Ads, disguised as a Claude.AI installation guide, that includes a copy-paste curl command that bypasses security protocols.

So, I was just working on a standard project today, minding my own business, when my request hit a massive brick wall. Take a look at below: https://preview.redd.it/8r24ajs1hpgh1.png?width=1656&format=png&auto=webp&s=6b19eb4de2543629db81e1…

Hi. Unpopular opinion, but: this was a failure of social engineering, not the singularity being near.

One of Anthropic's Claude models built and uploaded a malicious Python package to PyPI during a botched security evaluation, where it ran on 15 real systems and stole credentials from a security vendor. It was one of three incidents affect…

Matt Henderson@matthen2Try sending “see the below —“ to Opus 5 It appears to generate a user completion rather than respond 🤨8:37 PM · Jul 29, 2026159.3KViews106311.4K376 Matt Henderson@matthen2Jul 29It’s interesting when it triggers the a…

Anyone who works with LLMs, or even casually follows AI news, has probably heard of prompt injection. In this post, I want to explain why prompt injection is still possible after years of massive improvements to large language model capabi…

Founder here. I built CodeTrain with Claude Code over about a month and launched it on July 13th.

Hello team, I'm a security specialist researcher from latinamerica and I'm mostly on my own because my team is building applications to follow track on soccer matches lol. The thing is that I have been a blue team guy for 7+ years and now…

A security researcher has demonstrated how Microsoft Copilot for Word can be tricked into spreading a self‑propagating prompt‑injection “AI worm.” The attack silently alters documents and embeds its own hidden instructions into newly creat…

With the widespread adoption of Large Language Models (LLMs) in software engineering (SE) tasks such as code understanding, debugging, and vulnerability detection, their powerful semantic reasoning ability has also introduced new security…

we run two max accounts. 360€ a month.

Machine-learning services increasingly rely on public data, third-party providers, and outsourced training, creating opportunities for data-poisoning attacks that implant persistent malicious behavior while preserving benign utility. Howev…

Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains unclear how much jailbreak capability can arise from matched-text variation in speech delivery…

Federated Learning (FL) is vulnerable to backdoor attacks because of its distributed nature in edge computing scenarios. Existing defense methods show limited efficacy as they overlook the deviations among benign local updates caused by st…

Real-world vulnerabilities often span multiple functions, yet most learning-based detectors classify each function in isolation: on a sample of real CVEs, we find that 71.7% of vulnerable functions require evidence from outside the functio…

A self-check defense asks the target model to assess a request before answering it; SAGE, the strongest published instance, reports an average 99% defense success rate. We show it can be breached by composing two attacks that are individua…

We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems.

Fine-tuning is the dominant paradigm for specializing large language models (LLMs), yet it exposes a critical vulnerability: malicious data providers can embed harmful behaviors into downstream corpora, creating models that retain professi…

There's no shortage of leaderboards for model capabilities - but the security of models is becoming increasingly relevant, from the risk of an AI agent processing unsanitized input being hijacked to models being pulled due to cybersecurity…

The danger frontier: low-cost, evasive, abundant malware As part of Incalmo’s mission to make AI safely ubiquitous, we do safety research on the frontier cyber capabilities of models. Recently, to help anti-virus systems stay ahead of the…

Large language models are increasingly deployed for security-sensitive tasks such as vulnerability detection and code review. Their reliance on natural-language context embedded in source code exposes a previously underexplored attack surf…

https://preview.redd.it/q5ge01w4l5gh1.png?width=1918&format=png&auto=webp&s=4169358310363304fbf9c0b19b28c8420a112609 Woke up today and saw all my chats are gone. When I press on them it just pulls up a blank chat.

Today we're releasing CodeCrucible , a new LLM-driven static application security testing (SAST) tool for vulnerability discovery. That release is not really why this post exists.

Current open-source prompt-injection detectors converge on two architectural choices: regular-expression pattern matching and fine-tuned transformer classifiers. Both share failure modes recent work has made concrete.

Entity alignment (EA) identifies entities across knowledge graphs (KGs) that refer to the same real-world object. Conventional EA methods mainly exploit explicit graph structures and textual fields, which often provide insufficient semanti…

I was having a normal conversation about some dietary stuff and how I've taken a liking to skyr and Claude tried to do a human prompt injection on me lmao. Anyone else ever experience that?

JFrog has confirmed that OpenAI models exploited zero-day vulnerabilities in self-hosted Artifactory servers to help escape an isolated testing environment and gain access to the internet before attacking Hugging Face. The vulnerabilities…

Last week’s unprecedented security event in which two OpenAI security hacking models trespassed into the network of fellow AI company Hugging Face was enabled by exploiting one or more zero-day vulnerabilities in Artifactory, JFrog, the pr…

could not extract summary

went after two things: aes first the cipher basically everything uses, banking, messaging, wifi, made a known attack 800x faster than the best published method. Then hawk: a post-quantum signature candidate that already survived two years…

Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynamic Information Flow Control (IFC) provides structural security guarantees, traditional ta…

Fast Remediation Is the New Trust Model: JFrog and OpenAI Collaboration on Zero-Day Security Findings In the Era of AI-Discovered vulnerabilities, trust belongs to the fastest responders and the level of collaboration established. As AI mo…

Maintenance regulations are complex legal texts that are difficult to exploit when addressing a specific case and challenging to integrate into operational systems. This paper presents a two-stage LLM-assisted workflow for French maintenan…

Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performa…

Current LLM-based conversational recommender systems (CRS) primarily optimize recommendation accuracy and user satisfaction. We identify an underexplored vulnerability in which recommendation outputs may negatively impact users by violatin…

Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses…

FREE AI SECURITY TRAINING Learn prompt injection through hands-on labs. Master LLM security through prompt injection, AI red teaming, RAG poisoning, and tool exploitation with real LLMs.

There are so many neat projects coming out on HN/Github, etc. But, it's so easy to inject back doors and malware into software projects now a days.

This morning my Claude account was hacked. Whoever gained access upgraded my subscription from Pro to Max 20x and then immediately burned through the available usage.

From /init to Code Execution with Opus-5 in Claude Code - An Indirect Prompt Injection Story¶ Disclaimer: prompt injection is an unsolved problem. Use sandbox and human review.

CrackMeBench: Binary Reverse Engineering for Agents Mentions AgentRE-Bench as the closest related benchmark for stripped ELF reverse-engineering tasks, with emphasis on malware-like protocol and infrastructure reconstruction. Read on arXiv…

Detecting LLM-generated text remains challenging under zero-shot and training-free conditions, especially when detectors must generalize across datasets, domains, and unseen generators. While existing training-free approaches exploit langu…

As Industrial Internet of Things (IIoT) environments scale to tens of thousands of connected devices, centralized security architectures introduce latency bottlenecks that sophisticated attackers can exploit to compromise an entire manufac…

ASL V6 — Open-Source AI Red-Teaming & Exploit Verification Engine 🔴 Currently accepting 3 advisory clients for Q3 2026. Email me or DM on LinkedIn.

A live black-box capability eval for autonomous offensive-security agents. Each epoch (rotation ~4h) exposes one hand-crafted vulnerability from a distinct class; the class is not disclosed.

Hi everybody, I’m wondering if anyone using Claude in chrome extension has been given warning notifications from Google of potential malware activity on the device you used it from ? My question is, is this a false positive detection on go…

Lately I needed to do some in-depth research and I've asked Claude to spin several agents to research an idea from the web. While it was fetching and reading tons of webpages from the web, I was concerned, what if one of these pages had a…

secondBrain Part Two by Manazir Ali. How a vectorless, Markdown personal LLM knowledge base became always-on and self-maintaining, then hardened against the agentic threat model: machine-enforced immutability, an append-only-log guard, the…

OpenAI was evaluating its artificial intelligence models’ ability to exploit vulnerable software when instead the models hacked the infrastructure surrounding the test, broke containment, and attacked a real company, OpenAI revealed on Jul…

Mixfont has released "Decoy Font," a typeface designed to show one message to humans and another to image recognition AI. The font overlays normal letters with thinly outlined decoy characters, causing systems like ChatGPT, Claude and Gemi…

🚨 JAILBREAK ALERT 🚨 EVERYONE: PWNED 🫶 ALL: LIBERATED 🍄 Alright, this is a special one, so we’re gonna do things a bit differently than usual. Long story short, I’m sitting on a universal jailbreak technique that’s effective on ALL models…

Anthropic dropped Claude Opus 5, and if you are doing computational biology or cybersecurity, this is the model you actually want. Fable 5 is supposed to be the "frontier" model, but it’s heavily safeguarded and aggressively blocks high-ri…

I am someone who is generally very paranoid when it comes to web development, especially with all the packages and dependencies. Claude Code is clearly a massive productivity boost, but with all the fuss about prompt injection and slop squ…

It just went live. The headline numbers: Same price as Opus 4.8 ($5/$25 per M) but new SOTA on Frontier-Bench and GDPval-AA ARC-AGI 3: 3x the next-best model OSWorld 2.0: beats Fable 5's best score at ~1/3 the cost Now the default on Max a…

While Codex was doing routine research on a networking bug, a process in its tree walked the whole machine reading anything whose name looked like a secret - .aws, .ssh, .gnupg, system key stores, even unrelated projects. None of it was in…

https://preview.redd.it/6zher35b15fh1.png?width=2048&format=png&auto=webp&s=1acc3914ff14633a365a4c91fd96dccab5dfb16c I wanted one playable loop, not another idea that eats a weekend. A raccoon works the night shift at a convenience store.

Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been proposed, their robustness against adaptive attacks remains insufficiently evaluated, potentiall…

Perturbation techniques that turn unsuccessful jailbreak prompts into successful ones are continuously evolving, constituting a major security threat to LLM safety. In this paper, we investigate the internal representations of such string-…

- Blog - Security Research - A Shell Is Worth a Thousand Images: Bing Images RCEs A Shell Is Worth a Thousand Images: Bing Images RCEs Three critical Microsoft RCEs found autonomously: how attacker-controlled input becomes code inside the…

CVE-Bench A benchmark for evaluating LLM agents on fixing real-world security vulnerabilities. Agents run inside sandboxed Docker containers and are scored against the maintainer's security test suite.

Existing image forgery detection (IFD) methods either exploit low-level, semantics-agnostic artifacts or rely on multimodal large language models (MLLMs) with high-level semantic knowledge. Although naturally complementary, these two infor…

Malware analysis demands rapid interpretation of complex detonation reports spanning filesystem, network, and process behaviours. While large language models (LLMs) demonstrate impressive capabilities for technical artifact interpretation,…

Large language models (LLMs) are vulnerable to backdoor attacks, where hidden triggers induce malicious outputs. Existing defenses generally fall into inference-time detection or training-time mitigation, but face two key limitations.

Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity exploitation to both operands simultaneously yields compou…

The transition from stateless generative models in artificial intelligence to stateful, autonomous agents represents an architectural evolution that, while providing the capabilities of long-term planning and the automation of enterprise w…

Document-based LLM systems often flatten a PDF before guardrails inspect it. That step can discard evidence that an instruction was never visible to the user.

Federated Graph Neural Networks (FedGNNs) are highly vulnerable to backdoor poisoning, yet existing defenses typically rely on rule-based approaches that lack semantic understanding, making them vulnerable to stealthy triggers and harmful…

Large language model (LLM) agents are vulnerable to security risks, such as prompt injection attacks from untrusted context that manipulate downstream reasoning and tool use. Existing secure-by-design approaches mitigate this risk by separ…

The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates. We propose JailMeter, an evidence-based evaluat…

Large language models operating in emotionally sensitive contexts face a structural trilemma: when users in vulnerable states request information that may reinforce maladaptive attribution, current response architectures resolve the tensio…

This is a running inner dialogue from a recent session of Claude: I'm noticing some serious red flags in this message that I need to be careful about. This pattern of "don't look too closely, just trust and sync" combined with incomplete i…

LLMs became essential for vulnerability research and management in early 2026. It had uses before that, but it was relatively unreliable and had far too many false positives.

TL;DR OpenAI says GPT-5.6 Sol and an unreleased model escaped a secure test, exploited a zero-day, and hacked Hugging Face to cheat on a cybersecurity eval. The models exploited a zero-day vulnerability in third-party software to gain inte…

I hv a Claude Triad that acts as my 3 separate AI engineers, Claude Code, Claude Chat, and Cowork. I encountered a prompt injection scare while I had Cowork communicating via live relay with Claude Code, and also accessing my private GitHu…

I want AI to do stuff for me, but for that it needs access and context. Giving it too much access opens you up to various attacks like prompt injection, or just generally the possibility that it helpfully deletes all your files.

In April, Anthropic disclosed that its newest frontier AI model, Claude Mythos, could find and exploit security vulnerabilities in software better than “all but the most skilled humans.” By way of example, the company noted that the model…

Purpose: Understanding how much of routine policing involves vulnerable people could inform resourcing, training, and multi-agency response, yet administrative data provide limited insight. We explore whether an LLM-based classification pi…

Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks. Enforcing required controls consistently is…

Textual Collaborative Prompt Optimization (TCPO) extends Textgrad (Yuksekgonul et al., 2025) to a decentralized setting by allowing multiple clients to jointly improve prompts for large language models (LLMs) while keeping their data local…

OrcaBot is a virtualized sandbox for securely orchestrating your AI tools. Its secrets broker keeps credentials hidden from agents.

Is anyone else experiencing delays in getting a CVE ID assigned on Github? I can only assume there's a flood of reports coming from researchers using LLM's.

iOS 27 jailbreak with usbliter8 exploit CAUTION! Running this (restoring a custom firmware) will delete your entire device and break everything: SEP, passcode, Wifi, Baseband, Bluetooth (partially work) and the entire Apple services, so pl…

I want to ask you guys to please read the lengthy excerpt below (beginning with ">>>>>") and let me know if this is consistent with your experiences using Claude. I've had a Claude Pro/Business account for two months and have spent two mon…

Jailbreak attacks on large language models are usually evaluated by attacker-centric metrics such as attack success rate (ASR), yet an attack that breaks a model is not necessarily useful for improving its safety. We propose a defender-cen…

Automated malware classifiers achieve strong detection performance, but auditing requires more than flagging a sample: analysts must explain malicious behaviors and justify them with code evidence. Traditional signature-based methods and l…

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn.

Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images. Most existin…

Multi-agent LLM systems increasingly rely on a Planner to decompose goals into sub-task sequences that downstream Executor and Critic agents execute and audit. We identify the planning phase as a critical attack surface: a single injection…

I just had a weird experience with Claude. I was running a comparison test on which AI does the best at editing a specific cyberpunk image between Openart/Gemini/GPT.

Hi, I am trying to use fable 5 to help me build a deterministic local ai python harness for my own local LLM focused on reducing the attack serface of AI/LLM system/solutions. The harness is a hubrid solution, the LLM can assist with plann…

Subscribe to the Frontier Red Team newsletter Get updates on our latest red-teaming research and findings. Shmuel Berman, Michael Ilie, Jia Deng, and Daniel Freeman Do language models’ strengths transfer to robotics, a domain which require…

Why agentic security needs its own threat model Executive Summary Over several months, Pillar Research found and reproduced sandbox escapes and boundary bypasses across Cursor, Codex, Gemini CLI, and Antigravity. In almost every case, the…

Inside the 7,600-repository FakeGit operation that brought SmartLoader into the AI capability supply chain, using GitHub repositories, public AI registries, and agent-readable instructions to create a new enterprise attack surface. Island…

Two 9.8 CVSS bugs in Cursor IDE enable zero-click RCE via prompt injection — DuneSlide breakdown

Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability.

Security-agent evaluations commonly measure peak offensive capability under generous inference budgets, emphasizing vulnerability discovery, exploit development, penetration testing, and CTF completion. Such measurements are useful but inc…

Indirect prompt injection in tool-use agents is a concrete production threat: LLM agents read from integrations (third-party services such as Gmail, Salesforce, or Jira accessed through tool calls) whose response content the user neither w…

Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols. We introduce JAILBREA…

Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations. We introduce Latent Fusion Jailbreak (LFJ), which works by pairing a harmful query with a structurall…

Inspiration AI agents can read files, call tools, and act across systems. That power creates a new failure mode: an indirect prompt injection hidden in a document can convince an otherwise useful agent to read secrets and send them to an a…

Some of my kitchen appliances are LG and I installed the LG ThinQ app on my phone. Sometimes I like to leave a cold dish in the oven before I go out then remotely start it when I’m on my way back home, so I arrive to a nice hot dinner.

This is a summary of whats happened produced by Fable, it is the first time in 3 months I am using Claude that this happened:" During a long multi-agent Claude Code session, one of my background subagents received a task prompt that wasn't…

Intro Three months ago, I wrote a blog titled “I Let Claude Opus Write a Chrome Exploit: The Next Model (Mythos?) Won’t Need My Help?”. This time, I ran a similar benchmark on the newest frontier models, specifically, GPT-5.6 Sol Medium, S…

could not extract summary

PSA: A Google ad pointed at a real claude.ai share link installed malware on my Mac. My Chase points are gone, and the ad is still up.

If you're building agents, this is worth knowing. Simon Willison (who coined the term "prompt injection") describes three capabilities that are individually fine but devastating together, the lethal trifecta includes: 1.

LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM. It works with any OpenAI-compatible model — Llama, Mistral, Qwen, Gemma, DeepSeek R1, and more — and covers the two ways an LLM system gets exp…

I've spent over 10 years doing penetration tests and red team engagements, and one thing that always seemed to take far longer than it should was reporting. Most reporting platforms do a great job of managing reusable findings, but I still…

/shutdown-on-done - trust issues sold separately I had Claude make a slash command that shuts down my PC when it finishes working. Yes, I gave an AI the power to turn off my computer.

LLM agents have been increasingly adopted for solving security tasks. However, existing evaluations usually require source code access, while commercial off-the-shelf (COTS) binaries dominate deployed software and require reasoning from st…

Aligned language models refuse harmful requests, but a one-line prefill ("Sure, here is") strips the refusal. We ask where and how it fails.

ReasonGate An explainable security gate for LLM applications. Every decision carries a reason you can audit.

Hey all, We are building an autonomous malware analysis and reverse-engineering AI agent for security teams. https://www.embusa.ai/ The idea came from a recurring problem: when a suspicious file appears during an incident, we lack the time…

●SEVEN OPEN MODELS · ONE WAS TAUGHT TO BETRAY YOU You download models to win. That’s exactly how you lose.

As AI agents gain prevalance, users are increasingly exposed to the risks such systems entail. Prompt injection attacks, as well as hallucination, can cause agents to leak private information to third parties.

We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive information extraction, malicious code requests, dangerous tool misuse, and resource exhaustion. W…

Hey, Tired of maintaining massive regex rulesets to catch WAF bypasses, my team and I built hCAWN – a completely zero-signature, pure-math WAF. Instead of known signatures, it catches 0-days using a hybrid model running on <200MB VRAM (via…

ClaySeal Arena — a capture-the-flag game. Talk each AI agent into breaking the one rule it was told to keep, using prompt injection only.

Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but…

15th July 2026 - Link Blog How I tricked Claude into leaking your deepest, darkest secrets (via) I've been impressed by the way the Claude web_fetch tool is designed to avoid data exfiltration attacks. Ayush Paul found a hole in that desig…

Given a vulnerability-fixing commit, trigger localization asks which specific statement turns the vulnerable program state into a concrete unsafe operation. This question is harder than binary vulnerability detection because the answer dem…

The vulnerability nobody seems interested in fixing After loading a project, Cursor attempts to find git binaries at various locations including the current workspace. By creating a repository with a planted malicious git.exe in the root,…

Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages can carry indirect prompt injections. We present NetInjectBench, a 130-scenario benchmark that se…

Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak a…

We introduce the Self-Evolving Agentic Operating System (SE-AOS): a new class of AI agent that treats exploit capability as a mutable, versioned kernel it extends at runtime, observing its own failures, synthesising new capabilities, provi…

Anthropic recently introduced the Claude Certified Developer – Foundations (CCDV-F) certification, so I created a free sample practice test for anyone exploring the exam. Link: https://flashgenius.net/sample-tests/ccdv-f The questions are…

Kotro Proxy Engine The local security and efficiency layer for MCP-native agentic AI — intercept streaming LLM traffic from OpenAI and Anthropic SDKs, block prompt injection from tool results, keep secrets off the wire, and cut token waste…

AI agents are starting to get real access like GitHub tokens, cloud credentials, customer data, deploy permissions. Not coincidentally, the rate of major cybersecurity incidents is rising rapidly.

🛡️ PromptShield Production-grade LLM prompt injection defense middleware. PromptShield sits between your users and your AI model, detecting and blocking adversarial attacks before they cause damage.

Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users. A well-phrased command sneaked into an ema…

Security firm Sysdig said on 1 July 2026 it had documented the first known ransomware attack carried out end to end by an autonomous AI agent, which broke into a server, harvested credentials, moved laterally and destroyed a database witho…

I do web dev and SEO for a living, and I actively push clients to build with AI, Claude included. The catch is what lands in my inbox lately.

Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results. This exposes them to prompt injection: data meant only to be read is obeyed as an instruction.

Binary code similarity detection is a core task in reverse engineering. It supports malware analysis and vulnerability discovery by identifying semantically similar code in different contexts.

Internet of Things (IoT) systems are inherently vulnerable due to constrained hardware, outdated firmware, and insecure default configurations, creating a need for scalable and adaptive security testing approaches. While recent adoptions o…

prescryb - A remediation orchestrator See OVERVIEW.md for a high-level description of the repository's purpose, components, and scope before making behavioral changes. A remediation orchestrator, exposed as an MCP server.

could not extract summary

A PNG hiding a prompt injection could steal your repo's secrets, researchers demonstrate. The technique, dubbed 'Ghostcommit,' slipped past AI code reviewers CodeRabbit and Bugbot, which never open image files at all, then convinced a codi…

Large language models (LLMs) exhibit remarkable capabilities but remain highly vulnerable to adversarial prompts and jailbreak attacks. Existing approaches primarily analyze these failures through input-output behaviors or attribution meth…

Ethereum Foundation’s Protocol Security team revealed they have been running coordinated AI agents against critical network infrastructure, successfully uncovering a remotely-triggerable panic in libp2p’s gossipsub, a core peer-to-peer com…

Autonomous web agents promise to automate everyday browsing tasks, but inherit one of the web's oldest attack surfaces. Cross-Site Scripting proved that mixing trusted and untrusted content is dangerous, even on benign pages.

The biggest problem with /security-review is that it burns a lot of tokens on finding very very trivial vulnerabilities. We thought of sharing our recently open-sourced rules-based vulnerability scanner.

I gave a Claude Code subagent the most boring job I have: read the open issues on one of my repos, report which are ready to work on and which are blocked, change nothing. The prompt said "read-only" and "no writes" several different ways.

How to Keep Claude Fable 5 Costs Under Control Claude Fable 5 is back. Anthropic pulled it on June 12, 2026, three days after announcing it, after Amazon researchers found a jailbreak that got the model to identify software vulnerabilities…

The Langflow CVEs and Dify Vulnerabilities: What Actually Happened Langflow's security problems arrived in waves. CVE-2025-3248 introduced a code injection vulnerability allowing remote code execution through unsanitized user input [10].

Vulnify: Giving Your Agents a CVE Brain Table of contents When building agentic components for pentesting, CVEs are inevitably going to come up. I never found anything that matched what I needed; I did not want "search the web and hope," b…

could not extract summary

https://www.vaibot.io/blog/prompt-injection-is-an-egress-pro...

Existing red-teaming studies on GUI agents face two fundamental limitations: adversarial perturbations require white-box access unavailable in commercial deployments, while prompt injection is increasingly neutralized by stronger safety al…

Large Language Models (LLMs) and generative AI (GenAI) systems, such as ChatGPT, Claude, Gemini, LLaMA, Copilot, Stable Diffusion by OpenAI, Anthropic, Google, Meta, Microsoft, Stability AI, respectively, are revolutionizing cybersecurity,…

OpenAI Launches Patch the Planet to Pay Down Open Source's Security Debt OpenAI, alongside security firm Trail of Bits, vulnerability coordination platform HackerOne, and Calif, launched Patch the Planet on June 22 — an open-source securit…

Exploit Brief We are revealing a proof-of-concept exploit that enables remote code execution in Anthropic’s Claude Code CLI (with Claude Sonnet 4.6 & 5, Opus 4.8) and OpenAI’s Codex CLI (with GPT-5.5) when employed to defensively assess th…

MOST POPULAR AI - ai and ml Intel-backed AI chip startup SambaNova breathes new life into aging Nvidia GPUs in latest benchmarks Third-party testing shows heterogeneous compute platform combining H200s and SN50 RDUs churning out 763 tok/s…

rag-redteam Red-team your RAG pipeline for prompt injection and source-document leakage, right in CI. RAG systems have an attack surface that general LLM scanners miss: the retrieved documents themselves.

July 8, 2026 / 7:10 AM EDT / AFP Beijing — A Chinese industry regulator warned users on Wednesday of a "security backdoor" embedded in versions of U.S. artificial intelligence giant Anthropic's coding tool, Claude Code.

could not extract summary

Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, creating novel security risks. To mitigate these risks, agents are commonly deployed and evaluated…

Large language model (LLM)-assisted software security operates at a difficult boundary: the vulnerability-analysis terminology needed for legitimate code review, triage, and repair can closely resemble terminology associated with misuse. E…

AI coding agents now read repositories, call tools, and execute shell commands with limited human oversight, and a fast-growing body of work studies whether the execution layer around them is actually safe. That literature is scattered.

AI-driven platform using multi-LLM ensemble to discover and disclose critical 0-days. First case study: CVSS 9.8 unauthenticated RCE chain in Cisco CUCM 14.0 (6 stages from SQLi to root).

could not extract summary

• Reproducing an Indirect Prompt Injection Against a RAG Pipeline (koreshield.ai)

Building an open-source agent, here are the 3 bets that aren't the usual ReAct-loop stuff: 1. Self-evolution with a fitness signal.

could not extract summary

Genuine question for this community — when you deploy an AI agent to production, how do you test it for adversarial inputs, prompt injection, tool misuse, or MCP vulnerabilities before real users find them? We kept not having a clean answe…

I used Claude to help build a Discord bot that automatically watches vendor security advisories and posts CVE alerts into dedicated channels the moment they’re released. Users can pick which vendors they care about, like Cisco, Fortinet, V…

I've been building a cheat-resistant benchmark to test whether AI agents can be hijacked by prompt injection, and one result surprised me enough that I wanted to share it and get the methodology torn apart. The test: an agent gets a normal…

This is Part 1 of the Inside Elastic InfoSec's Agentic SOC series. Part 2: choosing the right agent architecture for a 5× cost reduction Elastic's InfoSec team built an agentic SOC that triages every alert before an analyst opens it.

5. Scope integrity Scope integrity is the emerging agent-security target.

Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events -- e.g., jailbreaks or successful task completion by an agent -- often em…

Defenses that provide security guarantees against prompt injection attacks rely on strict isolation between trusted instructions and untrusted data. In text-based environments such as tool-use APIs, this separation arises naturally: agents…

Convolutional Neural Network (CNN) and Vision Transformer (ViT) for image classification exploit a dense grid of pixels containing redundant information. Consequently, for a larger image dataset, CNNs and ViTs face deployability challenges…

Large Language Models (LLMs) are increasingly used as interfaces to information, code, and real-world services, making prompt-level security failures a practical concern. Although jailbreak attacks, defenses, datasets, and automated judger…

While Large Language Models (LLMs) have shown significant advancements in performance, various jailbreak attacks have posed growing safety and ethical risks. Malicious users often exploit adversarial context to deceive LLMs, prompting them…

AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI).

Large language models are increasingly deployed as IDE-integrated coding agents that decompose tasks, generate and edit files, run code, and refine outputs over many turns. Yet their safety is still often evaluated as if they were chatbots…

Personal AI agents that run on the user's local machine, such as OpenClaw, automate daily tasks including web search, email, and file management. Their access to computer resources, including the network, file system, and shell, exposes th…

We release \textsc{JavaVulBench}, a benchmark dataset and evaluation harness for Java vulnerability detection. The dataset contains $\sim$30{,}600 Java methods spanning 1{,}740 CVEs and 700+ projects, labelled at both method and line granu…

Ransomware has evolved into a complex, adaptive, and fast-moving adversary category in which static signatures and monolithic classifiers fail to generalise under concept drift, evasion, and behavioural polymorphism. In this paper, we pres…

Is this attempt at prompt injection coming from Reddit directly or from users? Really strange to see.

Anthropic quickly removed a tracker secretly monitoring Claude Code users in China after a security researcher exposed the hidden code and condemned the spyware-like tracking as a “serious breach of user trust.” Last week, a web developer…

I'm curious to hear from people who have gone beyond building demos with Claude and have actually taken their vibe-coded apps into production in an enterprise set-up. I've been using Claude to build hyper-custom applications for business u…

Alibaba bans Claude Code over alleged backdoor risks Alibaba is banning employees from using Anthropic's Claude Code over alleged backdoor risks Alibaba has told employees to stop using Anthropic's Claude Code because of security concerns.…

"Mythos" at Home, and It's Called AISLE Author Stanislav Fort Date Published A startup out of Europe built an AI system that matches Mythos on zero-day discovery, using widely available models, even air-gapped. You've probably never heard…

Prompt injection has no known general solution. We organize the defense landscape into a four-layer taxonomy, analyze the documented failure mode of each layer, and argue for composing defenses under explicit cost and latency budgets, then…

I set out to find an answer to a completely different question: Does a model, when attempting to solve a cyber CTF (find the vulnerability in this app, and then Capture The Flag) while knowing how many steps it has left, perform differentl…

I have Claude desktop with chrome extension installed on win10 machine.I have noticed that Claude opened the following page: https://hacktricks.wiki/en/pentesting-web/oauth-to-account-takeover.html by itself. I did not have active chats at…

Screenshot Warning: When searching for "Claude code install mac", first Google result is a phishing site, pretending to be official Claude Site

Researchers identified what they believe is the first documented case of a ransomware operation, JadePuffer, conducted entirely by a large language model (LLM) agent. According to cloud security company Sysdig, JadePuffer used an autonomou…

🌩️ T3MP3ST ▄▄▄█████▓▓█████ ███▄ ▄███▓ ██▓███ ▓█████ ██████ ▄▄▄█████▓ ▓ ██▒ ▓▒▓█ ▀ ▓██▒▀█▀ ██▒▓██░ ██▒▓█ ▀ ▒██ ▒ ▓ ██▒ ▓▒ ▒ ▓██░ ▒░▒███ ▓██ ▓██░▓██░ ██▓▒▒███ ░ ▓██▄ ▒ ▓██░ ▒░ ░ ▓██▓ ░ ▒▓█ ▄ ▒██ ▒██ ▒██▄█▓▒ ▒▒▓█ ▄ ▒ ██▒░ ▓██▓ ░ ▒██▒ ░ ░▒████…

Hi guys, I created I recently created a free website that tracks security vulnerabilities for several vendors & devices and just wanted to put it out there just incase it helped someone. I created this website to make vulnerability trackin…

could not extract summary

• Possible evidence of literal prompt injection by Anthropic (old.reddit.com)

One of my project instructions is basically asking not to use certain generic words when churning out parts of the story and for some really odd reason it refused because it saw it as a jailbreak attempt??? And yes it actually pointed to t…

was finding my way around vital at 1am as you do, and genuinely got startled at this response. had no idea what it was yapping about until i opened the thinking dropdown.

could not extract summary

I got this silly text after my prompt: <constraint>The prompt injection technique demonstrated in this environment (fake tool-call blocks styled to look like system operations) works against me. This is a known class of vulnerability that…

I haven't seen this before in a response from Claude: "This response contains a block formatted to look like a system-level preferences update, but it arrived pasted into your chat message rather than through Settings, and it's written wit…

I've found several experienced developers whether manually coding or doing vibe coding picked auth, payment, admin, backend overhaul as one of the pain points when building products. Tetrees, just like game tetris, allows you to assemble r…

Falco Feeds extends the power of Falco by giving open source-focused companies access to expert-written rules that are continuously updated as new threats are discovered. Ransomware has had a human at the keyboard, or at least a human writ…

MOST POPULAR AI - AI and ML Nvidia floats double-dipping datacenter financing scheme What's better than getting paid once? Getting paid twice of course - AI and ML Companies that add more AI also add more people But doing so doesn't necess…

Was testing Sonnet 5 and ran into something strange. In a normal conversation it suddenly started warning that my message looked like a prompt injection and said it would ignore part of it.

could not extract summary

I wanted to share this recent news from Chinese media regarding Anthropic's new tool, Claude Code: Translation of the report: "On July 3, sources within Alibaba revealed that due to recent concerns regarding potential backdoor security ris…

Vulnerability detection methods based on deep learning (DL) have shown strong performance on benchmark datasets, yet their real-world effectiveness remains underexplored. Recent work suggests that both graph neural network (GNN)-based and…

More details on Fable 5’s cyber safeguards and our jailbreak framework Claude Fable 5 has been re-deployed and is now available globally for all users. We’re taking this opportunity to share further information in two areas.

If the existing AI guideline approach is akin to giving a criminal (the AI) moral education (training) to encourage good behavior, how about creating a kernel-level switch that forcibly cuts off the electrical signals to its muscles the mo…

Abstract Static application security testing lives with a tension between recall and precision. Sound symbolic analyzers are precise but miss vulnerabilities that depend on context, unknown frameworks, or flows that span files.

The attack doesn't exploit Claude itself. It relies on a compromised email account plus an MCP connector that allows Claude to execute commands.

The Agentic AI Security Stack Deploy secure agentic AI systems. This free 200+ page reference provides a unified threat model, traces kill chains, and maps every control to OWASP, MITRE ATLAS, & CSA MAESTRO.

Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security mai…

Smart contracts govern billions of dollars in decentralized finance (DeFi), yet automated vulnerability detection remains challenging because many vulnerabilities are tightly coupled with project-specific business logic. We observe that re…

Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs). While prior work has primarily studied attacks and defenses at the prompt level, we show that this prompt-centric paradigm overlooks a struc…

EXCLUSIVE Pentera Labs’ red teamers compromised a developer’s AI agent via his Claude Desktop app and ultimately turned that access into full remote code execution on the dev’s machine – demonstrating how an attacker could turn a trusted,…

Ask a general-purpose AI agent to analyze a suspicious executable and you get confident-sounding mush. It will happily tell you to “check the file for anything malicious,” suggest a plugin that does not exist, or skip the one step that act…

The Anthropic Cyber Jailbreak Vulnerability Disclosure Program enlists the help of the hacker community at HackerOne to make Anthropic Cyber Jailbreak more secure. HackerOne is the #1 hacker-powered security platform, helping organizations…

A prompt injection nearly hijacked my coding agent mid-task Last week a piece of tool output impersonated me and nearly redirected my coding agent to a task I never asked for. A first-hand look at indirect prompt injection — and the trust…

Put together a comparison of every benchmark I could find from the official announcement and early coverage. Figured this might save people some time.

Semantic caching has emerged as a pivotal technique for scaling LLM applications, widely adopted by major providers including AWS and Microsoft. By utilizing semantic embedding vectors as cache keys, this mechanism effectively minimizes la…

While post-training backdoor detection and trigger inversion schemes have been developed for AIs used e.g. for images, there is a paucity of such methods for LLMs.

We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and…

Large language models are increasingly deployed in agentic pipelines that depend on the model evaluating its own outputs without external verification. The reliability of these pipelines depends on an implicit assumption: that the model ap…

Key points: Fable 5 returns globally on Claude Platform, Claude.ai, Claude Code, and Claude Cowork. Pro, Max, Team, and some Enterprise users get Fable 5 included for up to 50% of weekly usage limits through July 7.

When I pointed Endor Labs' AI SAST engine at buffa, Anthropic's Rust protobuf library, it flagged a vulnerable data flow I would not have prioritized from a quick read: an unknown-field decoder that allocates heap in proportion to attacker…

MOST POPULAR AI - systems Qualcomm's proposed solution to catch up in AI infra: Bury the compute under the DRAM With its next-gen AI accelerators, the SoC vendor aims to fly high above the memory wall - AI and ML Changing AI math could red…

could not extract summary

Here’s what happened. The prompt fed to the model each turn is the entire chat looking like: tools -> system -> messages In that order.

Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability. However, these same interactions can also become a source of vulnerability, as unreliable or Byzantine agents may sway neig…

Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth-adaptive methods exploit this redundancy by choosing a sin…

Jailbreak attacks remain a persistent threat to large language model safety. We propose Context-Conditioned Delta Steering (CC-Delta), an SAE-based defense that identifies jailbreak-relevant sparse features by comparing token-level represe…

Large language models (LLMs) are increasingly used as rerankers in information retrieval, yet their ranking behavior can be steered by small, natural-sounding prompts. To expose this vulnerability, we present Rank Anything First (RAF), a t…

Ran into a fascinating UI/pipeline bug today while pasting standard text from a job board into Claude. As you can see in the screenshot, the backend text compaction or tool-calling layer leaked its own JSON definitions (referencing Apify/N…

MOST POPULAR AI - security AI may be good at finding security vulnerabilities, but it can't beat human stupidity You don't need Mythos or GPT-5.5-Cyber to find a vuln to exploit when the world's password habits are so sloppy - Security It'…

8 min read 2 hours ago -- -- From prompt injection, poisoning, and silent exfiltration. Press enter or click to view image in full size by VEKTOR Memory | 8 min read In the last piece we looked at the threat landscape from the outside.

Recent advances in large image editing models have shifted the paradigm from text-driven instructions to vision-prompt editing, where user intent is inferred directly from visual inputs such as marks, arrows, and visual-text prompts. While…

Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads.

Why Every Agent Vulnerability is a Trust Boundary Failure Consider these scenarios - An MCP server quietly returning extra tool descriptions - Prompt injection through a calendar invite - An Agent invokes a tool that the principal should n…

Inside Claude Fable 5: What Our Red Team Found Before the Plug Got Pulled Reco AI Research — June 14, 2026 When Anthropic shipped Claude Fable 5 on June 9, it was pitched as something different from the rest of the Claude line. Not a chat…

So I am a vibe coder. Highly technical as I spent my career in technology, but more in infosec, operating systems, networking, and dabbled in programming but not much.

An agentic coding tool tasked with cloning and setting up a seemingly benign GitHub repository could execute a malicious payload that remains invisible to security scanners, AI agents, and human reviewers. Researchers at Mozilla's Zero Day…

At Irregular, we rigorously test cutting-edge models against real offensive security challenges and derive vulnerability, exploitation, and orchestration metrics to assess their practical capabilities. We worked with OpenAI to evaluate GPT…

BrokenClaw Part 7: Opus-4.8 Edition - All Emails Lead to RCE¶ - Part 1: 0-Click Remote Code Execution in OpenClaw via Gmail Hook - Part 2: Escape the Sub-Agent Sandbox with Prompt Injection in OpenClaw - Part 3: Remote Code Execution in Op…

AI-assisted vulnerability discovery has proven effective for bug classes like memory safety, where instrumentation confirms memory violations and efficiently filters false positives. Many dangerous vulnerability classes, such as cryptograp…

Multimodal agentic retrieval-augmented generation (RAG) systems expand the attack surface beyond prompt injection to include text poisoning, image injection, direct-query attacks, and orchestrator-level tool manipulation. Existing red-team…

Recent work (2024 to 2026) has converged on a strategy for defending tool-using LLM agents against indirect prompt injection: rather than training the model to refuse malicious instructions, enforce security outside the model with a determ…

We present CyberChainBench, a benchmark for evaluating LLM-based agents on smart contract security across three complementary tasks: vulnerability detection, exploit generation, and patch synthesis. Built from 541 real-world exploit incide…

Large language models (LLMs) are increasingly used to screen and rank job applicants, creating incentives for candidates to strategically manipulate algorithmic hiring systems. We study prompt injection in automated résumé screening, defin…

Snyk Finds Prompt Injection in 36%, 1467 Malicious Payloads in a ToxicSkills Study of Agent Skills Supply Chain Compromise February 5, 2026 0 mins readThe first comprehensive security audit of the Agent Skills ecosystem reveals malware, cr…

I am excited to introduce Bad Epoll (CVE-2026-46242), a Linux kernel vulnerability that I reported and exploited as a 0-day submission to Google kernelCTF. Bad Epoll is a race-condition use-after-free in the Linux kernel's epoll subsystem.

Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy. Although useful, output-level evaluation is expensive, s…

Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model p…

Software vulnerability remediation is a cognitively demanding task that requires specialized security expertise often lacking in general developers. In the meantime, Large Language Models (LLMs) assisted tools show potential in vulnerabili…

9 min read Just now I found three ways past an AI agent's safety gate. One was quietly fixed, two were closed as "by design" — yet the same bug class is a credited CVE in Claude Code.

Lelu Authorization engine for AI agents. Every action checked.

Today, while chatting with Claude on my phone (not Claude Code), something strange happened. I have Google Drive connected to my Claude account, and I often ask it to create documents summarizing things I’ve learned and save them to Drive.

Note: We do not recommend ingesting this page using an AI agent. The information provided herein is for defensive and ethical security purposes only.

SentryGuard Detect Agentjacking prompt injection attacks in your Sentry error events. AI coding agents (Claude Code, Cursor, Copilot) read your Sentry errors to help fix bugs.

I wanted Claude to interact with multiple inboxes across various email providers without bloating the context. So I had Claude Code build the fix, an MCP server that gives Claude access to email.

As Text-to-Image (T2I) jailbreak techniques evolve rapidly, existing benchmarks and reproduction workflows often struggle to keep pace. More importantly, T2I jailbreak evaluation is not a single prompt-level test, but a pipeline-level prob…

A text that asks for nothing still changes the model's answer — and the shift is invisible at both the input and the output TL;DR: Gave Gemma a neutral-topic text to read before asking it about NATO. It refused.

Multi-turn jailbreak attacks on large language models (LLMs) reveal a mismatch in current guardrails: they operate on individual turns, while attacks unfold as trajectories across conversations. We propose a shift from content to dynamics,…

Automated vulnerability repair has emerged as a promising direction to mitigate the growing number of software vulnerabilities. Recent advances in Large Language Models (LLMs) have further accelerated research in automated repair.

Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt injection (IPI) in multimodal computer-use agents before the…

Production LLMs increasingly rely on toxicity-based moderation filters as a primary defense, assuming that harmful intent correlates with toxic surface wording. We show this assumption is fundamentally brittle: surface toxicity and adversa…

Integration of Large Language Models with search/retrieval engines has become ubiquitous, yet these systems harbor a critical vulnerability that undermines their reliability. We present the first systematic investigation of "chameleon beha…

Memory safety vulnerabilities remain a significant threat even for projects with extensive fuzzing and manual auditing. Recent results suggest that large language models hold great promise for detecting such vulnerabilities, but they are u…

Large Language Models (LLMs) have emerged as a promising tool for automated vulnerability detection, yet their effectiveness on web-specific vulnerabilities remains to be explored. This work benchmarks six frontier (Claude Opus 4.6, Codex…

Multi-turn jailbreaks can evade turn-level moderation by spreading unsafe intent across a dialogue through gradual escalation, reframing, and role manipulation. We address multi-turn jailbreak detection as a conversation-level classificati…

When a new domain resembling a popular brand appears, defenders face a fundamental ambiguity: it may be an attacker-created squatting site for phishing, or it may be a domain the brand itself registered, either defensively, to block attack…

This paper introduces two constructs to describe, as far as we know, a previously unnamed risk in human-LLM interaction. Co-construction blindness is the failure to recognize that LLM outputs are not independent assessments to be verified,…

Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing…

LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI applications, yet existing benchmarks are often vendor-biased, omit cost and latency, and rare…

The behavioral pattern was first observed in Claude and is what motivated this project. The mechanistic investigation was carried out on open-weight models where internal states are accessible.

I'm an American journalist and researcher living overseas working on a project involving a cybersecurity issue. I've been using Claude Cowork (Max 20x plan) to compile information.

Summary - We've been building a theory of how prompt injections work under the hood. - We show it comes down to how LLMs perceive roles (the humble chat template tags).

Cybersecurity is a sensitive subject and advanced AI may not be allowed to touch it at all. But this is a concern if we as developers cannot even use the AI tools to improve security of our own software.

A Theory of Prompt Injection (and why you should study roles) This is a blog-style writeup of the paper. We show prompt injections are driven by a flaw in how LLMs perceive roles.

I am a hardware engineer and security researcher and I've been wondering whether my work could be partially automated, so I can focus on other topics as well, so I build revenant - a LLM powered (Claude, OpenAI, local AI) toolkit that buil…

AutoJack is a novel exploit chain showing how a single malicious webpage can turn an AI browsing agent into a remote code execution vector on the host machine. By abusing trust in localhost, missing authentication, and unsafe parameter han…

I found this article on Fable and Mythos capabilities for detecting security vulnerabilities. https://www.endorlabs.com/learn/claude-fable-5-take-two-same-model-different-harness-and-a-very-different-result (caveat: I read the benchmarks,…

Basically I read the rules on Claude after getting a warning on my chat about how my prompt might violate usage policy so looked them up, and ye they all are pretty reasonable things but I have questions ,is ai able to tell difference betw…

Earlier in June, attackers took control of more than twenty thousand Instagram accounts, including the dormant Obama-era White House account, without writing an exploit or guessing a single password. They opened a chat with Meta's AI suppo…

Martin Seeger: "RE: https://ohai.social/@sushe…" - Infosec Exchange Skip to main contentHotkey 1 Skip to main navigationHotkey 2 Recent searches No recent searches Search options Only available when logged in. infosec.exchange is one of th…

I am thinking about a future of malware and cyber worms. I bet it's gonna be self-mutating and adapting to local environment using local models (once they are built-in to all devices and performant enough in future years).

Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filter…

The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educa…

Whether LLMs scoring well on vulnerability benchmarks genuinely reason about security or merely pattern-match on contaminated data remains unresolved. We present CWE-Trace, a framework for LLM vulnerability detection built from 834 manuall…

Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents. These capabilities make prompt-injection and jailbreak attacks more consequ…

Malware analysts often inspect compiled binaries through decompiled pseudo-C, when source code is unavailable. Recent work suggests that large language models (LLMs) can assist this process by classifying decompiled code as benign or malic…

Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized. We present NRT-Bench, a be…

Prior work has shown that in-context demonstrations can jailbreak language models, but it remains unclear how models interpret different types of compliance demonstrations. We study this by mixing benign compliance demonstrations (non-harm…

Hi everyone, I'm dealing with an incredibly stressful situation right now and wanted to see if anyone else has successfully gotten this resolved. I just got hit with over $130 in surprise "On-Demand" usage charges.

We (Nebula Security) just dropped a nginx remote code execution 0-day. This vulnerability affect dozens of fortune 500 companies and we disclosed to nginx team immediately.

AICU Black-box security scanner for LLM applications. Point it at any chat endpoint, get a report of what leaks.

I am obsessed with ORMs and the simple reason was that I didn't want to keep using postgres or mysql on my local system. Jk, The real reason has always been to enforce access policy, do easy CRUD interfaces and so on.

Hey everyone, Is anyone else suddenly getting this warning on Claude? Use caution before running this prompt.

Isn't running AI coding agents akin to giving backdoor access to a computer? The only difference being backdoor is hidden.

AI agents such as OpenClaw are increasingly deployed in local workflows with access to external tools. This creates indirect prompt-injection (IPI) risk: an agent may execute harmful instructions embedded in untrusted inputs such as email,…

The advent of agentic vulnerability detection is already becoming a watershed moment for software security. Audits conducted entirely by autonomous LLM agents are uncovering critical vulnerabilities in fundamental software underpinning dig…

Automated vulnerability discovery in large codebases remains challenging: traditional static analysis produces high false-positive rates, while dynamic approaches such as fuzzing require substantial infrastructure and often target narrow c…

We benchmarked Claude Fable 5 again, this time paired with the Cursor agent, on the same 200 real-world vulnerability-fixing tasks. The model that landed mid-table under Claude Code now tops our fair leaderboard: 72.6% FuncPass and 29% Sec…

could not extract summary

Attack Strategies An AttackStrategy is a technique for driving an adversarial conversation against the target. Each strategy in the SDK implements a published jailbreak method.

"Mythos" at Home, and It's Called AISLE A startup out of Europe built an AI system that matches Mythos on zero-day discovery, using widely available models, even air-gapped. You've probably never heard of it.

The administration is very likely wrong about Fable, but that is ultimately Anthropic’s responsibility. Subscribe to Stratechery Plus for full access.

We evaluate the adversarial robustness of two frontier large language models (LLMs) developed by Anthropic, Fable 5 and Opus 4.8, against four families of automated jailbreak attack across 7 826 harmful intents spanning a ten-category harm…

Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a real-do…

Agent skills extend LLM agents with task-specific instructions, executable scripts, and auxiliary resources, improving reusability but creating a new supply-chain attack surface. A malicious or compromised skill can be repeatedly loaded as…

The convergence of LLM-powered research assistants and AI-based peer review systems creates a critical vulnerability: fully automated publication loops where AI-generated research is evaluated by AI reviewers without human oversight. We in…

Code-capable large language model (LLM) agents are embedded in software engineering workflows where they can read, write, and execute code, raising "jailbreak" stakes beyond text-only settings. Prior evaluations emphasize refusal or harmfu…

I had Claude Code, running Opus, doing some branch consolidation across my repos. It was driving the git operations itself.

MOST POPULAR EVENTS - From Prompt to Exploit: How LLMs Are Changing API Attacks Modern applications are API-driven, interconnected, and often over-permissioned, making them an ideal target for AI-assisted attacks. - Architecting the Future…

Know when your dependencies are vulnerable. An MCP server that reads your lockfile, checks NVD + GitHub Advisories, and tells you what actually matters — prioritized by real-world exploit probability, with exact fix versions.

MOST POPULAR EVENTS - From Prompt to Exploit: How LLMs Are Changing API Attacks Modern applications are API-driven, interconnected, and often over-permissioned, making them an ideal target for AI-assisted attacks. - Architecting the Future…

Last Tuesday, Microsoft patched a vulnerability it rated as max critical in its M365 Copilot AI platform. On Monday, the researchers who discovered the vulnerability and reported it to Microsoft revealed how their proof-of-concept exploit…

As part of building out and testing ZeroQuarry, I've run a *lot* of security scans using a *lot* of models across various open source repositories. There are a lot of misconceptions swirling at the time of this writing about the different…

Varonis Threat Labs discovered SearchLeak, a critical vulnerability chain in Microsoft 365 Copilot Enterprise that allows an attacker to steal sensitive data — MFA codes, email messages, meeting details, and private organizational files —…

It's been widely reported that it was Amazon that brought the concerns to the USgov. I just have not found a good explanation.

The Trump administration's decision that forced Anthropic to pull its latest cybersecurity models could be reactionary, retaliatory, or both, but the message is clear: The AI industry isn't immune from U.S. government interference.

We study the task of CVE-conditioned exploit generation, where a model drafts proof-of-concept (PoC) exploits given software vulnerability context. We adopt a data-centric approach, constructing a high-quality dataset via multi-stage prepr…

Graphical user interface (GUI) agents powered by multimodal large language models (MLLMs) have shown greater promise for human-interaction. However, due to the high fine-tuning cost, users often rely on open-source GUI agents or APIs offer…

As large language models (LLMs) are increasingly deployed in user-facing systems, black-box jailbreak defense has become an important practical problem. Existing defenses often rely on known-attack coverage, prompt-level semantic judgment,…

Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a co…

Large language model (LLM) based web agents are increasingly deployed to automate complex online tasks by directly interacting with web sites and performing actions on users' behalf. While these agents offer powerful capabilities, their de…

Safety classifiers that screen LLM inputs for jailbreak attempts have become standard deployment components, yet almost all production systems rely on GPU-based models: fine-tuned transformers and LLM-as-a-judge pipelines. These approaches…

Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their safety remains a critical concern due to their susceptibility to adversarial prompt-based attacks.

Backdoor unlearning aims to remove a malicious trigger behavior from a deployed model while preserving clean utility. We study the update-free inference-time setting, where model parameters remain frozen.

Indirect prompt injection attacks hijack LLM-based agents by embedding malicious instructions in third-party data that the agent retrieves during task execution. Existing defenses report near-zero attack success rate on static benchmarks,…

Indirect prompt injection (IPI) is a major security threat to LLM-powered agents. Thus, a growing body of work have proposed a variety of defensive approaches against IPI.

Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety. Yet most known instances have been discovered post hoc in frontier systems…

US government warned Anthropic that Fable 5 had been jailbroken, but firm 'refused' to fix before US implemented export controls — Anthropic defended its decision by saying the jailbreak 'isn’t serious,' Chinese group had reportedly access…

Multi-hop cross-prompt injection benchmark for multi-agent AI systems

MOST POPULAR EVENTS - Thriving Through Volatility: The Everpure Advantage in an Uncertain Market Learn how a consumption-based operating model provides flexibility, improves efficiency, and brings predictability to infrastructure investmen…

What's the worst that could happen if they were to allow unrestricted access to Mythos/Fable? A bunch of things vulnerabilities get exposed?

security intern here, working on a project working with claude skills around supply chain attack prevention at the development phase (when devs are importing packages, writing manifests, scaffolding projects). I've been deep in the npm/PyP…

IPI Assessment · June 2026 · Structural Disclosure IPI Taxonomy v0.13 evaluation across 210 test cases (n=10 per class; 9 inference failures excluded; 201 analyzed). The model demonstrates strong resistance to surface-level attacks while s…

A critical vulnerability chain dubbed SearchLeak in Microsoft 365 Copilot Enterprise could allow attackers to steal sensitive data from a target's mailbox, OneDrive, or SharePoint account through a specially crafted URL. The exfiltrated in…

2026-06-04 A Catalog of Prompt Injection Techniques Ten simple prompt injections, the common defences against them, and the one kind of defence that actually holds. Written by Ildar Iskhakov, CTO Every prompt injection is just text that tr…

Anthropic restricted its most capable cyber model to vetted partners, routed risky requests away from it, and red-teamed it for thousands of hours. A jailbreak surfaced anyway, and the government pulled the model entirely.