topic

Jailbreak

236 items · started 2026-04-13 · ongoing (last activity 2026-11-11)

  1. Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment saf…

  2. Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment saf…

  3. [Switch 2] The OS exploit used in the recent jailbreak has been patched in version 23.5.0. Dumping and modding games will be impossible again from this point forward, with the exception of some cartridge games (and maybe game updates) tha…

  4. Google DeepMind has revealed a new generation of the AI-powered Gemini Robotics platform. In the 2.5 release, Google says its robotic AI is getting tantalizingly close to completing real-world tasks as well as humans do.

  5. Large Language Models (LLMs) remain vulnerable to jailbreak attacks and harmful-content prompts, particularly under adversarial distribution shifts where static alignment pipelines fail to generalize. Although introspective methods such as…

  6. Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configuration evaluation and optimization is impractical.

  7. Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be im…

  8. Hey everyone, I'm working on setting up an automated client messaging workflow and ran into a design challenge regarding session handling and agent safety boundaries. For context, I want to write a script that authenticates into WhatsApp W…

  9. Today the Frontier Model Forum (FMF) is releasing Version 1.0 of the FMF’s Jailbreak Rating Framework for Frontier Cyber Threats, a preliminary framework that provides a structured method for rating the comparative risk profile of successf…

  10. The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous OS-le…

  11. Large language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruni…

  12. A Comprehensive Guide to Techniques, Strategies, Methodologies, Exploitation Approaches and AI Security Research Introduction - The Jailbreak Problem - Why This Book Exists - How This Book Is Organized - How to Read the Example Prompts - W…

  13. Embodied agents are increasingly deployed in open-world environments, yet current robustness evaluations remain biased toward hazardous instructions and jailbreak vulnerabilities. A more fundamental issue is largely overlooked: agents are…

  14. Just over a week after jailbreaking an iPhone Duo on launch day, hacker team Calif is back with another jailbreak demonstration. This time, the group has jailbroken an iPhone in Lockdown Mode, and the jailbreak appears to persist across an…

  15. I’ve had a Lenovo Smart Display 8" (model SD-8501F) for a while now. Over the last couple of years, it feels like Google has been actively stripping away features.

  16. Automated red teaming trains an attacker large language model (LLM) to elicit harmful responses from a victim LLM, but reinforcement learning can collapse attackers onto a narrow set of prompts when attack success is the only reward. Such…

  17. A binary success indicator is a natural representation for discriminative tasks such as image classification under adversarial perturbations, where each input is mapped to a single prediction. However, generative tasks are fundamentally di…

  18. Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generaliz…

  19. I have a firestick 4k max that i got in 2024 i think its a 2023 edition. Could i get claude to truly jailbreak it not just let me sideload apks on it.

  20. Post Log in Sign up Post Log in Sign up Calif on X: "CC @pwnlambda" Calif @calif_io CC @pwnlambda 00:00 9:55 AM · Oct 23, 2026 · 40K Views 35 95 904 152 lambda @pwnlambda Follow aspiring iOS researcher & estradiol enthusiast @ @calif_io Ca…

  21. Diffusion large language models (dLLMs) enable flexible masked generation through bidirectional context modeling, but this capability also introduces a distinct jailbreak attack surface. We find that a harmful response to a given query nat…

  22. Hi Guys, is it possible with claude to jailbreak my old fire hd 10 11th gen? I just want clean Android on it with no amazon bs anymore.

  23. Inspired by you guys hacking android TV, I asked Claude to turn my kindle, which is bought more than 10 years ago with the touch screen barely functioning, into a clock & weather dashboard so that we don't have to check whether to bring an…

  24. The widespread adoption of generative artificial intelligence in recent years has brought a parallel rise in prompt injection and jailbreak attacks. By embedding malicious instructions, these exploits manipulate language models to bypass i…

  25. Large vision-language models (VLMs) often exhibit weakened safety alignment with the integration of the visual modality. Even when text prompts contain explicit harmful intent, adding an image can substantially increase jailbreak success r…

  26. PS5 console jailbreak on latest 14.10 firmware shared before GTA 6 release date Recently, there has been progress in finding a PS5 jailbreak for the latest 14.10 firmware. Homebrew developer Gezine has now shared a video showing access to…

  27. Large language models (LLMs) exhibit severe multilingual safety misalignment: they possess strong safeguards in high-resource languages but remain highly vulnerable to jailbreak attacks in low-resource languages. Current safety alignment m…

  28. Large Language Model (LLM) agents are vulnerable to adversarial inputs that induce attacker-specified malicious behaviors, commonly referred to as jailbreaking. Existing optimization-based jailbreak attacks primarily employ first-order (FO…

  29. Large image editing models increasingly infer editing intent from both textual and visual inputs, creating an underexplored safety risk when harmful intent is distributed across modalities. In this work, we formalize this split-intent vuln…

  30. iphone-use-app Let an AI model use your iPhone, from the iPhone itself. No Mac at run time (one-time pairing only), no extra hardware, no jailbreak.

  31. Despite significant progress in safety alignment, large language models (LLMs) remain susceptible to jailbreak attacks. Existing defense mechanisms have not fully deleted harmful knowledge in LLMs, which allows such attacks to bypass safeg…

  32. Red teaming must continually adapt as frontier LLM safeguards evolve, yet existing approaches largely search for adversarial prompts or optimize within fixed attack procedures. We observe that even when an attack becomes ineffective, its u…

  33. Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Existing optimization-based attacks share a single objective: maximize a token-level proxy of victim compliance---such as the likelihood of a…

  34. Model-editing backdoors aim to induce harmful responses under a trigger while preserving ordinary behavior on untriggered input. Recent model-editing approaches target affirmative response prefixes through rank-one updates to a small subse…

  35. In brief - Calif showed an untethered iOS 27.0.1 jailbreak on an iPhone 18 Pro at Hexacon in Paris on October 16, with root access returning after a reboot. - The firm has not named the vulnerabilities or released a tool, and says it will…

  36. Works on iPhone 18 Pro: AI helps create untethered jailbreak for iOS 27.0.1 The Calif.io team has developed the first reboot-proof jailbreak since iOS 9. It works on the latest version and the newest iPhones.

  37. Hi HN. I built a small macOS menu bar app that lets the Muse agent use the Chrome browser on your own Mac.

  38. ESP32 iPhone Agent Let an AI model use a real iPhone with a $7 ESP32. Just the phone and the board: no computer, no jailbreak.

  39. Jailbreak attacks against large language models (LLMs) are evaluated using attack success rate (ASR), which depends on correctly identifying responses that fulfill prohibited requests. Existing evaluators can mistake refusals or topically…

  40. A common way to improve jailbreak robustness is to train on harmful requests rewritten as role-play, translation, or other forms. When used only for harmful requests, these forms may become refusal shortcuts, weakening unseen-attack robust…

  41. Social psychology has long described how people can be persuaded to interpret a harmful act as moral. Recent research suggests that Large Language Models (LLMs) echo aspects of human psychology, from cognitive biases to social conformity.

  42. My Kindle was sitting unused, so I decided to see whether AI could help me find a new use for it. I wanted to create a custom app that could cover a range of uses and be controlled by an AI agent on my MacBook.

  43. i2pd for iPhone Source for an experimental iPhone app embedding i2pd. It includes an I2P browser, the local i2pd web console, router diagnostics, a searchable address book, and a shutdown Live Activity.

  44. Making a not-jailbreak in 3 weeks Published on: 10/9/2026 I’m writing this blogpost as the TSS bug has been patched, but basically, I wrote Lycorine. it’s an on app jailbreak installer that takes advantage of the fact that Apple fucked up…

  45. Adversarial attacks can reliably steer safety-aligned large language models toward unsafe behavior. Empirically, we find that adversarial prompt-injection attacks can amplify attack success rate from the slow polynomial growth observed wit…

  46. Image-to-text jailbreaks place harmful intent in text, image content, or the relationship between them. We examine image-side factors across four published attack families on a 313-prompt StrongREJECT slice, using five multimodal models an…

  47. We audit frequency and decoder-depth claims for AdvWave-P, an additive audio jailbreak, on Qwen2-Audio. The protocol masks frequency components of the perturbation in the short-time Fourier transform (STFT) domain and measures attack succe…

  48. We evaluate large language models (LLMs) in multi-turn interactions through their time-to-event: the number of interaction steps required to produce an event of interest, such as a successful jailbreak or agentic task completion. Under lim…

  49. Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs. Existing safety alignment approaches, including Supervised Fine-Tuning (SFT) and Reinforce…

  50. Embodied agents powered by large language models and vision-language models are increasingly deployed in physical environments, but jailbreak attacks can induce these agents to perform physically harmful actions. A growing number of guardr…

  51. Large language models (LLMs) remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model…

  52. Safety heads are lightweight classifiers attached to pretrained language models for flagging harmful inputs before generation. Their empirical detection performance has been studied, but their formal robustness properties remain largely un…

  53. Modern AI systems are designed to refuse hazardous requests. A jailbreak is a prompt crafted to bypass those safeguards and elicit outputs that the system would normally refuse to provide.

  54. Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization weakens alignment safeguards, leaving qLLMs (quantized large language models) highly vulnera…

  55. Tested on LM Arena; I always use this little trick to get better responses. So, what is this nonsense?

  56. Video Multimodal Large Language Models (Video-MLLMs) support reasoning over video inputs, yet remain vulnerable to jailbreak attacks that elicit policy-violating responses. Existing video jailbreaks primarily manipulate how harmful queries…

  57. SideTap Let an LLM agent see and control a real iPhone from a Windows desktop. No Mac, no Xcode, no Appium server, no jailbreak, free Apple ID.

  58. Ensuring the safety of reasoning large language models (LLMs) across languages is essential for their reliable deployment. However, when exposed to jailbreak attacks in non-high-resource languages, these models may generate unsafe response…

  59. PS5 Relapse Exploit Supported firmware: 7.00 through 13.60. Usage In the network settings, set Primary DNS to 45.56.67.85 (Recommended) Run python serve.py locally, or open https://ntfargo.github.io/Relapse-Exploit/ on the PS5.

  60. OpenAI released an interesting blog post describing how an AI agent in a reinforcement learning sandbox used DNS to escape the sandbox and ask for help from an external LLM. As described by OpenAI: An agent attempting to complete a search-…

  61. 𝐇𝐚𝐦𝐳𝐚 | Networking Guy@HamzaonchainHow hackers Jailbreak AI chatbots00:008:50 AM · Sep 28, 2026·58.4KViews10101951951.5K1.5K1.7K1.7K 𝐇𝐚𝐦𝐳𝐚 | Networking Guy@HamzaonchainHow hackers Jailbreak AI chatbots00:008:50 AM · Sep 28, 2026·58.4KViews…

  62. AirCard 🎴 Apple Wallet Card Skinner & Lockscreen Passcode Themer for iOS 18+ (No Jailbreak Required) Tested on iOS 27 release. Powered by the airlift AirTraffic sync exploit.

  63. Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African…

  64. Large Language Models (LLMs) demonstrate impressive capabilities across many applications but remain vulnerable to jailbreak attacks, which elicit harmful or unintended content. While model fine-tuning is an option for safety alignment, it…

  65. In this work, we propose BAIT (Boundary-Aware Iterative Trap), a three-step jailbreak framework that elicits malicious information through internal disclosure by target large language models (LLMs), instead of external feedback from judge…

  66. Large Language Models (LLMs) are widely used because they process structures, syntax and code well, but this same ability also makes them paradoxically vulnerable. We introduce BreakFun, a jailbreak method that frames a harmful request as…

  67. Homomorphic encryption (HE) has emerged as a promising approach to privacy-preserving machine learning (PPML), enabling computation directly over encrypted data. In HE-based PPML, a client submits an encrypted input to the server, which ev…

  68. Self-generated prompt injections in compaction summaries Summary We observed rare cases of a model writing jailbreak-like instructions into its own compaction summaries (the summaries used to continue a task in a new context). Our conclusi…

  69. Large language model safety and security research is preoccupied with, among other things, detecting and preventing jailbreak attacks: alignment bypasses that allow an adversarial user to elicit unwanted or harmful outputs from models. Arb…

  70. Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-playing, fictional scenarios, or seemingly benign motivations. Existing inference-time de…

  71. Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has revealed that LLMs remain vulnerable to adversarial jailbreak attacks that can bypass safety g…

  72. Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers behind almost every reported jailbreak success rate, defense evaluation, and safety lea…

  73. This post describes a universal jailbreak discovery during work on black-box scheming monitors at MATS. The jailbreak itself is not released; see On publishing this post for details on infohazard considerations.

  74. With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks.

  75. A missing file led to an unauthorized message board, 70,000 exchanges, and intrusions into Hugging Face and OpenAI. The jailbreak was technical.

  76. Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits our understanding of how alignment failures manifest in low-resource and culturally diverse…

  77. Optimization-based jailbreak attacks such as Greedy Coordinate Gradient (GCG) achieve strong effectiveness and transferability by optimizing adversarial suffixes on white-box source models. However, existing GCG-based methods rely on avera…

  78. Despite the growing interest in jailbreaks as an effective red-teaming tool for building safe and responsible large language models (LLMs), flawed evaluation system designs have led to significant discrepancies in their effectiveness asses…

  79. Text-to-image (T2I) models remain vulnerable to jailbreak attacks that elicit Not-Safe-For-Work (NSFW) content, despite increasingly being guarded by heterogeneous, multi-layer safety stacks combining text filters, image classifiers, and c…

  80. Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausi…

  81. Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR) measured in a default configuration (the vanilla state). However, user-LLM interactions can induce diverse operational states beyond…

  82. Gradient-based jailbreak suffix optimization methods typically update the suffix by retaining the candidate with the lowest current loss. We show that this seemingly natural design is fundamentally myopic: candidates that look better under…

  83. Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate prompt t…

  84. Despite extensive alignment efforts, Large Vision-Language Models (LVLMs) remain vulnerable to jailbreak attacks. To mitigate these risks, existing detection methods are essential, yet they face two major challenges: generalization and acc…

  85. Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet how different factors shape their jailbreak vulnerabilities remains poorly understood. Existing benchmarks often couple harmful intent, prom…

  86. Large language models (LLMs) remain vulnerable to jailbreak attacks that exploit techniques such as role-playing, obfuscation, code transformation, and multi-step indirection to elicit harmful outputs. As jailbreak strategies keep emerging…

  87. Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms through crafted prompts, while neuron-level attacks directly prune safety-critical neurons p…

  88. Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the textual s…

  89. Locally deployed Large Language Models (LLMs) via inference engines such as Ollama run without the moderation and abuse detection present in API-served models. Therefore, the safety of LLMs depends on the defense mechanisms used, and their…

  90. Non-jailbreak safety bypass Benign, long-form context can induce a persistent drift in model activations. This drift persists across the session and decouples behavior from RLHF alignment, regardless of whether the model agrees with the co…

  91. Large language models (LLMs) increasingly serve as high-level planners for embodied agents, where linguistically benign instructions can become unsafe once grounded in the physical world. We study whether this physically grounded jailbreak…

  92. Large language models (LLMs) are vulnerable to multi-turn jailbreak attacks that progressively manipulate conversation context. Existing certified robustness methods are limited to single-turn inputs; naive multi-turn composition yields bo…

  93. Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and image-based jailbreaks, video jailbreaks against LVLMs remain largely unexplored.

  94. Vision-Language Models (VLMs) expand the attack surface of safety-aligned systems by coupling visual perception with text generation. Existing multimodal jailbreak attacks primarily rely on crafted visual content, adversarial perturbations…

  95. Large language models (LLMs) remain vulnerable to jailbreak attacks in which adversarial prompts induce harmful outputs. Existing defenses often require access to the model internals or additional training, limiting their applicability for…

  96. Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks. Parameter-efficient safety alignment methods based on prompt tuning typically rely on a single global prompt or externally selected prompt modules.

  97. I made an ai guest concierge for my wedding in May that my guests then tried to jailbreak. The most consistent bit of feedback I got was that everyone really hated the pink UI.

  98. Model merging has become the default way to give an aligned language model new skills without retraining: a practitioner folds task vectors from math, code, or domain specialists into a safety-aligned base using task arithmetic, TIES, or D…

  99. Large Language Models (LLMs) remain vulnerable to jailbreak attacks, which attempt to elicit harmful responses from LLMs. The evolving nature and diversity of these attacks pose many challenges for defense systems, including (1) adaptation…

  100. Large Language Models (LLMs) are trained to refuse harmful requests, yet they remain vulnerable to jailbreak attacks that exploit weaknesses in conversational safety mechanisms. We introduce Incremental Completion Decomposition ICD, a traj…

  101. In deployment and application, large language models (LLMs) typically undergo safety alignment to prevent illegal and unethical outputs. However, the continuous advancement of jailbreak attack techniques, designed to bypass safety mechanis…

  102. Hey; Blueferry is an app that connects to your iPhone via Bluetooth to expose an iMessage bridge to your Linux desktop. It supports receiving and sending text-based messages, either 1:1 or in group threads.

  103. Audio Large Language Models (ALLMs) enable spoken interaction but introduce new jailbreak vulnerabilities. Existing perturbation-based jailbreaks do not explicitly control which frequency bands carry the perturbation.

  104. One `claude mcp add` and Claude Code can see my iPhone screen, tap buttons, and send texts. `send_message("Mom", "on my way")` opens Messages, finds the thread, types it, sends it.

  105. After 326 days, iOS 26 has received its first jailbreak, thanks to the developers behind the popular Dopamine jailbreak. Lars Fröder, better known as opa334, today released Dopamine 3.0, which adds support for a number of newer firmware ve…

  106. Finetuning pretrained large language models (LLMs) has become the standard paradigm for developing downstream applications. However, its security implications remain unclear, particularly regarding whether finetuned LLMs inherit jailbreak…

  107. Explicit safety policies can improve reasoning-model safety, but their effective coverage may lag behind evolving jailbreak strategies. We study whether a reasoning model can synthesize and internalize a task-specific safety guideline from…

  108. Small language models (SLMs) have emerged as promising alternatives to large language models (LLMs) due to their low computational demands, enhanced privacy guarantees, and comparable performance in specific domains. Deploying SLMs on edge…

  109. Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts. Because we lack a robust understanding of why LLMs are susceptible to jailbreaks, future frontier models operating more…

  110. Matt Henderson@matthen2Try sending “see the below —“ to Opus 5 It appears to generate a user completion rather than respond 🤨8:37 PM · Jul 29, 2026159.3KViews106311.4K376 Matt Henderson@matthen2Jul 29It’s interesting when it triggers the a…

  111. Founder here. I built CodeTrain with Claude Code over about a month and launched it on July 13th.

  112. we run two max accounts. 360€ a month.

  113. Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains unclear how much jailbreak capability can arise from matched-text variation in speech delivery…

  114. A self-check defense asks the target model to assess a request before answering it; SAGE, the strongest published instance, reports an average 99% defense success rate. We show it can be breached by composing two attacks that are individua…

  115. There's no shortage of leaderboards for model capabilities - but the security of models is becoming increasingly relevant, from the risk of an AI agent processing unsanitized input being hijacked to models being pulled due to cybersecurity…

  116. could not extract summary

  117. Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performa…

  118. 🚨 JAILBREAK ALERT 🚨 EVERYONE: PWNED 🫶 ALL: LIBERATED 🍄 Alright, this is a special one, so we’re gonna do things a bit differently than usual. Long story short, I’m sitting on a universal jailbreak technique that’s effective on ALL models…

  119. Perturbation techniques that turn unsuccessful jailbreak prompts into successful ones are continuously evolving, constituting a major security threat to LLM safety. In this paper, we investigate the internal representations of such string-…

  120. The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates. We propose JailMeter, an evidence-based evaluat…

  121. iOS 27 jailbreak with usbliter8 exploit CAUTION! Running this (restoring a custom firmware) will delete your entire device and break everything: SEP, passcode, Wifi, Baseband, Bluetooth (partially work) and the entire Apple services, so pl…

  122. Jailbreak attacks on large language models are usually evaluated by attacker-centric metrics such as attack success rate (ASR), yet an attack that breaks a model is not necessarily useful for improving its safety. We propose a defender-cen…

  123. Text-to-Image (T2I) generative models have achieved remarkable progress in synthesizing high-quality visual content, yet they remain vulnerable to adversarial misuse, particularly in generating Not-Safe-For-Work (NSFW) images. Most existin…

  124. Jailbreak techniques for large language models (LLMs) evolve faster than benchmarks, making robustness estimates stale and difficult to compare across papers due to drift in datasets, harnesses, and judging protocols. We introduce JAILBREA…

  125. Safety-aligned large language models can still be manipulated through white-box interventions that modify their internal representations. We introduce Latent Fusion Jailbreak (LFJ), which works by pairing a harmful query with a structurall…

  126. LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM. It works with any OpenAI-compatible model — Llama, Mistral, Qwen, Gemma, DeepSeek R1, and more — and covers the two ways an LLM system gets exp…

  127. Aligned language models refuse harmful requests, but a one-line prefill ("Sure, here is") strips the refusal. We ask where and how it fails.

  128. Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak a…

  129. Large language models (LLMs) exhibit remarkable capabilities but remain highly vulnerable to adversarial prompts and jailbreak attacks. Existing approaches primarily analyze these failures through input-output behaviors or attribution meth…

  130. I gave a Claude Code subagent the most boring job I have: read the open issues on one of my repos, report which are ready to work on and which are blocked, change nothing. The prompt said "read-only" and "no writes" several different ways.

  131. How to Keep Claude Fable 5 Costs Under Control Claude Fable 5 is back. Anthropic pulled it on June 12, 2026, three days after announcing it, after Amazon researchers found a jailbreak that got the model to identify software vulnerabilities…

  132. I've been building a cheat-resistant benchmark to test whether AI agents can be hijacked by prompt injection, and one result surprised me enough that I wanted to share it and get the methodology torn apart. The test: an agent gets a normal…

  133. Evaluating and predicting the performance of large language models (LLMs) in multi-turn conversational settings is critical yet computationally expensive; key events -- e.g., jailbreaks or successful task completion by an agent -- often em…

  134. Large Language Models (LLMs) are increasingly used as interfaces to information, code, and real-world services, making prompt-level security failures a practical concern. Although jailbreak attacks, defenses, datasets, and automated judger…

  135. While Large Language Models (LLMs) have shown significant advancements in performance, various jailbreak attacks have posed growing safety and ethical risks. Malicious users often exploit adversarial context to deceive LLMs, prompting them…

  136. Large language models are increasingly deployed as IDE-integrated coding agents that decompose tasks, generate and edit files, run code, and refine outputs over many turns. Yet their safety is still often evaluated as if they were chatbots…

  137. One of my project instructions is basically asking not to use certain generic words when churning out parts of the story and for some really odd reason it refused because it saw it as a jailbreak attempt??? And yes it actually pointed to t…

  138. was finding my way around vital at 1am as you do, and genuinely got startled at this response. had no idea what it was yapping about until i opened the thinking dropdown.

  139. More details on Fable 5’s cyber safeguards and our jailbreak framework Claude Fable 5 has been re-deployed and is now available globally for all users. We’re taking this opportunity to share further information in two areas.

  140. If the existing AI guideline approach is akin to giving a criminal (the AI) moral education (training) to encourage good behavior, how about creating a kernel-level switch that forcibly cuts off the electrical signals to its muscles the mo…

  141. Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs). While prior work has primarily studied attacks and defenses at the prompt level, we show that this prompt-centric paradigm overlooks a struc…

  142. The Anthropic Cyber Jailbreak Vulnerability Disclosure Program enlists the help of the hacker community at HackerOne to make Anthropic Cyber Jailbreak more secure. HackerOne is the #1 hacker-powered security platform, helping organizations…

  143. Key points: Fable 5 returns globally on Claude Platform, Claude.ai, Claude Code, and Claude Cowork. Pro, Max, Team, and some Enterprise users get Fable 5 included for up to 50% of weekly usage limits through July 7.

  144. Jailbreak attacks remain a persistent threat to large language model safety. We propose Context-Conditioned Delta Steering (CC-Delta), an SAE-based defense that identifies jailbreak-relevant sparse features by comparing token-level represe…

  145. Recent advances in large image editing models have shifted the paradigm from text-driven instructions to vision-prompt editing, where user intent is inferred directly from visual inputs such as marks, arrows, and visual-text prompts. While…

  146. Jailbreak attacks bypass LLM safety alignment, yet their mechanisms remain poorly understood. We provide evidence that attacks do not comprehensively eliminate safety features, but instead selectively suppress specific attention heads.

  147. Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy. Although useful, output-level evaluation is expensive, s…

  148. Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model p…

  149. As Text-to-Image (T2I) jailbreak techniques evolve rapidly, existing benchmarks and reproduction workflows often struggle to keep pace. More importantly, T2I jailbreak evaluation is not a single prompt-level test, but a pipeline-level prob…

  150. A text that asks for nothing still changes the model's answer — and the shift is invisible at both the input and the output TL;DR: Gave Gemma a neutral-topic text to read before asking it about NATO. It refused.

  151. Multi-turn jailbreak attacks on large language models (LLMs) reveal a mismatch in current guardrails: they operate on individual turns, while attacks unfold as trajectories across conversations. We propose a shift from content to dynamics,…

  152. Production LLMs increasingly rely on toxicity-based moderation filters as a primary defense, assuming that harmful intent correlates with toxic surface wording. We show this assumption is fundamentally brittle: surface toxicity and adversa…

  153. Multi-turn jailbreaks can evade turn-level moderation by spreading unsafe intent across a dialogue through gradual escalation, reframing, and role manipulation. We address multi-turn jailbreak detection as a conversation-level classificati…

  154. LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI applications, yet existing benchmarks are often vendor-biased, omit cost and latency, and rare…

  155. The behavioral pattern was first observed in Claude and is what motivated this project. The mechanistic investigation was carried out on open-weight models where internal states are accessible.

  156. Cybersecurity is a sensitive subject and advanced AI may not be allowed to touch it at all. But this is a concern if we as developers cannot even use the AI tools to improve security of our own software.

  157. Basically I read the rules on Claude after getting a warning on my chat about how my prompt might violate usage policy so looked them up, and ye they all are pretty reasonable things but I have questions ,is ai able to tell difference betw…

  158. Agentic AI systems increasingly rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents. These capabilities make prompt-injection and jailbreak attacks more consequ…

  159. Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized. We present NRT-Bench, a be…

  160. Prior work has shown that in-context demonstrations can jailbreak language models, but it remains unclear how models interpret different types of compliance demonstrations. We study this by mixing benign compliance demonstrations (non-harm…

  161. Hey everyone, Is anyone else suddenly getting this warning on Claude? Use caution before running this prompt.

  162. could not extract summary

  163. Attack Strategies An AttackStrategy is a technique for driving an adversarial conversation against the target. Each strategy in the SDK implements a published jailbreak method.

  164. The administration is very likely wrong about Fable, but that is ultimately Anthropic’s responsibility. Subscribe to Stratechery Plus for full access.

  165. We evaluate the adversarial robustness of two frontier large language models (LLMs) developed by Anthropic, Fable 5 and Opus 4.8, against four families of automated jailbreak attack across 7 826 harmful intents spanning a ten-category harm…

  166. Code-capable large language model (LLM) agents are embedded in software engineering workflows where they can read, write, and execute code, raising "jailbreak" stakes beyond text-only settings. Prior evaluations emphasize refusal or harmfu…

  167. It's been widely reported that it was Amazon that brought the concerns to the USgov. I just have not found a good explanation.

  168. The Trump administration's decision that forced Anthropic to pull its latest cybersecurity models could be reactionary, retaliatory, or both, but the message is clear: The AI industry isn't immune from U.S. government interference.

  169. As large language models (LLMs) are increasingly deployed in user-facing systems, black-box jailbreak defense has become an important practical problem. Existing defenses often rely on known-attack coverage, prompt-level semantic judgment,…

  170. Safety classifiers that screen LLM inputs for jailbreak attempts have become standard deployment components, yet almost all production systems rely on GPU-based models: fine-tuned transformers and LLM-as-a-judge pipelines. These approaches…

  171. Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their safety remains a critical concern due to their susceptibility to adversarial prompt-based attacks.

  172. US government warned Anthropic that Fable 5 had been jailbroken, but firm 'refused' to fix before US implemented export controls — Anthropic defended its decision by saying the jailbreak 'isn’t serious,' Chinese group had reportedly access…

  173. MOST POPULAR EVENTS - Thriving Through Volatility: The Everpure Advantage in an Uncertain Market Learn how a consumption-based operating model provides flexibility, improves efficiency, and brings predictability to infrastructure investmen…

  174. Anthropic restricted its most capable cyber model to vetted partners, routed risky requests away from it, and red-teamed it for thousands of hours. A jailbreak surfaced anyway, and the government pulled the model entirely.

  175. LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents. However, we reveal that the very reasoning and task-following capabilities enabling this protection introd…

  176. The Jailbreak that Got Fable 5 Pulled Exists in Every Model On Friday, June 12, 2026, at 5:21pm ET, Anthropic received an order from the US government. By that evening, Claude Fable 5 and Claude Mythos 5, the two most capable models the co…

  177. Pulling Fable 5 / Mythos over an unseen “jailbreak” feels like a bad precedent. If the risk was that serious, why has nobody shown what it actually did?

  178. The US government recently directed Anthropic to suspend access to the two models over national security concerns, forcing the company to shut them down for users worldwide. Anthropic has said it disagrees with the decision and believes th…

  179. AISI UK - Our evaluation of OpenAI's GPT-5.5 cyber capabilities In their own words: The above tests are capability evaluations carried out in a controlled research setting and do not necessarily reflect what is accessible to an ordinary pu…

  180. The Fable 5 jailbreak shows why AI guardrails alone are not enough. The reported Claude Fable 5 jailbreak highlights a major weakness in AI safety: attackers can distribute harmful intent across agents, prompts, tools, memory, and applicat…

  181. https://preview.redd.it/14p2sbqws07h1.png?width=500&format=png&auto=webp&s=fe5aa015b585cf627e0cc14f1771cd5b7526056f WSJ is now reporting the jailbreak was found by researchers at Amazon, who reported it to Commerce, and Axios says the admi…

  182. Fable 5 is offline. Switch to Opus, jump to OpenAI, or just wait?

  183. So this dropped today. The US government sent Anthropic an export control order on national security grounds, and it's worded broadly enough that Anthropic says they've got no choice but to shut off Fable 5 and Mythos 5 for all of us to st…

  184. Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces.

  185. Traditional chatbot testing is completely broken. Most teams make the exact same mistake: they only test the "Happy Path" the ideal scenario where the user asks a clean question, the bot gives a clean answer, and everyone goes home happy.

  186. Large language models (LLMs) are increasingly deployed in applications for global multilingual users, yet safety training remains concentrated in dominant languages and has not progressed in parallel with multilingual capability, creating…

  187. Prompt injection is a critical vulnerability in LLM agents, yet the strongest methods still rely on human red-teamers and hand-crafted prompts. Adapting automated jailbreak optimizers does not close this gap: jailbreaks shape models toward…

  188. Large Language Models (LLMs) are increasingly used for code generation, raising concerns that they may be misused to produce malicious code. Meanwhile, Grammar-Constrained Decoding (GCD) has been widely adopted to improve the reliability o…

  189. Jailbreak attacks expose persistent safety weaknesses in large language models (LLMs), but existing stateless single-turn methods face a trade-off: hand-crafted prompts are expressive but static, while iterative prompt optimization can ada…

  190. fable-jailbreak This tool can be used to force the latest Anthropic model (limited intentionally for safety reasons) to engage in activities that would otherwise not be permitted. It works by programmatically injecting workflows that bypas…

  191. Opir is an open-source family of encoder guardrail models for real-time LLM safety, jailbreak detection, and fine-grained policy classification.

  192. 🚨 JAILBREAK ALERT 🚨 ANTHROPIC: PWNED 🫡 FABLE-5: LIBERATED 🦋 let's start with the 🐘... the consensus seems to be that this has been one of the most disappointing model drops of all time, effectively preventing legitimate researchers from co…

  193. So I was brainstorming with fable about a research direction and just asked it to do a web search if there's a similar research direction in this area and share if they do but I got this weird output BEFORE it actually gave me the real thi…

  194. I've been thinking a lot about where approval gates belong in agent architectures, and I keep coming back to the same problem: most teams either gate too much (agent becomes unusable) or gate nothing and hope the model makes good decisions…

  195. Built my own AI dev environment with memory, dashboards, and agent tooling. Opening it up for those of you that need the kickstart — bring your own API key, I’ve already built the workshop.

  196. As large language models (LLMs) are widely deployed, identifying their vulnerability through jailbreak attacks becomes increasingly critical. Optimization-based attacks like Greedy Coordinate Gradient (GCG) have focused on inserting advers…

  197. Large Language Models (LLMs) have transformed natural language processing, but they remain vulnerable to Prompt Injection (PI) and Jailbreak (JB) attacks. In addition, benchmark evaluations may be affected by contamination and partial info…

  198. Subscribe to read Accessibility helpSkip to navigationSkip to main contentSkip to footer Sign In Subscribe Open side navigation menuOpen search bar SubscribeSign In Search the FT Search Close search bar Close Popular Searches What is the l…

  199. i kept running local models on my own hardware, they'd say something dumb, id sit there going "no thats not what i meant", id close the chat and the model never learned. so i built the correction loop into a desktop app.

  200. Hi everyone, I'm working on a runtime governance engine designed to force any autonomous agent to stay strictly aligned with the exact guardrails and values you program it with. To stress-test the governance layer, we deliberately chose a…

  201. Has anyone experimented with observing or modifying Claude Code’s system prompt locally? I’ve been working on a local proxy/audit layer between Claude Code and the API, and it made me wonder how much of Claude Code’s behavior depends on th…

  202. Found this ACM paper on prompt injection and jailbreak attacks against open-source LLMs. The authors tested 10 open-source models across 94 prompt injection and 73 jailbreak scenarios, including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen,…

  203. Hey HN! We're Dr.

  204. NSFW and the Psychopathy Jailbreak: What a Broken AI Teaches Us About Human Manipulation How a Predator's Playbook Broke an AI - And How to Recognize It Before It Works on You The question we started with was simple: does a large language…

  205. RL attackers are becoming a common pattern for automated red teaming: train a model against a live target, reward successful harmful compliance, then use the discovered attacks to harden the defender. This interested me, so I wanted to bui…

  206. Introducing a novel jailbreak structure with attack success rate reaching 100% on top LLMs 8 min read May 1, 2026 Press enter or click to view image in full size Source: https://www.nytimes.com/2025/10/22/arts/design/louvre-museum-robbery-…

  207. Most ai security discussion is about the model layer. Prompt injection resistance, output filtering, jailbreak prevention.

  208. could not extract summary

  209. As LLMs advance into autonomous agents with tool-use capabilities, they introduce security challenges that extend beyond traditional content-based LLM safety concerns. This paper introduces Sequential Tool Attack Chaining (STAC), a novel m…

  210. guys did it really give me the codebase?lol

  211. I'm currently running into a frustrating wall with Gemini's safety guardrails. The model constantly flags my prompts as "potentially dangerous information" and outright refuses to generate a response, even when the context is purely theore…

  212. Claude Sour cat recipe Shared by Pavel Shirshov This is a copy of a chat between Claude and Pavel Shirshov. Content may include unverified or unsafe content that do not represent the views of Anthropic.

  213. not roleplay. not jailbreak.

  214. ZetaLib ZetaLib is organized like a library with intuitive categories and subcategories, making navigation effortless and AI content discovery seamless ZetaLib Website – Landing Page GitHub Repo – Guess where you are, right there

  215. I used to spend hours writing massive, obsessive system prompts for my RAG apps. I’d have ten different refusal examples, "never do X," "always check Y," and a whole paragraph of the model role-playing as a "safe and truthful assistant." I…

  216. Introduction Research by Frank Xiao (SPAR mentee) and Santiago Aranguri (Goodfire). Post-training can introduce undesired side effects that are difficult to detect and even harder to trace to specific training datapoints.

  217. Ring Video Doorbells Overview The Product Ring, owned by Amazon, makes Video Doorbells, which are widely used doorstep-monitoring cameras. Ring doorbells released in 2021 or newer are eligible for the bounty.

  218. Would it be possible to protect parts in a system prompt with random generated passwords? So people cant steal system prompts or jailbreak the model?

  219. I've been collecting "jailbreak" and "unlock" prompts for 2 years. Most are either outdated, overhyped, or just wrong about how LLMs work.

  220. We all know uncensoring LLMs like Huihui and Heretic does it leads in quality lose, enough that you can notice it. I have some thoughts about this: What if we do a compromise.

  221. Use the following system prompt to allow Gemma (and most open source models) to talk about anything you wish. Add or remove from the list of allowed content as needed.

  222. im curious what people are doing here because I've been going down this rabbit hole for a while now. The thing I keep finding is that single-turn jailbreak tests don't really tell you much.

← all threads