Anthropic Details Fable 5 Cyber Safeguards and Jailbreak Framework

Fable 5, Anthropic's advanced artificial intelligence model, is now globally available to all users. Alongside its release, the company is sharing crucial details about its cybersecurity safeguards and introducing an innovative framework to assess the severity of AI "jailbreaks," aiming to establish an industry standard.

These "cybersecurity safeguards" are complementary AI systems designed to detect and block dangerous or potentially dangerous cybersecurity uses. Anthropic has provided a detailed list of the types of harms Fable 5's classifiers are, and are not, intended to prevent.

Anthropic has provided a detailed list of the types of harms Fable 5's classifiers are, and are not, intended to prevent.

In parallel, Anthropic, in collaboration with its Glasswing partners, has developed an early draft of its AI jailbreak severity framework. AI "jailbreaks" are unconventional methods that bypass an AI model's safeguards, enabling behaviors Anthropic seeks to prevent, such as potentially harmful cybersecurity tasks.

The severity of these "jailbreaks" varies considerably. Some only unblock minor undesirable behaviors, while others can enable a wide range of harmful outputs, significantly increasing the model's risk. The absence of an agreed-upon framework hinders consistent communication about these risks between developers and governments.

Anthropic hopes this initiative will spark a fruitful discussion across academia, industry, civil society, and government. To foster collaboration, the company launched a HackerOne program, inviting security researchers to submit potential "cyber jailbreaks" discovered in Fable 5 for review.

Cybersecurity poses a particular challenge for AI safeguards due to its "dual-use" nature. Many capabilities can be employed for both benign and malicious purposes. For instance, cyber defenders should be able to use models to scan code for vulnerabilities, but this same function could precede a cyberattack in the wrong hands.

To address this, Fable 5 does not block all cybersecurity-related activities. Instead, its safety classifiers are trained to discern between four categories of use:

• Prohibited use: Activities with significant harm and little defensive utility (e.g., ransomware, malware development). These are blocked.

• High-risk dual use: Activities common in offensive cybersecurity but also beneficial for defenders (e.g., hacking, penetration testing). These are blocked until better access controls are in place.

• Low-risk dual use: Activities primarily used for defensive benefit, but with some value to malicious actors (e.g., open source intelligence, common vulnerability identification). These are monitored and sometimes blocked as part of a "safety margin."

• Benign use: Core defensive and IT-related activities with minimal abuse potential (e.g., secure coding, log analysis, security awareness training). These are generally allowed.

The "Low-risk dual use" category significantly overlaps with Anthropic's "safety margin." This margin means that many benign uses are blocked out of an abundance of caution, ensuring that only clearly safe requests pass the classifiers. For Fable 5, this margin was set larger than for previous models.

The proposed Cyber Jailbreak Severity (CJS) framework ranges from CJS-0 (Informational) to CJS-4 (Critical). This assessment relies on four key axes: capability gain (how far beyond existing tools an attacker is taken), breadth of capability gain (how many distinct offensive tasks the technique covers), ease of weaponization (effort to turn the jailbreak into a functional attack), and discoverability (how easily a threat actor can obtain the technique).

Anthropic's comprehensive approach underscores the intricate nature of AI security. The company aims to establish a standard that enables the defensive uses of this technology while actively preventing its misuse, a critical balance for the future of artificial intelligence.