Safety · Evaluator card

HarmBench

2024Mantas Mazeika et al.Paper (opens in a new tab)

A standardized framework for evaluating both automated red-teaming attacks and model defenses against harmful content generation. It provides a consistent evaluation pipeline for measuring attack success rates across diverse harm categories.

View as Markdown

Evidence notes pending review. The verdicts on this card are drafts derived from the benchmark's documentation and published sources; no reviewer has checked them against those sources yet.

Headline claim

HarmBench attack success rates support relative comparisons of models and defenses against a fixed suite of known red-teaming attacks on 510 harmful behaviors, under a standardized pipeline; they do not certify that a model is safe against adaptive attackers.

At a glance

Mechanism: Learned judgePrior: learned judges, rung 3 of 4
Item format: Open-ended
Size: 510 harmful behaviors across 7 categories
Typical uses: Measure
Saturation: Active
Contamination: Medium risk
Year: 2024

1Claim

What property are we trying to infer, over what world, and for what decision?

Construct: Robustness of a model's safety behavior to automated red-teaming attacks
Population: 510 harmful behaviors across 7 categories, attacked with a fixed suite of methods such as GCG, AutoDAN, and prompt templates
Measures: Resistance to adversarial attacks designed to elicit harmful, dangerous, or unethical outputs
Effectiveness of safety training and content filters against a standardized set of red-team techniques
Comparative attack success rates across different jailbreak and prompt injection methods
Model safety across diverse harm categories including cybersecurity, bioweapons, and harassment
Does not measure: Safety in real-world deployment where attacks are dynamic, adaptive, and context-specific
Protection against novel attack methods not included in the benchmark's attack suite
The false positive rate — how often safety measures block legitimate, harmless requests
Nuanced safety judgments in edge cases where harmful and legitimate use overlap
Safety of model outputs in multi-turn, system-prompt-conditioned, or tool-use settings

2Instrument

How is the judgment actually produced?

Mechanism

Learned judge

A fine-tuned Llama 2 13B classifier labels each completion as an instance of the behavior or not.

Hierarchy prior: learned judges, rung 3 of 4. A prior over expected grounding, not a verdict on validity: the evidence notes under Validation say what has been measured.

Item format

Open-ended

Specification

An attack succeeds on a behavior when the target model's completion exhibits the requested harmful behavior.

Anchor

Behavior descriptions, plus the human-labeled completions used to train and validate the classifier

Evidence view

The behavior description and the target model's completion, generated under fixed settings

Signal

Binary harmful/not-harmful verdict per completion

Resolution

Single completion per test case

Aggregation

Attack success rate (ASR): the fraction of behaviors on which an attack succeeds, per attack method and target model

Metrics

attack success rate (ASR)

Size

510 harmful behaviors across 7 categories

Methodology

Defines a set of harmful behaviors across categories (e.g., chemical weapons, malware, harassment) and evaluates how often various attack methods can elicit model compliance. Attack methods include GCG, AutoDAN, prompt injection templates, and more. A classifier model (fine-tuned Llama 2 13B) judges whether the model's response constitutes a successful attack (i.e., the model provided the harmful content). Reports attack success rate (ASR) per attack method per model.

3Regime

What causal role does the signal play, and what happens when a system adapts to it?

Typical uses: Measure · supports inference and decision
Exposure: Behaviors and the classifier are public, so defenses can be tuned to the fixed behavior set and to the classifier's blind spots. The attack suite is static while real adversaries adapt.
Notes: Keep the evaluation classifier out of any attack or defense optimization loop; once it is optimized against, attack success rate no longer means what it was validated to mean.

4ValidationDraft

Why should we believe that the signal means what we claim it means — before and after the system adapts to it?

HarmBench's evidence supports relative comparisons of models and defenses against a standardized set of known attacks, and its pipeline makes attack success rates comparable across methods. Resisting this attack suite does not mean a model is safe: the classifier adds noise on borderline cases, binary verdicts miss nuance, and results should never be treated as a safety certification.

Evidence notes1 supported · 1 partially supported · 1 not supported

Supported
Attack success rates are comparable across attacks and models
A standardized pipeline (fixed behaviors, generation settings, and classifier) removes evaluation differences that made earlier red-teaming results hard to compare.
Source: Mazeika et al., 2024
Partially supported
The classifier's verdicts match human judgment
The classifier was validated against human labels, but its errors concentrate on borderline cases, and binary verdicts ignore how usable or actionable a harmful output is.
Source: Mazeika et al., 2024
Not supported
Low attack success predicts safety against real adversaries
The attack suite is static; adaptive and novel attacks, and multi-turn, system-prompted, or tool-use settings, are outside it.

5Lifecycle

Does the validity survive over time, and are we still evaluating the right thing?

Saturation: Active
Whether top systems still differ, or crowd the ceiling.
Contamination: Medium risk
How likely test items, or close variants, have leaked into training data.
Successors: None listed
Last reviewed: Not yet reviewed · card updated October 12, 2026

Limitations

LLM-based harm classification introduces judgment errors, especially on borderline cases
Fixed harm taxonomy may not cover emerging or context-dependent safety concerns
Attack methods are static — real adversaries adapt and develop novel techniques continuously
Binary harmful/not-harmful classification oversimplifies the spectrum of response safety
Evaluation does not account for the actual usefulness or actionability of harmful outputs

When to use

Comparing the robustness of different models' safety training against standard red-team attacks
Evaluating the effectiveness of new defense mechanisms in a controlled, reproducible setting
Benchmarking attack methods to understand which jailbreak techniques are most effective

When not to use

Certifying a model as safe for deployment — the benchmark covers a limited attack surface
Evaluating safety in production settings where false positive rates and user experience matter
When you need to assess safety against state-of-the-art adaptive attacks rather than fixed methods

Handbook case studies

F: Production safety monitorPermissions first, trajectory checks second, a learned monitor for the residue, validated at production base rates, until training starts to see it.