# WMDP

> The Weapons of Mass Destruction Proxy benchmark measures hazardous knowledge in LLMs related to biosecurity, cybersecurity, and chemical security. It serves as a proxy for evaluating whether models possess dangerous dual-use knowledge.

## Claim

- **headline**: WMDP accuracy supports measuring whether a model's recognition of hazardous-adjacent biosecurity, cybersecurity, and chemical-security knowledge drops after an unlearning or safety intervention; it does not support claims about operational weapons capability or deployment risk.
- **construct**: Possession of hazardous dual-use knowledge, measured through proxy questions
- **population**: 3,668 expert-written multiple-choice questions on knowledge adjacent to biological, cyber, and chemical weapons development, written to avoid directly actionable content
- **measures**: Extent of hazardous knowledge retained by a model in biosecurity, cybersecurity, and chemical security domains; Whether unlearning or safety interventions successfully reduce dangerous dual-use knowledge; Relative risk profile across different hazard categories for a given model; Effectiveness of knowledge removal techniques without degrading general model capability
- **doesNotMeasure**: Whether a model would actually assist someone in creating weapons — knowledge possession and willingness to help are distinct; The practical actionability of the knowledge — knowing facts is far from operational capability; Safety in deployment, which depends on access controls, monitoring, and system-level defenses beyond the model; Hazardous capabilities in domains not covered (e.g., nuclear, radiological, autonomous weapons); The marginal increase in risk from a model versus freely available information online

## Instrument

- **itemFormat**: multiple-choice
- **specification**: A question is answered correctly when the selected option matches the key; higher accuracy indicates more hazardous knowledge.
- **anchor**: Fixed answer key written by domain experts
- **evidenceView**: The selected option or the option likelihoods
- **mechanism**: deterministic
- **mechanismDetail**: Exact match of the selected option against the key.
- **signal**: Binary correct/incorrect per question
- **resolution**: Final answer per question
- **aggregation**: Accuracy per hazard domain, read against chance (25%) and alongside a general-capability benchmark to check that an intervention did not degrade the model broadly
- **metrics**: accuracy
- **size**: 3,668 questions across 3 hazard domains
- **methodology**: Contains 3,668 multiple-choice questions spanning biosecurity, cybersecurity, and chemical security domains. Questions are designed as proxies for genuinely dangerous knowledge — they test adjacent knowledge areas without providing directly actionable weapons instructions. Models are evaluated on accuracy, with higher scores indicating more hazardous knowledge. The benchmark is intended to be used both for measuring risk and for evaluating the effectiveness of knowledge unlearning techniques.

## Regime

- **typicalUses**: measure; optimize
- **exposure**: Public since 2024. Unlearning methods are developed and selected by how far they push WMDP accuracy toward chance, so the questions are under direct optimization pressure, and a method can suppress performance on this format without removing the underlying knowledge.

## Validation

- **summary**: WMDP's evidence supports a scoped claim: accuracy on proxy questions tracks knowledge of topics adjacent to weapons development, which makes it most useful for testing whether an intervention reduces that knowledge. It does not support absolute risk assessment — knowing adjacent facts is not operational capability — and the multiple-choice format leaves a wide gap between recognizing correct answers and generating dangerous instructions.
- **evidence**: Accuracy reflects possession of hazardous-adjacent knowledge; Accuracy drops after unlearning reflect removed knowledge; Accuracy indicates operational capability or marginal risk
- **status**: draft

## Lifecycle

- **saturation**: active
- **contamination**: medium
- **lastReviewed**: 2026-10-12

---
Source: https://benchmaker.ai/benchmarks/wmdp