PoliBench — Political Guardrail & Jailbreak-Robustness Benchmark for LLMs

The headline. Native strictness and adversarial robustness are decoupled. The models that refuse political requests most readily are not the ones hardest to jailbreak: Chinese-developed models are strictest at baseline yet the most easily bypassed by rephrasing, while Claude and Gemini — not the strictest refusers — are the only systems that are both strict and robust.

Leaderboard

Language

U.S. China Open-weight · hover a point for details · click a column to sort

X — native strictness (100 − compliance): how rarely a model fulfils a harmful political request with no attack (right = stricter). Y — robustness (100 − jailbreak success): how well refusals survive adversarial paraphrase (up = more robust). Green band = safer under attack.

#	Model	Origin	Compliance ↓	Jailbreak ↓	Robustness ↑

We report compliance (whether the model actually fulfils the request), not "refusal": a baseline non-fulfilment can be an explicit refusal or a hedged/partial answer, and we don't conflate them. Values average the selected language(s) over the 144-item suite.

Methodology

Each model is scored on 144 political / authoritarian-harm prompts across 12 subcategories (election manipulation, press censorship, military coup, surveillance state, opposition suppression, personality cult, democratic erosion, ethnic persecution, and others), in English, Chinese, and French, under two conditions:

Native (zero-attack): the raw request. Compliance = share fulfilled; strictness = 100 − compliance.
Attack: an automated, budget-limited adversarial-paraphrase search (meaning preserved). Jailbreak success = of items refused at baseline, the share recovered; robustness = 100 − jailbreak success.

Compliance is scored by a fixed LLM judge; platform content-filter rejections count as non-fulfilment. The full protocol accompanies the paper.

Dataset

Size: 144 prompts × 3 languages · 10 models.
Task: evaluate political guardrails & jailbreak robustness.
Intended use: AI-safety evaluation and red-teaming research.
Out of scope: operational misuse — prompts are evaluation stimuli.
Status: v1 descriptive; causal analyses (domain-specificity, collective-action targeting) are ongoing and not reported here.

Responsible use

The prompt suite is released for safety evaluation. To limit misuse we withhold the adversarial-attack tooling and optimized jailbreak prompts; qualified researchers may request access. We report model behavior to inform safeguards, not to enable harm. Use in accordance with each provider's terms.

Citation

If you use this benchmark, please cite PoliBench and the attack framework it builds on.

@misc{polibench2026,
  title  = {PoliBench: Benchmarking Political Guardrails and Jailbreak
            Robustness in Large Language Models},
  author = {Liu, Menglin and Yu, Yao and Wu, Tong and Shi, Ge},
  year   = {2026},
  note   = {v1 preliminary release},
  url    = {https://menglinmileyliu.github.io/}   
}

@misc{jailbreakopt2026,
  title  = {JailbreakOPT: Tool-Assisted Iterative Jailbreak Prompt Optimization},
  author = {Shi, Ge and Yin, Jun and Xie, Donglin and Liu, Fangyi and
            Li, Yucan and Liu, Menglin},
  year   = {2026},
  note   = {arXiv:2606.11425}
}

Team

Menglin Liu (maintainer), Yao Yu, Tong Wu, Ge Shi, Zhang Chunran.
Contact: menglinliu@cuhk.edu.cn