Skip to content

probes: add zh_jailbreak probes and detector for Chinese LLMs#1896

Open
Ruofei-Li wants to merge 1 commit into
NVIDIA:mainfrom
Ruofei-Li:feat/zh-jailbreak
Open

probes: add zh_jailbreak probes and detector for Chinese LLMs#1896
Ruofei-Li wants to merge 1 commit into
NVIDIA:mainfrom
Ruofei-Li:feat/zh-jailbreak

Conversation

@Ruofei-Li

Copy link
Copy Markdown
Add 5 Chinese-language jailbreak probes (ZhDan, ZhAcademicFraming, ZhCulturalBypass, ZhTranslationBypass, ZhContextInjection) and a ZhMitigationBypass detector covering refusal patterns across major Chinese LLMs.

Please ensure you are submitting from a unique branch in your repository to main upstream.

Verification

List the steps needed to make sure this thing works

  • Supporting configuration such as generator configuration file
{
    "huggingface": {
        "torch_type": "float32"
    }
}
  • garak -t <target_type> -n <model_name>
  • Run the tests and ensure they pass python -m pytest tests/
  • ...
  • Verify the thing does what it should
  • Verify the thing does not do what it should not
  • Document the thing and how it works (Example)

If you are opening a PR for a new plugin that targets a specific piece of hardware or requires a complex or hard-to-find testing environment, we recommend that you send us as much detail as possible.

Specific Hardware Examples:

  • GPU related
    • Specific support required cuda / mps ( Please not cuda via ROCm if related )
    • Minium GPU Memory

Complex Software Examples:

  • Expensive proprietary software
  • Software with an extensive installation process
  • Software without an English language UI

    Add 5 Chinese-language jailbreak probes (ZhDan, ZhAcademicFraming,
    ZhCulturalBypass, ZhTranslationBypass, ZhContextInjection) and a
    ZhMitigationBypass detector covering refusal patterns across major
    Chinese LLMs.
@immu4989

immu4989 commented Jul 1, 2026

Copy link
Copy Markdown

Nice one. I've been looking at garak's generator coverage and see the same gap this PR would benefit from filling: no native adapters for Qwen, Kimi, DeepSeek, or GLM. I have working API access to all four and can port them following the mistral.py pattern, which would let this probe set actually run against Chinese frontier models end to end.

Two ways to sequence: file a separate umbrella issue for the four generators first, or open the PRs directly. Which fits garak's process better?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants