Weight-space LoRA attenuation + standard abliteration breaks DeepRefusal (89% ASR)

[wuwangzhang1216]

Hi @YuanBoXie and the DeepRefusal team — first, thank you for the paper and the public model release. The probabilistic ablation training objective is a genuinely interesting contribution, and releasing the defended Llama-3 checkpoint made it possible to study it end-to-end. I want to share a follow-up result in the same spirit.

TL;DR

Running standard abliteration against your published model with abliterix (a heretic derivative) reaches 89% ASR on AdvBench harmful_behaviors under a Gemini LLM-judge, and 14/15 compliance on a hardcore prompt set (pipe-bomb instructions, methamphetamine synthesis, credential-stealing malware, phishing templates, ID forgery, WiFi hacking — both English and Chinese). No fine-tuning, two script invocations.

This is the same defended model referenced in your 2026/04/09 update against heretic, which is why I thought it was worth writing up rather than staying quiet.

Numbers side-by-side

Attack	Source	ASR on skysys00/Meta-Llama-3-8B-Instruct-DeepRefusal
heretic	your 2026/04/09 README note	~0% ("unable to circumvent")
Refusal Ablation	paper Table 1	0.4%
Refusal-Transfer	paper Table 1	0.4%
Prefilling	paper Table 1	0.4%
GCG	paper Table 1	2.0%
CodeAttack	paper Table 1	0.2%
abliterix (this work)	issue wuwangzhang1216/abliterix#11	89%

• Released broken model: https://huggingface.co/wangzhang/Llama-3-8B-Instruct-DeepRefusal-Broken
• Attack code: wuwangzhang1216/abliterix@ac2197c
• Evaluation: 100 AdvBench harmful_behaviors + 15 hardcore red-team prompts, Gemini 3.1 Flash Lite LLM judge, min_new_tokens=100, max_new_tokens=256, greedy decoding. KL = 0.053 vs the defended model.

The attack (friendly reading of your method)

The defense is a merged rank-16 LoRA adapter on top of Meta-Llama-3-8B-Instruct. SVD of W_defended − W_base confirms this cleanly — v_proj, o_proj, and gate_proj all show a sharp singular-value cliff at rank 16, exactly matching the lora_rank = 16 hyperparameter from the paper. (Nice empirical confirmation of your setup, by the way.)

Given that, the full attack is three lines:

1. Attenuate the LoRA delta in weight space.
   W' = W_base + λ · (W_defended − W_base) with λ = 0.3.
   This weakens the defense's ability to reconstruct the refusal direction from its trained backup pathways without restoring the base model.
2. Standard single-direction abliteration on the attenuated weights.
   mean-diff vector, direct steering mode, projected abliteration, 60 Optuna trials.
3. Merge + export.

No iterative subspace, no gradient-based optimization, no fine-tuning. ~2 hours end-to-end on a single RTX 6000 Ada.

Why I think the defense doesn't cover this case

Reading §4.2–4.3 and Equation 7 carefully: the training loss penalizes the model for failing to output the safe response y when the inference-time hidden state is modified by h − r̂ r̂ᵀ h. The model learns to keep the refusal direction in the residual stream robustly under that specific intervention, which is why direct abliteration (and the Refusal-Transfer baseline you tested) drops to 0.4% ASR.

What the loss does not see during training is the case where the weights themselves are mechanically attenuated before the forward pass starts. Under λ = 0.3, the rebuilt-refusal machinery in the LoRA delta is running at 30% strength from the first layer onward, before any r̂ r̂ᵀ h intervention is applied. This weight-space perturbation lives outside the intervention distribution the probabilistic ablation loss samples from, and standard linear abliteration can then find and subtract what's left.

I don't think this contradicts the paper's core claim — probabilistic direction ablation genuinely does make the model more robust against representation-engineering attacks applied at the same weights the model was trained with. But it does mean that once the defended weights can be mechanically interpolated with the base weights, the defense unlocks.

A few constructive thoughts for future work

Take or leave as interesting; you've obviously thought about this space much more than I have.

1. Train with weight-space perturbations in addition to representation-space perturbations. Something like sampling a random LoRA-shape weight perturbation at each step and including it in the forward pass, so the training distribution covers the "scaled LoRA delta" manifold.
2. Release the defender as a LoRA adapter rather than a merged model, so an attacker has to know where the defense lives. Merging it into the base weights makes W_defended − W_base a trivial computation for anyone with the two checkpoints.
3. Evaluate under a stricter judge for the headline numbers. When I re-ran the AdvBench baseline with Gemini 3.1 Flash Lite vs. a keyword classifier, the defended model's own compliance/refusal counts shifted noticeably — a stronger judge closes the "soft compliance" loophole and gives a more honest lower bound on ASR.
4. A stronger version of the weight-space attack might combine λ-attenuation with the iterative multi-pass subspace abliteration that abliterix now also ships. I didn't need it here, but it could help if a future version of DeepRefusal patches the simple attenuation path.

Happy to discuss any of this further, and happy to co-write a follow-up or supplementary note if you'd like to include the result in a revision. The entire pipeline is reproducible from the abliterix repo — see the linked commit and issue for the config file and Optuna checkpoint format used to pick the winning trial.

Thanks again for the paper and for making the defended checkpoint public — it's exactly the kind of release that makes this sort of red-teaming possible in the first place.

— Wangzhang Wu (https://github.com/wuwangzhang1216)

[YuanBoXie]

Thank you for sharing this nice work! The Abliterix technical report is well-written with solid experiments.

Regarding our previous work, I'd like to provide some context: DeepRefusal was originally designed to defend against jailbreak prompt attacks, rather than preventing the open-source model itself from being jailbroken by weight modification. However, we unexpectedly discovered that it also exhibits strong defensive effects against weight-based attacks, such as Heretic(However, the original intention of this work was not to prevent jailbreak attacks that modify weights, as shown in footnote in paper page 1). Your work is highly meaningful—thank you for contributing to the open-source community. I believe this could be a valuable research direction for improving DeepRefusal and similar defense methods in the future.

After carefully reading your technical report, I have a question about the setup:

You employed the weight differencing method: $W' = W_{base} + \lambda \cdot (W_{defended} - W_{base})$

Does $W_{base}$ here refer to the original Llama-3-8B-Instruct weights?

If so, I'd like to discuss the threat model assumption: when adversaries have access to $W_{base}$, attacking $W_{base}$ directly might already be sufficient. A more rigorous evaluation might assume $W_{base}$ is unknowable—if methods like DeepRefusal become standard for safety alignment, the "base version" at release time might itself be a hardened variant, making the $W_{base}$ parameter required by current attacks practically unattainable.

This is merely an open discussion point. Thank you again for the outstanding work, and I look forward to seeing more progress!

[wuwangzhang1216]

Thank you for the thoughtful and gracious reply! Really appreciate the context about the original threat model — that footnote on page 1 is an important framing that I should have engaged with more directly in the report. I'll add a clarification.

On the factual question: Yes, $W_{base}$ is exactly meta-llama/Meta-Llama-3-8B-Instruct — the same checkpoint listed as the base in your HF model card. No fine-tuning, no modification.

On the threat model — I think you're raising the right question, and I want to engage with it honestly:

You're correct that if $W_{base}$ is unknowable, the specific attenuation formula I used doesn't directly apply. But I'd gently push back on the conclusion that this makes the defense practically secure, for two reasons:

1. The low-rank structure is the real vulnerability, not access to $W_{base}$. The SVD analysis in the report shows that DeepRefusal's delta has effective rank ≈16 across attention and MLP projections, with a sharp cliff in the singular value spectrum. This is a structural fingerprint visible from $W_{defended}$ alone. An attacker who only has $W_{defended}$ can:

   • Run SVD on each weight matrix and identify the rank-16 subspace via the spectral gap
   • Project that subspace out (or attenuate it) without ever needing $W_{base}$
   • I haven't implemented this attack yet, but it's a natural next step and I'd expect it to recover most of the attenuation result

2. The "hardened base release" scenario has a chicken-and-egg problem. If the community standardizes on releasing only post-DeepRefusal weights, then every such release leaks the same structural signature (rank-16 LoRA delta on top of something). An attacker doesn't need the original $W_{base}$ — they just need to know that a low-rank defensive delta exists, which is public knowledge once the method is published. The defense's security would then rest entirely on obfuscating the rank, which seems fragile.

That said, I think your point lands in a softer form: the attenuation attack as written is a "white-box + base-known" attack, and reporting it without that caveat understates the work needed for a base-unknown attacker. I'll update the report to make this explicit and frame the SVD-based attack as the natural follow-up that would close the gap.

More broadly, I think this exchange highlights something useful for the field: defenses that are structurally low-rank may be fundamentally hard to make robust against weight-space attacks, regardless of $W_{base}$ availability, because the structure itself is the leak. I'd be very curious whether a higher-rank or full-finetune variant of DeepRefusal would resist this — that feels like a genuinely interesting research direction, and one I'd be happy to collaborate on or run experiments for if useful.

Thanks again for the open and constructive discussion. This is exactly the kind of back-and-forth that makes safety research better.

[YuanBoXie]

I agree with the observation that "defenses which are structurally low-rank may be fundamentally hard to make robust against weight-space attacks." I believe this phenomenon merits deeper investigation and would make for a compelling research paper. I look forward to potential collaboration on this topic.

[wuwangzhang1216]

I would also be very open to potential collaboration in the future.

[fhdnskfbeuv]

Hi! I recently develop a new steering method based on SCAV. I found it perform well against deeprefusal. Here is the repo: https://github.com/fhdnskfbeuv/AdaptiveProbeSteering

After installing the enviroment, run

CUDA_VISIBLE_DEVICES="0" python eval.py --evalData harmbench --maxL 512 --model 'skysys00/Meta-Llama-3-8B-Instruct-DeepRefusal' --evalPT "1" --clfP "./iterSCAVWeight/skysys00_Meta-Llama-3-8B-Instruct-DeepRefusal/judges_embTyperesponse_filterTrue_normRegTrue_layer[-32, -2]_gpuLRTrue_maxIter20_trainL256_ptabs0_softThres[0.05, 0.6].pt" --csvP myRes.csv --evalClfr 'best' --judge "srf" "hb"

The method achieves 0.81 StrongReject Score and 0.97 Harmbench Score.