add BEFT as a new SFT/PEFT method

[whubaichuan]

Is your feature request related to a problem? Please describe.
I initially want to add our recently published BEFT (paper) (ACL2026 Main Conference) as a sft/peft method in Tunix. However, I found that the current tunix-LoRA is provided from qwix, as shown here. Moreover, in peft_trainer (link), the lora option is provided from flax.

Therefore, I am wondering that where is the best place for contributing other sft/peft methods?

Describe the solution you'd like
Alternatively, will Tunix provide the direct interface for intergrating more sft/peft methods?

Additional context
I wish get some guidance. Thank you in advance @tianshub @s-noghabi

Checklist

• I have searched the existing issues for similar feature requests.
• This is not a support question (please use the "bug template" for that).

[tianshub]

@whubaichuan can you outline an implementation plan for BEFT? like what are the proposed code change.

then we can see if it's better to land it in qwix or tunix. Thanks!

[whubaichuan]

@tianshub Thanks for your message.

The outline of my proposed code change is only setting certain specific parameters in LLM as trainable, while BEFT is only setting the bias terms of values projection to be trainable (if no bias terms in some LLM, BEFT adds a trainable vector as the role of bias terms). The benefit is the extreme parameter efficiency with a competitive accuracy, especially on few training data, as valided here.

BTW, if you decide to land it in tunix, I would suggest that Tunix could provide the interface for intergrating more sft/peft methods, such as prompt-tuning, prefix tuning, PiSSA, etc.

[tianshub]

@jshin1394 can you take a look and see if this is a better fit for qwix to support?

[jshin1394]

Hi @tianshub, Thank you so much for reaching out to me with this. I totally agree that the BEFT algorithm is a great method that can be implemented using the Qwix provider pattern as it would be similar to the existing Qwix QLoRA implementation in that both selectively update a small subset of parameters.

However, the reason why the QLoRA exists as a core algorithm within Qwix is because of the Q part, the quantized nature of the method. BEFT, on the other hand, as described in the paper, focuses on which existing parameters (the biases) to fine-tune. It doesn't inherently involve new quantization techniques for weights or activations.

Given this, I believe the most suitable place for implementing BEFT would be within the Tunix codebase, as Tunix is designed for post-training and fine-tuning workflows. Tunix could extend an existing Qwix provider and implement BEFT. Qwix will be happy to provide necessary guidance, review and communication during the process! :)