Skip to content

Fix Gemma SFT masking and DPO preference formatting - #8

Open
houyongsheng wants to merge 1 commit into
google-gemma:mainfrom
houyongsheng:fix/trainer-training-behavior
Open

Fix Gemma SFT masking and DPO preference formatting#8
houyongsheng wants to merge 1 commit into
google-gemma:mainfrom
houyongsheng:fix/trainer-training-behavior

Conversation

@houyongsheng

Copy link
Copy Markdown

Summary

  • keep DPO prompts and completions separate using the conversational preference format expected by TRL
  • load the saved SFT processor when available and set DPO beta explicitly
  • mask every assistant span in multi-turn SFT records while enforcing the configured maximum length
  • use FastVisionModel and UnslothVisionDataCollator for the optimized multimodal path
  • add dependency-free helpers and regression tests for formatting, truncation, and masking

Testing

  • Python unittest suite: 7 passed
  • AST parsing: all 6 gemma-trainer asset scripts passed
  • committed diff whitespace check passed

No model weights were downloaded and no GPU training run was performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant