Code

[Drft]Lora on dcu#9959

Code

Open

yongqiangma wants to merge 27 commits intoPaddlePaddle:developPaddlePaddle/PaddleNLP:developfrom yongqiangma:lorayongqiangma/PaddleNLP:loraCopy head branch name to clipboard

+488-74

Commits

Commits on Jan 26, 2025

• add modleing_pp

  DrownFish19

  committed
  741b8e7

  Copy full SHA for 741b8e7

Commits on Jan 27, 2025

• add modleing_pp for qwen2moe

  DrownFish19

  committed
  cf82bcc

  Copy full SHA for cf82bcc

Commits on Feb 1, 2025

• add flashmask and pp for Qwen2MoE and Deepseek

  DrownFish19

  committed
  d646dba

  Copy full SHA for d646dba

• remove

  DrownFish19

  committed
  3fcf2c1

  Copy full SHA for 3fcf2c1

• Merge remote-tracking branch 'paddlenlp/develop' into dev_20250126_add_pipeline_for_moe

  DrownFish19

  committed
  3a320cb

  Copy full SHA for 3a320cb

• fix fast_tokenizer save

  DrownFish19

  committed
  d55f559

  Copy full SHA for d55f559

Commits on Feb 6, 2025

• update for topk_weight of noaux_tc

  DrownFish19

  committed
  b104eaa

  Copy full SHA for b104eaa

• Merge branch 'PaddlePaddle:develop' into dev_20250126_add_pipeline_for_moe

  DrownFish19

  authored
  4c7f5d6

  Copy full SHA for 4c7f5d6

Commits on Feb 7, 2025

• fix for flashmask

  DrownFish19

  committed
  ecad2f1

  Copy full SHA for ecad2f1

• Merge remote-tracking branch 'paddlenlp/develop' into dev_20250126_add_pipeline_for_moe

  DrownFish19

  committed
  446b4da

  Copy full SHA for 446b4da

• add use_expert_parallel for pretrain

  DrownFish19

  committed
  80f5c98

  Copy full SHA for 80f5c98

Commits on Feb 10, 2025

• fix tokenizer test

  DrownFish19

  committed
  47628e4

  Copy full SHA for 47628e4

• modify model size for dev

  DrownFish19

  committed
  2651521

  Copy full SHA for 2651521

Commits on Feb 11, 2025

• Merge branch 'PaddlePaddle:develop' into dev_20250210_deepseek_v3_pretrained

  DrownFish19

  authored
  035b7fe

  Copy full SHA for 035b7fe

Commits on Feb 12, 2025

• Merge commit 'refs/pull/9838/head' of https://github.com/PaddlePaddle/PaddleNLP into develop

  phlrain

  committed
  a182411

  Copy full SHA for a182411

• update

  phlrain

  committed
  72bc6ee

  Copy full SHA for 72bc6ee

Commits on Feb 17, 2025

• mtp

  DrownFish19

  committed
  8f3028d

  Copy full SHA for 8f3028d

• MTP

  DrownFish19

  committed
  c75fd64

  Copy full SHA for c75fd64

• update deafult config

  DrownFish19

  committed
  175d735

  Copy full SHA for 175d735

• update MTP

  DrownFish19

  committed
  068e192

  Copy full SHA for 068e192

• update seq_aux_loss

  DrownFish19

  committed
  02f407f

  Copy full SHA for 02f407f

• Merge remote-tracking branch 'paddlenlp/develop' into dev_20250214_deepseek_mtp

  DrownFish19

  committed
  45b386e

  Copy full SHA for 45b386e

Commits on Feb 21, 2025

• Merge commit 'refs/pull/9876/head' of https://github.com/PaddlePaddle/PaddleNLP into develop

  yongqiangma

  committed
  c8aa47a

  Copy full SHA for c8aa47a

• Merge commit 'refs/pull/9849/head' of https://github.com/PaddlePaddle/PaddleNLP into develop

  yongqiangma

  committed
  0c13a3c

  Copy full SHA for 0c13a3c

Commits on Feb 27, 2025

• lora with use_flash_attention=false

  yongqiangma

  committed
  d1d8d9c

  Copy full SHA for d1d8d9c

Commits on Feb 28, 2025

• add config

  yongqiangma

  committed
  27579e7

  Copy full SHA for 27579e7

• use_flash_attention false

  yongqiangma

  committed
  471b172

  Copy full SHA for 471b172