Skip to content

recipe(cross-encoder): add MiniLM L2 text-classification recipes#1143

Open
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-cross-encoder-minilm-l2-recipe
Open

recipe(cross-encoder): add MiniLM L2 text-classification recipes#1143
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-cross-encoder-minilm-l2-recipe

Conversation

@kujin66

@kujin66 kujin66 commented Jul 20, 2026

Copy link
Copy Markdown

Adds catalog recipes for cross-encoder/ms-marco-MiniLM-L2-v2 as a BERT text-classification model. This is an L0 recipe-only contribution: current origin/main already builds the model via default BERT support, while the checked-in recipes make CPU/DML fp32/fp16 variants discoverable and reproducible. The producer-side Goal ladder reached L1 on reachable CPU and DML GPU EPs.

1. Recipe Path(s)

  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/cpu/cpu/text-classification_fp16_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/dml/gpu/text-classification_fp16_config.json

2. README Row

Added cross-encoder/ms-marco-MiniLM-L2-v2 | text-classification to examples/recipes/README.md; total count updated from 75 to 76.

3. Build Output Dir

  • Baseline auto-config: temp/baseline_cross_encoder_ms_marco_minilm_l2_v2_rebased
  • CPU fp32 recipe: temp/cross_encoder_ms_marco_minilm_l2_v2_cpu_fp32
  • CPU fp16 recipe: temp/cross_encoder_ms_marco_minilm_l2_v2_cpu_fp16_recipe
  • DML fp32 recipe: temp/cross_encoder_ms_marco_minilm_l2_v2_dml_fp32
  • DML fp16 recipe: temp/cross_encoder_ms_marco_minilm_l2_v2_dml_fp16_recipe

4. Build Log

Baseline command:

D:\WPDProjects\winml-cli\.venv\Scripts\python.exe -m winml.modelkit.cli build -m cross-encoder/ms-marco-MiniLM-L2-v2 -o temp/baseline_cross_encoder_ms_marco_minilm_l2_v2_rebased --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild

Baseline result: Build complete in 18.7s; final artifact temp\baseline_cross_encoder_ms_marco_minilm_l2_v2_rebased\model.onnx.

Recipe build results:

  • CPU fp32: Build complete in 11.2s; final artifact temp\cross_encoder_ms_marco_minilm_l2_v2_cpu_fp32\model.onnx.
  • CPU fp16: Build complete in 13.0s; FP16 stage ran; final artifact temp\cross_encoder_ms_marco_minilm_l2_v2_cpu_fp16_recipe\model.onnx.
  • DML fp32: Build complete in 11.9s; final artifact temp\cross_encoder_ms_marco_minilm_l2_v2_dml_fp32\model.onnx.
  • DML fp16: Build complete in 13.0s; FP16 stage ran; final artifact temp\cross_encoder_ms_marco_minilm_l2_v2_dml_fp16_recipe\model.onnx.

Structural validation:

CPU fp32: inputs input_ids/attention_mask/token_type_ids [1,512], output logits [1,1], FLOAT16 initializers 0/52
CPU fp16: inputs input_ids/attention_mask/token_type_ids [1,512], output logits [1,1], FLOAT16 initializers 46/52
DML fp32: inputs input_ids/attention_mask/token_type_ids [1,512], output logits [1,1], FLOAT16 initializers 0/52
DML fp16: inputs input_ids/attention_mask/token_type_ids [1,512], output logits [1,1], FLOAT16 initializers 46/52

5. Appended Findings

  • Lane A finding appended outside this model PR: model_knowledge/bert.json, bert-001.
  • Finding records: model has 15,615,745 parameters, 51 total modules, 19 traced modules, 38 execution steps, 137 ONNX nodes with 100% hierarchy tags.
  • Analyze note: winml analyze could not run on this host because src/winml/modelkit/analyze/rules/runtime_check_rules contains only README.md; exact CLI error was No runtime rule parquet files were found.

6. Optimum-Coverage Probe

{'vendor': ['feature-extraction', 'fill-mask', 'multiple-choice', 'question-answering', 'text-classification', 'token-classification'], 'after_winml': ['feature-extraction', 'fill-mask', 'multiple-choice', 'question-answering', 'text-classification', 'token-classification'], 'added_by_winml': []}

winml inspect -m cross-encoder/ms-marco-MiniLM-L2-v2 --format json resolved:

  • model_type: bert
  • architectures: BertForSequenceClassification
  • task: text-classification
  • exporter: BertIOConfig
  • WinML class: WinMLModelForSequenceClassification

7. Claimed (Effort, Goal, Outcome)

{
  "model_id": "cross-encoder/ms-marco-MiniLM-L2-v2",
  "model_type": "bert",
  "effort": "L0",
  "goal_ceiling": "L1",
  "outcome": "L0",
  "target_eps": ["cpu", "dml-gpu"],
  "optimum_probe": {
    "vendor": ["feature-extraction", "fill-mask", "multiple-choice", "question-answering", "text-classification", "token-classification"],
    "after_winml": ["feature-extraction", "fill-mask", "multiple-choice", "question-answering", "text-classification", "token-classification"],
    "added_by_winml": [],
    "verdict": "VENDOR-ONLY"
  },
  "catalog_gate": {
    "baseline_build": "PASS",
    "baseline_head": "238bd386d77381b70e03377295a1f40a69982576",
    "winml_version": "0.2.0",
    "verdict": "catalog-needs-numbers",
    "file_pr": true
  }
}

8. Goal-Ladder Verdict Table

Tier Verdict Evidence
L0 PASS (baseline-covered + recipe-verified) Baseline auto-config build on current origin/main PASS at 238bd386d77381b70e03377295a1f40a69982576; CPU fp32/fp16 and DML fp32/fp16 recipe builds all completed; ONNX structural validation matched recipe I/O.
L1 PASS Providers: ['DmlExecutionProvider', 'CPUExecutionProvider']. Baseline CPU fp32: 17.641 ms mean, 56.69 samples/s. CPU fp32 recipe: 18.827 ms mean, 53.12 samples/s. CPU fp16 recipe: 27.079 ms mean, 36.93 samples/s. DML fp32 recipe: 15.724 ms mean, 63.60 samples/s. DML fp16 recipe: 13.416 ms mean, 74.54 samples/s.

9. Methodology-Evolution Declaration

No methodology friction observed. The only host gap was missing analyze parquet rules; this is already covered by the reviewer guidance that accepts explicit analyze-unavailable evidence when the rules directory contains only README.md.

@kujin66
kujin66 force-pushed the kujin66/add-cross-encoder-minilm-l2-recipe branch from e882392 to f9b7b63 Compare July 20, 2026 08:43
@kujin66

kujin66 commented Jul 20, 2026

Copy link
Copy Markdown
Author

APPROVE (coverage: full for target EPs cpu and dml-gpu)

Reviewer checks performed:

  • PR exists and is draft: recipe(cross-encoder): add MiniLM L2 text-classification recipes #1143.
  • PR body contains the 9 required hand-off items: recipe paths, README row, build output dirs, build logs, findings, Optimum probe, claimed E/G/O, Goal ladder, and methodology declaration.
  • Baseline freshness: git fetch origin main; current origin/main is 238bd386d77381b70e03377295a1f40a69982576; PR body baseline head matches it; git rev-list --count 238bd386d77381b70e03377295a1f40a69982576..origin/main returned 0.
  • Diff scope for L0 recipe-only is correct: examples/recipes/README.md plus four JSON files under examples/recipes/cross-encoder_ms-marco-MiniLM-L2-v2/{cpu/cpu,dml/gpu}/; no source or test code included.
  • Recipe schema and layout are consistent with existing multi-EP cross-encoder L4/L6 recipes.
  • Evidence spot-check: DML fp16 perf JSON reports DmlExecutionProvider, precision fp16, mean latency 13.416 ms, throughput 74.54 samples/s.
  • Artifact validation evidence includes expected I/O and true FP16 initializer counts for fp16 artifacts (46/52).
  • Analyze was host-blocked by missing runtime rule parquet files; the PR body cites the exact CLI error and mines export_htp_metadata.json / winml_build_config.json instead, matching reviewer guidance.

No requested changes.

@kujin66
kujin66 marked this pull request as ready for review July 20, 2026 08:48
@kujin66
kujin66 requested a review from a team as a code owner July 20, 2026 08:48
@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants