Skip to content

Fangyangci/simplify rules zip#374

Merged
fangyangci merged 29 commits into
mainfrom
fangyangci/scale
Apr 24, 2026
Merged

Fangyangci/simplify rules zip#374
fangyangci merged 29 commits into
mainfrom
fangyangci/scale

Conversation

@fangyangci

@fangyangci fangyangci commented Apr 21, 2026

Copy link
Copy Markdown
Contributor
  1. Change update_zip to use a snapshot generation model of full baseline plus incremental updates, reducing duplicate data across opsets and shrinking rules_zip size.
  2. Unify the update entry point to just two parameters, ep + device, and remove the external version parameter to reduce misuse risk and ensure incremental-chain consistency.
  3. Move op_check_results/rules to rules_zip

fangyangci and others added 16 commits April 15, 2026 00:31
 into fangyangci/scale

# Conflicts:
#	src/winml/modelkit/analyze/runtime_checker/result_processor.py
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…-ModelKit into fangyangci/scale

# Conflicts:
#	src/winml/modelkit/analyze/runtime_checker/runner.py
…to fangyangci/scale

# Conflicts:
#	src/winml/modelkit/analyze/runtime_checker/case_runner.py
remove duplicate data between version
…to fangyangci/scale

# Conflicts:
#	src/winml/modelkit/analyze/runtime_checker/result_processor.py
#	src/winml/modelkit/analyze/runtime_checker/runner.py
Comment thread src/winml/modelkit/analyze/runtime_checker/runner.py Fixed
Comment thread src/winml/modelkit/analyze/runtime_checker/runner.py Fixed
Comment thread src/winml/modelkit/analyze/runtime_checker/runner.py Fixed
Comment thread src/winml/modelkit/analyze/runtime_checker/runner.py Fixed
fangyangci and others added 4 commits April 21, 2026 17:50
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@fangyangci fangyangci marked this pull request as ready for review April 21, 2026 09:51
@fangyangci fangyangci requested a review from a team as a code owner April 21, 2026 09:51
@xieofxie

Copy link
Copy Markdown
Contributor

could you explain more about baseline and delta?

  • For example we have version 12, 13, 14, do 13, 14 both base on 12 or 13 on 12, 14 on 13?
  • delta is per test case CRUD?
    Thanks

@fangyangci

fangyangci commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

could you explain more about baseline and delta?

  • For example we have version 12, 13, 14, do 13, 14 both base on 12 or 13 on 12, 14 on 13?
  • delta is per test case CRUD?
    Thanks

As each op has different since_version
every version is a diff with the previous version. 13 on 12, 14 on 13...

Load v22= Load v22+21+...+12

delta is NOT test case level.
it is op version level, this PR is try to remove the same op_check_result save in multi version.

@xieofxie

Copy link
Copy Markdown
Contributor

could you explain more about baseline and delta?

  • For example we have version 12, 13, 14, do 13, 14 both base on 12 or 13 on 12, 14 on 13?
  • delta is per test case CRUD?
    Thanks

As each op has different since_version every version is a diff with the previous version

Load v22= Load v22+21+...+12

delta is NOT test case level. it is op version level, this PR is try to remove the same op_check_result save in multi version.

In this sense, it is more easy to just convert raw result op data json to pandas dataframe json or other format and then load the since version in code? It could be even faster since we only load one needed. No need to combine them

@fangyangci

fangyangci commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

could you explain more about baseline and delta?

  • For example we have version 12, 13, 14, do 13, 14 both base on 12 or 13 on 12, 14 on 13?
  • delta is per test case CRUD?
    Thanks

As each op has different since_version every version is a diff with the previous version
Load v22= Load v22+21+...+12
delta is NOT test case level. it is op version level, this PR is try to remove the same op_check_result save in multi version.

In this sense, it is more easy to just convert raw result op data json to pandas dataframe json or other format and then load the since version in code? It could be even faster since we only load one needed. No need to combine them

our runtime queries many nodes/operators in a model, current combined snapshots are often better because they reduce file-open overhead and reuse in-memory data.
Any way, I don't think the performance bottleneck is load json file.

@timenick

timenick commented Apr 22, 2026

Copy link
Copy Markdown
Collaborator

Could you please update your PR title so it's easier to read? Please refer to other PRs' title format. Thanks.

@fangyangci fangyangci changed the title Fangyangci/scale Fangyangci/simplify rules zip Apr 23, 2026
@fangyangci fangyangci merged commit 0cf5559 into main Apr 24, 2026
9 checks passed
@fangyangci fangyangci deleted the fangyangci/scale branch April 24, 2026 08:17
ssss141414 pushed a commit that referenced this pull request Apr 27, 2026
1. Change update_zip to use a snapshot generation model of full baseline
plus incremental updates, reducing duplicate data across opsets and
shrinking rules_zip size.
2. Unify the update entry point to just two parameters, ep + device, and
remove the external version parameter to reduce misuse risk and ensure
incremental-chain consistency.
3. Move op_check_results/rules to rules_zip

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants