feat(demo): runnable 5-minute clean-gate demo#4
Merged
Conversation
examples/demo/ — a baseline card plus an honest candidate (ACCEPT) and an overfit candidate that games the public checks (contamination up, OOD down) and is REJECTed with reasons ood_regressed + dcr_increased. Includes baseline.json, candidate.json, candidate_overfit.json, the exact expected output, a redacted LIMITED_ROLLOUT assurance card, and a full walkthrough README (Run the demo / What this shows / What it does not / How to interpret the decision / links). README adds the 5-minute path; a CI test asserts ACCEPT exit 0 / REJECT exit 1. All synthetic/redacted; no private content.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a clean public demo path:
pip install "vlabs-sdk==0.0.2"thenvlabs clean-gateon a baseline + two candidates. The honest candidate ACCEPTs; the overfit candidate (highest public score, but contamination up + OOD down) REJECTs withood_regressed/dcr_increased— showing a higher public score is not a promotion. Includes fixtures, exact expected output, a redacted LIMITED_ROLLOUT assurance card, a walkthrough README with all required sections + links (PyPI/HF/W&B/docs) + the exact formal-scope statement, and a CI test asserting the ACCEPT(0)/REJECT(1) behaviour. Synthetic/redacted only; no private content; no forbidden claims.