fix: reconcile task registry into a single source of truth (#724)#986
Open
timenick wants to merge 2 commits into
Open
fix: reconcile task registry into a single source of truth (#724)#986timenick wants to merge 2 commits into
timenick wants to merge 2 commits into
Conversation
KNOWN_TASKS (validate_task / --list-tasks) and TASK_ABBREV (cache keys) were maintained independently and drifted: 13 task names lived only in TASK_ABBREV and were rejected by validate_task. Derive both from one _TASK_REGISTRY so they can no longer desync. - Add the 5 wired tasks (video-classification, keypoint-matching, table-question-answering, zero-shot-audio-classification, any-to-any) to the canonical set, keeping their existing abbreviations. - Drop the 8 unreferenced/stale task names from the abbreviation table. - Keep the 5 collapsed aliases out of KNOWN_TASKS but retain their abbreviations for cache-key / serve reverse-decode stability. - Add drift guards: registry/abbrev consistency, wired-task regression, stale-task audit lock, and an inference.tasks cross-source check. No abbreviation changes for any task still in use, so no cache churn.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #724.
Problem
loader/task.pyheld two hand-maintained task tables, created by different PRs from different authorities, that silently drifted apart:TASK_ABBREV(PR feat: add winml serve & winml run with MCP support #354) — canonical→abbreviation map for cache keys; intentionally broad (full HF taxonomy).KNOWN_TASKS+validate_task(PR fix(inspect): make --list-tasks return in <500 ms (#544) #717) — a hand-copied snapshot of optimum's exportable tasks for a fastwinml inspect --list-tasks, never cross-checked againstTASK_ABBREV.PR #717's
validate_taskgate consults onlyKNOWN_TASKS, so the pre-existing divergence became a user-visible bug: 13 task names that lived only inTASK_ABBREVwere rejected — including ones WinML genuinely supports through a third independent table,inference/tasks.py(TaskInputSpec). Root cause: no single source of truth for the canonical task set.Fix
Both public views now derive from one private registry, so they can no longer desync:
_TASK_REGISTRY: dict[str, str | None]— the single authoritative table of canonical task → abbreviation (Nonekeeps the existing 8-char truncation fallback).KNOWN_TASKS = frozenset(_TASK_REGISTRY)andTASK_ABBREV(non-Noneabbrevs + 5 collapsed aliases) are both derived.Audit of the 13 divergent names:
video-classification,keypoint-matching,table-question-answering,zero-shot-audio-classification,any-to-any— each has aTaskInputSpec, a composite-model registration, and/or an e2e testset tag.instance-segmentation,universal-segmentation,masked-image-modeling,text-encoding,audio-tokenization,multimodal-lm,backbone,time-series-prediction— unreferenced anywhere outside the old table;masked-image-modeling/time-series-predictionwere long-form duplicates of the existingmasked-im/time-series-forecasting.pretraining,sequence-classification,summarization,translation,zero-shot-classification) stay out ofKNOWN_TASKSbut keep their abbreviations, because some callers use the alias name directly as the resolved task (composite models registersummarization/translation;inference.tasksdefines a spec forzero-shot-classification).Zero cache churn: every abbreviation still in use is byte-for-byte unchanged. Net:
KNOWN_TASKS35→40,TASK_ABBREV47→39, and no abbreviation value modified — so no existing cache key changes.Drift guards added
tests/unit/loader/test_known_tasks.pylocks the invariant going forward:KNOWN_TASKSandTASK_ABBREVmust derive from_TASK_REGISTRY; abbreviations must be unique (theservereverse-decode map stays lossless).validate_taskand the click--taskcallback.inference.tasksTaskInputSpeckey must be a known task — the cross-source check that would have caught this bug.