Skip to content

fix: reconcile task registry into a single source of truth (#724)#986

Open
timenick wants to merge 2 commits into
mainfrom
timenick-zhiwang-reconcile-task-registry
Open

fix: reconcile task registry into a single source of truth (#724)#986
timenick wants to merge 2 commits into
mainfrom
timenick-zhiwang-reconcile-task-registry

Conversation

@timenick

Copy link
Copy Markdown
Collaborator

Fixes #724.

Problem

loader/task.py held two hand-maintained task tables, created by different PRs from different authorities, that silently drifted apart:

PR #717's validate_task gate consults only KNOWN_TASKS, so the pre-existing divergence became a user-visible bug: 13 task names that lived only in TASK_ABBREV were rejected — including ones WinML genuinely supports through a third independent table, inference/tasks.py (TaskInputSpec). Root cause: no single source of truth for the canonical task set.

Fix

Both public views now derive from one private registry, so they can no longer desync:

  • _TASK_REGISTRY: dict[str, str | None] — the single authoritative table of canonical task → abbreviation (None keeps the existing 8-char truncation fallback).
  • KNOWN_TASKS = frozenset(_TASK_REGISTRY) and TASK_ABBREV (non-None abbrevs + 5 collapsed aliases) are both derived.

Audit of the 13 divergent names:

  • 5 wired → added to the canonical set: video-classification, keypoint-matching, table-question-answering, zero-shot-audio-classification, any-to-any — each has a TaskInputSpec, a composite-model registration, and/or an e2e testset tag.
  • 8 stale → dropped: instance-segmentation, universal-segmentation, masked-image-modeling, text-encoding, audio-tokenization, multimodal-lm, backbone, time-series-prediction — unreferenced anywhere outside the old table; masked-image-modeling / time-series-prediction were long-form duplicates of the existing masked-im / time-series-forecasting.
  • 5 collapsed aliases (pretraining, sequence-classification, summarization, translation, zero-shot-classification) stay out of KNOWN_TASKS but keep their abbreviations, because some callers use the alias name directly as the resolved task (composite models register summarization/translation; inference.tasks defines a spec for zero-shot-classification).

Zero cache churn: every abbreviation still in use is byte-for-byte unchanged. Net: KNOWN_TASKS 35→40, TASK_ABBREV 47→39, and no abbreviation value modified — so no existing cache key changes.

Drift guards added

tests/unit/loader/test_known_tasks.py locks the invariant going forward:

  • KNOWN_TASKS and TASK_ABBREV must derive from _TASK_REGISTRY; abbreviations must be unique (the serve reverse-decode map stays lossless).
  • The 5 wired tasks must pass both validate_task and the click --task callback.
  • The 8 dropped tasks must stay out of every view.
  • Every inference.tasks TaskInputSpec key must be a known task — the cross-source check that would have caught this bug.

timenick added 2 commits June 26, 2026 14:58
KNOWN_TASKS (validate_task / --list-tasks) and TASK_ABBREV (cache keys)
were maintained independently and drifted: 13 task names lived only in
TASK_ABBREV and were rejected by validate_task. Derive both from one
_TASK_REGISTRY so they can no longer desync.

- Add the 5 wired tasks (video-classification, keypoint-matching,
  table-question-answering, zero-shot-audio-classification, any-to-any)
  to the canonical set, keeping their existing abbreviations.
- Drop the 8 unreferenced/stale task names from the abbreviation table.
- Keep the 5 collapsed aliases out of KNOWN_TASKS but retain their
  abbreviations for cache-key / serve reverse-decode stability.
- Add drift guards: registry/abbrev consistency, wired-task regression,
  stale-task audit lock, and an inference.tasks cross-source check.

No abbreviation changes for any task still in use, so no cache churn.
@timenick timenick requested a review from a team as a code owner June 26, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[task] Source-of-truth inconsistency: 13 task names in TASK_ABBREV are rejected by validate_task

1 participant