Make a sport-specific detector usable: custom classes, YOLOv8 head, pixel convention - #19
Merged
Merged
Conversation
COCO has 80 classes and none of them is a hoop, so toward_goal and activity_near_goal — 0.35 of the scoring weight — have never fired once, on any run, for any user. No amount of tuning reaches them; the class does not exist. Three things stopped a basketball model from being usable, and each of them fails silently rather than loudly: Class indices were hardcoded to COCO per sport, so a model whose index 1 is a hoop would have had it read as a bicycle. A model may now declare its own classes in a `<model>.classes.json` sidecar, as an ordered list or an explicit index map, with other people's vocabulary — basketball, rim, person, ref — renamed into ours rather than teaching the scorer every model's dialect. The head format differs. YOLOv8 emits [1, 4 + classes, anchors] with no objectness column and boxes already decoded to centre/size in input pixels; YOLOX emits [1, anchors, 5 + classes] needing grid and stride arithmetic. Feeding one to the other's decoder returns a full set of plausible boxes in the wrong places, so the head is now chosen by tensor shape. The pixel convention differs too. YOLOX takes raw 0-255 and YOLOv8 expects 0-1. Measured on the real export, feeding raw bytes to a YOLOv8 model produced 700 "basketball" detections at 1.00 confidence in a frame of an empty gym — every class saturated, nothing thrown. The convention is declared in the same sidecar and defaults to raw. Evaluated against E-BARD (CC-BY-4.0, YOLOv8n, classes basketball/hoop/player/ referee) on twelve frames pulled from the production volume: ball found in 9 of 12 frames, 0.35-0.74 (COCO managed roughly 3 in 15) hoop 1 of 12 full-frame, 3 of 12 tiled player 12 of 12 The ball result is the headline. The hoop is far below the 0.964 F1 that model reports on its own test set, which is domain gap: it was trained on different footage than a 7th-grade game shot from the stands. Non-zero where COCO is structurally zero, but not yet the fix that class needs. No GPU is involved. Inference stays on CPU, and the ONNX export was done once on the workstation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: run a sport-specific detector, so a hoop can exist at all
COCO has 80 classes and none of them is a hoop, so toward_goal and
activity_near_goal — 0.35 of the scoring weight — have never fired once, on any
run, for any user. No amount of tuning reaches them; the class does not exist.
Three things stopped a basketball model from being usable, and each of them
fails silently rather than loudly:
Class indices were hardcoded to COCO per sport, so a model whose index 1 is a
hoop would have had it read as a bicycle. A model may now declare its own
classes in a
<model>.classes.jsonsidecar, as an ordered list or an explicitindex map, with other people's vocabulary — basketball, rim, person, ref —
renamed into ours rather than teaching the scorer every model's dialect.
The head format differs. YOLOv8 emits [1, 4 + classes, anchors] with no
objectness column and boxes already decoded to centre/size in input pixels;
YOLOX emits [1, anchors, 5 + classes] needing grid and stride arithmetic.
Feeding one to the other's decoder returns a full set of plausible boxes in the
wrong places, so the head is now chosen by tensor shape.
The pixel convention differs too. YOLOX takes raw 0-255 and YOLOv8 expects
0-1. Measured on the real export, feeding raw bytes to a YOLOv8 model produced
700 "basketball" detections at 1.00 confidence in a frame of an empty gym —
every class saturated, nothing thrown. The convention is declared in the same
sidecar and defaults to raw.
Evaluated against E-BARD (CC-BY-4.0, YOLOv8n, classes basketball/hoop/player/
referee) on twelve frames pulled from the production volume:
ball found in 9 of 12 frames, 0.35-0.74 (COCO managed roughly 3 in 15)
hoop 1 of 12 full-frame, 3 of 12 tiled
player 12 of 12
The ball result is the headline. The hoop is far below the 0.964 F1 that model
reports on its own test set, which is domain gap: it was trained on different
footage than a 7th-grade game shot from the stands. Non-zero where COCO is
structurally zero, but not yet the fix that class needs.
No GPU is involved. Inference stays on CPU, and the ONNX export was done once
on the workstation.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com