Support per-field thresholds & regex validators in JSON schema (from_dict), plus per-task cls_threshold - #127
Open
Andrew-Chen-Wang wants to merge 1 commit into
Conversation
… per-task cls_threshold The Python builder (`Schema.structure(...).field(...)`) already accepts a per-field `threshold` and a list of `RegexValidator`s, and `classification(...)` accepts `cls_threshold`. None of these were reachable through the JSON `SchemaInput` format (`from_dict`/`from_json`), so callers building schemas from JSON could not set them. Wire them through the JSON boundary: - `schema_model.py`: new `ValidatorInput` model (pattern/mode/exclude/ignore_case) mirroring `RegexValidator`; add `threshold` + `validators` to `FieldInput` and `cls_threshold` to `ClassificationInput`, all range/regex validated. - `from_dict`: pass `threshold`/`validators` into `builder.field(...)` (converting `ValidatorInput` -> `RegexValidator`, `ignore_case` -> `re.IGNORECASE`) and `cls_threshold` into `classification(...)`. - `to_dict`: emit the same keys (defaults omitted) so the format round-trips. - Export `ValidatorInput`; add torch-free round-trip/validation tests. Backward compatible: all new fields are optional with the builders' existing defaults, and `to_dict` omits them when unset, so existing schemas serialize byte-for-byte as before. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PCpTQEuDz3WieZTGZstgCd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Schema.from_dict/from_jsonare the way to build a schema from JSON (e.g. when GLiNER2 is served behind an HTTP API and clients send aSchemaInputpayload). But the JSON format is missing three things the Python builder already supports, so they're unreachable from JSON:StructureBuilder.field(..., threshold=…)— per-field confidence thresholdStructureBuilder.field(..., validators=[RegexValidator(...)])— per-field regex span filtersSchema.classification(..., cls_threshold=…)— per-task classification thresholdFieldInputonly modeledname/dtype/choices/descriptionandClassificationInputonlytask/labels/multi_label, sofrom_dictsilently dropped these capabilities.Change
Wire the existing builder features through the JSON boundary:
schema_model.py: newValidatorInputmodel (pattern,mode,exclude,ignore_case) — a clean JSON mirror ofRegexValidatorthat avoids exposing rawreflag ints. Addthreshold+validatorstoFieldInputandcls_thresholdtoClassificationInput, with range/regex validation.from_dict: passthreshold/validatorsintobuilder.field(...)(convertingValidatorInput→RegexValidator,ignore_case→re.IGNORECASE) andcls_thresholdintoclassification(...).to_dict: emit the same keys (defaults omitted) so the dict round-trips throughfrom_dict.ValidatorInputfrom the package; add torch-free tests.Example
{ "structures": { "contact": { "fields": [ {"name": "name", "dtype": "str", "threshold": 0.7}, {"name": "email", "dtype": "str", "validators": [{"pattern": "[^@]+@[^@]+", "mode": "partial", "ignore_case": false}]} ] } }, "classifications": [ {"task": "sentiment", "labels": ["positive", "negative"], "cls_threshold": 0.8} ] }Backward compatibility
All new fields are optional and default to the builders' existing defaults;
to_dictomits them when unset, so existing schemas serialize byte-for-byte as before.from_dict(to_dict(schema))is idempotent (covered by a test).Tests
Added
tests/test_schema_input_thresholds_validators.py(torch-free — no model download): from_dict wiring, to_dict round-trip/idempotency, defaults-omitted, and rejection of out-of-range thresholds / invalid regex.🤖 Generated with Claude Code