Skip to content

Support per-field thresholds & regex validators in JSON schema (from_dict), plus per-task cls_threshold - #127

Open
Andrew-Chen-Wang wants to merge 1 commit into
fastino-ai:mainfrom
Andrew-Chen-Wang:andrew/jsoninput-threshold-regex
Open

Support per-field thresholds & regex validators in JSON schema (from_dict), plus per-task cls_threshold#127
Andrew-Chen-Wang wants to merge 1 commit into
fastino-ai:mainfrom
Andrew-Chen-Wang:andrew/jsoninput-threshold-regex

Conversation

@Andrew-Chen-Wang

Copy link
Copy Markdown

Motivation

Schema.from_dict / from_json are the way to build a schema from JSON (e.g. when GLiNER2 is served behind an HTTP API and clients send a SchemaInput payload). But the JSON format is missing three things the Python builder already supports, so they're unreachable from JSON:

  • StructureBuilder.field(..., threshold=…) — per-field confidence threshold
  • StructureBuilder.field(..., validators=[RegexValidator(...)]) — per-field regex span filters
  • Schema.classification(..., cls_threshold=…) — per-task classification threshold

FieldInput only modeled name/dtype/choices/description and ClassificationInput only task/labels/multi_label, so from_dict silently dropped these capabilities.

Change

Wire the existing builder features through the JSON boundary:

  • schema_model.py: new ValidatorInput model (pattern, mode, exclude, ignore_case) — a clean JSON mirror of RegexValidator that avoids exposing raw re flag ints. Add threshold + validators to FieldInput and cls_threshold to ClassificationInput, with range/regex validation.
  • from_dict: pass threshold/validators into builder.field(...) (converting ValidatorInputRegexValidator, ignore_casere.IGNORECASE) and cls_threshold into classification(...).
  • to_dict: emit the same keys (defaults omitted) so the dict round-trips through from_dict.
  • Export ValidatorInput from the package; add torch-free tests.

Example

{
  "structures": {
    "contact": {
      "fields": [
        {"name": "name", "dtype": "str", "threshold": 0.7},
        {"name": "email", "dtype": "str",
         "validators": [{"pattern": "[^@]+@[^@]+", "mode": "partial", "ignore_case": false}]}
      ]
    }
  },
  "classifications": [
    {"task": "sentiment", "labels": ["positive", "negative"], "cls_threshold": 0.8}
  ]
}

Backward compatibility

All new fields are optional and default to the builders' existing defaults; to_dict omits them when unset, so existing schemas serialize byte-for-byte as before. from_dict(to_dict(schema)) is idempotent (covered by a test).

Tests

Added tests/test_schema_input_thresholds_validators.py (torch-free — no model download): from_dict wiring, to_dict round-trip/idempotency, defaults-omitted, and rejection of out-of-range thresholds / invalid regex.

🤖 Generated with Claude Code

… per-task cls_threshold

The Python builder (`Schema.structure(...).field(...)`) already accepts a
per-field `threshold` and a list of `RegexValidator`s, and `classification(...)`
accepts `cls_threshold`. None of these were reachable through the JSON
`SchemaInput` format (`from_dict`/`from_json`), so callers building schemas from
JSON could not set them.

Wire them through the JSON boundary:

- `schema_model.py`: new `ValidatorInput` model (pattern/mode/exclude/ignore_case)
  mirroring `RegexValidator`; add `threshold` + `validators` to `FieldInput` and
  `cls_threshold` to `ClassificationInput`, all range/regex validated.
- `from_dict`: pass `threshold`/`validators` into `builder.field(...)` (converting
  `ValidatorInput` -> `RegexValidator`, `ignore_case` -> `re.IGNORECASE`) and
  `cls_threshold` into `classification(...)`.
- `to_dict`: emit the same keys (defaults omitted) so the format round-trips.
- Export `ValidatorInput`; add torch-free round-trip/validation tests.

Backward compatible: all new fields are optional with the builders' existing
defaults, and `to_dict` omits them when unset, so existing schemas serialize
byte-for-byte as before.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PCpTQEuDz3WieZTGZstgCd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant