Please open a private security advisory on the GitHub repository, or contact the maintainer directly, rather than filing a public issue. We aim to acknowledge within a few days.
goldset turns real traffic/traces into a versioned regression suite, so it ingests data that may contain PII — that shapes its trust model.
- PII redaction runs before anything is stored or displayed. The regex redactor is
built in (no extras); the optional
[pii]extra adds Presidio NER on top. Even so, redaction is best-effort — review your data-handling obligations before pointing goldset at production logs, and prefer already-anonymized corpora where possible. - Everything runs locally. goldset processes logs on your machine and does not send
them anywhere. Optional extras (
[embeddings],[cluster],[nli]) download model weights from the Hugging Face Hub; the core and the full test/validation path run offline and keyless with the deterministic embedder. - The golden gate fails closed. A case is never treated as "golden" until a human/oracle
confirms its expected outcome;
require_goldenblocks export otherwise. The tool is a curation accelerator, not a gold factory — this is enforced in the type system.
If you find a way to make goldset leak un-redacted PII, or to ship an unconfirmed suggestion as a golden case, that's a security-relevant bug — please report it.