bug: Schema loader allows double underscores in attribute names

[polmichel]

Component

API Server / GraphQL

Infrahub version

1.9.3

Current Behavior

When loading the a schema the schema loader and validator allows loading a schema where the attributes contain double underscores.

---
version: "1.0"
nodes:
  - name: TestUnderscore
    namespace: Test
    attributes:
      - name: name
        kind: Text
      - name: name__asc
        kind: Text

Probably there is the same problem with relationships. In isolation the above schema actually works but there a are a few places in the code that assumes that a double underscore will be a separator that is not connected to the name of the attribute, a common example is path values which will impact computed attributes. A common way to refer to the name attribute above would be name__value.

Expected Behavior

We should reject a schema that looks like this. A problem though is that we don't know if there is anyone who has such a schema in their database now and as such forbidding it would be a breaking change. This bug surfaced when considering the impact of opsmill#9190.

The best course of action might be to fix this bug in develop for Infrahub 1.10 and list the change as a breaking change in the release notes.

Steps to Reproduce

The above schema combined with a computed attribute that referred to name__asc should probably do the trick. There's also a PR with a failing test here: opsmill#9207

Additional Information

No response

[polmichel]

/bug-analyze

[test-bug-pipeline]

✅ Bug analyst agent completed successfully!

[test-bug-pipeline]

Root cause analysis for #20

Issue: bug: Schema loader allows double underscores in attribute names
Based on: ef11b629e51372553e9da84956fde9cd41ff6292
Bug clarity: CLEAR
Code identification: RESOLVED

Root cause

The pattern used to validate attribute and relationship names (NAME_REGEX = r"^[a-z0-9\_]+$") accepts any number of consecutive underscores, so names like name__asc pass validation even though __ is the reserved path separator used elsewhere in the codebase.

Affected files

• backend/infrahub/core/constants/__init__.py — line 412: NAME_REGEX = r"^[a-z0-9\_]+$" allows runs of underscores; this is the single source of truth fed into Pydantic field patterns.
• backend/infrahub/core/schema/definitions/internal.py — line 496: applies NAME_REGEX to AttributeSchema.name.
• backend/infrahub/core/schema/definitions/internal.py — line 715: applies NAME_REGEX to RelationshipSchema.name.
• backend/infrahub/core/schema/definitions/internal.py — line 757: applies NAME_REGEX to RelationshipSchema.identifier — must keep allowing __ because identifiers legitimately encode two kinds joined by __ (e.g. testcar__testperson).
• backend/infrahub/core/schema/basenode_schema.py — line 514: path.split("__") is one of the call sites that mis-parses paths once __ appears inside a name. Other call sites (backend/infrahub/core/validators/uniqueness/checker.py:35, backend/infrahub/graphql/mutations/node_getter/by_default_filter.py:30, by_hfid.py:36, backend/infrahub/core/node/__init__.py:810,816, backend/infrahub/core/query/node.py:1689,2056) share the same implicit contract.

Explanation

Infrahub uses __ throughout the codebase as a reserved separator in path expressions:

• <attribute>__<property> (e.g. name__value).
• <relationship>__<attribute>__<property>.
• <kind1>__<kind2> for relationship identifiers.

All path parsing code (parse_schema_path, default-filter resolution, HFID component splitting, uniqueness checker, computed-attribute dependency tracking, query filter parsing) does a plain str.split("__"). If an attribute or relationship is named name__asc, that string becomes indistinguishable from a two-segment path name/asc, and the wrong branch of the parser fires — for example, a computed attribute referencing name__asc will be interpreted as a property lookup on attribute name.

Because the regex at the schema-loader layer permits this, the bad name is accepted at schema-load time and only surfaces later as a confusing downstream parse failure (or worse, a silently wrong value). The fix belongs at the validation boundary, not in each parser.

Fix strategy

Approach: Tighten the name validation regex so that consecutive underscores are no longer allowed in attribute names or relationship names, while leaving relationship identifiers untouched.

1. In backend/infrahub/core/constants/__init__.py, add a new constant alongside the existing ones — e.g. SCHEMA_ELEMENT_NAME_REGEX = r"^[a-z0-9]+(_[a-z0-9]+)*$" (or an equivalent form using a negative lookahead such as ^(?!.*__)[a-z0-9_]+$). Keep the existing NAME_REGEX constant unchanged — it is still needed for relationship identifiers.
2. In backend/infrahub/core/schema/definitions/internal.py, switch the two name fields to the new regex: line 496 (AttributeSchema.name) and line 715 (RelationshipSchema.name). Leave line 757 (RelationshipSchema.identifier) using NAME_REGEX.
3. Regenerate the pydantic models: uv run invoke backend.generate. This updates backend/infrahub/core/schema/generated/attribute_schema.py and backend/infrahub/core/schema/generated/relationship_schema.py so the new pattern flows into the runtime validators.
4. Add a changelog/ fragment marking this as a breaking change for 1.10.

Scope: Two files edited by hand (constants + definitions), plus generated pydantic models updated via backend.generate, plus a changelog fragment. No changes needed in the __-splitting call sites — those become correct again once names with __ are rejected at load time.

Do NOT:

• Do not change NAME_REGEX itself. It is still applied to RelationshipSchema.identifier, where testcar__testperson style values are legitimate and pervasive across the test suite.
• Do not add a separate field_validator in attribute_schema.py / relationship_schema.py instead of fixing the regex pattern. The whole codebase expresses name constraints via regex= at the definition layer; bolting on a runtime validator would diverge from that pattern and skip JSON-schema enforcement.
• Do not try to patch the individual split("__") call sites to handle ambiguous names — there is no unambiguous interpretation once __ is permitted inside names, which is exactly why the schema must reject them.
• Do not write a migration that rewrites existing user data with __ in names. The issue explicitly accepts this as a breaking change for 1.10; flag affected users via release notes instead.

Notes for downstream steps

• The failing test referenced in the issue lives in Test for schema underscore bug opsmill/infrahub#9207 (a sibling PR, not in this repo's checkout). The test-writer should write equivalent failing tests here that:
  1. Construct AttributeSchema(name="name__asc", kind="Text") and assert it raises pydantic.ValidationError.
  2. Construct RelationshipSchema(name="peer__alt", ...) and assert it raises pydantic.ValidationError.
  3. Construct RelationshipSchema(..., identifier="testcar__testperson") and assert it still succeeds — this is the regression guardrail that protects the identifier carve-out.
  4. Optionally: load the YAML schema from the issue description through the schema loader (e.g. via SchemaRoot / SchemaBranch.load_schema) and assert it is rejected.
• Edge cases for the regex: leading underscore (_foo), trailing underscore (foo_), single underscore between alphanumerics (foo_bar). The recommended pattern ^[a-z0-9]+(_[a-z0-9]+)*$ rejects leading/trailing underscores too, which is a slight additional tightening — verify that no existing built-in attribute names rely on leading/trailing underscores before committing. Quick check: grep the definitions/core/*.py files for name="_ and name=".*_" patterns. If any exist, prefer the negative-lookahead form ^(?!.*__)[a-z0-9_]+$ instead, which only forbids the __ pattern.
• The Pydantic Field(pattern=...) only enforces the regex on input strings; existing serialized schemas in a database will not re-validate on read. The behavior change therefore lands at the next schema load. This is intentional per the issue and only needs to be called out in the changelog.
• Min length on attribute name is currently 3 (DEFAULT_KIND_MIN_LENGTH). A pattern like ^[a-z0-9]+(_[a-z0-9]+)*$ combined with that min-length cannot generate any false rejections of currently-valid 3+ char names that lack consecutive underscores.
• Once backend.generate is run, the generated files under backend/infrahub/core/schema/generated/ will diff — that diff is expected and should be committed alongside the source change.

AGENT_ANALYSIS_COMPLETE

Generated by Bug analyst agent for issue #20 · ● 10.9M · ◷

[polmichel]

/bug-tdd

[test-bug-pipeline]

❌ Bug test-writer agent failed. Please review the logs for details.

[test-bug-pipeline]

Draft test PR opened on branch ai-bug-pipeline-20-schema-double-underscore. Three unit tests added at backend/tests/unit/core/schema/test_schema_name_validation.py: two prove the bug by failing on the current code (consecutive underscores in AttributeSchema.name and RelationshipSchema.name are silently accepted), and one regression guardrail confirms RelationshipSchema.identifier still allows __. Awaiting reviewer.

Generated by Bug test-writer agent for issue #20 · ◷