Skip to content

[MAINTENANCE] Add integration coverage for quoted schema names on PostgreSQL - #12051

Draft
joshua-stauffer wants to merge 1 commit into
developfrom
m/quoted-schema-name-integration-test
Draft

[MAINTENANCE] Add integration coverage for quoted schema names on PostgreSQL#12051
joshua-stauffer wants to merge 1 commit into
developfrom
m/quoted-schema-name-integration-test

Conversation

@joshua-stauffer

Copy link
Copy Markdown
Collaborator

Summary

Wrapping an identifier in the dialect's quote characters is how a user asks GX to use that identifier verbatim rather than case-folding it. table_name honors that convention: a quoted value is unwrapped and rebuilt as a sqlalchemy.quoted_name(..., quote=True), so the quotes govern the emitted SQL instead of becoming part of the name.

A schema name never gets the same treatment. _effective_schema_name returns the string with the quote characters still embedded, so SQLAlchemy escapes them as data rather than honoring them as quoting:

sa.table("mytable", schema='"MySchema"')
# FROM """MySchema""".mytable          <- targets a schema whose name contains quotes

sa.table("mytable", schema=sa.quoted_name("MySchema", quote=True))
# FROM "MySchema".mytable              <- what was asked for

The practical consequence is that a case-sensitive schema is unreachable: quoting is the only way to preserve its case, and quoting is what breaks. The same mismatch also makes the schema-existence check in TableAsset.test_connection compare a quoted name against a bare schema listing, so it reports a schema that does exist as missing.

This PR adds coverage only — no behavior change.

What's here

Two PostgreSQL integration tests in tests/integration/data_sources_and_expectations/test_quoted_schema_names.py that differ in exactly one variable: whether the schema is handed to the asset bare or bracketed by quotes.

  • test_bare_schema_name_reaches_the_tablepasses. A control, so the companion test's failure can only be attributable to the quoting rather than to anything about naming a schema on the asset.
  • test_quoted_schema_name_reaches_the_tablexfail(strict=True). Documents the current behavior and converts into a hard failure the moment the identifier handling is corrected, so the fix cannot land without this test being updated deliberately.

The shared PostgreSQL setup targets its schema through the connection string's search_path, which never passes through GX's identifier handling, so these tests name the schema on the asset instead. That is also why PostgreSQL is the backend here: it runs locally, and the schema is created and dropped by the existing harness.

Verification

$ pytest tests/integration/data_sources_and_expectations/test_quoted_schema_names.py -m postgresql
1 passed, 1 xfailed

Against a local PostgreSQL container. With --runxfail, the xfailed test fails as described above:

TestConnectionError: Attempt to connect to table: ""gx_ci_test_17bef88060".expectation_test_table_..."
failed because the schema ""gx_ci_test_17bef88060"" does not exist.

ruff check, ruff format --check, and the repo's CI type-check configuration are all clean.

Notes for review

  • On PostgreSQL the only route to this code path is the asset's schema_name, which is deprecated but still supported. Backends that carry a schema in structured connection details (Snowflake, SQL Server) reach the same _effective_schema_name through their non-deprecated schema_ property, so the defect is not confined to the deprecated argument.
  • A fix would most likely normalize the schema at _effective_schema_name — the single point both routes pass through — mirroring what the table_name validator already does, including the dict() round-trip that re-adds the quote characters so stored configs keep their shape.

Wrapping an identifier in the dialect's quote characters is how a user asks
GX to use it verbatim rather than case-folding it. `table_name` honors that:
a quoted value is unwrapped and rebuilt as a quoted SQLAlchemy identifier, so
the quotes govern the emitted SQL instead of becoming part of the name. A
schema is not given the same treatment -- the quote characters stay inside the
string, and the emitted SQL targets a schema whose name literally contains
quotes.

Pin the behavior down with two PostgreSQL tests that differ only in whether
the schema is quoted. The bare case passes, establishing that naming a schema
on the asset works; the quoted case is marked xfail(strict=True), so it
reports as expected-fail today and turns into a hard failure the moment the
identifier handling is corrected.

The shared PostgreSQL setup targets its schema through the connection
string's search_path, which never passes through GX's identifier handling, so
these tests name the schema on the asset instead.
@netlify

netlify Bot commented Aug 10, 2026

Copy link
Copy Markdown

Deploy Preview for niobium-lead-7998 canceled.

Name Link
🔨 Latest commit 32459e0
🔍 Latest deploy log https://app.netlify.com/projects/niobium-lead-7998/deploys/6a7a1efd4e41ce000860c3f7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant