Skip to content

feat(kernel): honor WithPort on the kernel backend - #581

Open
cathleeny wants to merge 10 commits into
mainfrom
cathleeny/feat/kernel-honor-custom-port
Open

cathleeny wants to merge 10 commits into
mainfrom
cathleeny/feat/kernel-honor-custom-port

Conversation

@cathleeny

@cathleeny cathleeny commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

On the kernel backend, WithPort and the DSN port are now honored. Before this PR any value other than 0 or 443 was rejected. Thrift is unchanged.

Behavior

  • Adding the port:

    • A non-default port is added to the host as host:port, using net/url and net.JoinHostPort.
    • An IPv6 address, bare or bracketed, becomes [::1]:8443. That includes DSN hosts, which ParseDSN returns without brackets.
  • Host's port wins: a port already in the host wins over a different WithPort. A warning is logged, except when the explicit port is 443.

  • Defaults: 0 means the default port, and a trailing / is ignored.

  • Rejected with ErrInvalidKernelConfig:

    • an out-of-range or malformed port;
    • a non-default port with a host that has a path, query, fragment or credentials;
    • a password in the host, at any port.

    Error messages redact credentials.

  • OAuth: with OAuth and a non-default port, the kernel uses the same host:port for OAuth. This is documented.

Testing

  • gofmt, go vet and golangci-lint are clean. go test ./... with CGO off passes, and so does the kernel-tagged suite.
  • Live, DBSQL and Reyden: these pass:
    • WithPort(443);
    • a host with :443 and WithPort(0);
    • a host with :443 and WithPort(8443);
    • Thrift on 443.

Related PRs: databricks/databricks-sql-python#1071, databricks/databricks-sql-nodejs#636, https://github.com/databricks/databricks-driver-test/pull/2188

This pull request and its description were written by Isaac.

The kernel backend rejected any WithPort other than 443. The kernel has no port
setter, but it keeps a `host:port` authority in its host, so the connector now
folds a non-default port into the host it passes to the kernel.

- kernelHostWithPort adds a non-default port with net.JoinHostPort (IPv6-safe).
  The default 443 and an unset port add nothing.
- validateKernelHostPort rejects an out-of-range port, a host with unbalanced IPv6
  brackets (for example `[::1` or `h.databricks.com]`), and a supplied port that
  disagrees with a port embedded in the host. An unset port (WithDefaults fills
  443) defers to the embedded port. An explicit WithPort(443) or DSN port is
  supplied, so it must match an embedded port too (AC-P3: never drop a port).
- UserConfig gains an internal PortExplicit flag, set by WithPort and the DSN
  parser. It is not a public option: internal/ is not importable by users.
- Non-https protocols and the other kernel rejections are unchanged.

With OAuth (M2M and U2M) and a non-default port, the kernel reaches the OAuth
endpoints at the same host:port as the connection. Thrift keeps OAuth on the bare
host. This is documented in CONNECTION_PARAMETERS.md, the README and the
CHANGELOG, and is not guarded.

Tests: one table-driven TestKernelCustomPort covers the host mapping, the
validation (accept and reject, including explicit 443 and unbalanced brackets),
and the kernel Host. The config, connector and driver tests expect PortExplicit
where WithPort or the DSN set the port.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

Verdict: 1 Medium

Looks good: one low concern. The port folding, the validation and the PortExplicit plumbing check out, including DeepCopy and the DSN path. Nit: the PortExplicit disposition is listed as "forwarded", but it is only consumed by validation.

This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.

Comment thread kernel_config.go Resolved
proxyForEndpointFunc appended the port to cfg.Host even when the host already carried one, so host:8443 with Port 8443 built an invalid endpoint, the proxy was dropped and HTTPS_PROXY was ignored. It now uses the same authority as kernelHostWithPort, which leaves an embedded port alone and brackets IPv6 literals.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

Verdict: 1 Nit

Looks good, with 1 low concern. The port folding and validation are correct for IPv4, IPv6 and embedded-port cases, and PortExplicit is set by both entry points (WithPort and the DSN parser) and kept by DeepCopy. Nit: the out-of-range error message says "want 1..65535", but port 0 is accepted as unset.

Other findings

  • ⚪ Nit — The Reyden auto-recovery path (openSessionWithReydenFallback → getKernelBackend → buildKernelConfig) now folds a custom port into the kernel Host, where it was rejected before. That is reasonable, but no test or CHANGELOG line covers the recovery path. Add a recovery test with WithPort(8443) that asserts the kernel Host, and mention recovery in the CHANGELOG entry. (Found by Claude Opus 5.5 · confirmed by the judge)

Address the review nits on the kernel custom-port change:

- The out-of-range WithPort error now says 0 is accepted as the default.
- PortExplicit is classified as "validated", not "forwarded": it is consumed
  by validateKernelHostPort and never sent to the kernel.
- A Reyden recovery test with WithPort(8443) asserts that the kernel Host is
  host:8443 after the Thrift KP001 rejection. The CHANGELOG entry mentions the
  recovery path.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
@cathleeny

Copy link
Copy Markdown
Collaborator Author

Fixed the review nits in 263c741:

  • Port range message (review 5980950905): the out-of-range WithPort error now reads "want 1..65535, or 0 for the default", which matches the accepted unset value.
  • Reyden recovery coverage (review 5980950905): TestReydenReactiveRecovery has a new case. A Thrift KP001 rejection with WithPort(8443) recovers onto the kernel, and the case asserts that the kernel Host is reyden.example.com:8443 and that the port passes validation. The CHANGELOG entry now mentions the recovery path.
  • PortExplicit disposition (review 5980504145): the field is now classified as "validated", not "forwarded". This is the same label the experimental classification uses for TLSClientCertConfigured. The flag is consumed by validateKernelHostPort and never sent to the kernel.

CGO_ENABLED=0 go test -count=1 ./... passes for all packages, go vet ./... is clean, and gofmt is clean.

Automated reply from the driver feature factory (Isaac). Not reviewed by a human.

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

✅ No issues identified.

Comment thread kernel_config.go Resolved
cathleeny and others added 5 commits November 19, 2026 23:13
…hPort

Fold a trailing "/" in the host before any port handling, matching the
kernel's normalise_host, so WithServerHostname("https://h/") plus
WithPort(8443) connects to h:8443 instead of h/:8443, and
"https://h:8443/" keeps working.

Port precedence now matches the Node and Python kernel paths:
- A port embedded in the host wins over a differing WithPort. The driver
  logs a warning naming both ports and connects to the embedded port.
- A zero (unset) WithPort means the default, with no :443 added.
- A host with a path, query or fragment after the trimmed "/" is
  rejected with ErrInvalidKernelConfig. Base also rejected it: the kernel
  builds {host}/api/2.0/sql, so h/x sent requests to /x/api/2.0/sql.
- Out-of-range WithPort and out-of-range or malformed embedded ports stay
  rejected. Thrift also fails on them: an embedded port plus the default
  443 yields an unparseable https://h:8443:443 URL, -1 is an invalid port,
  and 70000 fails to dial.

Base behavior (origin/main): validateKernelConfigContext rejected any
WithPort other than 0 or 443 with ErrNotSupportedByKernel, and passed the
host through unchanged, so an embedded port connected to that port.

Tests: table cases for h/, h:8443/, h/x, [::1]/, embedded-vs-WithPort
(warn, embedded wins), explicit 443 against an embedded port, and an
invalid port (still rejected). Proxy endpoint test for a trailing-slash
host. Kernel loopback probe: embedded port dialed, explicit port never.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…xplicit 443

- A host with a path or query is rejected before the embedded-port parse, so
  "h.io/path:8443" with an explicit port cannot slip through. The old code passed
  the host verbatim to the kernel, which builds {host}/api/2.0/sql, so these
  configs never connected.
- The differing-port warning is skipped for an explicit 443 (the default).
- Thrift is unchanged. TestThriftEndpointUnchanged pins its current endpoint
  strings for the same port inputs, including the embedded-port-plus-443 case.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
- A host with a path, query or fragment is now rejected only when a
  non-default WithPort would have to be folded into it. The check still runs
  after the trailing "/" trim and before the embedded-port parse. With no port
  or 443 the host goes to the kernel unchanged, as on the base branch, where
  the kernel keeps a path prefix ({host}/api/2.0/sql). The base branch rejected
  every non-default WithPort, so nothing that worked before is rejected now.
- Remove the internal PortExplicit field. Once an explicit 443 stopped
  warning, nothing read it, so WithPort, ParseDSN, DeepCopy and their tests
  are back to the base branch.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Resolve the CHANGELOG.md conflict by keeping both Unreleased entries: this PR's
kernel port entry and main's non-finite float rendering entries. No code conflicts.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The kernel path folds WithPort into cfg.Host. The host is now parsed once per
call with net/url, as the https authority the kernel connects to, instead of
net.SplitHostPort plus hand-written bracket and path scans.

Rules (unchanged unless noted):
- A trailing "/" is trimmed first. With the default port (0 or 443) the host
  passes through unchanged.
- A port embedded in the host wins over a differing WithPort, with a warning.
  An explicit 443, or 0, is the default and is never warned about.
- WithPort outside 1..65535 is rejected. An embedded port that is empty,
  non-numeric or outside 1..65535 is rejected.
- With a non-default port, a host with a path, query, fragment (including an
  empty "h#" or "h?") or user info is rejected. Unbalanced brackets are rejected.
- Errors wrap ErrInvalidKernelConfig as before.

Behavior changes from the reviewed revision (52bd5ac):
- An IPv6 literal must be bracketed. A bare "::1" is rejected at any port
  (net/url cannot parse it). It never connected on main either: the kernel
  prepends https:// and its parse rejects it.
- A host with a password ("user:pw@h") is rejected at any port.
- A host net/url cannot parse (for example "h%zz" or a space) is rejected
  even at the default port, where it used to pass through to the kernel.

The parse-once wrappers and the separate warning helper are gone:
validateKernelHostPort logs the embedded-port warning, and kernelHostWithPort
is the only fold, so buildKernelConfig and resolveKernelProxy keep their
reviewed signatures.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

Verdict: 2 Medium · 1 Low

Mostly solid, with 1 medium and 1 low concern. The port folding and validation are well tested, and the Reyden recovery and production paths both go through newKernelBackend's validation. The main problem is that a password in the host can end up in error messages.

This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.

Comment thread kernel_config.go Resolved
Comment thread kernel_config.go Resolved
Comment thread kernel_config.go Resolved
cathleeny and others added 2 commits November 20, 2026 16:35
…rrors

Bracket a bare IPv6 literal before the kernel host is parsed, so "::1"
and "[::1]" both fold to "[::1]:8443" on every Go version. A DSN IPv6
host reaches the same path, since ParseDSN strips the brackets. ParseDSN
itself is unchanged.

Host-bearing errors now show user info as "<redacted>@h". url.Parse's
text is withheld when the host has user info, because it quotes the
whole URL and a bad escape quotes the password.

A password is still rejected at every port, except that the default
port keeps a host with a path, query or fragment unchanged, as before.
Document that rule in CHANGELOG, README and CONNECTION_PARAMETERS.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Resolve the CHANGELOG.md conflict by keeping both Unreleased entries: this
PR's kernel port entry and main's OAuth M2M token-request timeout entry. No
code conflicts.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

✅ No issues identified.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants