Add Kerberos Auth (#128) - #887
Conversation
Adds use_kerberos and kerberos_hostname_override connection parameters for both existing clients. Requires the new `kerberos` extra.
Stands up a real KDC and Kerberos-configured ClickHouse instance via docker-compose (behind a `kerberos` profile) for end-to-end testing of use_kerberos, plus a dedicated CI job.
bf4412e to
4092a75
Compare
|
Hi @pferate thanks for this PR as well. Before we commit to reviewing and maintaining this long term, could you share more about the environment driving it? Kerberos is excluded from ClickHouse Cloud, so this'll be a self-managed-only feature. Some of the other client maintainers are looking at the same question so your answers could help inform that broader decision. Specifically:
If you're more comfortable answering these outside of a public channel, let me know. |
|
Hi @joe-clickhouse Personally, I don't have a dog in this race, so I can't give you any useful answers to those questions. I was just looking through open issues and seeing where I can help contribute. I saw that #128 was open and stale, so I picked it. |
|
@pferate understood! Thanks so much for the context. |
# Conflicts: # CHANGELOG.md # docker-compose.yml
There was a problem hiding this comment.
Pull request overview
This PR adds experimental Kerberos authentication support for ClickHouse Connect over HTTP(S) by generating a fresh SPNEGO (Negotiate) token per HTTP request attempt (sync and async), backed by a new clickhouse_connect.driver.kerberos module and an optional kerberos extra.
Changes:
- Added
use_kerberosandkerberos_hostname_overrideconnection options, wiring them through the sync (HttpClient/HttpSyncBackend) and async (AsyncClient/HttpAsyncBackend) stacks. - Implemented SPNEGO token generation and mutual-auth validation via
pyspnego(imported asspnego) with lazy optional dependency loading. - Added unit and integration test coverage, plus Docker/CI plumbing to run Kerberos integration tests in GitHub Actions.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
clickhouse_connect/driver/options.py |
Adds lazy import + check_spnego() gate for optional spnego dependency. |
clickhouse_connect/driver/kerberos.py |
Introduces KerberosAuthContext to build Authorization: Negotiate ... and validate WWW-Authenticate response tokens. |
clickhouse_connect/driver/httpclient.py |
Adds Kerberos connection params, enforces auth exclusivity, passes Kerberos settings into the sync backend. |
clickhouse_connect/driver/asyncclient.py |
Adds Kerberos connection params, enforces auth exclusivity, passes Kerberos settings into the async backend. |
clickhouse_connect/driver/_backend/http_sync.py |
Generates a fresh Kerberos context per request attempt and validates mutual auth on successful responses; adjusts retry behavior. |
clickhouse_connect/driver/_backend/http_async.py |
Generates a fresh Kerberos context per request attempt and validates mutual auth on successful responses; adds redirect prevention and GET -> POST rewrite under Kerberos. |
clickhouse_connect/driver/__init__.py |
Documents new connection parameters in factory docstrings. |
tests/unit_tests/test_driver/test_kerberos.py |
Unit coverage for validation, construction, context behavior, and backend request wiring. |
tests/integration_tests/test_kerberos.py |
End-to-end Kerberos auth integration tests (token freshness, hostname override, stale token rejection). |
tests/integration_tests/kerberos_manage.py |
Orchestrates Kerberos test environment using Docker Compose and host-side kinit. |
tests/integration_tests/kerberos_conf/* |
Adds Kerberos KDC + ClickHouse config fixtures for integration tests. |
docker-compose.yml |
Adds Kerberos KDC and Kerberos-configured ClickHouse services under a kerberos Compose profile. |
.docker/kerberos-kdc/Dockerfile |
Provides a minimal image for the test KDC service. |
.github/workflows/on_push.yml |
Adds a dedicated Kerberos integration test job to CI and gates check-secret on it. |
setup.py |
Adds kerberos extra dependency (pyspnego[kerberos]). |
pyproject.toml |
Updates mypy config to ignore missing imports for spnego. |
README.md |
Documents Kerberos setup and usage at a high level. |
docs/index.mdx |
Lists the new kerberos extra in the install matrix. |
docs/driver-api.mdx |
Documents Kerberos options and behavior in the API reference. |
CONTRIBUTING.md |
Adds contributor instructions for running Kerberos integration tests. |
tests/test_requirements.txt |
Removes trailing whitespace line. |
CHANGELOG.md |
Adds an unreleased changelog entry describing the new Kerberos feature and constraints. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| if self.use_kerberos: | ||
| assert self.kerberos_hostname is not None | ||
| kerberos_context = KerberosAuthContext(self.kerberos_hostname) | ||
| headers["Authorization"] = kerberos_context.authorization_header |
| kerberos_context = None | ||
| if self.use_kerberos: | ||
| assert self.kerberos_hostname is not None | ||
| kerberos_context = KerberosAuthContext(self.kerberos_hostname) | ||
| req_headers["Authorization"] = kerberos_context.authorization_header |
|
|
||
| Each authenticated HTTP request attempt sends a fresh preemptive Kerberos token. The client also requires a `WWW-Authenticate: Negotiate <token>` header on every successful authenticated response and validates that token to complete mutual authentication. It raises `OperationalError` if the response token is missing, malformed, rejected, or does not complete the Kerberos context. | ||
|
|
||
| Do not combine `use_kerberos=True` with `username`, `password`, `access_token`, `token_provider`, or `client_cert`. Kerberos authentication supports the HTTP and HTTPS clients only. It does not support the chDB backend. |
|
Hey @pferate, after working through this a bit more I've decided to close the PR without merging for now. But I really appreciate all the effort and work you put into getting it here. The original issue is ~3.5 years old and hasn't seen any activity. The finished implementation adds >1K lines across several dozen files, along with additional client complexity, an optional dependency, and a dedicated CI job. Without any demand it's hard to justify the maintenance and complexity burden of another auth. This has nothing to do with quality, but added complexity without recent demand. I'm also planning broader simplification of the client API in the near future. That might be a better time to revisit this kind of broader auth support and decide how it should fit into the client. But thanks again for taking this on! The work is still available in the closed PR if/when we decide to revisit it later. For anyone else reading this who is interested in having kerberos auth support, please upvote or add some context to #128. |
Summary
This adds a Kerberos connection options that generates a SPNEGO ("Negotiate") token from the Kerberos credential cache for each HTTP request.
Adds use_kerberos and kerberos_hostname_override parameters to both the sync (http) and async clients, backed by a new kerberos.py module built on pyspnego/gssapi/krb5 (new kerberos extra). Because ClickHouse authenticates each HTTP request independently rather than caching auth for a session, a fresh token is generated per request rather than once at connect time.
Issue: #128
Checklist