feat(redis): add cluster-capable clients#357
Merged
Merged
Conversation
Managed providers can expose Redis Cluster rather than a direct or Sentinel endpoint. Add Cluster client construction, seed URL parsing, redirect and topology refresh settings, and DB zero validation to backend Redis config. URL schemes and credentials now survive into Cluster options while direct and Sentinel modes keep their existing path.
Cluster clients do not fit the concrete standalone Redis client type used by app wiring. Widen server, runtime, and health Redis fields to redis.UniversalClient while preserving duplicate-close handling. Application startup can pass direct, Sentinel, or Cluster clients through the same runtime slots.
HTTP edge code shares Redis clients with middleware and streaming coordination paths. Widen auth, rate limiting, and streamgate entry points from concrete clients to redis.UniversalClient. These callers can now use Cluster clients without changing command behavior.
Browser session coordination uses the shared Redis client set from runtime wiring. Widen coordination and continuity Redis fields and setters to redis.UniversalClient. Session locks, live state, and cleanup paths can now run on Cluster-capable clients.
Dashboard services fan Redis clients into cache and queue collaborators. Widen dashboard Redis setters so cache and queue clients can be direct, Sentinel, or Cluster. The facade no longer pins downstream services to standalone Redis clients.
Platform account state and cache stores share the managed Redis connection surface. Widen OAuth state and cache Redis clients to redis.UniversalClient. Account cache and OAuth flows can now use Cluster-aware clients transparently.
Media asset resolution stores cache entries and invalidates key families through Redis. Widen the media asset cache client and cache deletion helper to redis.UniversalClient. Resolve caching can use Cluster clients while keeping the existing command flow.
Project cache helpers scan and bump Redis generations across several cache families. Widen project cache fields and helpers to redis.UniversalClient. Project list and setup option caches can now run against Cluster-capable clients.
Asynq-backed queues need to share the same Redis client abstraction as runtime roles. Widen email, publish, and dashboard rebuild queue constructors to redis.UniversalClient. Queue workers can now be constructed from direct, Sentinel, or Cluster Redis clients.
Publish coordination and queue clients are supplied from the runtime Redis roles. Widen publish service Redis setters to redis.UniversalClient. Publishing can share Cluster-capable coordination and queue clients without extra wiring.
Stats overview caching uses Redis through the dashboard service graph. Widen the stats cache client and setter to redis.UniversalClient. Stats cache calls can now use the same Cluster-capable clients as other services.
Browser workers need the same managed Redis endpoint modes as backend session continuity. Add Cluster endpoint parsing, URL TLS and auth merging, redirect handling, topology refresh, and DB zero validation. Worker session state can now use direct, Sentinel, or Cluster Redis providers.
Collab sync previously supported direct and Sentinel Redis endpoints only. Add cluster config validation and node-redis createCluster options with seed nodes, TLS, auth, and redirection limits. Collab pubsub can now target Redis Cluster while keeping standalone and Sentinel rollback paths.
Backend Redis config needs coverage for Cluster mode and URL-derived settings. Add tests for Cluster options, seed URL parsing, URL TLS and auth, DB validation, and direct-client assertions. The tests lock the managed Cluster contract while preserving existing direct and Sentinel expectations.
Runtime tests still inspect direct client options for role-specific Redis baselines. Assert the configured coordination client is a direct redis.Client before reading its options. The test remains precise after runtime wiring moves to redis.UniversalClient.
Browser worker Redis state config now supports Cluster mode. Add tests for Cluster options, URL TLS and auth, DB validation, and conflicting URL credentials. The worker state store contract is covered for direct, Sentinel, and Cluster endpoints.
Collab Cluster mode needs option-level coverage before integration environments are available. Add tests for cluster root nodes, TLS and auth defaults, redirection limits, and DB validation. The tests verify node-redis receives the same provider settings as direct and Sentinel modes.
The shared env contract now advertises Redis Cluster seed lists. Add the cluster endpoint mode and validate REDIS_ADDR as comma-separated hostport or URL entries. Env doctor can now check direct, Sentinel, and Cluster Redis deployment inputs from the same schema.
Generated env examples must reflect the updated Redis endpoint contract. Regenerate dev and deploy examples with the Cluster seed-list guidance comment. Operators see the Cluster mode requirement without hand-editing generated files.
Redis deployment docs still described only direct and Sentinel endpoint modes. Document Cluster seed-node configuration and the REDIS_DB zero requirement in module and Kubernetes baseline docs. Readers can configure Cluster providers while keeping direct and Sentinel rollback paths clear.
KurodaKayn
force-pushed
the
feat/348-redis-cluster-clients
branch
from
0b67479 to
4b017ed
Compare
Cluster SCAN routing depends on a hash tag in the match pattern. Reuse the dashboard project-list hash tag for the invalidation scan pattern and cache keys. Project list invalidation now scans the same slot as the cached list entries in Cluster mode.
The project-list cache key contract needs to include the invalidation pattern. Assert the scan pattern shares the dashboard project-list hash tag with generated cache keys. Future changes cannot accidentally route Cluster invalidation scans to an arbitrary shard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature Description
REDIS_ENDPOINT_MODE=clusterfor backend, browser-worker, and collab-service Redis clients.REDIS_DB=0in Cluster mode and documents comma-separated seed-node config.Implementation Approach
redis.UniversalClientso direct, Sentinel, and Cluster clients share service paths and Asynq queues.createClusteroptions for collab-service.Testing
go test ./...(backend)go test ./...(browser-worker)pnpm --dir collab-service run lint && pnpm --dir collab-service run format:checkpnpm --dir collab-service run type-check && pnpm --dir collab-service testruby script/env/generate_examples.rb --checkgit diff --checkruby script/kubernetes/test_validate_rendered_manifests.rbnot run successfully: local Ruby is 2.6.10 and cannot parse existing Ruby 3 keyword argument shorthand inscript/kubernetes/overlay_image_pinner.rb; CI uses Ruby 3.3.