redis · ndyakov · Apr 29, 2025 · Apr 29, 2025 · Apr 29, 2025 · Apr 30, 2025
diff --git a/.github/wordlist.txt b/.github/wordlist.txt
@@ -65,4 +65,75 @@ RedisGears
 RedisTimeseries
 RediSearch
 RawResult
-RawVal
+RawVal
+proto
+REdis
+bufio
+bool
+lifecycle
+Lifecycle
+ConnPool
+mutexes
+SingleConnPool
+StickyConnPool
+backoff
+PubSub
+Reconnection
+reconnection
+resubscription
+Backoff
+GOMAXPROCS
+MaxActiveConns
+MaxIdleConns
+MinIdleConns
+PoolSize
+PoolTimeout
+ReadTimeout
+tcp
+Suboptimal
+deallocation
+goroutine
+Mutex
+HGET
+HSET
+LPUSH
+RPOP
+SADD
+SMEMBERS
+ZADD
+ZRANGE
+EVAL
+EVALSHA
+JSON
+SLOWLOG
+stateful
+BLPOP
+BRPOP
+roundtrip
+HooksMixin
+Unwatch
+UNWATCH
+NewClient
+baseClient
+connPool
+dialHook
+func
+init
+newConnPool
+cmdable
+hooksMixin
+Pooler
+newConn
+parentHooks
+statefulCmdable
+Tx
+NewStickyConnPool
+newTx
+tx
+AfterEach
+Gomega's
+Makefile
+Gomega
+codebase
+structs
+unwatch
diff --git a/README.md b/README.md
@@ -46,6 +46,7 @@ in the `go.mod` to `go 1.24` in one of the next releases.
 
 - [English](https://redis.uptrace.dev)
 - [简体中文](https://redis.uptrace.dev/zh/)
+- [AI-Generated Developer Documentation](./docs) - Detailed technical documentation about the client's implementation, architecture, and internals
 
 ## Resources
 
@@ -295,39 +296,9 @@ vals, err := rdb.Eval(ctx, "return {KEYS[1],ARGV[1]}", []string{"key"}, "hello")
 res, err := rdb.Do(ctx, "set", "key", "value").Result()
 ```
 
-## Run the test
+## Testing
 
-go-redis will start a redis-server and run the test cases.
-
-The paths of redis-server bin file and redis config file are defined in `main_test.go`:
-
-```go
-var (
-	redisServerBin, _  = filepath.Abs(filepath.Join("testdata", "redis", "src", "redis-server"))
-	redisServerConf, _ = filepath.Abs(filepath.Join("testdata", "redis", "redis.conf"))
-)
-```
-
-For local testing, you can change the variables to refer to your local files, or create a soft link
-to the corresponding folder for redis-server and copy the config file to `testdata/redis/`:
-
-```shell
-ln -s /usr/bin/redis-server ./go-redis/testdata/redis/src
-cp ./go-redis/testdata/redis.conf ./go-redis/testdata/redis/
-```
-
-Lastly, run:
-
-```shell
-go test
-```
-
-Another option is to run your specific tests with an already running redis. The example below, tests
-against a redis running on port 9999.:
-
-```shell
-REDIS_PORT=9999 go test <your options>
-```
+The project includes comprehensive test coverage and follows best practices for testing Redis operations. Tests are run using Ginkgo and Gomega for behavior-driven development. For detailed information about running tests, writing tests, and test coverage, please refer to the [Testing Guide](./docs/redis_testing.md). The testing documentation covers test environment setup, common test patterns, and best practices for writing effective tests.
 
 ## See also
 

diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,70 @@
+# Redis Client Documentation
+
+This documentation is AI-generated and provides a comprehensive overview of the Redis client implementation. The documentation is organized into several key files, each focusing on different aspects of the Redis client.
+
+## Documentation Structure
+
+### 1. General Architecture ([`general_architecture.md`](general_architecture.md))
+- High-level architecture of the Redis client
+- Core components and their interactions
+- Connection management
+- Command processing pipeline
+- Error handling
+- Monitoring and instrumentation
+- Best practices and patterns
+
+### 2. Connection Pool Implementation ([`redis_pool.md`](redis_pool.md))
+- Detailed explanation of the connection pool system
+- Pool configuration options
+- Connection lifecycle management
+- Pool statistics and monitoring
+- Error handling in the pool
+- Performance considerations
+- Best practices for pool usage
+
+### 3. Command Processing ([`redis_command_processing.md`](redis_command_processing.md))
+- Command interface and implementation
+- Command execution pipeline
+- Different execution modes (single, pipeline, transaction)
+- Command types and categories
+- Error handling and retries
+- Best practices for command usage
+- Monitoring and debugging
+
+### 4. Testing Framework ([`redis_testing.md`](redis_testing.md))
+- Test environment setup using Docker
+- Environment variables and configuration
+- Running tests with Makefile commands
+- Writing tests with Ginkgo and Gomega
+- Test organization and patterns
+- Coverage reporting
+- Best practices for testing
+
+### 5. Clients and Connections ([`clients-and-connections.md`](clients-and-connections.md))
+- Detailed client types and their usage
+- Connection management and configuration
+- Client-specific features and optimizations
+- Connection pooling strategies
+- Best practices for client usage
+
+## Important Notes
+
+1. This documentation is AI-generated and should be reviewed for accuracy
+2. The documentation is based on the actual codebase implementation
+3. All examples and code snippets are verified against the source code
+4. The documentation is regularly updated to reflect changes in the codebase
+
+## Contributing
+
+For detailed information about contributing to the project, please see the [Contributing Guide](../CONTRIBUTING.md) in the root directory.
+
+If you find any inaccuracies or would like to suggest improvements to the documentation, please:
+1. Review the actual code implementation
+2. Submit a pull request with the proposed changes
+3. Include references to the relevant code files
+
+## Related Resources
+
+- [Go Redis Client GitHub Repository](https://github.com/redis/go-redis)
+- [Redis Official Documentation](https://redis.io/documentation)
+- [Go Documentation](https://golang.org/doc/) 
diff --git a/docs/clients-and-connections.md b/docs/clients-and-connections.md
diff --git a/docs/general_architecture.md b/docs/general_architecture.md
diff --git a/docs/redis_command_processing.md b/docs/redis_command_processing.md
@@ -0,0 +1,164 @@
+# Redis Command Processing
+
+This document describes how commands are processed in the Redis client, including the command pipeline, error handling, and various command execution modes.
+
+## Command Interface
+
+The core of command processing is the `Cmder` interface:
+
+```go
+type Cmder interface {
+    Name() string          // Command name (e.g., "set", "get")
+    FullName() string      // Full command name (e.g., "cluster info")
+    Args() []interface{}   // Command arguments
+    String() string        // String representation of command and response
+    readTimeout() *time.Duration
+    readReply(rd *proto.Reader) error
+    SetErr(error)
+    Err() error
+}
+```
+
+## Command Processing Pipeline
+
+### 1. Command Creation
+- Commands are created using factory functions (e.g., `NewCmd`, `NewStatusCmd`)
+- Each command type implements the `Cmder` interface
+- Commands can specify read timeouts and key positions
+
+### 2. Command Execution
+The execution flow:
+1. Command validation
+2. Connection acquisition from pool
+3. Command writing to Redis
+4. Response reading
+5. Error handling and retries
+
+### 3. Error Handling
+- Network errors trigger retries based on configuration
+- Redis errors are returned directly
+- Timeout handling with configurable backoff
+
+## Command Execution Modes
+
+### 1. Single Command
+```go
+err := client.Process(ctx, cmd)
+```
+
+### 2. Pipeline
+```go
+pipe := client.Pipeline()
+pipe.Process(ctx, cmd1)
+pipe.Process(ctx, cmd2)
+cmds, err := pipe.Exec(ctx)
+```
+
+### 3. Transaction Pipeline
+```go
+pipe := client.TxPipeline()
+pipe.Process(ctx, cmd1)
+pipe.Process(ctx, cmd2)
+cmds, err := pipe.Exec(ctx)
+```
+
+## Command Types
+
+### 1. Basic Commands
+- String commands (SET, GET)
+- Hash commands (HGET, HSET)
+- List commands (LPUSH, RPOP)
+- Set commands (SADD, SMEMBERS)
+- Sorted Set commands (ZADD, ZRANGE)
+
+### 2. Advanced Commands
+- Scripting (EVAL, EVALSHA)
+- Pub/Sub (SUBSCRIBE, PUBLISH)
+- Transactions (MULTI, EXEC)
+- Cluster commands (CLUSTER INFO)
+
+### 3. Specialized Commands
+- Search commands (FT.SEARCH)
+- JSON commands (JSON.SET, JSON.GET)
+- Time Series commands (TS.ADD, TS.RANGE)
+- Probabilistic data structures (BF.ADD, CF.ADD)
+
+## Command Processing in Different Clients
+
+### 1. Standalone Client
+- Direct command execution
+- Connection pooling
+- Automatic retries
+
+### 2. Cluster Client
+- Command routing based on key slots
+- MOVED/ASK redirection handling
+- Cross-slot command batching
+
+### 3. Ring Client
+- Command sharding based on key hashing
+- Consistent hashing for node selection
+- Parallel command execution
+
+## Best Practices
+
+1. **Command Batching**
+   - Use pipelines for multiple commands
+   - Batch related commands together
+   - Consider transaction pipelines for atomic operations
+
+2. **Error Handling**
+   - Check command errors after execution
+   - Handle network errors appropriately
+   - Use retries for transient failures
+
+3. **Performance**
+   - Use appropriate command types
+   - Leverage pipelining for bulk operations
+   - Monitor command execution times
+
+4. **Resource Management**
+   - Close connections properly
+   - Use context for timeouts
+   - Monitor connection pool usage
+
+## Common Issues and Solutions
+
+1. **Timeout Handling**
+   - Configure appropriate timeouts
+   - Use context for cancellation
+   - Implement retry strategies
+
+2. **Connection Issues**
+   - Monitor connection pool health
+   - Handle connection failures gracefully
+   - Implement proper cleanup
+
+3. **Command Errors**
+   - Validate commands before execution
+   - Handle Redis-specific errors
+   - Implement proper error recovery
+
+## Monitoring and Debugging
+
+1. **Command Monitoring**
+   - Use SLOWLOG for performance analysis
+   - Monitor command execution times
+   - Track error rates
+
+2. **Client Information**
+   - Monitor client connections
+   - Track command usage patterns
+   - Analyze performance bottlenecks
+
+## Future Improvements
+
+1. **Command Processing**
+   - Enhanced error handling
+   - Improved retry mechanisms
+   - Better connection management
+
+2. **Performance**
+   - Optimized command batching
+   - Enhanced pipelining
+   - Better resource utilization 
diff --git a/docs/redis_pool.md b/docs/redis_pool.md
@@ -0,0 +1,271 @@
+# Redis Connection Pool Implementation
+
+## Overview
+
+The Redis client implements a sophisticated connection pooling mechanism to efficiently manage Redis connections. This document details the implementation, features, and behavior of the connection pool system.
+
+## Core Components
+
+### 1. Connection Interface (`Pooler`)
+
+```go
+type Pooler interface {
+    NewConn(context.Context) (*Conn, error)
+    CloseConn(*Conn) error
+
+    Get(context.Context) (*Conn, error)
+    Put(context.Context, *Conn)
+    Remove(context.Context, *Conn, error)
+
+    Len() int
+    IdleLen() int
+    Stats() *Stats
+
+    Close() error
+}
+```
+
+The `Pooler` interface defines the contract for all connection pool implementations:
+- Connection lifecycle management
+- Connection acquisition and release
+- Pool statistics and monitoring
+- Resource cleanup
+
+### 2. Connection Pool Options
+
+```go
+type Options struct {
+    Dialer func(context.Context) (net.Conn, error)
+
+    PoolFIFO        bool
+    PoolSize        int
+    DialTimeout     time.Duration
+    PoolTimeout     time.Duration
+    MinIdleConns    int
+    MaxIdleConns    int
+    MaxActiveConns  int
+    ConnMaxIdleTime time.Duration
+    ConnMaxLifetime time.Duration
+}
+```
+
+Key configuration parameters:
+- `PoolFIFO`: Use FIFO mode for connection pool GET/PUT (default LIFO)
+- `PoolSize`: Base number of connections (default: 10 * runtime.GOMAXPROCS)
+- `MinIdleConns`: Minimum number of idle connections
+- `MaxIdleConns`: Maximum number of idle connections
+- `MaxActiveConns`: Maximum number of active connections
+- `ConnMaxIdleTime`: Maximum idle time for connections
+- `ConnMaxLifetime`: Maximum lifetime for connections
+
+### 3. Connection Pool Statistics
+
+```go
+type Stats struct {
+    Hits     uint32 // number of times free connection was found in the pool
+    Misses   uint32 // number of times free connection was NOT found in the pool
+    Timeouts uint32 // number of times a wait timeout occurred
+
+    TotalConns uint32 // number of total connections in the pool
+    IdleConns  uint32 // number of idle connections in the pool
+    StaleConns uint32 // number of stale connections removed from the pool
+}
+```
+
+### 4. Main Connection Pool Implementation (`ConnPool`)
+
+#### Structure
+```go
+type ConnPool struct {
+    cfg *Options
+
+    dialErrorsNum uint32 // atomic
+    lastDialError atomic.Value
+
+    queue chan struct{}
+
+    connsMu   sync.Mutex
+    conns     []*Conn
+    idleConns []*Conn
+
+    poolSize     int
+    idleConnsLen int
+
+    stats Stats
+
+    _closed uint32 // atomic
+}
+```
+
+#### Key Features
+1. **Thread Safety**
+   - Mutex-protected connection lists
+   - Atomic operations for counters
+   - Thread-safe connection management
+
+2. **Connection Management**
+   - Automatic connection creation
+   - Connection reuse
+   - Connection cleanup
+   - Health checks
+
+3. **Resource Control**
+   - Maximum connection limits
+   - Idle connection management
+   - Connection lifetime control
+   - Timeout handling
+
+4. **Error Handling**
+   - Connection error tracking
+   - Automatic error recovery
+   - Error propagation
+   - Connection validation
+
+### 5. Single Connection Pool (`SingleConnPool`)
+
+```go
+type SingleConnPool struct {
+    pool      Pooler
+    cn        *Conn
+    stickyErr error
+}
+```
+
+Use cases:
+- Single connection scenarios
+- Transaction operations
+- Pub/Sub subscriptions
+- Pipeline operations
+
+Features:
+- Dedicated connection management
+- Error state tracking
+- Connection reuse
+- Resource cleanup
+
+### 6. Sticky Connection Pool (`StickyConnPool`)
+
+```go
+type StickyConnPool struct {
+    pool   Pooler
+    shared int32 // atomic
+    state  uint32 // atomic
+    ch     chan *Conn
+    _badConnError atomic.Value
+}
+```
+
+Features:
+- Connection stickiness
+- State management
+- Error handling
+- Thread safety
+- Connection sharing
+
+### 7. Connection Health Checks
+
+```go
+func (p *ConnPool) isHealthyConn(cn *Conn) bool {
+    now := time.Now()
+
+    if p.cfg.ConnMaxLifetime > 0 && now.Sub(cn.createdAt) >= p.cfg.ConnMaxLifetime {
+        return false
+    }
+    if p.cfg.ConnMaxIdleTime > 0 && now.Sub(cn.UsedAt()) >= p.cfg.ConnMaxIdleTime {
+        return false
+    }
+
+    if connCheck(cn.netConn) != nil {
+        return false
+    }
+
+    cn.SetUsedAt(now)
+    return true
+}
+```
+
+Health check criteria:
+- Connection lifetime
+- Idle time
+- Network connectivity
+- Protocol state
+
+### 8. Error Types
+
+```go
+var (
+    ErrClosed = errors.New("redis: client is closed")
+    ErrPoolExhausted = errors.New("redis: connection pool exhausted")
+    ErrPoolTimeout = errors.New("redis: connection pool timeout")
+)
+
+type BadConnError struct {
+    wrapped error
+}
+```
+
+### 9. Best Practices
+
+1. **Pool Configuration**
+   - Set appropriate pool size based on workload
+   - Configure timeouts based on network conditions
+   - Monitor pool statistics
+   - Adjust idle connection settings
+
+2. **Connection Management**
+   - Proper connection cleanup
+   - Error handling
+   - Resource limits
+   - Health monitoring
+
+3. **Performance Optimization**
+   - Connection reuse
+   - Efficient pooling
+   - Resource cleanup
+   - Error recovery
+
+4. **Monitoring**
+   - Track pool statistics
+   - Monitor connection health
+   - Watch for errors
+   - Resource usage
+
+### 10. Known Issues and Limitations
+
+1. **Performance Considerations**
+   - Lock contention in high-concurrency scenarios
+   - Connection creation overhead
+   - Resource cleanup impact
+   - Memory usage
+
+2. **Resource Management**
+   - Connection leaks in edge cases
+   - Resource cleanup timing
+   - Memory fragmentation
+   - Network resource usage
+
+3. **Error Handling**
+   - Error recovery strategies
+   - Connection validation
+   - Error propagation
+   - State management
+
+### 11. Future Improvements
+
+1. **Performance**
+   - Optimize lock contention
+   - Improve connection reuse
+   - Enhance resource cleanup
+   - Better memory management
+
+2. **Features**
+   - Enhanced monitoring
+   - Better error handling
+   - Improved resource management
+   - Advanced connection validation
+
+3. **Reliability**
+   - Better error recovery
+   - Enhanced health checks
+   - Improved state management
+   - Better resource cleanup 
diff --git a/docs/redis_testing.md b/docs/redis_testing.md
@@ -0,0 +1,146 @@
+# Redis Testing Guide
+
+## Running Tests
+
+### 1. Setup Test Environment
+
+```bash
+# Start Docker containers for testing
+make docker.start
+
+# Stop Docker containers when done
+make docker.stop
+```
+
+### 2. Environment Variables
+
+```bash
+# Redis version and image configuration
+CLIENT_LIBS_TEST_IMAGE=redislabs/client-libs-test:rs-7.4.0-v2  # Default Redis Stack image
+REDIS_VERSION=7.2  # Default Redis version
+
+# Cluster configuration
+RE_CLUSTER=false  # Set to true for RE testing
+RCE_DOCKER=false  # Set to true for Docker-based Redis CE testing
+
+```
+
+### 3. Running Tests
+
+```bash
+
+# Run tests with race detection, as executed in the CI
+make test.ci
+
+
+### 4. Test Coverage
+
+```bash
+# Generate coverage report
+go test -coverprofile=coverage.out
+
+# View coverage report in browser
+go tool cover -html=coverage.out
+```
+
+## Writing Tests
+
+### 1. Basic Test Structure
+
+```go
+package redis_test
+
+import (
+    . "github.com/bsm/ginkgo/v2"
+    . "github.com/bsm/gomega"
+    "github.com/redis/go-redis/v9"
+)
+
+var _ = Describe("Redis Client", func() {
+    var client *redis.Client
+    var ctx = context.Background()
+
+    BeforeEach(func() {
+        client = redis.NewClient(&redis.Options{
+            Addr: ":6379",
+        })
+    })
+
+    AfterEach(func() {
+        client.Close()
+    })
+
+    It("should handle basic operations", func() {
+        err := client.Set(ctx, "key", "value", 0).Err()
+        Expect(err).NotTo(HaveOccurred())
+
+        val, err := client.Get(ctx, "key").Result()
+        Expect(err).NotTo(HaveOccurred())
+        Expect(val).To(Equal("value"))
+    })
+})
+```
+
+### 2. Test Organization
+
+```go
+// Use Describe for test groups
+Describe("Redis Client", func() {
+    // Use Context for different scenarios
+    Context("when connection is established", func() {
+        // Use It for individual test cases
+        It("should handle basic operations", func() {
+            // Test implementation
+        })
+    })
+})
+```
+
+### 3. Common Test Patterns
+
+#### Testing Success Cases
+```go
+It("should succeed", func() {
+    err := client.Set(ctx, "key", "value", 0).Err()
+    Expect(err).NotTo(HaveOccurred())
+})
+```
+
+#### Testing Error Cases
+```go
+It("should return error", func() {
+    _, err := client.Get(ctx, "nonexistent").Result()
+    Expect(err).To(Equal(redis.Nil))
+})
+```
+
+#### Testing Timeouts
+```go
+It("should timeout", func() {
+    ctx, cancel := context.WithTimeout(ctx, time.Millisecond)
+    defer cancel()
+
+    err := client.Ping(ctx).Err()
+    Expect(err).To(HaveOccurred())
+})
+```
+
+### 4. Best Practices
+
+1. **Test Structure**
+   - Use descriptive test names
+   - Group related tests together
+   - Keep tests focused and simple
+   - Clean up resources in AfterEach
+
+2. **Assertions**
+   - Use Gomega's Expect syntax
+   - Be specific in assertions
+   - Test both success and failure cases
+   - Include error checking
+
+3. **Resource Management**
+   - Close connections in AfterEach
+   - Clean up test data
+   - Handle timeouts properly
+   - Manage test isolation