When a Lua script containing a write command is executed against a
read-only replica in a Redis Cluster, the error may contain
"-READONLY You can't write against a read only replica" embedded
within a larger error message rather than starting with "READONLY ".
The existing code only checked for the "READONLY " prefix, missing
this format. This caused shouldRetry() and isReadOnlyError() to fail,
preventing automatic retry and connection pool cleanup.
The fix checks for the exact substring
"-READONLY You can't write against a read only replica" which is
specific enough to avoid false positives while catching the Lua
script error format.
Closes#2770
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* ring: remove external rendezvous dependency by inlining implementation
Replace usage of github.com/dgryski/go-rendezvous with an internal
implementation to eliminate the external dependency and avoid licence
propagation concerns.
The implementation preserves the exact hashing algorithm and behaviour:
- same hash combination: khash ^ nhash
- same mixing function (xorshift* variant)
- same node ordering semantics
No functional changes intended.
* Refactor TestRendezvous to remove randomString function
Replaced function with inline string generation for test cases.
* test: refactor RendezvousHash tests to use Ginkgo and Gomega
* test: improve RendezvousHash test for per-instance determinism and distribution balance
* test: enhance distribution verification in RendezvousHash tests
* Added repository, ref and client image tag logic
* Remove benchmark from build.yml for testing
* Trigger build.yml with push to current branch
* Restore checkout code in build.yml
* Use client iamge test tag input
* Change redis-version input to be not required
* Reset build workflow changes to master
- pubsub.go: use clear() to empty maps
- search_commands.go: use slices.Clone() and maps.Clone()
- command.go: use maps.Clone() in XMessageSliceCmd.Clone()
- options.go: use maps.Keys() + slices.Collect() + slices.Sort()
- osscluster.go: use slices.SortFunc() instead of custom sort type
- proxy_fault_injector_server.go: remove custom min() function
* fix(client): Do not try to access pointer field without a nil check
* fix(client): always make copy of options to prevent access client internals
* Client: return options copy
* ClusterClient: add missing nil options check, copy passed options, return
options copy
* Ring: copy passed options, return options copy
* SentinelClient: copy passed options
* fix(tests): some tests relied on client internal field access
timeout tests now use reflect and unsafe to access clients internal
options
* fix(redisotel): call options method only once for every client
* revert options cloning, but leave nil check in NewClusterClient
reverts e170a61, 9ffb47e, most of the changes in d986ff7
* chore(doc): add note about alteration of options returned from Options()
* revert leftover change, update doc for clients
---------
Co-authored-by: Nedyalko Dyakov <[email protected]>
* Add DoRaw and DoRawWriteTo for raw RESP protocol access
* fix augment comments
* Pipeline retry fix
* Fix Missing NoRetry check in cluster and ring retry loops
* Add DoRaw and DoRawWriteTo for raw RESP protocol access
* fix augment comments
* Pipeline retry fix
* Fix Missing NoRetry check in cluster and ring retry loops
* add dial retry backoff function.
* add readme
* add backoff to wordlist
* add logical consistency check
* fix(cluster): propagate DialerRetryBackoff to per-node clients
ClusterOptions.clientOptions() was not propagating DialerRetryBackoff,
causing the backoff function to be silently ignored when set on a cluster
client.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
---------
Co-authored-by: Nedyalko Dyakov <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
* test: add race test for Options.clone() vs initConn MaintNotificationsConfig write
Demonstrates a data race where Options.clone() reads MaintNotificationsConfig
without acquiring optLock while initConn writes to it concurrently.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: data race in Options.clone() vs initConn MaintNotificationsConfig write
Add baseClient.cloneOpt() that acquires optLock.RLock before cloning options,
preventing a race with initConn which writes MaintNotificationsConfig.Mode
under the same lock. Update withTimeout and newTx to use cloneOpt().
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Ruben de Vries <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: connection closure metrics and enable all metric groups by default
This commit addresses three issues in the redisotel-native metrics implementation:
1. Fix connection closure metrics not appearing in dashboards
- Root cause: CloseConn() in internal/pool/pool.go was not calling
the metricConnectionClosedCallback when closing stale connections
- Added callback invocation in CloseConn() to record 'stale' closures
2. Verify tracing and metrics compatibility
- Added TestTracingAndMetricsCompatibility to document that
redisotel (tracing via AddHook) and redisotel-native (metrics via
SetOTelRecorder) can coexist without conflicts
3. Enable all metric groups by default
- Changed default MetricGroups from connection-basic+resiliency
to MetricGroupAll for comprehensive observability out of the box
Uses the name, desc, unit of `db.client.operation.duration`, `db.client.connection.count`, `db.client.connection.create_time`, `db.client.connection.wait_time`, `db.client.connection.pending_requests` defined by otel/semconv/v1.38.0/dbconv.
The resolves the metric description part of #3730.
Co-authored-by: ofekshenawa <[email protected]>
Adds ClientSetName(ctx, name string) error to *PubSub, allowing
callers to label the underlying connection via CLIENT SETNAME.
The name appears in CLIENT LIST output, useful for identifying
PubSub connections when debugging.
Follows the same pattern as Ping() — acquires mutex, gets conn,
writes command, releases conn.
Note: if the connection drops and reconnects, the name will be
lost and must be set again.
Closes#3708
* Dial timeout is per-retry
Use dial timeout for each dial attempt, rather than
all attempts together.
* some tests
* additional tests
* use parent timeout so we can test our changes
---------
Co-authored-by: Nedyalko Dyakov <[email protected]>
* chore(go): update go version to 1.24
* update readmes and ci workflows
* update dockerfile for proxy-fi-server
* address changed test behaviour
* address changed test behaviour
* First draft. One metric - command duration.
Signed-off-by: Elena Kolevska <[email protected]>
* Adds connection state metrics
Signed-off-by: Elena Kolevska <[email protected]>
* wip
Signed-off-by: Elena Kolevska <[email protected]>
* Add phase 1 metrics
* Add client api configurations
* remove redundent code
* refactor: move otel-metrics example to separate module
* refactor: move otel-metrics example to separate module
* remove internal test
* Resolve merge conflicts with upstream/master
* Fix linting issues
* Fix pool idle connection counter bug
* Add OTel performance benchmarking suite
* Improve perf compatison script
* Add phase 2,3 metrics
* Remove local benchmark tests
* Add new fields to tests
* Change db.client.connection.count and db.client.connection.pending_request metric type from UpAndDownCounter to Async gauge.
* fix golangci lint
* Add piplined commands metrics support
* Redesign otel init method
* Address comments
* Fix golangci-lint errors
* Fix passing nil pointer
* Add metric prefix to callbacks and atomic batch registration
* Add unique name for pool.name attr
* Fix unique pool name for pubsub
* Refactor all hardcoded OTel attribute strings to use constants
* Refactor all hardcoded OTel attribute strings to use constants
* extract metric names and pool names as constants, improve error handling
---------
Signed-off-by: Elena Kolevska <[email protected]>
Co-authored-by: Elena Kolevska <[email protected]>
* chore(tests): add tls tests
* fix: Use ShardPicker for keyless commands instead of random slot
- cmdSlot now returns -1 for keyless commands instead of RandomSlot()
- This allows cmdNodeWithShardPicker to use the configured ShardPicker
(default: RoundRobinPicker) for keyless command routing
- Updated mapCmdsByNode to handle slot == -1 by using ShardPicker
- Updated test to verify round robin distribution works correctly
* chore(image): update clients-test-lib image
* chore(tests): enable tls user auth test
* chore(docker): use proper envs for test image
* chore(image): use latest build
* chore(image): use latest build
* fix(tests): use the correct root cert
* feat(tls): add example app
* chore(docker): address pr comments
* chore(tests): address pr comments
* chore(doc): add README.md in the example app
* fix(tests): do not skip tests
* fix(tests): skip only because of redis version
* Extracts default image name in a var (including osscluster-tls)
Signed-off-by: Elena Kolevska <[email protected]>
* fix(tlstests): remove redundent test
---------
Signed-off-by: Elena Kolevska <[email protected]>
Co-authored-by: Elena Kolevska <[email protected]>
* Add support for Redis HOTKEYS commands
* Skip hotkeys tests for redis version lower than 8.6
* chore(image): use proper ci image for 8.6
* feat(command): hotkeys wip
* fix(tests): cluster hotkeys
* chore(lint): fmt
* Ensure HOTKEYS commands are only available on standalone Client and NOT on ClusterClient, Ring, or pooled clients
* fix(hotkeys): address pr comments
* chore(ci): use latest build
* fix(hotkeys): fix typo in response
---------
Co-authored-by: Nedyalko Dyakov <[email protected]>
Co-authored-by: Nedyalko Dyakov <[email protected]>
Co-authored-by: Elena Kolevska <[email protected]>
* fix(conn): guard against nil MaintNotificationsConfig in initConn
In initConn, line 540 accessed MaintNotificationsConfig.EndpointType
unconditionally, even though line 538 correctly nil-checked
MaintNotificationsConfig. When MaintNotificationsConfig is nil, this
caused a nil pointer dereference panic and left optLock.RLock held,
leading to a deadlock on the next call.
Move the EndpointType access inside a nil guard to match the existing
nil check for the Mode field.
Fixes#3675
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* refactor: use maintNotifEnabled for readability
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* chore(ci): use 8.6-pre
* chore(ci): add 8.6 in ci matrix
* fix(xinfostream): added new fields in the response
* fix(doctests): fix expected output for XInfoStream
* fix(xinfostreamfull): same as xinfostream
* fix(doctests): fix expected output for XInfoStream
* Support hybrid search params vectors
* Support xinfo stream change in tests
* Revert "Support xinfo stream change in tests"
This reverts commit fde9315b50dd495f96bdaac5300c9828ecdee90b.
* Support xinfo stream change in tests
* Fix stream doctest to support the changes in 8.6
---------
Co-authored-by: ofekshenawa <[email protected]>
Co-authored-by: ofekshenawa <[email protected]>
Add DialerRetries and DialerRetryTimeout fields to ClusterOptions, RingOptions,
and FailoverOptions to allow users to configure connection retry behavior for
cluster, ring, and sentinel clients.
These options were previously only available for the standard Client and were
added to connection pools in #3518. This change ensures feature parity across
all client types.
Changes:
- Add DialerRetries and DialerRetryTimeout fields to ClusterOptions
- Add DialerRetries and DialerRetryTimeout fields to RingOptions
- Add DialerRetries and DialerRetryTimeout fields to FailoverOptions
- Update clientOptions() methods to pass through these values
- Update sentinelOptions() method to pass through these values
- Add tests to verify correct pass-through behavior
Fixes issue where ClusterClient, Ring, and FailoverClient users could not
customize connection retry behavior and were stuck with defaults (5 retries,
100ms backoff).
Co-authored-by: Nedyalko Dyakov <[email protected]>
