Files
zeekay 6994cf05aa kv, not redis: rename the package, the extras and the URL scheme
We do not use Redis. The client was a go-redis fork and still said so
everywhere: `package redis`, `redis.NewClient`, `redis://`, `extra/redisotel`.
Consumers wrote `redis.Client` against a Valkey-backed store called KV.

  package redis            -> package kv          (93 decls)
  redis.X                  -> kv.X                (2,794 call sites)
  "redis: …" errors        -> "kv: …"             (206)
  extra/redis{otel,cmd,prometheus,census} -> extra/kv{otel,cmd,prometheus,census}
  redis:// , rediss://     -> kv:// , kvs://      (ParseURL)
  Redis*/‌*Redis* idents    -> KV*/*KV*
  REDIS_VERSION, REDIS_ENDPOINTS_CONFIG_PATH -> KV_*

I previously argued this rename was unsafe because much of it was "wire
contract". That was wrong, and the categorisation shows why: RESP command names
are GET/SET/HSET — they never contain the word. Of 5,771 occurrences only two
kinds were server-visible at all: the ParseURL scheme (we choose what we pass;
our configs already use kv://) and the CLIENT SETINFO lib-name label, which the
server merely records. Nothing here changes a byte of protocol.

Two things deliberately keep the old spelling:
  RediSearch  — names a specific external module the FT.* commands map to.
  StoreDist   — contains the letters by accident (sto-reDis-t), not the word.

Breaking for consumers by design: every import site must move redis.X -> kv.X.
Minor bump, never a major — this module stays v9 forever.
2026-07-26 08:34:19 -07:00

6.5 KiB

E2E Test Scenarios for Push Notifications

This directory contains comprehensive end-to-end test scenarios for KV push notifications and maintenance notifications functionality. Each scenario tests different aspects of the system under various conditions.

⚠️ Important Note

Maintenance notifications are currently supported only in standalone KV clients. Cluster clients (ClusterClient, FailoverClient, etc.) do not yet support maintenance notifications functionality.

Introduction

To run those tests you would need a fault injector service, please review the client and feel free to implement your fault injector of choice. Those tests are tailored for KV Enterprise, but can be adapted to other KV distributions where a fault injector is available.

Once you have fault injector service up and running, you can execute the tests by running the run-e2e-tests.sh script. there are three environment variables that need to be set before running the tests:

  • KV_ENDPOINTS_CONFIG_PATH: Path to KV endpoints configuration
  • FAULT_INJECTION_API_URL: URL of the fault injector server
  • E2E_SCENARIO_TESTS: Set to true to enable scenario tests

Test Scenarios Overview

1. Basic Push Notifications (scenario_push_notifications_test.go)

Original template scenario

  • Purpose: Basic functionality test for KV Enterprise push notifications
  • Features Tested: FAILING_OVER, FAILED_OVER, MIGRATING, MIGRATED, MOVING notifications
  • Configuration: Standard enterprise cluster setup
  • Duration: ~10 minutes
  • Key Validations:
    • All notification types received
    • Timeout behavior (relaxed/unrelaxed)
    • Handoff success rates
    • Connection pool management

2. Endpoint Types Scenario (scenario_endpoint_types_test.go)

Different endpoint resolution strategies

  • Purpose: Test push notifications with different endpoint types
  • Features Tested: ExternalIP, InternalIP, InternalFQDN, ExternalFQDN endpoint types
  • Configuration: Standard setup with varying endpoint types
  • Duration: ~5 minutes
  • Key Validations:
    • Functionality with each endpoint type
    • Proper endpoint resolution
    • Notification delivery consistency
    • Handoff behavior per endpoint type

3. Database Management Scenario (scenario_database_management_test.go)

Dynamic database creation and deletion

  • Purpose: Test database lifecycle management via fault injector
  • Features Tested: CREATE_DATABASE, DELETE_DATABASE endpoints
  • Configuration: Various database configurations (simple, with modules, clustered)
  • Duration: ~10 minutes
  • Key Validations:
    • Database creation with different configurations
    • Database creation with KV modules (ReJSON, search, timeseries, bf)
    • Database deletion
    • Complete lifecycle (create → use → delete)
    • Configuration validation

See DATABASE_MANAGEMENT.md for detailed documentation on database management endpoints.

4. Timeout Configurations Scenario (scenario_timeout_configs_test.go)

Various timeout strategies

  • Purpose: Test different timeout configurations and their impact
  • Features Tested: Conservative, Aggressive, HighLatency timeouts
  • Configuration:
    • Conservative: 60s handoff, 20s relaxed, 5s post-handoff
    • Aggressive: 5s handoff, 3s relaxed, 1s post-handoff
    • HighLatency: 90s handoff, 30s relaxed, 10m post-handoff
  • Duration: ~10 minutes (3 sub-tests)
  • Key Validations:
    • Timeout behavior matches configuration
    • Recovery times appropriate for each strategy
    • Error rates correlate with timeout aggressiveness

5. TLS Configurations Scenario (scenario_tls_configs_test.go)

Security and encryption testing framework

  • Purpose: Test push notifications with different TLS configurations
  • Features Tested: NoTLS, TLSInsecure, TLSSecure, TLSMinimal, TLSStrict
  • Configuration: Framework for testing various TLS settings (TLS config handled at connection level)
  • Duration: ~10 minutes (multiple sub-tests)
  • Key Validations:
    • Functionality with each TLS configuration
    • Performance impact of encryption
    • Certificate handling (where applicable)
    • Security compliance
  • Note: TLS configuration is handled at the KV connection config level, not client options level

6. Stress Test Scenario (scenario_stress_test.go)

Extreme load and concurrent operations

  • Purpose: Test system limits and behavior under extreme stress
  • Features Tested: Maximum concurrent operations, multiple clients
  • Configuration:
    • 4 clients with 150 pool size each
    • 200 max connections per client
    • 50 workers, 1000 queue size
    • Concurrent failover/migration actions
  • Duration: ~15 minutes
  • Key Validations:
    • System stability under extreme load
    • Error rates within stress limits (<20%)
    • Resource utilization and limits
    • Concurrent fault injection handling

Running the Scenarios

Prerequisites

  • Set environment variable: E2E_SCENARIO_TESTS=true
  • KV Enterprise cluster available
  • Fault injection service available
  • Appropriate network access and permissions
  • Note: Tests use standalone KV clients only (cluster clients not supported)

Individual Scenario Execution

# Run a specific scenario
E2E_SCENARIO_TESTS=true go test -v ./maintnotifications/e2e -run TestEndpointTypesPushNotifications

# Run with timeout
E2E_SCENARIO_TESTS=true go test -v -timeout 30m ./maintnotifications/e2e -run TestStressPushNotifications

All Scenarios Execution

./scripts/run-e2e-tests.sh

Expected Outcomes

Success Criteria

  • All notifications received and processed correctly
  • Error rates within acceptable limits for each scenario
  • No notification processing errors
  • Proper timeout behavior
  • Successful handoffs
  • Connection pool management within limits

Performance Benchmarks

  • Basic: >1000 operations, <1% errors
  • Stress: >10000 operations, <20% errors
  • Others: Functionality over performance

Troubleshooting

Common Issues

  1. Enterprise cluster not available: Most scenarios require KV Enterprise
  2. Fault injector unavailable: Some scenarios need fault injection service
  3. Network timeouts: Increase test timeouts for slow networks
  4. TLS certificate issues: Some TLS scenarios may fail without proper certs
  5. Resource limits: Stress scenarios may hit system limits

Debug Options

  • Enable detailed logging in scenarios
  • Use dump = true to see full log analysis
  • Check pool statistics for connection issues
  • Monitor client resources during stress tests