go/maintnotifications/e2e/DATABASE_MANAGEMENT.md

Database Management with Fault Injector

This document describes how to use the fault injector's database management endpoints to create and delete KV databases during E2E testing.

Overview

The fault injector now supports two new endpoints for database management:

1. CREATE_DATABASE - Create a new KV database with custom configuration
2. DELETE_DATABASE - Delete an existing KV database

These endpoints are useful for E2E tests that need to dynamically create and destroy databases as part of their test scenarios.

Action Types

CREATE_DATABASE

Creates a new KV database with the specified configuration.

Parameters:

• cluster_index (int): The index of the cluster where the database should be created
• database_config (object): The database configuration (see structure below)

Raises:

• CreateDatabaseException: When database creation fails

DELETE_DATABASE

Deletes an existing KV database.

Parameters:

• cluster_index (int): The index of the cluster containing the database
• bdb_id (int): The database ID to delete

Raises:

• DeleteDatabaseException: When database deletion fails

Database Configuration Structure

The database_config object supports the following fields:

type DatabaseConfig struct {
    Name                            string                   `json:"name"`
    Port                            int                      `json:"port"`
    MemorySize                      int64                    `json:"memory_size"`
    Replication                     bool                     `json:"replication"`
    EvictionPolicy                  string                   `json:"eviction_policy"`
    Sharding                        bool                     `json:"sharding"`
    AutoUpgrade                     bool                     `json:"auto_upgrade"`
    ShardsCount                     int                      `json:"shards_count"`
    ModuleList                      []DatabaseModule         `json:"module_list,omitempty"`
    OSSCluster                      bool                     `json:"oss_cluster"`
    OSSClusterAPIPreferredIPType    string                   `json:"oss_cluster_api_preferred_ip_type,omitempty"`
    ProxyPolicy                     string                   `json:"proxy_policy,omitempty"`
    ShardsPlacement                 string                   `json:"shards_placement,omitempty"`
    ShardKeyRegex                   []ShardKeyRegexPattern   `json:"shard_key_regex,omitempty"`
}

type DatabaseModule struct {
    ModuleArgs string `json:"module_args"`
    ModuleName string `json:"module_name"`
}

type ShardKeyRegexPattern struct {
    Regex string `json:"regex"`
}

Example Configuration

Simple Database

{
  "name": "simple-db",
  "port": 12000,
  "memory_size": 268435456,
  "replication": false,
  "eviction_policy": "noeviction",
  "sharding": false,
  "auto_upgrade": true,
  "shards_count": 1,
  "oss_cluster": false
}

Clustered Database with Modules

{
  "name": "iokv-cluster",
  "port": 11112,
  "memory_size": 1273741824,
  "replication": true,
  "eviction_policy": "noeviction",
  "sharding": true,
  "auto_upgrade": true,
  "shards_count": 3,
  "module_list": [
    {
      "module_args": "",
      "module_name": "ReJSON"
    },
    {
      "module_args": "",
      "module_name": "search"
    },
    {
      "module_args": "",
      "module_name": "timeseries"
    },
    {
      "module_args": "",
      "module_name": "bf"
    }
  ],
  "oss_cluster": true,
  "oss_cluster_api_preferred_ip_type": "external",
  "proxy_policy": "all-master-shards",
  "shards_placement": "sparse",
  "shard_key_regex": [
    {
      "regex": ".*\\{(?<tag>.*)\\}.*"
    },
    {
      "regex": "(?<tag>.*)"
    }
  ]
}

Usage Examples

Example 1: Create a Simple Database

ctx := context.Background()
faultInjector := NewFaultInjectorClient("http://127.0.0.1:20324")

dbConfig := DatabaseConfig{
    Name:           "test-db",
    Port:           12000,
    MemorySize:     268435456, // 256MB
    Replication:    false,
    EvictionPolicy: "noeviction",
    Sharding:       false,
    AutoUpgrade:    true,
    ShardsCount:    1,
    OSSCluster:     false,
}

resp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
if err != nil {
    log.Fatalf("Failed to create database: %v", err)
}

// Wait for creation to complete
status, err := faultInjector.WaitForAction(ctx, resp.ActionID,
    WithMaxWaitTime(5*time.Minute))
if err != nil {
    log.Fatalf("Failed to wait for action: %v", err)
}

if status.Status == StatusSuccess {
    log.Println("Database created successfully!")
}

Example 2: Create a Database with Modules

dbConfig := DatabaseConfig{
    Name:           "modules-db",
    Port:           12001,
    MemorySize:     536870912, // 512MB
    Replication:    true,
    EvictionPolicy: "noeviction",
    Sharding:       true,
    AutoUpgrade:    true,
    ShardsCount:    3,
    ModuleList: []DatabaseModule{
        {ModuleArgs: "", ModuleName: "ReJSON"},
        {ModuleArgs: "", ModuleName: "search"},
    },
    OSSCluster:                   true,
    OSSClusterAPIPreferredIPType: "external",
    ProxyPolicy:                  "all-master-shards",
    ShardsPlacement:              "sparse",
}

resp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
// ... handle response

Example 3: Create Database Using a Map

dbConfigMap := map[string]interface{}{
    "name":            "map-db",
    "port":            12002,
    "memory_size":     268435456,
    "replication":     false,
    "eviction_policy": "volatile-lru",
    "sharding":        false,
    "auto_upgrade":    true,
    "shards_count":    1,
    "oss_cluster":     false,
}

resp, err := faultInjector.CreateDatabaseFromMap(ctx, 0, dbConfigMap)
// ... handle response

Example 4: Delete a Database

clusterIndex := 0
bdbID := 1

resp, err := faultInjector.DeleteDatabase(ctx, clusterIndex, bdbID)
if err != nil {
    log.Fatalf("Failed to delete database: %v", err)
}

status, err := faultInjector.WaitForAction(ctx, resp.ActionID,
    WithMaxWaitTime(2*time.Minute))
if err != nil {
    log.Fatalf("Failed to wait for action: %v", err)
}

if status.Status == StatusSuccess {
    log.Println("Database deleted successfully!")
}

Example 5: Complete Lifecycle (Create and Delete)

// Create database
dbConfig := DatabaseConfig{
    Name:           "temp-db",
    Port:           13000,
    MemorySize:     268435456,
    Replication:    false,
    EvictionPolicy: "noeviction",
    Sharding:       false,
    AutoUpgrade:    true,
    ShardsCount:    1,
    OSSCluster:     false,
}

createResp, err := faultInjector.CreateDatabase(ctx, 0, dbConfig)
if err != nil {
    log.Fatalf("Failed to create database: %v", err)
}

createStatus, err := faultInjector.WaitForAction(ctx, createResp.ActionID,
    WithMaxWaitTime(5*time.Minute))
if err != nil || createStatus.Status != StatusSuccess {
    log.Fatalf("Database creation failed")
}

// Extract bdb_id from output
var bdbID int
if id, ok := createStatus.Output["bdb_id"].(float64); ok {
    bdbID = int(id)
}

// Use the database for testing...
time.Sleep(10 * time.Second)

// Delete the database
deleteResp, err := faultInjector.DeleteDatabase(ctx, 0, bdbID)
if err != nil {
    log.Fatalf("Failed to delete database: %v", err)
}

deleteStatus, err := faultInjector.WaitForAction(ctx, deleteResp.ActionID,
    WithMaxWaitTime(2*time.Minute))
if err != nil || deleteStatus.Status != StatusSuccess {
    log.Fatalf("Database deletion failed")
}

log.Println("Database lifecycle completed successfully!")

Available Methods

The FaultInjectorClient provides the following methods for database management:

CreateDatabase

func (c *FaultInjectorClient) CreateDatabase(
    ctx context.Context,
    clusterIndex int,
    databaseConfig DatabaseConfig,
) (*ActionResponse, error)

Creates a new database using a structured DatabaseConfig object.

CreateDatabaseFromMap

func (c *FaultInjectorClient) CreateDatabaseFromMap(
    ctx context.Context,
    clusterIndex int,
    databaseConfig map[string]interface{},
) (*ActionResponse, error)

Creates a new database using a flexible map configuration. Useful when you need to pass custom or dynamic configurations.

DeleteDatabase

func (c *FaultInjectorClient) DeleteDatabase(
    ctx context.Context,
    clusterIndex int,
    bdbID int,
) (*ActionResponse, error)

Deletes an existing database by its ID.

Testing

To run the database management E2E tests:

# Run all database management tests
go test -tags=e2e -v ./maintnotifications/e2e/ -run TestDatabase

# Run specific test
go test -tags=e2e -v ./maintnotifications/e2e/ -run TestDatabaseLifecycle

Notes

• Database creation can take several minutes depending on the configuration
• Always use WaitForAction to ensure the operation completes before proceeding
• The bdb_id returned in the creation output should be used for deletion
• Deleting a non-existent database will result in a failed action status
• Memory sizes are specified in bytes (e.g., 268435456 = 256MB)
• Port numbers should be unique and not conflict with existing databases

Common Eviction Policies

• noeviction - Return errors when memory limit is reached
• allkeys-lru - Evict any key using LRU algorithm
• volatile-lru - Evict keys with TTL using LRU algorithm
• allkeys-random - Evict random keys
• volatile-random - Evict random keys with TTL
• volatile-ttl - Evict keys with TTL, shortest TTL first

Common Proxy Policies

• all-master-shards - Route to all master shards
• all-nodes - Route to all nodes
• single-shard - Route to a single shard