mirror of
https://github.com/hanzoai/datastore.git
synced 2026-09-26 13:55:07 +00:00
zlareb1andClaude Sonnet 4.6
35e48faf43
Adds a stress testing framework for ClickHouse Keeper under `tests/stress/keeper/`, wired into CI (PR gate + nightly workflows). Key components: - Workloads: `keeper_bench.py` runner with YAML-defined workload profiles (`prod_mix.yaml`, `read_requests.yaml`, `write_requests.yaml`, etc.) - Scenarios: `core_no_faults.yaml` and `core_faults.yaml` with netem-based network fault injection (latency, jitter, packet loss) - Fault framework: `faults/network.py` (netem), `faults/base.py` (step dispatcher with `kind: parallel` support) - Metrics: `MetricsSampler` collects keeper stats, `sink.py` writes JSONL, ingested into CIDB via `ingest_keeper_metrics.py` post-hook - Gates: scenario pass/fail conditions based on RPS, p99 latency, error rate - Grafana dashboards for run details, 1vs1 comparison, historical progression CI integration: - PR job: 3 no-fault scenarios (prod-mix, read-multi, write-multi), ARM_LARGE, 15 min each, runs when `src/Coordination/` or stress test files change - Nightly (no-faults): all no-fault scenarios, default + rocks backends, ARM_LARGE - Nightly (faults): all fault scenarios, default + rocks backends, ARM_LARGE - Single `keeper_stress_job` config; mode detected via `Info().pr_number` and `Info().workflow_name` inside the job script Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
35 lines
1.4 KiB
Python
35 lines
1.4 KiB
Python
from keeper.framework.core.settings import CLIENT_PORT
|
|
|
|
|
|
def servers_arg(nodes, in_container=False):
|
|
"""Build server address string for keeper-bench.
|
|
|
|
Keeper-bench always runs on host (in_container=False): uses keeper_client_host_port
|
|
when the cluster published the client port; otherwise ip_address and client_port.
|
|
For ZooKeeper (ZooNodeWrapper), that yields host-reachable IP:2181 per node.
|
|
|
|
Args:
|
|
nodes: List of ClickHouseCluster node instances or ZooNodeWrapper.
|
|
in_container: If True and ZooKeeper nodes, return Docker hostnames (zoo1:2181).
|
|
Kept for compatibility; callers typically use False.
|
|
|
|
Returns:
|
|
Space-separated string of "host:port" addresses.
|
|
"""
|
|
if in_container and nodes and getattr(nodes[0], "is_zookeeper", False):
|
|
# Use zk_name when set (ZKBackedNode) so we get zoo1:2181, not keeper1:2181
|
|
return " ".join(
|
|
f"{getattr(n, 'zk_name', n.name)}:{getattr(n, 'client_port', CLIENT_PORT)}" for n in nodes
|
|
)
|
|
addrs = []
|
|
for n in nodes or []:
|
|
hp = getattr(n, "keeper_client_host_port", None)
|
|
if hp is not None:
|
|
addrs.append(f"localhost:{int(hp)}")
|
|
else:
|
|
ip = getattr(n, "ip_address", None)
|
|
if ip:
|
|
port = getattr(n, "client_port", None) or CLIENT_PORT
|
|
addrs.append(f"{ip}:{int(port)}")
|
|
return " ".join(addrs)
|