New `--storage` flag switches keeper-bench into an in-process mode that exercises
`KeeperStorage` directly, without the rest of the keeper server (no network, no
raft, no state machine).
Threading model mirrors a simplified Keeper commit path:
* N generator threads (`--concurrency`, default 4) produce requests using the
existing `Generator` (so `setup` + `generator` config is reused).
* One "preprocess" thread calls `preprocessRequest` for every write (stands in
for the path that, in real keeper, runs under `raft_server::lock_`).
* One "commit" thread calls `processRequest` for writes. After each committed
write, with probability `1/writes_per_read_batch` it drains the pending read
queue and processes it in one `processLocalRequests` batch.
Inter-thread queues are `NonblockingBoundedQueue` (copied from commit
6a91c30115) with 100µs polling — no condition
variables. The preprocess->commit queue has a small configurable bound
(`storage.preprocess_commit_queue_size`, default 32).
Periodic reports include (for the period): committed writes/s and reads/s,
preprocess / commit-write / commit-read thread utilization, znode count, RSS,
snapshot-mode flag, and per-op-num average preprocess/process ns and ns-per-
list-name.
Every `storage.snapshot_toggle_periods` periods the runner toggles
`enableSnapshotMode`/`disableSnapshotMode` + `clearGarbageAfterSnapshot`, so the
snapshot overhead shows up in the timings.
Refactoring: `Generator`/`PathGetter` previously took a `Coordination::ZooKeeper
&` to resolve `children_of` paths. They now take a `ListChildrenFn` callback;
the existing Runner passes a lambda wrapping the ZooKeeper client, the new
StorageRunner passes a lambda that calls `KeeperStorage::processLocalRequests`.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Add `// Ok` comments to `catch (...)` blocks in `Runner.cpp` that handle
exceptions via `handle_request_exception` (which logs and counts errors)
but were flagged by the `catch_all` style check.
Fix seed collision in `MultiRequestGenerator::setSeedImpl`: the inner
`RequestGetter::setSeed` distributed sub-seeds as `seed + i + 1`, which
collided with siblings assigned by the outer `RequestGetter` using the
same scheme. Use a large prime offset (500009) to separate the inner
seed namespace.
https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=98906&sha=latest&name_0=PR
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Previously only `CreateRequestGenerator::rng` was seeded from the
configured seed. Four other RNGs — in `NumberGetter::getNumber`,
`StringGetter::getString` (via `generateRandomString`),
`PathGetter::getPath`, and `RequestGetter::getRequestGenerator` —
used independent `thread_local pcg64(randomSeed())` instances,
making runs non-deterministic even with a fixed seed.
Replace all `thread_local` RNGs with per-instance `mutable pcg64`
members and propagate the seed through the full generator hierarchy:
`Generator` → `RequestGetter` → each `RequestGenerator` subclass →
their `PathGetter`, `StringGetter`, and `NumberGetter` members.
Each component uses a distinct sub-seed offset to avoid correlation.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Add `pipeline_depth` and `queue_depth` settings to allow asynchronous
in-flight requests per consumer thread, hiding round-trip latency
- Add `warmup_seconds` to discard early stats while caches warm up
- Add `generator.seed` for reproducible workloads
- Track per-operation-type stats (Create, Remove, Get, Set, List, Multi)
in both stderr reports and JSON output
- Add `on_failure_callbacks` to clean up pending paths on failed creates
- Replace `paths_created` set with O(1) swap-and-pop vector+index structure
for random path removal
- Fix `static` → `thread_local` for RNG instances to avoid data races
- Fix `continue_on_errors` → `continue_on_error` (match actual config key)
- Validate `remove_factor` range, zero weights, and empty path getters
- Add retry limit for unique path name generation
- Simplify `removeRecursive` and deduplicate default ACL construction
- Document all new settings in README
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Mikhail Artemenko