14 Commits
Author SHA1 Message Date
Michael Kolupaev 0fb7678e3a Merge remote-tracking branch 'origin/master' into kref 2026-07-09 19:17:24 -07:00
uber fbeebfaf10 doing safe config getting for non-Poco applications 2026-07-07 18:24:25 +03:00
Michael KolupaevandClaude Opus 4.8 3daf5230a8 Remove default arguments from virtual methods because someone at google judged them to be too error-prone for everyone everywhere.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-26 12:45:14 -07:00
Michael Kolupaev 23e1df421c Split KeeperStorage into 3 parts 2026-06-25 23:28:06 -07:00
Michael KolupaevandClaude Opus 4.8 5a82478ee2 Pull KeeperStorageStats instead of pushing
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-25 23:28:06 -07:00
Michael KolupaevandClaude Opus 4.8 34189cf6e5 Refactor snapshot serialization
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-25 23:28:06 -07:00
Michael KolupaevandClaude Opus 4.8 20ad6cbe09 Clean up leftovers after removing Keeper RocksDB storage
Follow-up cleanup pass over the previous commit, removing dispatch
machinery that became trivial once `KeeperStorage` stopped being a
template:

- Unwrap the always-true `if constexpr (std::is_same_v<KeeperStorage,
  KeeperStorage>)` in `KeeperStateMachine::apply_snapshot` and drop the
  now-dead RocksDB snapshot-apply branch. De-templating had reduced the
  former `std::is_same_v<Storage, KeeperMemoryStorage>` to a degenerate
  comparison.

- Merge `IKeeperStateMachine` into `KeeperStateMachine`. The abstract
  base existed only to hide the `KeeperStorage` template parameter behind
  a non-templated interface; with a single storage type it is pointless.
  `KeeperStateMachine` now derives from `nuraft::state_machine` directly
  and the former pure-virtual methods become plain members.

- Remove the storage template parameter from the Coordination unit-test
  fixture: `TestParam` and `CoordinationTest` no longer carry a `Storage`
  type (only the compression flag), and the `addNode` / `getUncommittedACLs`
  helpers take `KeeperStorage` directly. Also drop the unused
  `CompressionParam`.

- Fix `keeper-bench`'s `StorageRunner`, which still referenced the
  removed `KeeperMemoryStorage` typedef and `KeeperContext::setRocksDBOptions`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-19 14:00:16 -07:00
Michael Kolupaev c72b547bb3 Print total stats at the end 2026-06-10 14:27:27 -07:00
Michael Kolupaev a589d92f59 README and fix 2026-06-09 12:31:22 -07:00
Michael Kolupaev a859e0d084 Review comments 2026-06-09 00:44:21 -07:00
Michael Kolupaev 879d219071 Review comments 2026-06-01 21:46:23 -07:00
Michael Kolupaev 708734108b lint 2026-05-05 17:07:22 -07:00
Michael Kolupaev b485997a39 lint 2026-05-05 16:54:02 -07:00
Michael KolupaevandClaude Opus 4.7 392b35c425 keeper-bench: add KeeperStorage-only benchmark mode
New `--storage` flag switches keeper-bench into an in-process mode that exercises
`KeeperStorage` directly, without the rest of the keeper server (no network, no
raft, no state machine).

Threading model mirrors a simplified Keeper commit path:
  * N generator threads (`--concurrency`, default 4) produce requests using the
    existing `Generator` (so `setup` + `generator` config is reused).
  * One "preprocess" thread calls `preprocessRequest` for every write (stands in
    for the path that, in real keeper, runs under `raft_server::lock_`).
  * One "commit" thread calls `processRequest` for writes. After each committed
    write, with probability `1/writes_per_read_batch` it drains the pending read
    queue and processes it in one `processLocalRequests` batch.

Inter-thread queues are `NonblockingBoundedQueue` (copied from commit
6a91c30115) with 100µs polling — no condition
variables. The preprocess->commit queue has a small configurable bound
(`storage.preprocess_commit_queue_size`, default 32).

Periodic reports include (for the period): committed writes/s and reads/s,
preprocess / commit-write / commit-read thread utilization, znode count, RSS,
snapshot-mode flag, and per-op-num average preprocess/process ns and ns-per-
list-name.

Every `storage.snapshot_toggle_periods` periods the runner toggles
`enableSnapshotMode`/`disableSnapshotMode` + `clearGarbageAfterSnapshot`, so the
snapshot overhead shows up in the timings.

Refactoring: `Generator`/`PathGetter` previously took a `Coordination::ZooKeeper
&` to resolve `children_of` paths. They now take a `ListChildrenFn` callback;
the existing Runner passes a lambda wrapping the ZooKeeper client, the new
StorageRunner passes a lambda that calls `KeeperStorage::processLocalRequests`.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-18 15:36:53 -07:00