19 Commits
Author SHA1 Message Date
János Benjamin Antal dbaaf36920 Even more finds 2026-05-20 10:04:07 +00:00
Michael Kolupaev eb325bb0f5 Make the periodically printed stats show throughput from the last period rather than the whole run. Remove per-operation latencies (they should all be the same because of batching anyway). 2026-04-07 23:26:11 -07:00
Michael Kolupaev 19a5e151f6 keeper-bench: watches 2026-04-04 20:03:23 -07:00
Michael Kolupaev ce58390c45 Fix 2026-03-25 20:05:07 -07:00
Michael Kolupaev 160fcd4503 Fixes 2026-03-25 12:45:10 -07:00
Michael Kolupaev 0ede0f9c11 keeper-bench: go faster 2026-03-24 19:07:33 -07:00
Antonio AndelicandClaude Opus 4.6 f39a28314e Guard against division by zero in Stats::writeJSON
`Stats::report` already returns early when `elapsed.elapsedSeconds()` is
zero, but `Stats::writeJSON` did not, causing `getThroughput(0)` to
produce infinity values that render invalid JSON.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-10 12:47:34 +01:00
Antonio AndelicandClaude Opus 4.6 a3cea2d34a Fix keeper-bench throughput and latency measurement
Two fixes:

1. Measure request latency in the response callback instead of when
   `collect_request` drains the in-flight deque. Previously the
   `Stopwatch` ran from submission until collection, which at high
   `pipeline_depth` included time the request sat completed in the
   deque waiting to be collected. Now a shared `Stopwatch` is captured
   by the callback and read when the response actually arrives.

2. Compute RPS using wall-clock elapsed time (`Stopwatch`) instead of
   deriving it from `sum(individual_latencies) / concurrency`. The old
   formula did not account for pipelining: with `pipeline_depth=64` it
   reported ~2k RPS while actual wall-clock throughput was ~140k RPS.
   `getThroughput` now takes elapsed seconds directly, and the
   `concurrency` parameter is removed from `report` and `writeJSON`.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-10 11:20:56 +01:00
Antonio AndelicandClaude Opus 4.6 31601a2e87 Batch removeRecursive deletions and lock mutex in report/writeJSON
Restore batched child removal in `removeRecursive` using `multi` requests
in chunks of 1000, reducing network round trips from O(N) to O(N/1000).

Add `std::lock_guard` in `Stats::report` and `Stats::writeJSON` so that
collector fields are consistently protected by `Stats::mutex` regardless
of caller context.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-08 21:43:50 +01:00
Antonio AndelicandClaude Opus 4.6 019c17de1a Fix missing mutex lock in Stats::addOp and correct comment
`addOp` was modifying `op_collectors` without acquiring `Stats::mutex`,
unlike `addRead` and `addWrite` which both lock it. Currently safe
because callers hold `Runner::mutex`, but would race if used from the
log-replay path where only `Stats::mutex` is relied upon.

Also update the comment to reflect the actual invariant: access is
protected by `Stats::mutex`, not `Runner::mutex`.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-06 22:51:42 +01:00
Antonio Andelic 36daeb474d Merge branch 'master' into keeperbench-improv
# Conflicts:
#	programs/keeper-bench/Runner.cpp
#	programs/keeper-bench/Stats.cpp
#	programs/keeper-bench/Stats.h
2026-03-06 20:36:24 +01:00
zlareb1andClaude Sonnet 4.6 c58be47e0e keeper-bench: fix assertion failure and data race in log replay mode
Fix two issues reported by Cursor Bugbot in `Stats::report` and
`requestFromLogExecutor`:

1. `Stats::report` called `getThroughput` unconditionally on both
   collectors after only checking if both are zero. A log with only read
   requests would hit `assert(requests != 0)` inside
   `write_collector.getThroughput`. Now each collector's throughput is
   computed only when its request count is non-zero.

2. `requestFromLogExecutor` only waited for the last request's promise.
   When an executor processes requests across multiple ZooKeeper
   connections (one per session_id), callbacks on earlier connections
   could still be running on I/O threads after `last_request->wait()`
   returned, causing a data race with `report`/`writeJSON` which read
   `Stats` fields without `Stats::mutex`. Fixed by collecting all
   per-request futures and waiting for all of them before returning.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-03-06 20:09:49 +05:30
zlareb1 317e7f247f accommodate claude bot suggestions 2026-03-06 20:09:49 +05:30
zlareb1 03ae870353 Enable JSON metrics for log mode and report errors in metrics reported file 2026-03-06 20:09:48 +05:30
Antonio AndelicandClaude Opus 4.6 33ba2fc5c7 Fix CI: add warmup to codespell ignore list, mark getThroughput const
- codespell flagged `Warmup` as a misspelling; added to ignore list
- clang-tidy `readability-make-member-function-const` on `getThroughput`
- Removed unused `<vector>` include from Stats.h

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-06 11:02:32 +01:00
Antonio AndelicandClaude Opus 4.6 17344a6d17 Improve keeper-bench with pipelining, warmup, per-op stats, and reproducible seeds
- Add `pipeline_depth` and `queue_depth` settings to allow asynchronous
  in-flight requests per consumer thread, hiding round-trip latency
- Add `warmup_seconds` to discard early stats while caches warm up
- Add `generator.seed` for reproducible workloads
- Track per-operation-type stats (Create, Remove, Get, Set, List, Multi)
  in both stderr reports and JSON output
- Add `on_failure_callbacks` to clean up pending paths on failed creates
- Replace `paths_created` set with O(1) swap-and-pop vector+index structure
  for random path removal
- Fix `static` → `thread_local` for RNG instances to avoid data races
- Fix `continue_on_errors` → `continue_on_error` (match actual config key)
- Validate `remove_factor` range, zero weights, and empty path getters
- Add retry limit for unique path name generation
- Simplify `removeRecursive` and deduplicate default ACL construction
- Document all new settings in README

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-06 10:32:03 +01:00
Antonio Andelic dc3d47d689 Enable implicit-int-float-conversion check 2026-01-20 09:35:57 +01:00
Raúl Marín d6ac63c8d2 Style: Forbid the use of quotes for includes (partial) 2025-06-24 13:20:10 +02:00
Nikita Mikhaylov dc7bc42223 Moved some of utils to programs 2024-12-09 21:35:31 +00:00