New keeper-only system table with one row per on-disk Raft log file
(changelog_<from>_<to>.bin[.zstd]) tracked by the in-process Keeper.
Mirrors the plumbing of system.keeper_snapshots.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
When set in the server config, this option overrides the session setting
`allow_experimental_analyzer` for `MutationsInterpreter`. Absent (default)
means the session setting is used.
The override is parsed once per config reload in `Server.cpp` and stored as
a tri-state atomic on the shared context, so each mutation only pays an
atomic load instead of re-reading the config.
Add a server setting (`min_allocation_size_to_throw_on_memory_limit`) to
allow any allocation to throw `MEMORY_LIMIT_EXCEEDED`, this should help
with preventing OOMs
Right now, any `new` cannot throw, this is due exception safety issues,
and only explicitly marked can throw `MEMORY_LIMIT_EXCEEDED`.
But this can lead to real OOMs, i.e. when kernel will kill
clickhouse-server, even though we are trying hard to track memory
properly.
Add a server setting `min_allocation_size_to_throw_on_memory_limit`
(UInt64 bytes, default 0). When non-zero, the throwing `operator new`
variants are allowed to raise `MEMORY_LIMIT_EXCEEDED`. _The assumption
is that code with big enough allocations does not exception safety
issues._
Note, places that allows (Allocator)/never allows (malloc)
`MEMORY_LIMIT_EXCEEDED` before are not affected.
ClickHouse-side `CacheBase` adapter that implements RocksDB's `Cache`
API for the UNIQUE KEY index SST block cache. Includes Context
registration, server-side wiring, server settings, ProfileEvents, and
CurrentMetrics.
Settings (read by Server.cpp at startup + reload):
- `unique_key_index_cache_policy` (default `SLRU`)
- `unique_key_index_cache_size_bytes` (default `1 GiB`)
- `unique_key_index_cache_size_ratio` (default `0.5`)
Setting size to 0 disables the cache; `getUniqueKeyIndexCache` returns
nullptr. On `USE_ROCKSDB=0` builds the registration is a graceful no-op.
ProfileEvents:
- `UniqueKeyIndexCacheLookupMicroseconds` — combined Lookup + Insert
wall-clock inside the adapter.
- `UniqueKeyIndexCacheHits` / `UniqueKeyIndexCacheMisses` — per-lookup
hit/miss counters.
CurrentMetrics:
- `UniqueKeyIndexCacheBytes`, `UniqueKeyIndexCacheEntries`.
`updateUniqueKeyIndexCacheConfiguration` logs an INFO line when the
configured cache size changes.
Tests (`gtest_unique_key_index_cache.cpp`): 5 cases pinning wrapper-only
behaviour — insert/lookup with hit/miss counter assertion, slice-key
erase, release-erase semantics, identity-aware replacement protection,
concurrency smoke. Tests that re-verify the dependency contracts
(CacheBase eviction, atomic ref counting) are intentionally not included.
Reviewer: PR 10 is the first production reader consumer of this cache;
the SST-reader-cache invalidation race helpers (removeWithToken*,
replaceIfPresent) and their gtest coverage are out of scope here and
travel with PR 10.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- `arrayAggregation.cpp`, `distanceTransposed.cpp`, `randDistribution.cpp`,
`toDecimalString.cpp`, `geohashesInBox.cpp`, `seriesPeriodDetectFFT.cpp`:
drop the file-wide `-Wdouble-promotion` pragma in favor of explicit
`static_cast` at the actual conversion sites or qualified `std::sqrt`.
- `KeeperDataDumper.cpp`, `LocalServer.cpp`,
`ChecksumForCompressedBlock.cpp`, `tcp_protocol_fuzzer.cpp`,
`Server.cpp`: replace `static` on free functions with anonymous
namespaces, matching the project preference Algunenano flagged.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The query profiler was disabled under sanitizers because libunwind's
async stack unwinding interferes with sanitizer internals — sanitizers
modify %rbp during their own frame-walking, and a profiling signal
arriving mid-unwind would crash in libunwind.
Replace libunwind with abseil's frame-pointer-based stack unwinding
under sanitizers. Abseil's `GetStackTrace` / `GetStackTraceWithContext`
use `NO_SANITIZE_ADDRESS/MEMORY/THREAD` attributes and are safe.
Key changes:
StackTrace.cpp: use `absl::GetStackTrace[WithContext]` under SANITIZER
instead of `unw_backtrace` / libunwind.
QueryProfiler.cpp: remove the SANITIZER throw guards that disabled the
profiler. Skip `hasPHDRCache` check under sanitizers (not needed for
frame-pointer unwinding). Skip `sigsetjmp` protection under sanitizers
(TSan wraps signal delivery, making `siglongjmp` across its frames
unsafe).
Server.cpp: bypass `hasPHDRCache` for `TraceCollector` init under
sanitizers. Under TSan the PHDR cache is not created (TSan intercepts
`dl_iterate_phdr`), but frame-pointer unwinding does not need it.
Defines.h, SettingsQuirks.cpp: enable profiler default sample rate and
remove the quirk that force-zeroed profiler settings under sanitizers.
build_clang_builtin.cmake: build compiler-rt with
`-fno-omit-frame-pointer` and `-DCOMPILER_RT_HAS_FOMIT_FRAME_POINTER_FLAG=OFF`
so that sanitizer runtime frames (e.g. TSan interceptors) have frame
pointers, allowing abseil's walker to traverse them.
sleep.cpp: under TSan, use `nanosleep` instead of `clock_nanosleep`.
TSan intercepts `nanosleep` (setting `in_blocking_func`), which causes
profiling signals to be delivered immediately rather than deferred.
Without this, TSan defers the signal and the saved `ucontext_t` may
point to stale stack memory.
Tests: remove no-tsan/no-asan/no-ubsan/no-msan tags from query
profiler tests. Add a real-time profiler test using `numbers_mt` which
reliably produces meaningful traces under TSan.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
