On the experimental ReaderExecutor path, `initDecryption` read the encryption
header (`N * Header::kSize` bytes) from the source on every open of an encrypted
file — a network GET per column-stream open. Add a process-global cache of the
raw header bytes, keyed by the file's storage path, so repeated opens of the same
file skip that read.
The cache stores opaque bytes (not parsed `FileEncryption::Header`s), so it has no
SSL dependency and compiles in every build; on a hit the executor re-parses the
tiny header. It is a `CacheBase` (SLRU), sized by the new
`encryption_header_cache_size` server setting (default 50 MiB, hot-reloadable) and
cleared by `SYSTEM DROP ENCRYPTION HEADERS CACHE` (also cleared with the other
path-keyed caches, since a reused path could otherwise return stale headers).
Only disk reads populate/consult it: `DiskEncrypted::prepareRead` marks the
pipeline via `ReadPipeline::allowEncryptionHeaderCache`, and the executor receives
the cache only then — url / external-storage reads never cache, so a reused
external path can never serve stale headers.
Adds `CurrentMetrics::EncryptionHeaderCache{Bytes,Entries}`,
`ProfileEvents::EncryptionHeaderCache{Hits,Misses}`, a gtest and a functional test
(`04509`), and the `SYSTEM_DROP_ENCRYPTION_HEADERS_CACHE` access type.
Note, I've tried to use librseq, but looks like it does not worth it,
for read it may get up to 16%, while contended atomic vs uncontended is
30x.
Though rseq can also help with contention between write and read sides,
but it is also neglectible, it is only around 40ns win for p99+, while
complete read of all counters (for metric_log) is 2us vs 38us, yes, it
is 19x more, but it is still only 38us, and unlikely the contention is a
problem there.
v2: switch users to shared counter - 30x faster
v3: use NPROC over MAX_CPUs
v4: remove atomic pointer in ctor
v5: rename shard → per_cpu
v6: cache-line-align ProfileEvents per-CPU rows
v7: Add benchmark for ProfileEvents under 500 threads × 50 groups
v8: Drop librseq from ProfileEvents per-CPU sharding (does not give lots of benefit)
per-cpu atomic
==============
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------------------
BM_ProfileEvents/1000/process_time/real_time 0.948 ms 26.1 ms 694 groups=50 incs/thread/step=1.44092 items_per_second=527.21M/s ns/inc=1.89678ns threads=500 users=1
BM_ProfileEvents/10000/process_time/real_time 5.13 ms 213 ms 110 groups=50 incs/thread/step=90.9091 items_per_second=975.257M/s ns/inc=1025.37ps threads=500 users=1
BM_ProfileEvents/100000/process_time/real_time 38.3 ms 1862 ms 18 groups=50 incs/thread/step=5.55556k items_per_second=1.30592G/s ns/inc=765.745ps threads=500 users=1
rseq
==============
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------------------
BM_ProfileEvents/1000/process_time/real_time 0.875 ms 22.3 ms 753 groups=50 incs/thread/step=1.32802 items_per_second=571.651M/s ns/inc=1.74932ns threads=500 users=1
BM_ProfileEvents/10000/process_time/real_time 4.64 ms 175 ms 156 groups=50 incs/thread/step=64.1026 items_per_second=1076.95M/s ns/inc=928.552ps threads=500 users=1
BM_ProfileEvents/100000/process_time/real_time 34.6 ms 1576 ms 20 groups=50 incs/thread/step=5k items_per_second=1.44688G/s ns/inc=691.142ps threads=500 users=1
v9: Add user_profile_events_per_cpu server setting
v10: Cache-line-align the per-user ProfileEvents counter allocation
v11: Resolve sched_getcpu once per logical ProfileEvents increment
v12: More benchmarks
Address review: the fail-closed migration message for
`insert_deduplication_version` only described the replicated, time-based
window (`replicated_deduplication_window_seconds`). Non-replicated
`MergeTree` deduplication uses `MergeTreeDeduplicationLog`, whose window is
count-based (`non_replicated_deduplication_window` blocks) and persisted on
local disk; a one-hour wait does not drain it. Without running
`compatible_double_hashes` for at least that many inserts, stale legacy
block ids can survive the upgrade and a retried insert may be accepted as
new on non-replicated tables.
Extend both the startup exception text and the `insert_deduplication_version`
server-setting description to spell out the count-based non-replicated window.
Squashes prior branch commits:
- Make `to_throw` settings changeable without restart
- Respect settings in local `clickhouse`
- Add runtime settings update test
- Extend tests and respect `server_settings`
- Condense duplicated `Context` code with X-macros
- Fix style check and warning format messages
getEffectiveSampleProbability reads only ThreadStatus's cached sample config, so it
goes stale whenever the effective tracker parent/config changes without a refresh:
threads parented to total_memory_tracker with no query context, the initiating thread
after ProcessList::insert applies the query's SETTINGS (e.g. memory_profiler_sample_probability
in the SQL SETTINGS clause), detach reparenting back to total_memory_tracker, the main
thread after the server applies total_memory_tracker_sample_probability (set after
MainThreadStatus was created), and MemoryTrackerSwitcher scopes — each kept a stale
value and stopped emitting MemorySample. Re-resolve the cache at every such point:
ThreadStatus construction, applyGlobalSettings (every attach), applyQuerySettings,
ProcessList::insert, detachFromGroup, MemoryTrackerSwitcher (save/re-resolve/restore),
and Server after the total-sampler setters.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Threads defer (de)allocations in a thread-local untracked_memory and flush to the
global MemoryTracker only past max_untracked_memory, so it undercounts by up to
nthreads * max_untracked_memory — tens of GiB with thousands of threads, enough to
overshoot max_server_memory_usage and OOM.
Bound it per-CPU instead. Each CPU keeps two counters, allocated (>= 0) and freed
(<= 0); a thread folds its untracked_memory into the counter for its sign, batched by
per_cpu_untracked_memory_thread_buffer (32 KiB), and must flush once a counter leaves
[-max_per_cpu_untracked_memory, max_per_cpu_untracked_memory]. The deferred total is
then bounded by ncpu * max_per_cpu_untracked_memory regardless of thread count, parked
threads included. The two sides are independent so a deferred free cannot admit a
deferred allocation (which the free would then strand untracked on its own flush).
max_per_cpu_untracked_memory (default 8 MiB, 0 disables) is a new server setting;
max_untracked_memory stays per-thread. Linux only (needs sched_getcpu).
v2: reimplemented — v1 bounded via sum % SLICE on one global counter, which cannot
bound the per-CPU deferred bytes; replaced with the PerCPUMemory class.
v3: two counters instead of one signed net — a signed net let a deferred free cancel
a deferred allocation, stranding it untracked when the free flushed.
v4: restore the per-CPU contribution unconditionally on a failed flush; a gated
re-reserve could drop still-live bytes if another thread took the budget meanwhile.
v5: release the per-CPU contribution only in the caller, not in reserve().
v6: Rename PerCPUMemory::reserve to sync (it sets an absolute value, not a reservation)
v7: Rename net_per_cpu member to slots
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The fail-closed startup message and the setting doc told operators to run
`compatible_double_hashes` for "the longest deduplication window", which is
ambiguous and reads as the week-long async-block retention. The unified hashes
the unified-only build relies on are retained for
`replicated_deduplication_window_seconds` (one hour by default), and the default
windows keep all inserts for that window, which is considered enough for an
insert retry loop. State the one-hour sync window explicitly so the migration
guidance is unambiguous.
Resolve include-region conflicts in programs/local/LocalServer.cpp and
programs/server/Server.cpp: master added <Functions/pointInPolygon.h>
and this branch added <Functions/UserDefined/UserDefinedSQLFunctionFactory.h>
at the same location; keep both (both are used in each file).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The runtime reload guard validated after `config().replace`, so a rejected
`SYSTEM RELOAD CONFIG` had already swapped in the new configuration:
`system.server_settings` would then report an unsupported legacy value even
though the reload failed, and `ConfigReloader` has no rollback hook. Validate
the incoming configuration before `config().replace`, so a rejected reload
leaves the previous valid config untouched. The integration test now asserts
`system.server_settings` still reports `new_unified_hash` after the rejected
reload.
The fail-closed guard validated only the first loadSettingsFromConfig, so a legacy
`insert_deduplication_version` arriving via a ZooKeeper config include (reloaded during startup) or a
runtime `SYSTEM RELOAD CONFIG` could slip past it, leaving the server running unified-only while the
setting still named a legacy mode. Factor the check into a helper and run it after every
server-settings load: the initial load, the ZooKeeper-include reload, and the config reloader.
Throwing in the reloader rejects the offending reload and keeps the previous valid config.
Extend test_insert_deduplication_version_guard with a reload case asserting that SYSTEM RELOAD CONFIG
to a legacy value is rejected and the server stays up on the valid value.
This build writes only the unified insert deduplication hash. An operator left
on a legacy `insert_deduplication_version` (`old_separate_hashes` or
`compatible_double_hashes`) would otherwise silently lose the mixed-version
deduplication contract during a rolling upgrade. Validate the server setting at
startup and refuse to start, with a migration message, unless it is
`new_unified_hash`.
The setting is therefore no longer obsolete: restore it as a normal server
setting, drop it from the obsolete-settings reference, and stop the
`casa_del_dolor` fuzzer from assigning it a value that would abort startup.
Rework `test_parallel_replicas_protocol` to populate the shared table from a
single replica and propagate by replication; the new build no longer shares an
insert deduplication hash with the old replicas, so inserting identical data
from every replica can no longer rely on cross-version deduplication. Add an
integration test asserting the guard fails closed.
CustomResourceManager was the original config-XML-based resource scheduler:
scheduling hierarchies were described in the server config via the `resources`
and `workload_classifiers` sections. It has been superseded by
WorkloadResourceManager, which builds the same hierarchies from SQL
`CREATE RESOURCE` / `CREATE WORKLOAD` entities, so the config-based path is
redundant.
This removes:
- `CustomResourceManager` and `ClassifiersConfig`, and the dispatcher entry in
`createResourceManager`.
- The legacy config-string node-construction machinery that only existed to
support it: `SchedulerNodeFactory`, `registerSchedulerNodes`, the per-node
`register*` functions, every `(config, prefix)` constructor (down through
`SchedulerNodeInfo`, `ISchedulerNode` and the `ITimeSharedNode` /
`ISchedulerConstraint` / `ISchedulerQueue` / `ISpaceSharedNode` bases), and
`emptyConfig`. The scheduler node classes themselves are unchanged and are
still created programmatically by `WorkloadNode`.
- The now-dead `equals(ISchedulerNode *)` configuration-diff virtual (used only
by `CustomResourceManager`) and its overrides, plus the unused
`IResourceManager::updateConfiguration`.
- The `CUSTOM_RESOURCE_MANAGER` thread name.
The `resources` and `workload_classifiers` config sections are now obsolete;
the server logs a warning and ignores them on startup/reload.
Unit tests (`ResourceTest` helper and the time-shared node gtests) now build
nodes via `SchedulerNodeInfo` constructors instead of XML snippets.
The `test_scheduler` integration suite is renamed to `test_scheduler_io` (it is
IO-specific, exercising `READ DISK` / `WRITE DISK` resources, matching the
`test_scheduler_*` naming). Its config-based tests are reworked to define
resources and workloads via SQL; tests already covered by existing SQL-based
tests were dropped rather than duplicated.
Documentation: the config-based `resources` hierarchy and the deprecated
workload classifiers sections are removed from workload-scheduling.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Branch was 1157 commits behind master and the only CI red was the chronic
master-wide `Hung check failed, possible deadlock found` flake (13-70/day on
master itself; this PR is a compile-time no-op under `SANITIZER` so it cannot
affect the `arm_asan_ubsan` stress run). Re-merge to refresh CI on the current
master and to re-run the AI review against the already-updated PR description.
The merge conflicted only in `src/Common/MemoryWorker.cpp`: master evolved the
`memory_worker_dynamic_hard_limit` apply block to skip adjustments smaller than
`min_change_ratio` (1%) of the current limit, still naming the local
`current_hard_limit`. The branch had renamed that local to `installed_hard_limit`
to avoid shadowing the `current_hard_limit` the speculative-reserve block reads
earlier in `updateResidentMemoryThread` (`-Wshadow`/`-Werror`). Kept master's new
threshold logic verbatim and renamed its local to `installed_hard_limit`; the
`LOG_TRACE` already references `installed_hard_limit`. No behavior change.
Verified the resolved `MemoryWorker.cpp` compiles cleanly (`-fsyntax-only`,
`-Werror -Wall -Wextra`) against the merged tree.
The merge with master brought in the multistage distributed queries
feature, whose `ExchangeServer` startup code in `Server.cpp` throws
`Exception(ErrorCodes::NETWORK_ERROR, ...)`. This branch had removed the
`extern const int NETWORK_ERROR;` declaration from `Server.cpp` because
its only previous use (in `Server::createServer`) was extracted into
`src/Server/createServer.cpp`. Restore the declaration so the new use
compiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
`max_remote_read_connections` (the bound on concurrently held long source
connections) was read once at first use, so a config reload never took effect.
Apply it on reload via `Context::reloadLongConnectionLimitConfig`, called from the
server config-reload handler next to the throttler reloads, mirroring
`reloadRemoteThrottlerConfig`. The update routes through `getLongConnectionLimit`
so a first use racing a reload cannot both construct the limit; capacity is set
lock-free via `LongConnectionLimit::setCapacity`.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Mikhail Artemenko