When set in the server config, this option overrides the session setting
`allow_experimental_analyzer` for `MutationsInterpreter`. Absent (default)
means the session setting is used.
The override is parsed once per config reload in `Server.cpp` and stored as
a tri-state atomic on the shared context, so each mutation only pays an
atomic load instead of re-reading the config.
Add a server setting (`min_allocation_size_to_throw_on_memory_limit`) to
allow any allocation to throw `MEMORY_LIMIT_EXCEEDED`, this should help
with preventing OOMs
Right now, any `new` cannot throw, this is due exception safety issues,
and only explicitly marked can throw `MEMORY_LIMIT_EXCEEDED`.
But this can lead to real OOMs, i.e. when kernel will kill
clickhouse-server, even though we are trying hard to track memory
properly.
Add a server setting `min_allocation_size_to_throw_on_memory_limit`
(UInt64 bytes, default 0). When non-zero, the throwing `operator new`
variants are allowed to raise `MEMORY_LIMIT_EXCEEDED`. _The assumption
is that code with big enough allocations does not exception safety
issues._
Note, places that allows (Allocator)/never allows (malloc)
`MEMORY_LIMIT_EXCEEDED` before are not affected.
- `arrayAggregation.cpp`, `distanceTransposed.cpp`, `randDistribution.cpp`,
`toDecimalString.cpp`, `geohashesInBox.cpp`, `seriesPeriodDetectFFT.cpp`:
drop the file-wide `-Wdouble-promotion` pragma in favor of explicit
`static_cast` at the actual conversion sites or qualified `std::sqrt`.
- `KeeperDataDumper.cpp`, `LocalServer.cpp`,
`ChecksumForCompressedBlock.cpp`, `tcp_protocol_fuzzer.cpp`,
`Server.cpp`: replace `static` on free functions with anonymous
namespaces, matching the project preference Algunenano flagged.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The query profiler was disabled under sanitizers because libunwind's
async stack unwinding interferes with sanitizer internals — sanitizers
modify %rbp during their own frame-walking, and a profiling signal
arriving mid-unwind would crash in libunwind.
Replace libunwind with abseil's frame-pointer-based stack unwinding
under sanitizers. Abseil's `GetStackTrace` / `GetStackTraceWithContext`
use `NO_SANITIZE_ADDRESS/MEMORY/THREAD` attributes and are safe.
Key changes:
StackTrace.cpp: use `absl::GetStackTrace[WithContext]` under SANITIZER
instead of `unw_backtrace` / libunwind.
QueryProfiler.cpp: remove the SANITIZER throw guards that disabled the
profiler. Skip `hasPHDRCache` check under sanitizers (not needed for
frame-pointer unwinding). Skip `sigsetjmp` protection under sanitizers
(TSan wraps signal delivery, making `siglongjmp` across its frames
unsafe).
Server.cpp: bypass `hasPHDRCache` for `TraceCollector` init under
sanitizers. Under TSan the PHDR cache is not created (TSan intercepts
`dl_iterate_phdr`), but frame-pointer unwinding does not need it.
Defines.h, SettingsQuirks.cpp: enable profiler default sample rate and
remove the quirk that force-zeroed profiler settings under sanitizers.
build_clang_builtin.cmake: build compiler-rt with
`-fno-omit-frame-pointer` and `-DCOMPILER_RT_HAS_FOMIT_FRAME_POINTER_FLAG=OFF`
so that sanitizer runtime frames (e.g. TSan interceptors) have frame
pointers, allowing abseil's walker to traverse them.
sleep.cpp: under TSan, use `nanosleep` instead of `clock_nanosleep`.
TSan intercepts `nanosleep` (setting `in_blocking_func`), which causes
profiling signals to be delivered immediately rather than deferred.
Without this, TSan defers the signal and the saved `ucontext_t` may
point to stale stack memory.
Tests: remove no-tsan/no-asan/no-ubsan/no-msan tags from query
profiler tests. Add a real-time profiler test using `numbers_mt` which
reliably produces meaningful traces under TSan.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
`DNSResolver::Impl::Impl()` was eagerly running
`Poco::Net::DNS::hostName()` followed by `hostByName` on the local
hostname. The cached result is consumed only by
`updateHostNameAndAddresses` (a server-only periodic task) and by
`getHostName`, which has its own lazy fallback. Yet the work happened
inside a process-wide Meyers singleton, so every program that calls
`DNSResolver::instance` paid the cost — including `clickhouse-client`,
`-local`, `-keeper`, and `-disks`.
When the local hostname could not be resolved (a normal occurrence on
fresh CI/EC2 runners where the auto-generated `ip-X-X-X-X` name is
neither in `/etc/hosts` nor served by the VPC resolver), the catch
block in the constructor logged a stack trace at error level. The
fast-test runner treats any stderr output as a test failure, so this
infrastructure flake masqueraded as a per-test failure.
Move the eager priming out of the singleton constructor:
* `Impl::Impl` is now defaulted; non-server programs no longer touch
the local hostname during startup.
* `programs/server/Server.cpp` calls `updateHostNameAndAddresses`
explicitly before starting `DNSCacheUpdater`, so the first cache
tick on a healthy server still sees a populated value and does not
spuriously emit `IPs of host name X have been changed`.
Behaviour from PR #93726 (cluster config reload on host IP change) is
preserved — verified by `test_reload_cluster_config_if_host_address_change`,
which still passes.
Failing fast tests in https://github.com/ClickHouse/ClickHouse/pull/103689
(report:
https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=103689&sha=3a119a57269adba3557204ce44e58a65054e31df&name_0=PR&name_1=Fast%20test
)
include `Tests/02001_hostname_test`, `Tests/03213_deep_json`,
`Tests/03638_merge_max_dynamic_subcolumns_in_compact_part`, and
`Tests/02433_default_expression_operator_in`, all hitting the same
`Cannot resolve host (ip-172-31-33-233)` startup path.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Drop two heuristics added during PR development that override explicit
user-configured ratios on small systems:
- Forced uplift of `max_server_memory_usage_to_ram_ratio` to `0.95` when
RAM < 4 GiB.
- Forced cap of `merges_mutations_memory_usage_to_ram_ratio` to `0.1`
when RAM < 4 GiB.
Per reviewer feedback: too ad-hoc, removing for now. The remaining
auto-tuning (Parquet watermark, INSERT block size, background pool,
jemalloc dirty decay) is sufficient for the low-memory case and does not
silently override explicit operator settings.
