The prior sweep kept " in the exclusion set to guard the SQL keyword "clickhouse", which
over-protected ALL quoted display strings ("clickhouse " << app in --help, install-tool defaults
like DEFAULT_..._SERVER_USER="clickhouse"/getpwnam("clickhouse")/binary-name fallbacks). Redid the
pass dropping " from the exclusion while still skipping the 4 SQL dict-source keyword lines, URLs,
and /clickhouse + .clickhouse.hash paths (kept . / - in the boundary). Renamed the 2 fuzzer files
clickhouse_fuzzer.cpp / clickhouse-template.g (not built; completeness).
Remaining lowercase clickhouse (526) = URLs, SQL SOURCE(CLICKHOUSE) keyword, and ZK/fs path
conventions (/var/lib/clickhouse, /clickhouse/tables) + internal .clickhouse.hash ELF section —
all protocol/external/convention, intentionally kept.
The correct, safe broad rebrand: renames "ClickHouse" only as a whole word (boundary on
BOTH sides), so it rewrites display strings/messages/comments but leaves every compound
C++ symbol intact — `ClickHouseDictionarySource`, `getInfoIfClickHouseDictionarySource`,
`mainEntryClickHouseServer`, `ClickHouseRevision`, `ClickHouse_SOURCE_DIR` are untouched
(an earlier one-sided heuristic mangled suffix symbols like `manageClickHouse`; reverted).
1677 occurrences across 482 files. Compile-safe (string/comment content; symbols preserved).
Still NOT renamed (would be a build-coupled symbol rename, needs build-host iteration): the
199 compound identifiers containing ClickHouse, the `DB::` namespace, the `clickhouse_*`
CMake targets/macros, the lowercase `clickhouse` SQL dictionary-source keyword. Needs build
verification.
programs/main.cpp isClickhouseApp now matches both clickhouse-<app> and datastore-<app>
symlink names (additive — the subcommand form 'datastore server' already worked via the
argv[1] check). So a binary invoked as datastore-server/datastore-client dispatches
correctly. Compile-safe; the CMake alias rename (clickhouse-* -> datastore-* symlinks)
is the structural batch that needs the build host.
The `unit_tests_dbms` gtest binary did not define
`__lsan_default_suppressions`, so OpenSSL EVP method-store allocations
(cached in the provider method store and never freed at exit) were
reported as leaks. This surfaced through the `FunctionsStress` fuzzer,
which registers the `HMAC` function and calls `EVP_MD_fetch` during
registration.
Move the sanitizer default options and LSan suppressions into a shared
`base/base/sanitizer_options.h` so `programs/main.cpp`, the gtest binary,
and the clickhouse fuzzer reuse a single definition. Broaden the
suppressions to anchor on `evp_generic_fetch`, `EVP_MD_fetch`, and
`ossl_method_construct` so the whole fetch path is covered (including
`ossl_provider_set_operation_bit`, which the previous list missed).
CI report:
https://s3.amazonaws.com/clickhouse-test-reports/PRs/106096/69616e065ac177580a0b55b49d448450b117b6b0/unit_tests_asan_ubsan_function_prop_fuzzer/unit_tests_asan_ubsan_function_prop_fuzzer.log
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Without this carve-out, `clickhouse --help` matches the
`startsWith(argv[i], "-h")` rule below it and gets routed into
`clickhouse-client`, which treats anything starting with `-h` as a
`--host` specification. This produces confusing behavior: either a
"Host argument requires value" error or an unintended host parse of
the string `--help`.
Explicit early routing to `mainEntryHelp` ensures that when the sole
argument is `--help`, `-h`, or `-?` the dispatcher prints its help
listing all subcommands and exits with code 0.
Extend the `03823_clickhouse_help_return_codes` test to cover the
three short forms.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
- `printHelp` now takes an `std::ostream &` parameter: explicit `help`
writes to stdout, while `printHelpOnError` (mistyped commands) writes
to stderr, preserving the error-output contract for scripts.
- Shell-quote binary path and user:group in the `--no-sudo` branch of
`clickhouse start` to prevent command injection via crafted CLI args.
- Make the keeper availability check in the help return codes test more
precise: match `clickhouse keeper [args]` exactly instead of bare
`keeper` which could false-match `keeper-converter` etc.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Resolve merge conflicts:
- `base/base/coverage.cpp`: take master's version which removed the old
sanitizer coverage guard/trace functions (replaced by LLVM profile-based
coverage)
- `src/Client/BuzzHouse/AST/SQLProtoStr.cpp`: take master's unified
`CONV_FN(SQLIdentifier, ident)` replacing the individual per-type
converter functions
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The condition on the "unknown file path" error path compared `main_func`
(type `MainFunc`, i.e. `int (*)(int, char **)`) against `printHelp`
(type `void ()`), which is an incompatible function-pointer comparison
and made the branch unreachable.
Compare against `printHelpOnError` instead, which has the correct type
and is the actual default value assigned to `main_func`.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The `__asan_default_options`, `__lsan_default_options`,
`__lsan_default_suppressions`, `__msan_default_options`,
`__tsan_default_options`, and `__ubsan_default_options` functions need
forward declarations to satisfy `-Wmissing-prototypes`.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add `static` to file-local functions and forward declarations
for cross-TU functions to satisfy `-Wmissing-prototypes`.
This ensures all functions have visible prototypes before their
definitions, improving code quality and catching accidental
linkage issues.
Reference: https://github.com/ClickHouse/ClickHouse/issues/76944
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Implements self-contained, per-test source coverage collection using LLVM's
profiling runtime APIs and the binary's own `__llvm_covfun`/`__llvm_covmap`
ELF sections. No external tools (llvm-cov, llvm-profdata) or build artifacts
are required at runtime.
**How it works:**
At server startup (lazily on first `SYSTEM SET COVERAGE TEST` call):
- `readLLVMCoverageMapping("/proc/self/exe")` parses `__llvm_covfun` and
`__llvm_covmap` ELF sections, building a map of `NameRef → (file, line_start, line_end)`.
- `__llvm_covmap` filename tables are matched to `__llvm_covfun` records via
`FilenamesRef = MD5Hash(raw_filename_blob)`, matching LLVM's own scheme.
- The inline coverage mapping per function is decoded (ULEB128 virtual file
mapping, expression table, region sub-arrays) to extract each function's
entry region file and line range.
At each `SYSTEM SET COVERAGE TEST 'name'` call:
- `getCurrentCoveredNameRefs()` iterates `__llvm_profile_data` records,
uses the `CounterPtr` relative offset to find the function's entry counter,
and returns `NameRef` values for all functions with a non-zero entry counter.
- The `CoverageFlushCallback` resolves each `NameRef` to `(file, line_start, line_end)`
via the pre-built map, deduplicates, and inserts a row into `system.coverage_log`.
- `__llvm_profile_reset_counters()` resets all counters for the next test.
**Validated locally with 100 stateless tests:**
- 100 rows in `system.coverage_log`, one per test
- Line ranges validated: 0 inverted ranges, min_line=5, max_line=10900, avg_span≈14
- Test discrimination: `SELECT 1` covers 0 files unique to it; `ALTER TABLE`
covers 1372 files not in SELECT 1; hot-path files (ProfileEvents.cpp,
MergeTreeBackgroundExecutor.cpp) appear in all 100 tests as expected
**New files:**
- `src/Common/LLVMCoverageMapping.h/cpp`: ELF+covmap parser
- `src/Common/CoverageCollection.h/cpp`: lazy map loading + DB insertion
**Changed:**
- `base/base/coverage.h/cpp`: `CoverageFlushCallback` uses `vector<uint64_t>`
(NameRefs) instead of function addresses; `getCurrentCoveredNameRefs()` replaces
`getCurrentCoveredFunctionAddresses()` (FunctionPointer is null in LLVM 21)
Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Two issues remained after the previous fix:
1. The ASan stack-size warning block (4 lines) was only partially
filtered in `clickhouse_proc.py`. The last line:
"For details see https://github.com/google/sanitizers/issues/189"
was not excluded, leaving `sanitizer_hits` non-empty and triggering
a spurious "Unknown error / Lost connection to server" result for
jobs that had no real sanitizer errors. Add the missing exclusion.
2. OpenSSL's EVP method cache (`evp_md_new`, `construct_evp_method`,
`CRYPTO_THREAD_lock_new`) is never freed at process exit. These
objects are allocated when the S3 client initializes HMAC via
`AWS SDK -> HMAC_Init_ex -> EVP_MD_fetch`. LSan reports them as
leaks, causing the server to abort with SIGABRT (`halt_on_error=1`),
which `clickhouse-test` records as "Unknown error / Lost connection
to server". Add suppressions to `__lsan_default_suppressions`.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
`clickhouse local` (used for post-test log collection) was aborting at
exit under ASan+LSan with 45 KB of leaks from OpenSSL provider
initialization (`ossl_provider_new`, `OSSL_PROVIDER_try_load_ex`, etc.).
OpenSSL 3.x intentionally does not free all global state at exit, so
these are false positives.
Added `__lsan_default_suppressions` alongside the existing
`__lsan_default_options` in `programs/main.cpp` to suppress the five
known OpenSSL leak patterns. This fixes the "Unknown error / Lost
connection to server" failures in:
- Stateless tests (amd_asan_ubsan, distributed plan, parallel, 1/2)
- Stateless tests (amd_asan_ubsan, distributed plan, parallel, 2/2)
- Stateless tests (arm_asan_ubsan, azure, sequential)
CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=99657&sha=24b9445c3f9c8dfc420b55f1a2fb369ae17220be&name_0=PR
Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Simplify jemalloc integration by replacing the three-mechanism approach
(`JEMALLOC_IS_MALLOC` + `JEMALLOC_OVERRIDE___LIBC_*` + linker `--wrap`)
with a cleaner two-step design:
1. Compile jemalloc with `je_` prefix so it only exports `je_malloc`,
`je_free`, etc. (matching what Darwin already does).
2. Define our own `malloc`/`free` in `malloc.cpp` that perform memory
tracking and delegate to `je_malloc`/`je_free`. Provide `__libc_*`
aliases on glibc to intercept internal calls.
Changes:
- Enable `JEMALLOC_PREFIX "je_"` in all non-Darwin `jemalloc_internal_defs.h.in`
- Undef `JEMALLOC_IS_MALLOC` and all `JEMALLOC_OVERRIDE___LIBC_*` defines
- Remove `JEMALLOC_OVERRIDE_MEMALIGN`/`VALLOC`/`PVALLOC` from `jemalloc_defs.h`
- Add `JEMALLOC_NO_RENAME` interface define so consumer code sees `je_*` names
- Prefix all jemalloc API calls (`mallctl` -> `je_mallctl`, etc.) in ClickHouse
- Rewrite `memory.h` to call `je_malloc`/`je_free` directly instead of `__real_*`
- Rewrite `malloc.cpp` with tracking wrappers calling `je_*` underneath
- Rewrite `AllocationInterceptors.h` to map `__real_*` -> `je_*` or `::*`
- Remove `__wrap_*` functions from `AllocationInterceptors.cpp`
- Remove `--wrap` linker flags from `CMakeLists.txt`
- Add jemalloc + clickhouse_common_io deps to `clickhouse_malloc` OBJECT library
- And also fix JEMALLOC_CPREFIX for Darwin, that was the only platform that uses JE_MALLOC_CONF (over MALLOC_CONF)
Co-Authored-By: Claude <[email protected]>
- Add `clickhouse docker-init` C++ subcommand that replaces `entrypoint.sh`
for shell-free distroless environments (fork/exec, POSIX-only, no shell deps)
- Add `docker/server/Dockerfile.distroless` and `docker/keeper/Dockerfile.distroless`
as parallel build targets alongside existing ubuntu/alpine variants
- Add `--no-distroless` CI flag and fix package selector to use `.deb` for distroless
- Add structural smoke-test harness under `docker/tests/distroless/`
The distroless images use `gcr.io/distroless/cc-debian12:nonroot` as base,
expose the same ports as ubuntu, run as uid/gid 101, and include a debug
variant (`distroless-debug`) with a busybox shell for troubleshooting.
Commit 66aeef7f59 introduced a reference to `printHelpOnError` which
doesn't exist — the function is called `printHelp`. This caused a
build failure.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
When running `clickhouse path/to/nonexistent.sql` or
`clickhouse local /tmp/aaa.rep`, the error messages were confusing:
the main binary showed the generic "Use one of the following commands"
help, and `clickhouse-local` showed "Positional option is not supported".
Now, if the argument looks like a file path (contains `/` or `.`) but
the file doesn't exist, a clear "no such file" error is shown instead.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
This commit improves the help output for all ClickHouse applications:
- Add `clickhouse help` command that lists all available subcommands
- Return exit code 0 for all help messages (previously some returned 1)
- Output help messages to stdout instead of stderr
- Use "clickhouse <cmd>" format instead of argv[0] in help messages
- Add dynamic `app_name` detection for `clickhouse client` and `clickhouse local`
to show correct invocation name based on how the binary was called
- Add `--no-sudo` option for `clickhouse start` and `clickhouse restart`
for environments without sudo (e.g., Docker containers)
- Add detailed help headers and footers for client and local with usage examples
Closes#58244
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Previously it was only shipped as part of the client package, now it
will be bundled into the binary, and will be available as:
- clickhouse chdig
- clickhouse-chdig
- chdig
Note, that since now chdig will not be linked, we can remove all
quirks for this, that has been done in #79666
zooqueen