13,838 upstream commits, forward from the 26.6.1.1 fork point. 129 conflicts:
37 resolved mechanically (take upstream's file, re-apply the brand sweep) and
92 by hand where our fork and upstream had both changed the same code.
The rule throughout: upstream owns the engine, we own the naming and a small set
of deliberate deviations. Never resolve by discarding an upstream change — a
dropped rename or a missed new parameter becomes a compile error a thousand
files away.
Deviations verified intact after the merge:
ENABLE_XRAY still defaults OFF (CMakeLists.txt:105)
X-Datastore-User read by the server (authenticateUserByHTTP.cpp)
X-ClickHouse-* in src/ 0
hanzo/research present
system.*_log retention bound present
ArrowFlight rip held against upstream re-adding it
Two things worth naming, because neither shows up as a conflict:
src/CMakeLists.txt gained an UNGUARDED
`add_object_library(datastore_storages_arrowflight Storages/ArrowFlight)` from
upstream, for a directory our gRPC rip deleted. It sat outside every conflict
hunk, so a resolver reading only the markers would sail straight past it into
a CMake configure error. Removed; the Server/ArrowFlight sibling at line 474
stays because it is inside `if (TARGET ch_contrib::arrow_flight)`.
The brand sweep had rewritten github.com/ClickHouse/ClickHouse issue links to
github.com/Datastore/Datastore — a dead host. Those point at upstream's own
tracker and are restored, alongside the contributors list, sql.clickhouse.com
and the ClickHouse Inc copyright: attribution is theirs, not ours to rename.
base/glibc-compatibility/memcpy/memcpy.h is deleted, not kept: upstream dropped
it for LLVM-libc's, and our only change to it was a comment rebrand.
`Client::login` read `hosts_and_ports.front()`, but `hosts_and_ports` is only
filled from explicit `--host` arguments; the default host is added later, in
`Client::connect`. Running `clickhouse client --login` without `--host` (e.g.
with the host coming from `--connection`, the config file, or the
`CLICKHOUSE_HOST` environment variable) failed a libc++ hardening assertion:
`front() called on an empty vector`.
Resolve the host from the configuration when `hosts_and_ports` is empty, the
same way `connect` does.
Also fix the client docs, which listed the option as `login` instead of
`--login`.
Closes: https://github.com/ClickHouse/ClickHouse/issues/103603
Co-Authored-By: Claude Fable 5 <[email protected]>
The per-query session_timezone seed reads client_local_timezone, captured
once in Client::connect(). It was captured via DateLUT::instance(), which
folds in session_timezone: starting the client with an explicit
--session_timezone (on a host whose real zone differs) cached that override
value instead of the machine-local zone. A later SET session_timezone =
DEFAULT then reseeded future queries from the cached override, so server-side
literal parsing (async INSERT, SELECT) never returned to true
use_client_time_zone semantics.
Use DateLUT::serverTimezoneInstance().getTimeZone() instead, which reads the
process default directly and ignores session_timezone. On the client with
use_client_time_zone=1 setDefaultTimezone() is never called, so the process
default stays the machine-local zone captured at DateLUT construction.
Extends 04401_async_insert_use_client_time_zone to cover the
--session_timezone=UTC start followed by SET session_timezone = DEFAULT.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The previous change seeded session_timezone once in Client::connect(), so it
only reflected the use_client_time_zone value at connect time. On an already
open connection, SET use_client_time_zone / per-query SETTINGS change the
setting without reconnecting, leaving two holes:
- turning use_client_time_zone on mid-session did not seed session_timezone,
so server-side literal parsing kept using the server time zone;
- turning it off left the auto-seeded session_timezone in place, so
server-side parsing never switched back.
Capture the client local time zone once on the first connect (before the
branch that may switch the process default to the server time zone), and seed
session_timezone from it in processParsedSingleQuery for every query whose
effective use_client_time_zone is set and that does not set session_timezone
explicitly. The seed is applied inside the transient per-query settings scope,
so it is recomputed for each query and reverted afterwards, tracking
use_client_time_zone changes in both directions without leaking state.
Extend the regression test to cover the command line flag, a mid-session SET,
a per-query SETTINGS, and turning use_client_time_zone back off.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
With --use_client_time_zone=1 the client parses DateTime string literals in its
local time zone, but literals interpreted server-side (asynchronous INSERT, SELECT
literals) use the session_timezone setting, which the client left empty. The server
then fell back to its own default time zone, so an async INSERT of a string literal
stored a different instant than the equivalent synchronous INSERT.
Set session_timezone to the client local time zone in Client::connect() when
use_client_time_zone is enabled and the user has not set session_timezone explicitly,
so server-side interpretation matches the client-side (synchronous) path. An explicit
--session_timezone still takes priority.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The correct, safe broad rebrand: renames "ClickHouse" only as a whole word (boundary on
BOTH sides), so it rewrites display strings/messages/comments but leaves every compound
C++ symbol intact — `ClickHouseDictionarySource`, `getInfoIfClickHouseDictionarySource`,
`mainEntryClickHouseServer`, `ClickHouseRevision`, `ClickHouse_SOURCE_DIR` are untouched
(an earlier one-sided heuristic mangled suffix symbols like `manageClickHouse`; reverted).
1677 occurrences across 482 files. Compile-safe (string/comment content; symbols preserved).
Still NOT renamed (would be a build-coupled symbol rename, needs build-host iteration): the
199 compound identifiers containing ClickHouse, the `DB::` namespace, the `clickhouse_*`
CMake targets/macros, the lowercase `clickhouse` SQL dictionary-source keyword. Needs build
verification.
Implement https://github.com/ClickHouse/ClickHouse/issues/95479.
Previously, the client echoed queries differently depending on the mode
with no way to control it granularly: in interactive mode it always
printed a formatted and highlighted version of the query and the
`query_id`, while in batch mode `--echo` printed the raw query text only.
This change makes the behavior configurable while keeping the defaults
identical to before:
- `--echo` now takes an optional boolean value and controls echoing of
queries in both interactive and batch mode. It defaults to on in
interactive mode and off in batch mode.
- `--echo-formatted` is a new boolean option controlling whether the
echoed query is formatted. It defaults to on in interactive mode and
off in batch mode, so highlighting non-formatted queries is now
possible as well.
- `--echo-query-id` is a new boolean option controlling whether the
`query_id` is printed before execution. It defaults to on in
interactive mode and off in batch mode. When enabled in batch mode, a
fresh `query_id` is generated per query so that the printed value
matches the one used for execution.
- `--hilite` (`--highlight`) now controls highlighting of both the
command prompt and the echoed queries. Highlighting is only applied
when writing to a terminal.
The echoing logic for all modes is unified in the new `echoQuery`
method, and the per-client defaults are resolved in
`setupEchoAndHighlightSettings`.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
A positional file argument is converted to --queries-file inside
ClientApplicationBaseParser via options.emplace, which populates the
boost options map (and the queries_files member) but never the Poco
config. The conflict check in processConfig used config.has("queries-file"),
so the positional case slipped through and the -q query was silently
dropped instead of erroring.
Check queries_files directly so both forms are rejected consistently.
Closes#104735
* Make client rainbow parentheses optional
* Make client rainbow parentheses optional
* Make client rainbow parentheses optional
* Make client rainbow parentheses optional
* Fix style and clang-tidy CI failures
- In `programs/format/Format.cpp`, replace `options.count(...)` with
`options.contains(...)` to satisfy the `readability-container-contains`
clang-tidy check that fires in the `arm_tidy` build.
- In `src/Client/ClientBaseHelpers.cpp`, drop redundant braces around
single-statement `if (!rainbow_parentheses) ...` blocks. One of them
had the opening brace on the same line, which the C++ style check
rejects (Allman braces are required).
CI report: https://github.com/ClickHouse/ClickHouse/pull/103851
---------
Co-authored-by: Alexey Milovidov <[email protected]>
`Client::getHelpFooter` interpolated `{0}` (= `app_name`) into the note
about the dashed alias. When invoked as `clickhouse client --help`,
`app_name` is `"clickhouse client"`, so the line read
"use 'clickhouse client' invocation with a dash" - self-contradictory
and not pointing at the actual dashed alias.
Use the literal `clickhouse-client` for that specific reference so the
note is accurate regardless of how the client is invoked.
"or as interactive client" -> "or as an interactive client"
in `Client::getHelpHeader`.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The original PR #96025 was reverted (#98038) because:
1. The install completion message showed wrong relative paths like
`var/run/clickhouse-server` instead of absolute `/var/run/clickhouse-server`.
This happened because `options.contains()` returns true even for options
with default values in boost::program_options. Fix: use `defaulted()`
to only include options that the user explicitly specified.
2. The install completion message did not suggest `sudo` for the
start/restart command. This happened because `getuid() != 0` is false
when install runs as root via sudo, but the user returns to their
unprivileged shell afterward. Fix: always suggest sudo (matching the
original behavior).
Also address review feedback:
- Fix `clickhouse-client` help: remove misleading "initial table definition"
(a `clickhouse-local` concept) from usage line
- Fix typo: "as command line tool" → "as a command line tool"
- Fix `clickhouse-local` help grammar: "After you can execute" → "After that,
you can execute", "in first query" → "in the first query"
- Make keeper `--help` test robust for builds without `ENABLE_CLICKHOUSE_KEEPER`
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- FreeBSD: guard `collapseDemangledNames` with `#if` matching its call site
(the function is only used when `!OS_FREEBSD`)
- ppc64le: suppress `-Wdisabled-macro-expansion` for poco PCRE (third-party code)
- musl: suppress `-Wdisabled-macro-expansion` in `Client.cpp` (musl defines
`stderr` as a self-referential macro)
- arm_fuzzers: add forward declarations for `LLVMFuzzerTestOneInput` and
`LLVMFuzzerInitialize` to satisfy `-Wmissing-prototypes`
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Processing many small inserts over the native protocol is suboptimal because
the server replies with the table structure and column defaults for every query.
This adds:
- A setting `send_table_structure_on_insert_with_inline_data` (default true). When disabled,
the server will parse inline data in INSERT queries itself instead of sending
table structure back to the client.
- A `--inline-insert-data` command-line option for `clickhouse-client`. When set,
the client sends the data as is in the INSERT query text instead of converting
it to blocks in the native format, and routes the query through the ordinary
query path (no round-trip for table structure).
The server already has the machinery to process inline INSERT data (used by the
HTTP interface and async inserts). This change reuses it for the native protocol.
https://github.com/ClickHouse/ClickHouse/issues/91630
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add `static` to file-local functions and forward declarations
for cross-TU functions to satisfy `-Wmissing-prototypes`.
This ensures all functions have visible prototypes before their
definitions, improving code quality and catching accidental
linkage issues.
Reference: https://github.com/ClickHouse/ClickHouse/issues/76944
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add named `Throttler` instances with LOG_TRACE logging to diagnose
throttling issues (follow-up to #95127). The `Throttler` class now
takes a `std::string_view` name as its first constructor parameter.
Logging covers all state mutations:
- Sleeping: amount, count, tokens, max_speed, block_ns, max_block_ns, sleep_ns
- Not sleeping (when throttler is active): amount, count, tokens, max_speed
- `reset`: count, tokens, max_burst before clearing
- `setMaxSpeed`: old/new speed, tokens
Fix the flaky test by removing the `native_copy` case — with
`allow_s3_native_copy=1` S3 performs a server-side copy and ClickHouse
never reads/writes the data, so throttling cannot apply and wall clock
time is unpredictable.
https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=95118&sha=ab1adaa5d88f38a4934fac9fc8ffb91e97ef5a6f&name_0=PR&name_1=Stateless%20tests%20%28arm_binary%2C%20parallel%29
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Remove `CurrentThread.h` from high-fan-out headers: `ProcessList.h`,
`IQueryPlanStep.h`, `DDLWorker.h`, `TCPHandler.h`, `AggregatingTransform.h`,
`ParallelParsingInputFormat.h`, `StorageTableFunction.h`, `parseSyslogLevel.h`,
`EmbeddedRocksDBBulkSink.h`, `BlockIO.h`, `LocalConnection.h`,
`StorageMaterializedView.h`, `HTTPHandler.h`
- Extract `QueryScope` from `CurrentThread` into standalone `Common/QueryScope.h`
with its own `.cpp`, replacing the nested `CurrentThread::QueryScope` class
- Extract `getCurrentThreadGroup` free function into `Common/ThreadGroupSwitcher.h`,
replacing `CurrentThread::getGroup()` in template headers:
`threadPoolCallbackRunner.h`, `ParallelFormattingOutputFormat.h`,
`HashedDictionaryParallelLoader.h`, `UniqExactSet.h`, `HashedDictionary.h`,
`StatusRequestsPool.h`
- De-inline `MemoryTrackerSwitcher` ctor/dtor into `MemoryTrackerSwitcher.cpp`
- De-inline `ResourceGuard::Request::wait` into `ResourceGuard.cpp`
- De-inline `ManyAggregatedData` destructor into `AggregatingTransform.cpp`
- De-inline `QueryStatus::updateProgressIn/Out` into `ProcessList.cpp`
- Add explicit `CurrentThread.h` includes to `.cpp` files that relied on
transitive includes through the trimmed headers
Co-Authored-By: Claude Opus 4.6 <[email protected]>
clang-tidy `bugprone-empty-catch` fires on catch blocks with empty
bodies regardless of the exception type. Re-add the NOLINT annotation
on the catch lines that were previously suppressed with `catch (...)`
and now use specific exception types.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replace blanket `catch (...)` with specific exception types based on
analysis of what each try block can actually throw:
- `const Exception &` for catch blocks where only ClickHouse operations
are involved (Field operations, expression evaluation, parsing, etc.)
- `const std::exception &` for catch blocks involving standard library,
Poco, or third-party code (filesystem, HTTP, thread creation, etc.)
Blocks that genuinely need `catch (...)` are left unchanged:
- try-pattern functions (`tryGetPosition`, `tryDeserializeText`, etc.)
- Exception formatting utilities in `Exception.cpp`
- Signal/terminate handlers
- FFI callbacks (DeltaLake)
- Dual-handler patterns (already have specific + catch-all)
- `noexcept` functions with user-supplied callbacks
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Enhance the `catch_all` style check in two ways:
1. Recognize more legitimate exception-handling patterns as "handled":
`getCurrentExceptionMessage`, `getCurrentExceptionCode`,
`getCurrentExceptionMessageAndPattern`, `ExecutionStatus::fromCurrentException`,
`onBackgroundException`, `storeException`, `STDERR_FILENO`, `writeRetry`,
`handle_exception`, `handleException`, `finishWithException`.
2. Fix function-try-block detection for destructors like
`Exception::~Exception() try { } catch (...) { }` — the enclosing-scope
scan now falls through to the function-try-block scan when it hits a
namespace/class scope (no parentheses in signature).
For the remaining 108 catch-all blocks, add `// Ok: <reason>` comments
explaining why each is intentional. Categories include:
- Try-patterns (`tryDeserializeText`, `tryGetPosition`, etc.)
- Exception handling/reporting utilities (`Exception.cpp`)
- Non-essential system probes (`Server.cpp` Linux /proc and /sys checks)
- C callbacks where exceptions cannot propagate (zip archives)
- Background thread exception saving (`onBackgroundException`)
- Fallback/retry logic (`DatabaseOverlay`, `ReadSchemaUtils`)
- Best-effort cleanup or error reporting
- Tests that expect exceptions
Replace `// NOLINT(bugprone-empty-catch)` with `// Ok:` comments on all
catch blocks that were flagged by the new check. Fix Allman-style brace
violations in `convertFieldToType.cpp` and `ClientBase.cpp`.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Adapted the `ssh-key-file` check to the new retry loop structure:
added `!config().has("ssh-key-file")` to the `should_ask_password` condition.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Wrap long lines in `getHelpHeader` and `getHelpFooter` for `clickhouse client`
and `clickhouse local` to keep lines within reasonable width,
fixing the 00910_client_window_size_detection test
- Use `createOptionsDescription` instead of raw `po::options_description` constructor
in `git-import` and `static-files-disk-uploader` to handle zero terminal width
properly, fixing the `git-import --help` abort in debug builds
(03823_clickhouse_help_return_codes test)
Co-Authored-By: Claude Opus 4.6 <[email protected]>
This commit improves the help output for all ClickHouse applications:
- Add `clickhouse help` command that lists all available subcommands
- Return exit code 0 for all help messages (previously some returned 1)
- Output help messages to stdout instead of stderr
- Use "clickhouse <cmd>" format instead of argv[0] in help messages
- Add dynamic `app_name` detection for `clickhouse client` and `clickhouse local`
to show correct invocation name based on how the binary was called
- Add `--no-sudo` option for `clickhouse start` and `clickhouse restart`
for environments without sudo (e.g., Docker containers)
- Add detailed help headers and footers for client and local with usage examples
Closes#58244
Co-Authored-By: Claude Opus 4.5 <[email protected]>
zeekay
Alexey Milovidov