Commit Graph
3125 Commits
Author SHA1 Message Date
Alexey MilovidovandClaude Opus 4.6 ab7bc27ae3 Bundle xterm.js libraries as embedded static files
Embed xterm.js 5.5.0, addon-fit 0.10.0, and addon-web-links 0.11.0
(minified JS and CSS) into the server binary, served at /js/*.
This removes the dependency on CDN and makes the web terminal
page fully self-contained, like the dashboard.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 14:28:05 +01:00
Alexey MilovidovandClaude Opus 4.6 1d5c996a8f Change web terminal background to black
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 14:16:57 +01:00
Alexey MilovidovandClaude Opus 4.6 e3be53f779 Clean up "Connecting..." message on successful connection
Erase the "Connecting..." line before PTY output starts, so the
terminal content renders from a clean line. Also erase it on
auth failure before showing the error prompt.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 14:15:14 +01:00
Alexey MilovidovandClaude Opus 4.6 e63c5babd2 Track #embed file dependencies in CMake
The C++ `#embed` directive includes file contents at compile time,
but the build system does not automatically track these as
dependencies. When an embedded file (e.g., `play.html`) changes,
the .cpp file that embeds it is not recompiled.

Fix this by adding `OBJECT_DEPENDS` properties to all source files
that use `#embed`, covering: HTML pages, JavaScript files, XML
configs, and NLP data files.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 13:43:47 +01:00
Alexey MilovidovandClaude Opus 4.6 6c6f77eefe Rework web terminal to use in-terminal authentication prompts
- Remove the overlay login form; the terminal fills the entire page
- Auto-connect with default credentials (or credentials from URL params)
- On auth failure, prompt for username/password directly in the terminal
- If user was provided in the URL, only prompt for password
- After session ends, prompt for credentials again to reconnect
- Hide the scrollbar while keeping the terminal scrollable

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 13:29:15 +01:00
Alexey MilovidovandClaude Opus 4.6 b0bf29a673 Add web terminal interface to clickhouse-server
Add a WebSocket-based terminal protocol (webterminal) that runs
a pseudoterminal with embedded clickhouse-client functionality,
accessible via a browser at /webterminal.

The implementation reuses the same `PtyClientDescriptorSet` and
`ClientEmbeddedRunner` infrastructure as the SSH protocol, but
connects via WebSocket instead of SSH. Authentication works the
same way as in the HTTP protocol (URL params, Basic auth,
X-ClickHouse headers).

The HTML page uses xterm.js for terminal emulation and connects
to the server via WebSocket, bridging data bidirectionally
between the browser and the PTY.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 13:11:13 +01:00
Alexey MilovidovandGitHub c7d4825d6f Merge pull request #98975 from Onyx2406/fix/dashboard-chart-title-empty-result
Show chart title when query returns empty result in dashboard
2026-03-13 19:54:18 -07:00
Alexey MilovidovandGitHub 7856d64bcf Merge pull request #96573 from ClickHouse/revert-96548-revert-96305-fix-relative-config-paths-after-chdir
Revert "Revert "Fix relative config paths resolving against wrong directory after chdir""
2026-03-13 00:19:36 -07:00
Alexey MilovidovandGitHub a0bfc4b994 Merge pull request #99313 from azat/dashboard-fixes
Fix dashboard loading (via Reload button) after unsuccessfull load
2026-03-13 06:17:08 +00:00
Raúl MarínandClaude Opus 4.6 6aa187da03 Reduce CurrentThread.h include fan-out to cut compilation cascade
- Remove `CurrentThread.h` from high-fan-out headers: `ProcessList.h`,
  `IQueryPlanStep.h`, `DDLWorker.h`, `TCPHandler.h`, `AggregatingTransform.h`,
  `ParallelParsingInputFormat.h`, `StorageTableFunction.h`, `parseSyslogLevel.h`,
  `EmbeddedRocksDBBulkSink.h`, `BlockIO.h`, `LocalConnection.h`,
  `StorageMaterializedView.h`, `HTTPHandler.h`
- Extract `QueryScope` from `CurrentThread` into standalone `Common/QueryScope.h`
  with its own `.cpp`, replacing the nested `CurrentThread::QueryScope` class
- Extract `getCurrentThreadGroup` free function into `Common/ThreadGroupSwitcher.h`,
  replacing `CurrentThread::getGroup()` in template headers:
  `threadPoolCallbackRunner.h`, `ParallelFormattingOutputFormat.h`,
  `HashedDictionaryParallelLoader.h`, `UniqExactSet.h`, `HashedDictionary.h`,
  `StatusRequestsPool.h`
- De-inline `MemoryTrackerSwitcher` ctor/dtor into `MemoryTrackerSwitcher.cpp`
- De-inline `ResourceGuard::Request::wait` into `ResourceGuard.cpp`
- De-inline `ManyAggregatedData` destructor into `AggregatingTransform.cpp`
- De-inline `QueryStatus::updateProgressIn/Out` into `ProcessList.cpp`
- Add explicit `CurrentThread.h` includes to `.cpp` files that relied on
  transitive includes through the trimmed headers

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-12 14:54:34 +01:00
Azat KhuzhinandClaude 8f73aade9b Fix dashboard loading (via Reload button) after unsuccessfull load
Previously it was always hanged on the main screen with Reload button,
but it does send requests.

Co-Authored-By: Claude <[email protected]>
2026-03-12 11:24:55 +01:00
Daniil IvanikandGitHub aa00dcbd13 Merge pull request #98140 from grantholly-clickhouse/parquet_metadata_cache_v2
Parquet metadata cache v2
2026-03-09 11:29:31 +00:00
Onyx2406 8ea9377f06 Set chart title before error check so it is visible on failure
Move the title-setting code above the error branch so the title
text is populated even when the query returns an empty result.
Previously, the title div was shown but empty because the title
text was only set after the early return on error.
2026-03-08 00:12:46 +05:30
Onyx2406 9e06e4972f Show chart title when query returns empty result in dashboard
Previously, when a dashboard query returned an empty result or
encountered an error, both the chart title and the error message
were shown in the same space with the title hidden. This made it
impossible to identify which chart had the issue.

Keep the chart title visible in error state and position the error
message below it to avoid overlap.

Fixes #85225
2026-03-08 00:04:51 +05:30
divanik 9fbd84c21c Merge branch 'master' into parquet_metadata_cache_v2 2026-03-05 11:08:33 +01:00
Alexey MilovidovandGitHub 653b0449a9 Merge master into style-check-empty-catch-all 2026-03-05 03:23:35 +01:00
Alexey MilovidovandGitHub 250dcae7b0 Merge master into revert-96548-revert-96305-fix-relative-config-paths-after-chdir 2026-03-05 03:23:34 +01:00
Alexey MilovidovandGitHub e0146eb204 Merge master into fix-keeper-tcp-handler-shutdown-hang 2026-03-04 09:46:03 +01:00
Alexey MilovidovandGitHub be5df3810b Merge master into style-check-empty-catch-all 2026-03-04 00:35:29 +01:00
Grant HollyandGitHub 93bf0c4cb5 Merge branch 'master' into parquet_metadata_cache_v2 2026-03-03 08:20:53 -08:00
Alexey MilovidovandGitHub 3104a83f3f Merge master into revert-96548-revert-96305-fix-relative-config-paths-after-chdir 2026-03-03 00:42:28 +01:00
Grant HollyandGitHub d87331e6a3 Merge branch 'master' into parquet_metadata_cache_v2 2026-03-02 13:22:59 -08:00
Alexey MilovidovandClaude Opus 4.6 aa0d31a68c Add signalKeeperDispatcherShutdown method to Context
Move the `#if USE_NURAFT` guard into Context, avoiding preprocessor
directives inside `SCOPE_EXIT_SAFE` macro arguments (which is undefined
behavior). This also fixes the FreeBSD build where NuRaft is not
available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 20:52:30 +01:00
Alexey MilovidovandClaude Opus 4.6 6621ba6c42 Wrap signalShutdown call in #if USE_NURAFT guard
`tryGetKeeperDispatcher` is only available when NuRaft is enabled,
which is not the case on FreeBSD builds.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 20:44:59 +01:00
Alexey MilovidovandGitHub dca263066b Merge master into style-check-empty-catch-all 2026-03-02 20:15:24 +01:00
Alexey MilovidovandClaude Opus 4.6 8c8cbe6ffc Fix Keeper TCP handler preventing graceful server shutdown
During shutdown, `KeeperTCPHandler` connections run an infinite loop
with no shutdown detection. When the server stops the Keeper TCP
listening socket, existing handler threads keep processing heartbeats
indefinitely. `KeeperDispatcher::shutdown` (which sets the shutdown
flag) is only called after waiting for connections to close, creating
a circular dependency that blocks shutdown until the external timeout
kills the server.

Add a `shutting_down` flag to `KeeperDispatcher` that is set before
waiting for connections. `KeeperTCPHandler` checks this flag after
each `poll` return and exits cleanly when set.

https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=98447&sha=79d30b439bc793fc9aa276b28aec105750847880&name_0=PR&name_1=Stateless%20tests%20%28arm_asan%2C%20azure%2C%20parallel%29

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 18:46:50 +01:00
Anton Popov e3ceb4e141 Merge remote-tracking branch 'upstream/master' into HEAD 2026-03-02 12:26:51 +00:00
Alexey MilovidovandClaude Opus 4.6 6897b4e8a4 Add NOLINT(bugprone-empty-catch) to empty catch blocks with specific types
clang-tidy `bugprone-empty-catch` fires on catch blocks with empty
bodies regardless of the exception type.  Re-add the NOLINT annotation
on the catch lines that were previously suppressed with `catch (...)`
and now use specific exception types.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 04:40:52 +00:00
Alexey MilovidovandClaude Opus 4.6 6415834976 Narrow catch (...) to specific exception types where possible
Replace blanket `catch (...)` with specific exception types based on
analysis of what each try block can actually throw:

- `const Exception &` for catch blocks where only ClickHouse operations
  are involved (Field operations, expression evaluation, parsing, etc.)
- `const std::exception &` for catch blocks involving standard library,
  Poco, or third-party code (filesystem, HTTP, thread creation, etc.)

Blocks that genuinely need `catch (...)` are left unchanged:
- try-pattern functions (`tryGetPosition`, `tryDeserializeText`, etc.)
- Exception formatting utilities in `Exception.cpp`
- Signal/terminate handlers
- FFI callbacks (DeltaLake)
- Dual-handler patterns (already have specific + catch-all)
- `noexcept` functions with user-supplied callbacks

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 03:55:15 +00:00
Alexey MilovidovandGitHub 1c7c44d587 Merge pull request #98232 from ClickHouse/jemalloc-setup-once
Make jemalloc profiling cover all threads when enabled
2026-03-02 03:48:51 +00:00
Alexey MilovidovandClaude Opus 4.6 b60662ea1c Annotate all catch (...) blocks flagged by the style check
Enhance the `catch_all` style check in two ways:

1. Recognize more legitimate exception-handling patterns as "handled":
   `getCurrentExceptionMessage`, `getCurrentExceptionCode`,
   `getCurrentExceptionMessageAndPattern`, `ExecutionStatus::fromCurrentException`,
   `onBackgroundException`, `storeException`, `STDERR_FILENO`, `writeRetry`,
   `handle_exception`, `handleException`, `finishWithException`.

2. Fix function-try-block detection for destructors like
   `Exception::~Exception() try { } catch (...) { }` — the enclosing-scope
   scan now falls through to the function-try-block scan when it hits a
   namespace/class scope (no parentheses in signature).

For the remaining 108 catch-all blocks, add `// Ok: <reason>` comments
explaining why each is intentional. Categories include:

- Try-patterns (`tryDeserializeText`, `tryGetPosition`, etc.)
- Exception handling/reporting utilities (`Exception.cpp`)
- Non-essential system probes (`Server.cpp` Linux /proc and /sys checks)
- C callbacks where exceptions cannot propagate (zip archives)
- Background thread exception saving (`onBackgroundException`)
- Fallback/retry logic (`DatabaseOverlay`, `ReadSchemaUtils`)
- Best-effort cleanup or error reporting
- Tests that expect exceptions

Replace `// NOLINT(bugprone-empty-catch)` with `// Ok:` comments on all
catch blocks that were flagged by the new check. Fix Allman-style brace
violations in `convertFieldToType.cpp` and `ClientBase.cpp`.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-02 03:06:11 +00:00
Alexey Milovidov 7547751354 Merge branch 'master' into style-check-empty-catch-all 2026-03-02 02:47:11 +00:00
Antonio AndelicandClaude Opus 4.6 c527e0bb99 Fix Jemalloc::setup crash after watchdog fork in debug builds
The previous implementation used `std::call_once` which fired in the
parent process during `BaseDaemon::initialize`. After the watchdog
`fork`, jemalloc state (e.g. `background_thread`) resets in the child
but the `once_flag` remains set, so the verification check saw a
mismatch and fired `chassert`.

Split `setup` and `verifySetup` into separate functions:
- `setup` always applies settings (idempotent), called in
  `BaseDaemon::initialize` and re-applied after watchdog fork
- `verifySetup` checks jemalloc state matches server settings,
  called from `Server::main` and `Keeper::main`

Also extract config key names into constants in `Jemalloc.h` to keep
them in sync between `BaseDaemon` and `ServerSettings`.

https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=98232&sha=b69293471789706abde22ec86f33104dbfaf0382&name_0=PR&name_1=Stateless%20tests%20%28amd_debug%2C%20parallel%29

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-01 21:58:03 +01:00
Amos Bird a4cb6c38dd Allow custom http_handlers per protocol endpoint
Add optional <handlers> config key to type=http entries in <protocols>,
so each HTTP port can use a different set of http_handlers. This enables
use cases like running an alternative HTTP API (e.g. ES-compatible) on
a separate port with its own routing rules, without affecting the
default HTTP port.

Config example:
  <protocols>
    <alt_http>
      <type>http</type>
      <port>8124</port>
      <handlers>http_handlers_alt</handlers>
    </alt_http>
  </protocols>
  <http_handlers_alt>
    <rule>...</rule>
  </http_handlers_alt>

If <handlers> is omitted, the default <http_handlers> is used (fully
backward compatible). Config reload (updateServers) also respects the
per-protocol handlers key.
2026-03-01 22:14:58 +08:00
Alexey MilovidovandGitHub 1fad019d22 Merge pull request #98229 from amosbird/disable-netlink-metric-collection
Remove NetlinkMetricsProvider, use procfs exclusively for taskstats
2026-03-01 04:22:49 +00:00
Alexey MilovidovandGitHub aef174f17d Merge pull request #98349 from ClickHouse/jemalloc-html-max-block-size
Set `max_block_size` to 500 for collapsed jemalloc profile output
2026-03-01 04:18:40 +00:00
Alexey MilovidovandGitHub cc6a29736e Revert "Call OpenSSL cleanup before safeExit to fix LeakSanitizer false positives" 2026-03-01 00:21:39 +01:00
Antonio AndelicandClaude Opus 4.6 9b905689c8 Set max_block_size to 500 for collapsed jemalloc profile output
Minimize memory usage when querying `system.jemalloc_profile_text` in
collapsed format by streaming results in small blocks instead of
buffering up to 65K rows.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-28 23:48:19 +01:00
Amos Bird b503accc7f Remove NetlinkMetricsProvider, use procfs exclusively for taskstats
Netlink-based metric collection is problematic in containerized environments
(Docker, LXC, Kubernetes) where it can hang indefinitely. Benchmarks show
procfs has better tail latency and lower total CPU overhead under contention.
Since all production deployments already use the procfs path, remove netlink
entirely rather than adding a setting to disable it.
2026-02-28 11:32:25 +08:00
Alexey MilovidovandClaude Opus 4.6 354cfbae6c Call OpenSSLInitializer::cleanup before safeExit to fix LeakSanitizer false positives
When the server takes the forced shutdown path via `safeExit` → `_exit`,
static destructors and atexit handlers are bypassed, so
`OpenSSLInitializer::cleanup` / `OPENSSL_cleanup` never runs.
ASan intercepts `_exit` and runs LSan, which reports OpenSSL's global
cached state (EVP method cache, legacy provider context) as leaked memory.

https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=97652&sha=c9d73d1a85abfa220c6ddad38d81e7acb72be2e9&name_0=PR&name_1=AST%20fuzzer%20%28arm_asan%29

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-28 04:03:34 +01:00
Antonio AndelicandGitHub dce5b36218 Merge pull request #98168 from ClickHouse/add-jemalloc-link
Add jemalloc link to server index page and update allocation profiling docs
2026-02-27 12:52:44 +00:00
Anton Popov ccc73b9a44 Merge remote-tracking branch 'upstream/master' into HEAD 2026-02-27 10:56:35 +00:00
Antonio AndelicandClaude Opus 4.6 b543a1bec4 Add jemalloc link to server index page and update allocation profiling docs
Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-27 10:20:43 +01:00
RinChanNOWWW d36d5bb840 Support auxiliary keeper for DatabaseReplicated.
Mark getObjectStorage as const.

Add docs and tests.

Fix.

Fix drop replica.

Fix getQueryStatus.

Fix test.

Fix zookeeper usage.

Fix codes review.

Split the tests.
2026-02-27 10:37:23 +08:00
Alexey MilovidovandGitHub 3e0d079016 Merge pull request #98038 from ClickHouse/revert-96025-clickhouse-help
Revert "Improve help output for ClickHouse applications"
2026-02-27 00:48:53 +01:00
Grant HollyandGitHub 836b4fa2a0 Merge branch 'master' into parquet_metadata_cache_v2 2026-02-26 13:54:38 -08:00
Anton Popov 8f1b9a5f6c Merge remote-tracking branch 'upstream/master' into HEAD 2026-02-26 12:38:20 +00:00
Antonio AndelicandGitHub 711912ccb5 Merge pull request #97863 from ClickHouse/jemalloc-dash-better-check
Check actual jemalloc profiling state via `system.asynchronous_metrics`
2026-02-26 10:05:59 +00:00
Alexey MilovidovandGitHub e8654c51eb Revert "Improve help output for ClickHouse applications" 2026-02-26 07:51:17 +01:00
Antonio AndelicandGitHub 4c91652224 Merge pull request #97945 from ClickHouse/jemalloc-profiler-sampling-rate
Add `jemalloc_profiler_sampling_rate` server setting
2026-02-25 17:09:38 +00:00