Commit Graph
24 Commits
Author SHA1 Message Date
Raúl MarínandGitHub b089d73797 Merge pull request #105299 from Algunenano/jemalloc-html-escape-error-messages
Escape error messages in jemalloc.html
2026-05-20 11:43:07 +00:00
Raúl MarínandClaude Opus 4.7 6af2f16ae2 Escape error messages in jemalloc.html
Route the four `innerHTML` error-message renderers in `jemalloc.html`
through the existing `escapeHtml` helper. The interpolated values
(`error.message`, user-typed `queryId`) are trust-anchored so this is
not a security boundary, but escaping is hygienic and matches the rest
of the file.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-19 10:18:18 +00:00
Alexey MilovidovandClaude Opus 4.7 0848414e02 Web pages: empty user input falls back to server default
The "user" input on `play.html` and on the other web pages
(`dashboard.html`, `jemalloc.html`, `merges.html`, `binary.html`,
`webterminal.html`) previously came pre-filled with the literal text
`default`. Replace that with an empty value and a `user` placeholder
hint. When the value is empty, the page does not send a `user=` query
parameter (or, in the WebTerminal protocol, omits the `user` field
from the auth JSON), letting the server fall back to its configured
default user — which is not necessarily named `default`.

For the WebTerminal, `parseAuthMessage` now treats the `user` field as
optional: when omitted or empty, the caller-provided default is
preserved. The call site seeds it with `default`, mirroring the HTTP
fallback in `authenticateUserByHTTP`.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-18 18:53:07 +02:00
murphy-4o 9f5abb4163 Address reviewer feedback: reuse symbolizeJemallocHeapProfile, fix build deps, remove dead code
- Add `OBJECT_DEPENDS` for `KeeperJemallocHandler.cpp` in
  `src/CMakeLists.txt` so that changes to the `#embed`-ed
  `jemalloc.html` trigger recompilation, matching the existing pattern
  for other handlers.

- Replace the 130-line `heapProfileToCollapsedStacks` in
  `Common/Jemalloc.cpp` with a call to the existing
  `JemallocProfileSource` collapsed-stack pipeline via the new
  `symbolizeJemallocHeapProfileToString` helper.  This eliminates
  duplicated heap-dump parsing and address symbolization logic.

- Remove the unused `instanceName` JavaScript variable from
  `jemalloc.html` (assigned but never referenced in any template).

https: //github.com/ClickHouse/ClickHouse/pull/100606
Made-with: Cursor
2026-04-15 07:03:36 +00:00
murphy-4o 815ff3ba68 Fix CI failures and address remaining review comments
- Fix `attachStrictPath` not matching URIs with query parameters: replace
  with a custom filter that matches the path portion, ignoring `?...`
  suffixes.  This caused `/jemalloc/profile?format=bad` to 404 in
  non-jemalloc builds instead of returning 501.

- Fix HEAD test assertions: HEAD responses carry no body, so checking
  `response.text` for "not available" fails on 501.  Only assert the
  status code for HEAD.

- Add HEAD short-circuit to `KeeperJemallocStatsHandler` to avoid the
  expensive `je_malloc_stats_print` call when no body is needed.

- Handle `status.errors` in the web UI's `checkProfilingEnabled`: when
  the `/jemalloc/status` response includes errors, treat profiling state
  as unknown rather than silently defaulting to disabled.

Made-with: Cursor
2026-03-27 06:42:55 +00:00
murphy-4o fb54ffeb41 Fix review comments: log mallctl failures in handleStatus, remove duplicate comment
- `handleStatus` no longer silently swallows `mallctl` exceptions.
  Failures are logged via `tryLogCurrentException` and surfaced in the
  JSON response as an `errors` array so operators can distinguish
  "profiling disabled" from "status unknown".
- Remove triple-duplicated "Substitute user name" comment in
  `jemalloc.html`.

Made-with: Cursor
2026-03-24 14:38:49 +00:00
murphy-4o 77d68fd242 Add jemalloc profiling web UI for ClickHouse Keeper
Expose jemalloc heap profiling and statistics through Keeper's HTTP
control interface, matching the functionality already available in the
ClickHouse Server web UI.

New REST API endpoints on the Keeper HTTP control port:
- GET /jemalloc — serves the interactive web UI (shared with server via
  `isKeeper` mode flag injected at serve time)
- GET /jemalloc/stats — returns raw `malloc_stats_print` output
- GET /jemalloc/status — returns profiling state as JSON
- GET /jemalloc/profile?format={collapsed|raw} — flushes a heap profile,
  symbolizes it, and returns collapsed stacks for flame graph rendering

The web UI reuses the server's `jemalloc.html` with a thin `isKeeper`
abstraction that swaps SQL-based data fetching for REST API calls and
hides server-specific features (Query Profiler, connection params, output
format options). This avoids duplicating ~3000 lines of HTML/CSS/JS.

New shared helpers in `Jemalloc::heapProfileToCollapsedStacks` (parses
raw jemalloc heap dump, symbolizes addresses via `StackTrace::forEachFrame`,
aggregates into FlameGraph-compatible collapsed stacks) and
`Jemalloc::getStats` (captures `malloc_stats_print` output as a string).

Closes https://github.com/ClickHouse/ClickHouse/issues/100544

Made-with: Cursor
2026-03-24 14:17:13 +00:00
Antonio AndelicandClaude Opus 4.6 9b905689c8 Set max_block_size to 500 for collapsed jemalloc profile output
Minimize memory usage when querying `system.jemalloc_profile_text` in
collapsed format by streaming results in small blocks instead of
buffering up to 65K rows.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-28 23:48:19 +01:00
Antonio AndelicandClaude Opus 4.6 bc7e8e3c51 Fix queryId TDZ error and rename getQueryFlameGraphProfile to getQueryCollapsedProfile
Remove duplicate `const queryId` declaration inside the `try` block of
`getQueryCollapsedProfile` that shadowed the outer one, causing a
"Cannot access 'queryId' before initialization" ReferenceError every
time the function was called.

Rename the function from `getQueryFlameGraphProfile` to
`getQueryCollapsedProfile` to better reflect what it does: it fetches
collapsed stack data and renders it as text with action buttons,
rather than generating the flame graph itself.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-25 17:32:20 +01:00
Antonio AndelicandClaude Opus 4.6 57bebd0a51 Show query profile output as text with action buttons in jemalloc dashboard
Change the Query Profiler's Flame Graph section to show the collapsed
profile data in a text box (matching the Global Profiler output style)
instead of directly generating and downloading an SVG. The button is
now "Get Profile" and the output card includes Copy All, Save,
Generate Flame Graph, and Close buttons.

Extract shared helper functions parameterized by result div ID to
deduplicate profile output code between Global and Query Profiler:
`getFullProfileText`, `copyProfileOutput`, `saveProfileOutput`,
`closeProfileOutput`, `showMoreProfileLines`,
`downloadFlameGraphFromProfile`, and `renderProfileOutputCard`.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-25 15:18:58 +01:00
Antonio AndelicandClaude Opus 4.6 e5fdb872cd Improve query profiler in jemalloc dashboard
- Add `SETTINGS allow_introspection_functions = 1` to the `flameGraph`
  query so it works without requiring the setting globally.
- Handle `SYSTEM FLUSH LOGS` failure gracefully: show a warning instead
  of aborting, since the user may lack privileges (e.g. readonly mode).
- Preserve the flush warning when no trace log data is found, so the
  user knows the data may appear after a delay.
- Fix empty result detection: `min`/`max` return default values (0)
  on empty sets, not empty strings.
- Darken light theme `--warning-color` for better readability.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-24 16:55:09 +01:00
Antonio AndelicandClaude Opus 4.6 1eedff02b5 Check actual jemalloc profiling state via system.asynchronous_metrics
Query `jemalloc.prof.thread_active_init` from `system.asynchronous_metrics`
instead of the config value from `system.server_settings` to reflect
whether profiling is truly active at the jemalloc level.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-24 14:56:48 +01:00
Antonio AndelicandClaude Opus 4.6 a2a9eec8d2 Add connection params UI and improve auth in jemalloc.html
- Add URL, user, and password input fields to the header, matching
  the pattern used in `dashboard.html`
- Extract `buildUrl` helper and shared `fetchOptions` to deduplicate
  URL building across all fetch call sites
- Add `add_http_cors_header` and `enable_http_compression` to requests
- Add `Authorization: never` header to suppress browser auth popups
- Change profiling check to query `system.server_settings` for
  `jemalloc_enable_global_profiler` instead of `system.asynchronous_metrics`
- Wrap initialization in `start()` function for compatibility with
  dataplanectl credential injection

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-24 10:04:18 +01:00
Antonio AndelicandClaude Opus 4.6 2974936649 jemalloc.html: rename jemalloc_profile to jemalloc_profile_text, set default auto-refresh to 60s
Rename `system.jemalloc_profile` to `system.jemalloc_profile_text` and
the corresponding settings (`jemalloc_profile_output_format`,
`jemalloc_profile_symbolize_with_inline`, `jemalloc_profile_collapsed_use_count`)
to their `jemalloc_profile_text_*` equivalents, following the server-side rename.

Also change the default auto-refresh interval from 10s to 60s.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-23 13:58:41 +01:00
Antonio Andelic b8df9f1e0c better 2026-02-20 15:35:06 +01:00
Antonio Andelic a314fd66bb Better 2026-02-19 17:01:26 +01:00
Antonio Andelic 8e76ede05e Add tip for Claude 2026-02-19 15:32:56 +01:00
Antonio Andelic 4b56c65910 Add query profiler 2026-02-18 12:58:28 +01:00
Antonio Andelic 3aa1fac84f Better output 2026-02-18 11:42:09 +01:00
Antonio Andelic 5debb878c2 generate flamegraph 2026-02-18 11:25:16 +01:00
Antonio Andelic 12185df54e Add button 2026-02-18 10:33:11 +01:00
Antonio Andelic 7c83545d48 Fetch profiles in UI 2026-02-18 10:18:35 +01:00
Antonio Andelic 0e6691aa50 More improvements 2026-02-17 10:42:09 +01:00
Antonio Andelic 77173af567 Add visualization 2026-02-16 15:36:49 +01:00