33 Commits
Author SHA1 Message Date
zeekayandHanzo Dev 64c8e40f76 the face is Zen, and it ships from @hanzo/font
One family. Font stacks name Zen and fall back to the native faces, tokens
name the role rather than a face, and the packages that used to supply a
face now point at @hanzo/font.

Self-hosted @font-face blocks are removed rather than renamed: rewriting the
family in one would leave a src pointing at a file this repo no longer ships.
@hanzo/font carries the binaries and declares the faces, so the local copies
go with the declaration that used them.

Comments naming a face we no longer use are deleted, not reworded.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-08 10:25:22 -07:00
zeekayandHanzo Dev ff3d14fcff delete the vendored upstream docs tree, and the NonCommercial snippets it fed
docs/ was 502M and 23,841 files: upstream's entire documentation site, in ten
languages, with their branding throughout. Nothing in our pipeline touches it —
the image builds from the root Dockerfile and cmake compiles no documentation,
and our CI is one image build rather than upstream's Praktika jobs, so the doc
jobs and the style check that reads this tree never run for us. docs.hanzo.ai
serves our documentation and 404s /datastore, so none of it reached a reader
either.

The licence is the part worth knowing. docs/LICENSE is CC BY-NC-SA 4.0 —
NONCOMMERCIAL — which is not this repository's Apache-2.0. So that tree was
never ours to ship in a commercial product, independent of whose name is on it.

That exposure did not stop at the tree: programs/server/docs.html inlined four
of those snippets (a JSON-type guide, two settings tables, a data-type mapping)
and that page IS served by the binary. Deleting the tree would have left the
copies behind, so DOC_SNIPPETS is emptied — the resolver still iterates it, so
an entry importing one renders without it rather than failing, and writing our
own is a matter of adding entries back.

The root LICENSE and NOTICE are untouched and still carry the upstream Apache
copyright. The eleven legal files that went with the tree were docs/LICENSE
itself and ten translations of one system-table reference page.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-14 14:03:51 -07:00
zeekayandHanzo Dev 32c3ef8786 rebuild Lexer.wasm so the pages call exports that exist
play.html's SQL highlighting was dead. The C API had been renamed to
datastore_lexer_* in Lexer.cpp and play.html's calls renamed with it, but the
base64 payload embedded in the page was never rebuilt — it still exported
clickhouse_lexer_*. So play.html asked WebAssembly for six functions the module
does not have. docs.html kept the old call names, which is the only reason its
highlighting still worked.

This is the failure mode worth naming: rename either side of a binary contract
alone and nothing errors loudly — the highlighter just stops highlighting.

Rebuilt from src/Parsers/Lexer.cpp with the flags src/Parsers/CMakeLists.txt
uses (-Os -fno-exceptions -fno-rtti -DLEXER_STANDALONE_BUILD --target=wasm32
-flto -nostdlib -Wl,--no-entry -Wl,--export-all, plus -std=c++20 for the fold
expressions). 6720 bytes, exporting datastore_lexer_{size,create,next_token,
token_is_error,token_is_end,token_is_significant}. Embedded in BOTH pages and
both now call those names — verified by instantiating the payload and checking
every symbol each page uses is present, which is the check that would have
caught this when the rename happened.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-14 11:56:18 -07:00
zeekayandHanzo Dev e53ccdf6d5 the pages call the server this fork actually is
Three of these were not branding at all — they were live breakage, because the
server was renamed and its consumers were not:

- dashboard.html parsed `ClickHouseAsyncMetrics_` while PrometheusMetricsWriter
  emits `DatastoreAsyncMetrics_`. Paste the Prometheus output into the metrics
  editor and it matched nothing.
- trace-visualizer queried `attribute['clickhouse.thread_id']` while
  OpenTelemetryTraceContext records `datastore.thread_id`. The trace tab found
  no spans.
- both visualizers loaded `js/query-clickhouse.js`; the file on disk is
  `query-datastore.js`, so the script 404'd and the query function never
  existed.

The rest is the name: titles, navbar brands, the status line a user reads after
connecting, links into upstream's repo and docs site, their Cloud badges, and
tests-visualizer pulling a logo from their CDN and querying their public demo
server. Local symbols and DOM ids went with them (queryDatastore,
parseDatastoreTrace, #tabDatastore).

LICENSE, NOTICE and every copyright notice are untouched — no removed line in
this diff carries a ©, an SPDX id or an attribution.

`clickhouse_lexer_*` in docs.html STAYS, and a comment at the embed site says
why: those export names are baked into the base64 WebAssembly payload, so the
JS must match the blob. Renaming means renaming the C API, rebuilding
Lexer.wasm, re-embedding it and updating the calls in one change; rename either
side alone and syntax highlighting fails silently.

Co-authored-by: Hanzo Dev <[email protected]>
2026-08-14 10:17:16 -07:00
Shaun Struwig 9fe065fe21 Sync embedded Avro documentation snippets 2026-07-19 12:57:41 +02:00
Alexey Milovidov 77b3585680 Render the plan-gating badges' message on the built-in help surfaces
A prop-bearing badge such as `<ScalePlanFeatureBadge feature="S3 Role-Based Access" />` (used by
the `S3Queue` page) collapsed to just its name-derived label on both help surfaces, while on the
website the component renders a substantive message from its attributes
(`docs/snippets/components/ScalePlanFeatureBadge`): which plan the feature requires and how to get
it. Add an identical `badgePayload` to the terminal `help` renderer
(`TerminalMarkdownRenderer.cpp`) and the `/docs` web UI (`docs.html`), mirroring `badgeLabel`:
`ScalePlanFeatureBadge` and `EnterprisePlanFeatureBadge` now render their message (built from the
`feature`, `support`, and `linking_verb_are` attributes exactly as the website components do)
after the badge label, and both get the website's exact label casing as known badges.

New gtest `PlanFeatureBadgeRendersItsMessage` covers the attribute combinations; new stateless
tests drive both surfaces end to end: `04606` asserts `help S3Queue` renders the message through
the real render path, and `04607` asserts the `/docs` page ships `badgePayload` and the message
text.

Addresses the review finding in
https://github.com/ClickHouse/ClickHouse/pull/110195#discussion_r3603607579
2026-07-17 19:05:56 +00:00
Alexey MilovidovandClaude Opus 4.8 b06df4ca90 Render *Badge MDX components as readable labels in the /docs web UI
The built-in `/docs` renderer's `preprocessMarkdown` dropped every `*Badge`
MDX component outright. This PR moves many component reference pages into
`system.documentation`, and a number of them rely on those badges for real
status and availability information: for example `file` opens with
`<ExperimentalBadge/>` / `<CloudNotSupportedBadge/>` and `S3Queue` uses
`<ScalePlanFeatureBadge/>`. Stripping them lost the "experimental", "not
supported in ClickHouse Cloud", or plan-gated warning entirely from the
browser help surface.

The terminal `help` renderer (`TerminalMarkdownRenderer`) already renders a
badge as readable text via `badgeLabel` (e.g. `[Experimental]`). Mirror that
in `docs.html`: add an equivalent `badgeLabel` and render any `*Badge`
component as a bold label (`**[...]**`) instead of dropping it. Known badges
get a descriptive label; any other `*Badge` the website adds later falls back
to its name with the camel case split (e.g. `ScalePlanFeatureBadge` ->
`Scale Plan Feature`), matching the terminal exactly. Matching by the `*Badge`
name (rather than the collected `import`) keeps importless badges working and
still prevents a self-closing badge from swallowing the rest of the document
into its subtree (which the sanitizer would then drop). Imported badges are
therefore skipped by the generic component-strip loop so they reach the label
step instead of being removed.

Rename `04493_docs_web_ui_badge_strip` to `04493_docs_web_ui_badge_label` and
rewrite it to assert the new label rendering, and update
`04415_docs_web_ui_mdx_components_stripped` and
`04418_docs_web_ui_importless_mdx_badge` to reflect that a status `*Badge` is
now rendered as a label while non-badge MDX components are still stripped.

Addresses review comment:
https://github.com/ClickHouse/ClickHouse/pull/110195#discussion_r3602007589

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-17 13:14:25 +00:00
Alexey Milovidov 45adfaf14c Resolve known documentation snippet imports on the /docs page and help surfaces
Converted pages `import` common snippets (a settings table, a data-type
mapping) and use them as a self-closing tag, e.g. `<PrettyFormatSettings/>`.
Unlike a decorative badge, dropping such a tag loses real content, so this
teaches `preprocessMarkdown` (built-in `/docs` page) and `TerminalMarkdownRenderer`
(client `help`) to resolve a known snippet import to its actual content
instead of stripping it like an unrecognized self-closing component. Matched
by the imported path, so any local binding name resolves (e.g. `Avro` and
`AvroConfluent` import the same `data-types-matching.mdx` snippet under
different names).

Addresses the AI review finding that `JSON`, `Pretty*`, and `RowBinary*`
entries lost their `<WhenToUseJson />` / `<PrettyFormatSettings/>` /
`<RowBinaryFormatSettings/>` sections on both help surfaces.
2026-07-16 13:05:29 +00:00
Alexey Milovidov 14b9b9539c Render Mintlify MDX components on the built-in help surfaces
Embedded documentation pages converted from the website's Mintlify sources
carry Mintlify components that the two existing `system.documentation`
consumers did not understand: the built-in `/docs` page dropped a `<Tab>`
(an unknown element is sanitized away with its whole subtree, losing e.g.
all three syntax variants of `azureBlobStorage`), and the client `help`
renderer printed the raw JSX for `<Note>` / `<Tab>` / `<Card>` literally.

Teach both presentation layers the component set the converted bodies use,
keeping `system.documentation` itself as the raw single source of truth:

- `docs.html`: map `<Note>`/`<Warning>`/`<Tip>`/`<Info>`/`<Check>`/
  `<Danger>`/`<Caution>` onto the `:::type` admonition syntax (rendered
  by the existing admonition extension), keep `<Tab title>`/`<TabItem
  label>`/`<Card title>` titles as bold lines, strip the remaining known
  component tags while preserving their children, and strip any leftover
  self-closing PascalCase component (an unimported website snippet would
  otherwise swallow the rest of the document as its subtree).
- `TerminalMarkdownRenderer`: render a whole-line `<Note>`...`</Note>`
  like its `:::note` equivalent, keep `Tab`/`Card` titles as bold lines,
  drop the other known component tag lines and self-closing components,
  and keep rendering unknown non-self-closing tags literally so `<...>`
  placeholders in prose survive.

Covered by new `TerminalMarkdownRenderer` unit tests and the stateless
test `04601_docs_web_ui_mintlify_components`.
2026-07-15 12:27:45 +00:00
Alexey MilovidovandClaude Opus 4.8 001538226f Resolve more relative documentation links to their docs route in /docs
The `/docs` Web UI rewrites a relative documentation link that does not
point to a documented entity into a `https://clickhouse.com/docs` URL via
`toDocsURL`. Two more shapes from the current `system.documentation`
corpus still fell back to the bare docs root, losing the target page and
its `#anchor`:

- A standalone settings page linked with the extension and a stray slash
  before the fragment, e.g. `materialize_skip_indexes_on_insert` links to
  `merge-tree-settings.md/#materialize_skip_indexes_on_merge`. `toDocsURL`
  now drops a trailing slash before stripping the `.md`/`.mdx` extension
  (so the same fix also repairs the matching absolute links such as
  `/sql-reference/statements/alter/skipping-index.md/#materialize-index`),
  and `merge-tree-settings` is mapped in `DOCS_PAGE_ROUTE`.
- A section overview linked with a source-relative path, e.g.
  `regionToPopulation` links to `../dictionaries#embedded-dictionaries`;
  `dictionaries` is mapped in `DOCS_SECTION_ROUTE`.

Other uncovered relative links in the corpus (format, aggregate-function
`reference/*`, and `mergetree-family/*` pages) resolve to a documented
entity as in-app links and never reach `toDocsURL`, so they are
unaffected.

New test `04492_docs_web_ui_settings_page_link_route`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-30 22:56:50 +00:00
Alexey MilovidovandClaude Opus 4.8 2da4d1f65b Address review: resolve standalone-page doc links instead of the docs root
The `/docs` web UI's `toDocsURL` resolved a relative documentation link to a
canonical docs route only when its leading path segment named a known section
(`DOCS_SECTION_ROUTE`) or was already a route root (`DOCS_ROUTE_ROOTS`). A link
to a standalone page that is neither — and whose docs section the relative path
does not reveal, because the entity's embedded `source` is a code path, not a
docs path — fell back to the bare `https://clickhouse.com/docs` root, losing the
target page and its `#anchor`.

Several current `system.documentation` rows hit this: `toWeek`/`toYearWeek` link
to `type-conversion-functions.md#parseDateTime64BestEffort`, `like` links to
`../syntax.md#string`, and the settings docs link to `../query-cache.md`.

Add a `DOCS_PAGE_ROUTE` map (mirroring `DOCS_SECTION_ROUTE`) that maps these
standalone pages to their canonical routes, consulted before the docs-root
fallback, so the link resolves to the intended page (keeping its
`#fragment`/`?query` suffix). New test `04491_docs_web_ui_page_link_route`
covers the `type-conversion-functions.md#parseDateTime64BestEffort` and
`../syntax.md#string` links.

Addresses the AI review "Request changes" on relative-link routing.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-30 15:27:54 +00:00
Alexey Milovidov 4e8497e6e8 Resolve /docs same-directory relative links to the docs root instead of a fabricated path
Addresses the review on PR #108345: `toDocsURL` resolved the leading segment of
an unmappable relative link from the docs root, so a bare same-directory link such
as `textindexes.md` became `https://clickhouse.com/docs/textindexes`, which 404s
(the real route is `/engines/table-engines/mergetree-family/textindexes`). Such a
link carries no section information, and an entity's embedded `source` is a code
path, not a docs path, so its docs route cannot be determined.

`toDocsURL` now falls back to the docs root for any relative link whose route it
cannot map, matching the function's own stated intent ("fall back to the docs root")
and the fail-close principle of not emitting a fabricated path. The few embedded
documentation entries that used bare same-directory links now spell them as absolute
docs routes that resolve correctly in both the `/docs` page and the website:
`textindexes`/`annindexes` in the `MergeTree` table-engine docs and
`memory-overcommit` in the memory-limit settings.

New test `04489_docs_web_ui_same_directory_link` covers the same-directory case.
2026-06-29 21:14:40 +00:00
Alexey MilovidovandClaude Opus 4.8 2761900342 Resolve /docs documentation links correctly: same-named entities and relative cross-section paths
Two gaps in the `/docs` renderer's link handling, both losing or misrouting real
existing documentation:

- `rewriteLinks` collapsed every entity type to one lower-cased `name`, but names
  are not unique across types in `system.documentation` (for example `JSON` is both
  a data type and a format). A link such as `[JSON](/interfaces/formats/JSON)` was
  rewritten to the `JSON` data type instead of the `JSON` format. Keep all
  same-named entities (`all_names` now maps a name to the list that share it) and
  disambiguate by the link's target route (`typeFromDocsHref` / `resolveDocEntity`);
  when the route gives no hint, leave the link as an external docs link rather than
  guessing. `linkifyCode` (for `Related:`/`Alias of` references, which carry no
  route) linkifies only an unambiguous single match.

- `toDocsURL` resolved every relative fallback as if it were rooted at `/docs` by
  stripping the leading `../`, so a source-relative cross-section link such as
  `mortonEncode`'s `../data-types/int-uint.md` became the broken
  `/docs/data-types/int-uint` instead of `/docs/sql-reference/data-types/int-uint`.
  Resolve the first segment of a relative path to its canonical section route
  (`DOCS_SECTION_ROUTE` / `DOCS_ROUTE_ROOTS`).

Adds `04419_docs_web_ui_link_disambiguation` and
`04420_docs_web_ui_relative_link_route`.

Addresses review on https://github.com/ClickHouse/ClickHouse/pull/108345

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-29 00:23:49 +00:00
Alexey MilovidovandClaude Opus 4.8 2150831146 Allow <details>/<summary> disclosure elements in the /docs Markdown sanitizer
The `/docs` page renders Markdown from `system.documentation` through an allowlist
sanitizer that drops elements not on the list together with their whole subtree.
Some embedded docs wrap a note in an ordinary `<details><summary>...</summary>...</details>`
disclosure block, for example the "Implementation details" sections of the `uniq`
and `uniqCombined` aggregate functions. Because `details` and `summary` were not on
the allowlist, those sections were removed from the rendered page entirely.

Add `details` and `summary` to the tag allowlist and `open` to the attribute
allowlist; these are safe, standard HTML5 disclosure elements.

Adds `04417_docs_web_ui_disclosure_details`.

Addresses review on https://github.com/ClickHouse/ClickHouse/pull/108345

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-29 00:23:35 +00:00
Alexey MilovidovandClaude Opus 4.8 a9e435d522 Strip importless MDX component tags in the /docs renderer
Some embedded documentation entries open with an MDX status badge that the text
does not `import` locally: `transactionID`, `transactionLatestSnapshot`, and
`transactionOldestSnapshot` start with a bare `<ExperimentalBadge/>` /
`<CloudNotSupportedBadge/>`. `preprocessMarkdown` only stripped the components
that were actually imported, so these survived. The HTML parser then treats the
self-closing custom tag as an unclosed non-void element that swallows the rest of
the document, and the sanitizer drops that whole subtree, so the entry renders as
an empty page.

Seed the strip set with a curated allowlist of known MDX components
(`MDX_COMPONENTS`) so they are removed whether or not they were imported, while
still preserving literal `<...>` placeholders in prose (e.g. a `<SearchPhrase>` in
an XML example).

Adds `04418_docs_web_ui_importless_mdx_badge` and updates
`04415_docs_web_ui_mdx_components_stripped` for the changed initialization.

Addresses review on https://github.com/ClickHouse/ClickHouse/pull/108345

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-29 00:23:27 +00:00
Alexey MilovidovandClaude Opus 4.8 f903bf6380 Keep line-height 1 in /docs code blocks
The relaxed prose line-height (1.5 on `#doc`) also applied to `pre` code
blocks, spacing out their lines. Reset it to 1 for code blocks.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 21:03:41 +02:00
Alexey MilovidovandClaude Opus 4.8 db34a09d5d Strip imported MDX components in the /docs renderer
Experimental and other website-facing entities embed MDX machinery in their
documentation: `import` statements pulling in components such as
`<ExperimentalBadge/>` or `<CloudNotSupportedBadge/>`. `preprocessMarkdown`
already stripped the `import` lines, but not the component usages.

A self-closing custom tag like `<ExperimentalBadge/>` is parsed by the HTML
parser as an *unclosed* element (the self-closing slash is ignored for
non-void elements), so it swallows the rest of the document as its children.
The sanitizer then drops that whole subtree, and the entity renders as an
empty page (only its heading survives). This affected `MaterializedPostgreSQL`
and many other entities, e.g. all experimental features.

`preprocessMarkdown` now collects the local names of imported MDX components
and strips their opening, closing, and self-closing tags, keeping any nested
content. Only imported names are removed, so literal `<...>` placeholders in
the prose (e.g. `<SearchPhrase>` in an XML example) are preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 21:03:32 +02:00
Alexey Milovidov 9a18117b70 Better line height 2026-06-27 20:49:05 +02:00
Alexey MilovidovandClaude Opus 4.8 faa37cb7e1 Limit the /docs result list to 30% width on narrow screens
The fixed-width left panel dominated mobile viewports. Cap it at
`max-width: 30%` so it shrinks on narrow screens while staying `24rem`
on wide ones.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 20:45:43 +02:00
Alexey MilovidovandClaude Opus 4.8 643ffea9ad Mark aliases with a badge in the /docs result list
Following a review suggestion: an entity that is only an alias of another (its whole
documentation is the generated "Alias of `name`." string) now shows a small "alias"
badge next to its name in the search results, with the canonical name in the tooltip,
so aliases are recognizable without opening them. The badge is tinted with the same
per-type accent as the row and adapts to the light/dark theme.

`04414_docs_web_ui_alias_badge` guards the badge styling and the detection/rendering
logic on the served page, and that `system.documentation` exposes aliases for it to mark.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 18:09:46 +00:00
Alexey MilovidovandClaude Opus 4.8 f22b901f54 Address review: reject same-origin src and CSS url() in the /docs sanitizer
The `/docs` Markdown sanitizer dropped scripts and unsafe URL schemes, but it
still kept resource sinks that the browser auto-loads as the document renders.
A row returned by a user-pointed remote `system.documentation` could render
`![x](/?query=SELECT%201)` or raw HTML with `style="background:url('/?query=…')"`,
making the browser issue an unintended query against the very ClickHouse origin
that holds the connection credentials (outside the explicit `fetch` calls that
are protected with `Authorization: never`).

A `Content-Security-Policy` header cannot cleanly close this: the query endpoint
is same-origin with the page, which legitimately needs same-origin scripts,
styles and `connect-src` (plus arbitrary remote origins when `/docs` is pointed
at another server), so no policy distinguishes the query path from the page's
own assets. The fix therefore tightens the sanitizer itself:

- reject a `src` that resolves to this page's own origin (relative URLs such as
  `/?query=…` included); cross-origin images are still allowed;
- drop a `style` attribute that uses a resource-loading CSS function
  (`url(...)`, `image-set(...)`, `cross-fade(...)`) or hides one behind a
  backslash escape. KaTeX's inline styles never use these, so this does not
  affect math rendering.

Extends `04412_docs_web_ui_markdown_sanitized` to guard the new rules.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 16:41:32 +00:00
Alexey MilovidovandClaude Opus 4.8 52d9a5652e Fix /docs URL and link construction in the Web UI
Address two review findings on `programs/server/docs.html`:

- `buildURL` always appended the documentation parameters with `?`, so a
  configured server URL that already carried a query string (e.g.
  `http://host:8123/?database=db`) became `...?database=db?default_format=...`:
  `default_format` was folded into the existing parameter, the database name was
  corrupted, and the response could come back in a non-JSON format. Append with
  `&` when the URL already contains `?`, exactly like `play.html`.

- `toDocsURL`, the fallback for relative links that cannot be resolved to a
  documented entity in-app, preserved the repository `.md`/`.mdx` extension.
  Embedded docs link to source files like `/operations/workload-scheduling.md`,
  and those extension URLs 404 on `clickhouse.com/docs` while the extensionless
  routes work. Strip the extension from the path part while preserving any
  `#fragment`/`?query` suffix.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-27 09:51:31 +00:00
Alexey MilovidovandClaude Opus 4.8 6da68429fa Sanitize rendered Markdown in the /docs Web UI
The `/docs` documentation search page rendered the Markdown stored in
`system.documentation` by feeding the output of `marked.parse` straight into
`innerHTML`. `marked` is not a sanitizer: it passes raw HTML through verbatim
and keeps any URL scheme in links and images. Because this page intentionally
queries the server `URL` the user enters and renders the response in the same
HTTP origin that holds the connection credentials, an untrusted or compromised
`system.documentation` could inject `<img src=x onerror=...>` or
`[x](javascript:...)` that runs in this origin, reads the `user`/`password`
fields, and issues same-origin queries.

Sanitize every rendered document against an allowlist before it reaches the DOM:
elements not on the list are dropped with their subtree, attributes not on the
list (including all `on*` event handlers) are removed, and `href`/`src` are kept
only for safe URL schemes (`http`, `https`, `mailto`, relative, and `#`
fragments) — `javascript:`, `data:`, `vbscript:`, and the like are rejected, with
ASCII whitespace and control characters stripped first so tricks like
`java\tscript:` are still caught. The allowlist covers what Markdown, the
admonition and math extensions, and KaTeX (its HTML and the parallel MathML)
legitimately produce, so syntax highlighting, math, admonitions, and cross-links
are unaffected.

Add `04412_docs_web_ui_markdown_sanitized` to guard that the served page routes
rendered Markdown through the sanitizer, keeps the element/attribute allowlist
and the URL-scheme policy, and no longer contains the direct-`innerHTML` sink.

Addresses the review at
https://github.com/ClickHouse/ClickHouse/pull/108345#pullrequestreview-4567656354

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-26 20:43:51 +00:00
Alexey Milovidov e344e246fe Tiny adjustments 2026-06-26 22:24:48 +02:00
Alexey MilovidovandClaude Opus 4.8 48f8274969 Address review: scope the /docs search responses to the current query and connection
`runSearch` checked `request_seq` only after `fetch` returned, but then awaited
`response.text()` a second time without re-checking. A newer keystroke (bumping
`request_seq`) or a connection change (bumping `connection_seq` and clearing
`all_names`) during that second await let the stale body still replace `current`
and render against the now-current `URL`/credentials, mixing results from the
previous connection into the rebuilt cross-link cache.

Snapshot `connection_seq` at the start of `runSearch` (mirroring `loadAllNames`)
and re-check both `request_seq` and that snapshot after each `await`, so a
superseded response is dropped instead of rendered.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-25 23:15:39 +00:00
Alexey MilovidovandClaude Opus 4.8 267d5c7897 Address review: scope the /docs cross-link cache to the connection that produced it
The cross-link name cache (`all_names`) was published unconditionally at the
end of `loadAllNames`. If the user changed the connection (server URL, user, or
password) while the request was in flight, the `change` handler cleared
`all_names`, but the older in-flight request then resolved and re-populated the
cache with names fetched from the previous server or user. Subsequent searches
ran against the new connection while `rewriteLinks`/`linkifyReferences` used the
stale names, so in-app documentation links could point to entities that do not
exist or mean something different on the active server.

Snapshot a connection generation counter (`connection_seq`, bumped on every
connection change, mirroring the existing `request_seq` pattern used to discard
superseded keystrokes) before issuing the request, and publish into `all_names`
only when the snapshot still matches. The immediate caller still receives the
fetched map, but a stale in-flight response is no longer cached.

Addresses the AI review finding on `programs/server/docs.html`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-25 03:37:28 +00:00
Alexey MilovidovandClaude Opus 4.8 f0e7fee308 Always request CORS headers on the /docs page
When the page is opened as a local file (`file://`) and pointed at a remote
ClickHouse server, the queries are cross-origin (the browser sends
`Origin: null`), and ClickHouse only adds `Access-Control-Allow-Origin` when
`add_http_cors_header=1` is set. The previous condition suppressed the header
in exactly the `file://` case that needs it, so searches and `loadAllNames`
were blocked by CORS. Request the header unconditionally, like `/play` and
`/schema`.

Addresses review feedback.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 20:19:10 +00:00
Alexey MilovidovandClaude Opus 4.8 b3db462b47 Show the new documentation categories and a source link on the /docs page
The merged \`system.documentation\` work added more kinds of documented
entities (compression codecs, profile events, current and asynchronous
metrics, and the system tables themselves) and a \`source\` column with the
repository-relative path of the file each entity's documentation comes from.

Color-code the new entity kinds in the result list, and select \`source\`
in the search query so it is available for rendering. Next to the title of
the open entity, show a pencil link to its source file on GitHub (the
\`master\` branch); it is omitted when the source location is unknown (e.g.
an alias of an undocumented function).

Test 04411 checks that the served page selects \`source\`, points the pencil
link at GitHub, and color-codes the new entity kinds.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 20:18:45 +00:00
Alexey MilovidovandClaude Opus 4.8 47250be02b Connect as the default user on the /docs page, like play.html
The `/docs` page issued its queries without an `Authorization` header, so the
browser could attach stored HTTP Basic credentials and the page would
authenticate as a cached `default` login instead of the empty credentials in the
form. Send `Authorization: never` on the fetches, exactly as `play.html` does,
so that empty `user`/`password` fields connect as the default user with an empty
password rather than as a cached `default` login.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 17:43:09 +00:00
Alexey MilovidovandClaude Opus 4.8 a69d8d6f3b Use yellow links in the dark theme on the /docs page
In the dark theme of the `/docs` documentation search page, links used a blue
(`#4AA8FF`) that does not fit the dark palette. Use the ClickHouse yellow
(`#FF0`) — the same accent the page header already uses — instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 17:42:50 +00:00
Alexey MilovidovandClaude Opus 4.8 614b995ba7 Do not poison the /docs cross-link cache on a failed request
`loadAllNames` on the `/docs` page caches the entity-name map even when the
request to `system.documentation` failed or returned a non-OK response. On a
server that requires credentials, the background call during initial page load
runs before the user enters them, stores an empty map, and then `rewriteLinks`
and `linkifyReferences` keep using that stale empty cache even after later
searches succeed with the new credentials, until the page is reloaded.

Publish `all_names` only after a successful response (a failed or errored
request returns an empty map for that call but leaves the cache unset so a later
call retries), and invalidate the cache when the connection URL, user, or
password changes so it is rebuilt against the new connection.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 17:17:52 +00:00
Alexey MilovidovandClaude Opus 4.8 7880f905ee Serve the /docs page renderer assets locally instead of from a CDN
The built-in `/docs` documentation search page loaded the `marked` (Markdown)
and `katex` (TeX math) renderers from `cdn.jsdelivr.net`. That executes
third-party network code inside the ClickHouse HTTP origin, which also handles
the `user`/`password` credentials entered on the page, so a CDN or package
compromise could read those credentials or issue and exfiltrate queries as the
user. It also broke the page's self-contained/offline contract.

Vendor `marked.min.js`, `katex.min.js`, and `katex.min.css` under
`programs/server/js/` and serve them from the binary via the existing
`JavaScriptWebUIRequestHandler` `/js/` route. The KaTeX fonts are inlined into
the stylesheet as `data:` URIs, so the page pulls no external font files either.
`DocsWebUIRequestHandler` rewrites the CDN links to the same-origin `/js/` paths
when serving the page, mirroring how `DashboardWebUIRequestHandler` handles
`uplot.js`/`lz-string.js`; the CDN URLs are kept in the source so the page still
works when opened as a local file against a remote server.

Add a stateless test `04409_docs_web_ui_local_assets` that checks the served
page references the local `/js/` assets, loads no `cdn.jsdelivr.net` code, and
that the assets are served from the binary with the fonts inlined.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 17:17:32 +00:00
Alexey MilovidovandClaude Opus 4.8 4a99d1ca55 Add a documentation search Web UI backed by system.documentation
Add `/docs` ("ClickHouse Reference"), an embedded single-page Web UI for
instant search over `system.documentation`, rendering each entity's Markdown
(with KaTeX math, ClickHouse SQL syntax highlighting via the embedded
`Lexer.wasm`, `:::note` admonitions, heading anchors, and in-app links to
related/aliased entities). Wired up like the other built-in pages (`/play`,
`/merges`, ...) in `WebUIRequestHandler` and `HTTPHandlerFactory`.

Also fix the embedded documentation so it does not emit empty fenced code
blocks: `examplesAsString` no longer emits a `Query`/`Response` block when that
field is empty, and the `filesystem`/`format` table-function examples no longer
double-wrap their already-fenced queries. Covered by a new stateless test.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-24 02:13:06 +02:00