The S3 engine and table-function docs handed readers `'minioadmin',
'minioadminpassword'` five times as the way to authenticate, plus the
integration fixture's own `ClickHouse_Minio_P@ssw0rd` in the s3Cluster
examples. Copy-pasteable weak credentials in the reference docs are a worse
problem than the branding: they get pasted. Those are placeholders now, and the
endpoints point at Hanzo S3 (`hanzoai/s3` — SeaweedFS-derived, Apache-2.0, and
never a MinIO fork).
`registerStorageObjectStorage.cpp` also linked a docker-compose recipe at
github.com/Datastore/examples, a URL the ClickHouse->Datastore rebrand invented
and that has never resolved. The sentence no longer depends on it.
Scope is deliberately docs only, both copies of them: `docs/en/**` — the one
doc tree `docs/AGENTS.md` says to edit — and the `src/**` `R"DOCS_MD(...)"`
blocks that mirror it into the binary. Every changed line in a .cpp is inside a
documentation raw-string (verified by delimiter-span check, 21/21); the only
header change is a `///` comment. No executable code, no test, no fixture, no
build config.
The integration-test harness keeps MinIO on purpose, and LLM.md now records
why so this is not re-litigated or sed'd: `minio==7.2.20` is a real dependency
behind `from minio import Minio` in helpers/cluster.py, 60 test files call 16
distinct client methods, 136 tests pass with_minio=True, ~120 storage XMLs
hardcode the fixture credential, and the compose file leans on MinIO-only
surface (--certs-dir TLS, prometheus auth mode, console address, virtual-host
bucket aliases) fronted by a registry-pulled clickhouse/s3-proxy image. That
port is a real project whose acceptance test is the S3 suite going green, and
it cannot be run here: ghcr.io/hanzoai/s3 is linux/amd64 only and this is
arm64.
Co-authored-by: Hanzo Dev <[email protected]>
Move the full page body of each table-function reference page into the
FunctionDocumentation attached at registerFunction time, and convert the
pages to be regenerated from system.table_functions (like table engines,
data types and formats already are).
55 of the 58 table-function pages are now autogenerated. The description of
each function is set to its reference-page body, resolved to the correct
registerFunction<Class>({...}) site by the function name (Class::name),
including the object-storage family (s3/gcs/azureBlobStorage/hdfs/iceberg/
deltaLake/hudi/paimon and their *Cluster variants, via Definitions) and the
templated/conditional-name functions (values, zeros, generateSeries). The
string-name overload functions (remote, cluster) and the doc-variable form
(format) are handled at their own sites. A page->name alias maps the
generate() page to the generateRandom function.
timeSeriesData is intentionally left hand-written: it is a registerAlias of
timeSeriesSamples, so the two share a single description, but their pages
differ (each uses its own name in examples) and an alias cannot carry its
own doc.
Verified against a from-master build: every function's
system.table_functions.description matches its master page body
byte-for-byte, and each regenerated page equals origin/master (body
identical, only the AUTOGENERATED markers added) with zero broken links.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
An explicit object storage cluster function (s3Cluster, azureBlobStorageCluster,
hdfsCluster, ...) nested inside an outer distributed query did not reach the
nested-cluster-function gate: on the secondary query it gated distributed reads
on collaborate_with_initiator, which is false under clusterAllReplicas, so it
silently read the whole object on every outer replica and multiplied the row
count instead of being rejected. urlCluster and fileCluster always request a
distributed read on a secondary query and are correctly rejected.
Make the object storage cluster functions request a distributed read on a
secondary query too, so they reach RemoteQueryExecutor::processReadTaskRequest
and the initiator decides uniformly for every *Cluster function: it serves the
read task when it installed an iterator (the legitimate top-level and
INSERT INTO distributed SELECT FROM *Cluster dispatch) and rejects it with
BAD_ARGUMENTS otherwise (the nested shapes). This supersedes the earlier
defensive gate that avoided the LOGICAL_ERROR by skipping distributed reads,
now that the gate raises a normal query error instead of aborting.
03850 now also covers nested clusterAllReplicas(..., s3Cluster(...)).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add `static` to file-local functions and forward declarations
for cross-TU functions to satisfy `-Wmissing-prototypes`.
This ensures all functions have visible prototypes before their
definitions, improving code quality and catching accidental
linkage issues.
Reference: https://github.com/ClickHouse/ClickHouse/issues/76944
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Remove `CurrentThread.h` from high-fan-out headers: `ProcessList.h`,
`IQueryPlanStep.h`, `DDLWorker.h`, `TCPHandler.h`, `AggregatingTransform.h`,
`ParallelParsingInputFormat.h`, `StorageTableFunction.h`, `parseSyslogLevel.h`,
`EmbeddedRocksDBBulkSink.h`, `BlockIO.h`, `LocalConnection.h`,
`StorageMaterializedView.h`, `HTTPHandler.h`
- Extract `QueryScope` from `CurrentThread` into standalone `Common/QueryScope.h`
with its own `.cpp`, replacing the nested `CurrentThread::QueryScope` class
- Extract `getCurrentThreadGroup` free function into `Common/ThreadGroupSwitcher.h`,
replacing `CurrentThread::getGroup()` in template headers:
`threadPoolCallbackRunner.h`, `ParallelFormattingOutputFormat.h`,
`HashedDictionaryParallelLoader.h`, `UniqExactSet.h`, `HashedDictionary.h`,
`StatusRequestsPool.h`
- De-inline `MemoryTrackerSwitcher` ctor/dtor into `MemoryTrackerSwitcher.cpp`
- De-inline `ResourceGuard::Request::wait` into `ResourceGuard.cpp`
- De-inline `ManyAggregatedData` destructor into `AggregatingTransform.cpp`
- De-inline `QueryStatus::updateProgressIn/Out` into `ProcessList.cpp`
- Add explicit `CurrentThread.h` includes to `.cpp` files that relied on
transitive includes through the trimmed headers
Co-Authored-By: Claude Opus 4.6 <[email protected]>
The `deltaLakeAzureCluster` table function was registered under `#if USE_AZURE_BLOB_STORAGE`
but the corresponding `DeltaLakeAzure` storage engine requires `USE_DELTA_KERNEL_RS`. This
mismatch caused a `LOGICAL_ERROR` exception when the table function tried to look up its
engine for access checking, crashing the server in debug builds.
Fixes:
- Add `USE_DELTA_KERNEL_RS` guard to `deltaLakeAzureCluster` registration (.cpp, .h, template)
- Change `getSourceAccessObject` from `LOGICAL_ERROR` to `UNKNOWN_STORAGE` for unknown engines
- Fix copy-paste in documentation: "Iceberg" → "DeltaLake" for `deltaLakeAzureCluster`
- Skip Delta Lake tests when `DeltaLake` engine is not available (query `system.table_engines`)
- Conditionally handle Delta Lake entries in `test_mask_sensitive_info` based on engine availability
- Skip timing-sensitive and memory-measurement tests under MSan:
`test_ttl_compatibility`, `test_memory_limit_success`, `test_async_query_sending`,
`test_when_s3_timeout_at_listing`
Co-Authored-By: Claude Opus 4.6 <[email protected]>
zeekayandHanzo Dev