Commit Graph
698 Commits
Author SHA1 Message Date
Alexey MilovidovandClaude Opus 4.7 17bdae2f4b Fix -Wdisabled-macro-expansion and -Wmissing-prototypes in two more build targets
- musl build: `stderr` is defined as a recursive macro `#define stderr (stderr)`,
  which triggers `-Wdisabled-macro-expansion` when used as a function argument.
  Suppress the warning in `LocalServer.cpp` the same way it is already handled
  in `FuzzLoop.cpp`.

- arm_fuzzers build: the `DEFINE_BINARY_PROTO_FUZZER` macro from
  `libprotobuf-mutator` expands to definitions of `LLVMFuzzerCustomMutator`,
  `LLVMFuzzerCustomCrossOver`, and `LLVMFuzzerTestOneInput` without preceding
  prototypes, and the generated `out.cpp` defines `GenerateSentence` without
  one either. Add `-Wno-missing-prototypes` to the compile flags for the
  codegen fuzzer sources where we cannot easily add declarations.

CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=100976&sha=latest&name_0=PR

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-19 12:02:04 +00:00
Alexey Milovidov c8be129a5b Merge remote-tracking branch 'origin/master' into fix-compiler-warnings
# Conflicts:
#	src/Common/WKB.cpp
#	src/Functions/array/arrayDotProduct.cpp
#	src/Interpreters/InterpreterSelectIntersectExceptQuery.cpp
#	src/Interpreters/InterpreterSelectQueryAnalyzer.cpp
#	src/Interpreters/InterpreterSelectWithUnionQuery.cpp
#	src/Server/KeeperHTTPHandlerFactory.cpp
#	src/Storages/MergeTree/MergeTreeReaderTextIndex.cpp
2026-04-19 11:57:10 +00:00
Alexey Milovidov e76298231e Merge remote-tracking branch 'origin/master' into fix-compiler-warnings
# Conflicts:
#	src/Interpreters/InterpreterDropQuery.cpp
#	src/Processors/QueryPlan/Optimizations/optimizeLazyMaterialization.cpp
2026-04-09 20:13:51 +00:00
kssenii a39cafba2a Move FileCache to separate dir 2026-04-09 11:33:20 +02:00
Konstantin Bogdanov ded76ea441 Merge branch 'master' into update-llvm-22 2026-04-07 23:49:25 +02:00
Alexey MilovidovandClaude Opus 4.6 8b47d200d5 Enable -Wmissing-prototypes compiler warning
Add `static` to file-local functions and forward declarations
for cross-TU functions to satisfy `-Wmissing-prototypes`.

This ensures all functions have visible prototypes before their
definitions, improving code quality and catching accidental
linkage issues.

Reference: https://github.com/ClickHouse/ClickHouse/issues/76944

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 20:11:07 +00:00
Alexey MilovidovandClaude Opus 4.6 f04c87153a Start MemoryWorker in clickhouse-local
`MemoryWorker` was only started in `clickhouse-server` but not in
`clickhouse-local`. This caused three problems:

1. The userspace page cache stayed stuck at `page_cache_min_size`
   (100 MiB by default) and never grew to `page_cache_max_size`,
   because `PageCache::autoResize` — which adjusts the cache size
   based on available memory — was never called. The result was severe
   cache thrashing: blocks evicted immediately after fetching, turning
   every "hot" run into a cold run with full S3 re-downloads.

2. RSS was never reported to the global `MemoryTracker`, so memory
   tracking in `clickhouse-local` was less accurate.

3. Jemalloc dirty pages were never purged under memory pressure.

Start `MemoryWorker` unconditionally (not only when page cache is
enabled), matching `clickhouse-server` behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 23:13:17 +00:00
Alexey MilovidovandClaude Opus 4.6 71e0229050 Initialize page cache in clickhouse-local
The userspace page cache (`page_cache_max_size`) was never initialized
in `clickhouse-local`. The `setPageCache` call only existed in
`programs/server/Server.cpp` but was missing from
`programs/local/LocalServer.cpp`.

This adds the same page cache initialization (including validation
of `page_cache_min_size` vs `page_cache_max_size`) to `clickhouse-local`,
enabling the page cache for S3/object storage queries.

Closes https://github.com/ClickHouse/ClickHouse/issues/99499

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-14 08:59:43 -07:00
Konstantin Bogdanov 287828bb58 Merge remote-tracking branch 'origin/master' into update-llvm-22 2026-03-09 16:48:57 +01:00
Konstantin Bogdanov 64b0f70e7a Fix build errors caused by LLVM 22 upgrade
- `Intrinsic::getDeclaration` renamed to `Intrinsic::getOrInsertDeclaration`
  in LLVM 22 (abs.cpp, bitCount.cpp)
- `createTargetMachine` first argument changed from `StringRef` to
  `const Triple&` — construct `llvm::Triple` explicitly (CHJIT.cpp)
- `std::__gcd` internal removed from libc++ — use `std::gcd` from
  `<numeric>` (AggregateFunctionKolmogorovSmirnovTest.cpp)
- Added `(void)` casts for new `[[nodiscard]]` on `std::stod`,
  `std::filesystem::is_directory`, and `future.get()`
  (KQLGeneralFunctions.cpp, LocalServer.cpp,
  ZooKeeperRemoveByList.cpp, EphemeralLockInZooKeeper.cpp)

Assisted-by: Claude Opus 4.6 via GitHub Copilot
2026-03-06 12:15:26 +01:00
Grant HollyandGitHub 93bf0c4cb5 Merge branch 'master' into parquet_metadata_cache_v2 2026-03-03 08:20:53 -08:00
Grant HollyandGitHub d87331e6a3 Merge branch 'master' into parquet_metadata_cache_v2 2026-03-02 13:22:59 -08:00
Anton Popov ccc73b9a44 Merge remote-tracking branch 'upstream/master' into HEAD 2026-02-27 10:56:35 +00:00
Grant HollyandGitHub 836b4fa2a0 Merge branch 'master' into parquet_metadata_cache_v2 2026-02-26 13:54:38 -08:00
Anton Popov 8f1b9a5f6c Merge remote-tracking branch 'upstream/master' into HEAD 2026-02-26 12:38:20 +00:00
Alexey MilovidovandGitHub e8654c51eb Revert "Improve help output for ClickHouse applications" 2026-02-26 07:51:17 +01:00
Antonio AndelicandClaude Opus 4.6 a0564970cf Fix missing extern declarations and add jemalloc.prof.lg_sample async metric
- Add missing `extern` for `jemalloc_profiler_sampling_rate` in
  `Keeper.cpp`, `LocalServer.cpp`, and `Server.cpp`
- Pass `jemalloc_profiler_sampling_rate` as 5th argument to
  `Jemalloc::setup` in `Keeper.cpp` and `LocalServer.cpp`
- Expose `prof.lg_sample` via `system.asynchronous_metrics` as
  `jemalloc.prof.lg_sample`
- Add integration test `test_jemalloc_profiler_sampling_rate`

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-25 14:31:07 +01:00
Daniil IvanikandGrant Holly 500341abd6 reintroducing Parquet footer cache 2026-02-24 14:41:00 -08:00
Anton Popov 07d1f5a026 Merge remote-tracking branch 'upstream/master' into HEAD 2026-02-24 13:45:33 +00:00
Anton Popov 271ac98247 add cache for single tokens 2026-02-20 20:38:43 +01:00
Anton Popov fb7135602c optimize reading from the text index dictionary 2026-02-20 19:29:25 +01:00
Raúl MarínandClaude Opus 4.6 9aa8e4ed27 Move ErrnoException from Exception.h to its own header
Extract `ErrnoException` into `Common/ErrnoException.h` to reduce the
include footprint of `Exception.h`. This removes `<base/errnoToString.h>`
and `<cerrno>` from `Exception.h`, which is included by ~95% of all
translation units.

All files referencing `ErrnoException` now include the new header
directly. Files that used `errnoToString` without `ErrnoException`
got an explicit `<base/errnoToString.h>` include.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-20 13:47:23 +01:00
Alexey Milovidov 223e53c76d Revert "Merge pull request #89750 from grantholly-clickhouse/parquet_footer_cache"
This reverts commit 5243ef786e, reversing
changes made to 6e3a5ad955.
2026-02-13 01:03:02 +01:00
divanik 5c33f119b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parquet_footer_cache 2026-02-09 13:28:20 +01:00
Grant Holly d4179cbd85 merging in upstream master 2026-02-07 13:16:29 -08:00
Alexey MilovidovandClaude Opus 4.6 9412039119 Read access_control_improvements from config in clickhouse-local
Allow overriding the default access control improvements via an
explicitly provided config file, instead of unconditionally hardcoding
them to true.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-07 01:07:40 +01:00
Alexey Milovidov 77c13f8916 Merge branch 'master' into embedded-server-sql-access-storage 2026-02-07 00:49:43 +01:00
Alexey MilovidovandClaude Opus 4.6 f31adb4a49 Fix CI failures: wrap long help text lines and fix git-import --help abort
- Wrap long lines in `getHelpHeader` and `getHelpFooter` for `clickhouse client`
  and `clickhouse local` to keep lines within reasonable width,
  fixing the 00910_client_window_size_detection test
- Use `createOptionsDescription` instead of raw `po::options_description` constructor
  in `git-import` and `static-files-disk-uploader` to handle zero terminal width
  properly, fixing the `git-import --help` abort in debug builds
  (03823_clickhouse_help_return_codes test)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-05 22:21:04 +01:00
divanik e52b0adfd1 Merge branch 'master' into parquet_footer_cache 2026-02-05 11:42:17 +01:00
Alexey MilovidovandClaude Opus 4.5 62ccfbd3aa Improve help output for ClickHouse applications
This commit improves the help output for all ClickHouse applications:

- Add `clickhouse help` command that lists all available subcommands
- Return exit code 0 for all help messages (previously some returned 1)
- Output help messages to stdout instead of stderr
- Use "clickhouse <cmd>" format instead of argv[0] in help messages
- Add dynamic `app_name` detection for `clickhouse client` and `clickhouse local`
  to show correct invocation name based on how the binary was called
- Add `--no-sudo` option for `clickhouse start` and `clickhouse restart`
  for environments without sudo (e.g., Docker containers)
- Add detailed help headers and footers for client and local with usage examples

Closes #58244

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 07:12:44 +01:00
Yakov OlkhovskiyandGitHub dd53f068fc fix 2026-02-03 19:32:47 -05:00
Alexey Milovidov 80b735a2f7 I think 1 Gi is enough 2026-02-04 00:44:03 +01:00
Alexey MilovidovandClaude Opus 4.5 73802adf52 Enable all access control improvements in embedded server and clickhouse-local
Set all `access_control_improvements` options to true for stricter
access control behavior in the embedded server config and clickhouse-local.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-29 03:37:32 +01:00
Grant Holly 1fcac4d2ab Merge branch 'master' into parquet_footer_cache 2026-01-28 13:47:14 -08:00
Antonio Andelic dc3d47d689 Enable implicit-int-float-conversion check 2026-01-20 09:35:57 +01:00
Grant Holly 7d51bbf3fc bringing in upstream 2026-01-19 12:12:50 -08:00
Robert Schulze c6486aa334 Minor readability improvements for client options 2026-01-18 14:27:09 +00:00
Alexey Milovidov 7f33341c50 Continuation 2026-01-17 23:13:01 +01:00
Alexey MilovidovandClaude Opus 4.5 caf664f3e9 Enable access management for default user in clickhouse-local
The default user in clickhouse-local was missing the access_management
privilege, which caused operations like DROP ROW POLICY IF EXISTS to
fail with ACCESS_DENIED error, even though the user should be unrestricted.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-01-17 21:49:30 +01:00
Grant Holly adaa6273ab bringing in upstream master 2026-01-04 12:16:24 -08:00
Cole Smith 91fa9c52d0 nit: comments
nit: comments
2025-12-31 11:00:23 -05:00
Cole Smith ba515bfbad no longer print time on initial query
whitespace
2025-12-30 23:10:54 -05:00
Grant Holly e8b2af6311 removing references to v2 Arrow cache 2025-12-13 19:59:09 -08:00
Grant Holly 8dd831c781 Merge branch 'master' into parquet_footer_cache 2025-11-29 17:34:07 -08:00
Kirill Nikiforov 3d297c7dea cleanup: a bit better handling of database data/store/metadata paths 2025-11-21 09:54:50 +04:00
Grant Holly 3e11b555fb bringing in upstream 2025-11-14 15:36:49 -08:00
Grant Holly 396e53ae3b more cleanup 2025-11-07 15:00:35 -08:00
Grant Holly d334c8bfe0 more cleanup 2025-11-07 14:52:10 -08:00
Grant Holly bc81218d1b moving all settings under parquet_metadata_cache_* and no longer having separate options for v3 caches 2025-11-07 14:49:15 -08:00
Grant Holly 4f007ae6a8 fixing copy paste error 2025-11-07 14:35:25 -08:00