76 Commits
Author SHA1 Message Date
Alexey MilovidovandGitHub 842b133f8d Merge pull request #100399 from ClickHouse/clang-tidy-init-variables
Enable clang-tidy check for uninitialized variables
2026-05-31 21:56:29 -07:00
Alexey MilovidovandClaude Opus 4.6 f41bf1d609 Fix CI failures: add static to functions missing prototypes, fix double-promotion warnings
- `Server.cpp`: Make `checkForUsersNotInMainConfig` static (removes need for style-violating forward declaration)
- `LibSSHLogger.cpp`: Include header in `#else` branch to provide prototype for `initialize`
- `Lexer.cpp`: Include `clickhouse_lexer.h` in standalone build to provide prototypes for `extern "C"` functions
- `compressor.cpp`/`decompressor.cpp`: Make all file-local functions static
- `phdr_cache.cpp`: Add NOLINT for intentional reserved identifier `__lsan_ignore_object`
- BuzzHouse: Add `static` to `CONV_FN`/`CONV_FN_QUOTE` macros and individual functions
- Test/example files: Add `static` to helper functions across ~50 files
- Fix `double-promotion` in `compressor.cpp`, `gtest_compressionCodec.cpp`, `FuzzConfig.cpp`

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 14:57:32 +00:00
Alexey MilovidovandClaude Opus 4.6 f5ee191db8 Initialize variables in utils/self-extracting-executable
Fix clang-tidy `cppcoreguidelines-init-variables` and
`cppcoreguidelines-pro-type-member-init` warnings in compressor.cpp
and decompressor.cpp.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 18:48:47 +00:00
Antonio Andelic 2832928464 Merge branch 'master' into implicit-int-conversion 2026-01-13 13:01:28 +01:00
Josh Carp 352407239b utils: switch to posix-standard api and fix free space calculation.
This patch switches from the non-portable statfs API to the
POSIX-standard statvfs API. We also fix the free space calculation,
which was previously only correct by coincidence. The previous
calculation used `f_blocks * st_blksize` to calculate available space.
However, `f_blocks` counts the number of blocks on the filesystem, not
the number of available blocks. And `st_blksize` is an advisory field,
not the actual block size. Using `st_blksize` will yield correct results
when it happens to be set to the same value as the true block size, but
this isn't always the case.

This patch uses f_bavail (blocks available) rather than f_blocks (total
blocks on fs), and f_frsize (block size) rather than st_blksize
(preferred block size).

Fixes #93896.
2026-01-13 04:24:37 +00:00
Raúl MarínandGitHub aeb743df51 Revert "Improve posix portability for illumos." 2026-01-12 13:17:05 +01:00
Antonio Andelic 36c4767a2a Merge branch 'master' into implicit-int-conversion 2026-01-12 09:31:50 +01:00
Josh Carp b0a6dcb295 Improve posix portability for illumos.
- Guard RLIMIT_NPROC usage; not defined on illumos.
- Include sys/filio.h for FIONREAD on illumos (not in sys/ioctl.h).
- Use posix statvfs in decompressor.
- Use DNS::hostName() instead of DNS::thisHost() on illumos.
- Fix fpclass() call for long double.

See also:
- https://github.com/oxidecomputer/garbage-compactor/blob/master/clickhouse/patches/0017-Use-statvfs-on-illumos.patch
- https://github.com/oxidecomputer/garbage-compactor/blob/master/clickhouse/patches/0018-Do-not-try-to-increase-the-thread-limit-via-proc.patch

Part of #23777.
2026-01-11 05:08:58 +00:00
Antonio Andelic e8f89fe9bc Fix build 2026-01-10 15:00:45 +01:00
Raúl Marín d6ac63c8d2 Style: Forbid the use of quotes for includes (partial) 2025-06-24 13:20:10 +02:00
Konstantin Bogdanov c844c4b613 Prepare for clang20 2025-04-25 15:06:45 +02:00
Nikita Mikhaylov c0ff1298ea Misc fixes 2024-12-23 10:58:20 +00:00
Nikita MikhaylovandNikita Mikhaylov a6d5854caa Fix decompressor 2024-12-22 00:48:16 +01:00
Nikita TaranovandGitHub 5e21ecba27 Merge pull request #73119 from ClickHouse/fix_build
Build with -Wvla-cxx-extension
2024-12-12 16:36:36 +00:00
Nikita Taranov 5eae4e2294 fix 2024-12-11 22:55:43 +01:00
Robert Schulze 7d6fbe5a57 Some clang-tidy fixes 2024-12-11 16:12:16 +00:00
Nikita Taranov 0b3f7d18ef fix 2024-12-11 16:46:02 +01:00
Yakov Olkhovskiy 3aefd0f50b copy on mac instead renaming 2024-02-12 18:01:43 +00:00
Yakov Olkhovskiy 648fa52583 move instead copy 2024-02-06 18:48:57 +00:00
Antonio Andelic fce276c441 Print info while decompressing 2023-11-06 09:29:02 +00:00
Robert Schulze e4c081652d Suppress more checks 2023-09-26 11:01:35 +00:00
Robert Schulze 50c51c2854 Fix new clang-tidy-s 2023-09-20 22:17:39 +00:00
Azat Khuzhin 1fb7605fb4 Fix self extracting binaries under qemu linux-user (qemu-$ARCH-static)
The problem was that the decompressor uses realpath(/proc/self/exe)
instead of readlink(/proc/self/exe), while realpath() does lots of
trickerly [1] which leads to bypassing qemu linux-user override [2] of
/proc/self/exe to the executable with with it had been called -- and
the reason for this is that the getpid() after unshare returns 1, while
reading /proc/self returns the pid that was before unshare (from the
chroot) [3].

  [1]: https://github.com/bminor/glibc/blob/4290aed05135ae4c0272006442d147f2155e70d7/stdlib/canonicalize.c#L223
  [2]: https://github.com/qemu/qemu/blob/ed8ad9728a9c0eec34db9dff61dfa2f1dd625637/linux-user/syscall.c#L8634
  [3]: https://gist.github.com/azat/fcbd8b6c26afd505ae5f3387fc15f0e2

But note, that even after this patch qemu without binfmt will not work,
due to internally the code calls execv() while qemu does not handle it
(see [4]).

  [4]: https://patchwork.kernel.org/project/qemu-devel/patch/[email protected]/

Signed-off-by: Azat Khuzhin <[email protected]>
2023-07-17 13:10:40 +02:00
Azat Khuzhin 16165d9498 Improve error messages for decompressor
Signed-off-by: Azat Khuzhin <[email protected]>
2023-07-17 13:10:40 +02:00
Azat Khuzhin c374653c7b Revert "Merge pull request #52138 from azat/decompressor-inode"
This reverts commit 6524031348, reversing
changes made to 9bf114f9a3.

This was not a good idea, since the underlying problem was that
`/proc/self/exe` was pointing to `qemu-$ARCH-static` (because the code
uses realpath() over normal interface readlink(), which is not caught by
the qemu linux-user).

And this means that later, it will try to overwrite incorrect binary and
then execute some garbage.

Signed-off-by: Azat Khuzhin <[email protected]>
2023-07-17 13:10:29 +02:00
Azat Khuzhin 20a671b8cf Skip protection from double decompression if inode from maps cannot be obtained
Under some circumstances, like using qemu-$ARCH-static, /proc/self/maps
will not contain information about /proc/self/exe.

Well, strictly speaking it does contains, however qemu will not pass it
to the user program:

<details>

<summary>strace</summary>

    $ sudo strace -s10000 -f arch-chroot . /qemu-riscv64-static /clickhouse
    ...
    execve("/qemu-riscv64-static", ["/qemu-riscv64-static", "/clickhouse"], 0x7fffffffe458 /* 20 vars */) = 0
    readlinkat(AT_FDCWD, "/proc/self/exe", "/qemu-riscv64-static", 4096) = 20
    openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 4
    [pid  3126] read(4, "00010000-00111000 r--p 00000000 fe:01 30312571                           /clickhouse\n00111000-00119000 r--p 00100000 fe:01 30312571                           /clickhouse\n00119000-0011a000 rw-p 00108000 fe:01 30312571                           /clickhouse\n0011a000-0013d000 rw-p 00000000 00:00 0 \n4000000000-4000001000 ---p 00000000 00:00 0 \n4000001000-4000801000 rw-p 00000000 00:00 0 \n4000801000-400081a000 r--p 00000000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081a000-400081b000 ---p 00000000 00:00 0 \n400081b000-400081c000 r--p 00019000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081c000-400081e000 rw-p 0001a000 fe:01 30316932                       /lib/riscv64-linux-gnu/ld-2.32.so\n400081e000-400081f000 r--p 00000000 00:00 0 \n400081f000-4000922000 r--p 00000000 fe:01 30316935 /lib/riscv64-linux-gnu/libc-2.32.so\n4000922000-4000926000 r--p 00102000 fe:01 30316935                       /lib/riscv64-linux-gnu/libc-2.32.so\n4000926000-4000928000 rw-p 00106000 fe:01 30316935                       /lib/riscv64-linux-gnu/libc-2.32.so\n4000928000-400092d000 rw-p 00000000 00:00 0 \n400092d000-40009af000 r--p 00000000 fe:01 30316943                       /lib/riscv64-linux-gnu/libm-2.32.so\n40009af000-40009b0000 r--p 00081000 fe:01 30316943                       /lib/riscv64-linux-gnu/libm-2.32.so\n40009b0000-40009b1000 rw-p 00082000 fe:01 30316943 /lib/riscv64-linux-gnu/libm-2.32.so\n40009b1000-40009c5000 r--p 00000000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c5000-40009c6000 r--p 00013000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c6000-40009c7000 rw-p 00014000 fe:01 30316946                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n40009c7000-40009cb000 rw-p 00000000 00:00 0 \n40009cb000-40009cd000 r--p 00000000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009cd000-40009ce000 r--p 00001000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009ce000-40009cf000 rw-p 00002000 fe:01 30316939                       /lib/riscv64-linux-gnu/libdl-2.32.so\n40009cf000-40009d1000 rw-p 00000000 00:00 0 \n7fffe8000000-7fffeffff000 rwxp 00000000 00:00 0 \n7fffeffff000-7ffff0000000 ---p 00000000 00:00 0 \n7ffff0000000-7ffff0021000 rw-p 00000000 00:00 0 \n7ffff0021000-7ffff4000000 ---p 00000000 00:00 0 \n7ffff6b4b000-7ffff6b5b000 rw-p 00000000 00:00 0 \n7ffff71ff000-7ffff7200000 ---p 00000000 00:00 0 \n7ffff7200000-7ffff7a00000 rw-p 00000000 00:00 0\n7ffff7a00000-7ffff7a3c000 r--p 00000000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7a3c000-7ffff7c74000 r-xp 0003c000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7c74000-7ffff7d77000 r--p 00274000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7d77000-7ffff7dce000 r--p 00377000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7dce000-7ffff7df7000 rw-p 003ce000 fe:01 30316953                   /qemu-riscv64-static\n7ffff7df7000-7ffff7e0c000 rw-p 00000000 00:00 0                          [heap]\n7ffff7e0c000-7ffff7e70000 rw-p 00000000 00:00 0                          [heap]\n7ffff7f42000-7ffff7ff9000 rw-p 00000000 00:00 0 \n7ffff7ff9000-7ffff7ffd000 r--p 00000000 00:00 0                          [vvar]\n7ffff7ffd000-7ffff7fff000 r-xp 00000000 00:00 0                          [vdso]\n7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]\nffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]\n", 4096) = 3608
    [pid  3126] read(4, "", 1024)           = 0
    [pid  3126] close(4)                    = 0
    [pid  3126] write(3, "10000-111000 r-xp 00000000 fe:01 30312571", 41) = 41
    [pid  3126] write(3, "                                /clickhouse\n", 44) = 44
    [pid  3126] write(3, "111000-119000 r--p 00100000 fe:01 30312571", 42) = 42
    [pid  3126] write(3, "                               /clickhouse\n", 43) = 43
    [pid  3126] write(3, "119000-11a000 rw-p 00108000 fe:01 30312571", 42) = 42
    [pid  3126] write(3, "                               /clickhouse\n", 43) = 43
    [pid  3126] write(3, "11a000-13d000 rw-p 00000000 00:00 0", 35) = 35
    [pid  3126] write(3, "                                      \n", 39) = 39
    [pid  3126] write(3, "4000000000-4000001000 ---p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "4000001000-4000801000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              [stack]\n", 38) = 38
    [pid  3126] write(3, "4000801000-400081a000 r-xp 00000000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57 <unfinished ...>
    [pid  3127] <... clock_nanosleep resumed>0x7ffff79ff060) = 0
    [pid  3126] <... write resumed>)        = 57
    [pid  3127] clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000},  <unfinished ...>
    [pid  3126] write(3, "400081a000-400081b000 ---p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400081b000-400081c000 r--p 00019000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57) = 57
    [pid  3126] write(3, "400081c000-400081e000 rw-p 0001a000 fe:01 30316932", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/ld-2.32.so\n", 57) = 57
    [pid  3126] write(3, "400081e000-400081f000 r-xp 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400081f000-4000922000 r-xp 00000000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000922000-4000926000 r--p 00102000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000926000-4000928000 rw-p 00106000 fe:01 30316935", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libc-2.32.so\n", 59) = 59
    [pid  3126] write(3, "4000928000-400092d000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "400092d000-40009af000 r-xp 00000000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009af000-40009b0000 r--p 00081000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009b0000-40009b1000 rw-p 00082000 fe:01 30316943", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libm-2.32.so\n", 59) = 59
    [pid  3126] write(3, "40009b1000-40009c5000 r-xp 00000000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c5000-40009c6000 r--p 00013000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c6000-40009c7000 rw-p 00014000 fe:01 30316946", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libpthread-2.32.so\n", 65) = 65
    [pid  3126] write(3, "40009c7000-40009cb000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31
    [pid  3126] write(3, "40009cb000-40009cd000 r-xp 00000000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009cd000-40009ce000 r--p 00001000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009ce000-40009cf000 rw-p 00002000 fe:01 30316939", 50) = 50
    [pid  3126] write(3, "                       /lib/riscv64-linux-gnu/libdl-2.32.so\n", 60) = 60
    [pid  3126] write(3, "40009cf000-40009d1000 rw-p 00000000 00:00 0", 43) = 43
    [pid  3126] write(3, "                              \n", 31) = 31

</details>

Signed-off-by: Azat Khuzhin <[email protected]>
2023-07-16 08:17:14 +02:00
Yakov OlkhovskiyandGitHub 07a3988749 geteuid is enough 2023-03-08 08:26:07 -05:00
Yakov OlkhovskiyandGitHub aacd05e34b bugfix + review suggestion 2023-03-06 21:24:04 -05:00
Yakov OlkhovskiyandGitHub a8ceab1366 NOLINT for getenv 2023-03-03 09:24:58 -05:00
Yakov Olkhovskiy df41a83ddb set uid gid to file's original 2023-03-02 14:49:49 +00:00
Yakov Olkhovskiy 566a0e166f preserve uid gid if running with sudo 2023-03-02 00:42:02 +00:00
Azat Khuzhin 6587d11885 Check return value of close() in self-extracting-executable
And also add description into perror() over callign simply
perror(nullptr).

Signed-off-by: Azat Khuzhin <[email protected]>
2023-02-07 11:28:22 +01:00
Yakov Olkhovskiy c8f635e40b WSL1 inconsistency 2023-01-16 18:32:41 +00:00
Azat Khuzhin 15bcd6250a Fix -Wshorten-64-to-32 for darwin builds
Signed-off-by: Azat Khuzhin <[email protected]>
2022-10-21 13:25:42 +02:00
Azat Khuzhin 8414ea0691 Fix -Wshorten-64-to-32 in self extracting executable
Signed-off-by: Azat Khuzhin <[email protected]>
2022-10-21 13:25:42 +02:00
Yakov Olkhovskiy 1338f7d65e disable inode lock for macos and freebsd 2022-09-05 15:41:52 +00:00
Yakov Olkhovskiy a9cab86a73 typo 2022-08-30 01:46:17 +00:00
Yakov Olkhovskiy 7a85a943f1 fix close lock on exec, refactoring, comments added 2022-08-30 01:11:04 +00:00
Yakov Olkhovskiy ae82c68725 truncate lock 2022-08-25 10:52:56 +00:00
Yakov Olkhovskiy 6c1a7205e3 fix style 2022-08-24 21:47:37 +00:00
Yakov Olkhovskiy f043d03053 fix style 2022-08-24 21:28:52 +00:00
Yakov Olkhovskiy 6afd9c176b inode based singleton 2022-08-24 21:05:00 +00:00
Yakov OlkhovskiyandGitHub d11d494587 Merge pull request #40426 from ClickHouse/util-compressor-exec
Self-extracting: add --exec param, allow compressed to be renamed
2022-08-23 11:20:28 -04:00
Yakov Olkhovskiy 088a969500 comments added 2022-08-22 12:14:29 +00:00
Yakov Olkhovskiy 47d5a64f13 fix style 2022-08-21 22:06:11 +00:00
Yakov Olkhovskiy 32717a844e allow empty --exec to signify no executable to run, add some stats printout 2022-08-21 21:39:24 +00:00
Yakov Olkhovskiy 5be782a7e6 fix buffer overflow 2022-08-20 21:02:36 +00:00
Alexey Milovidov 74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Yakov Olkhovskiy d81ec25027 add --exec param, allow compressed to be renamed 2022-08-19 20:43:10 +00:00
Yakov Olkhovskiy 2c3732ed3a extract real path of executable instead of argv[0] 2022-08-08 17:56:16 -04:00