mirror of
https://github.com/hanzoai/datastore.git
synced 2026-09-26 13:55:07 +00:00
ManuelandClaude Fable 5
6b80338f3b
wide_integer_from_builtin and wide_integer_from_wide_integer sign-extend
through an `if (rhs < 0)` branch that selects between two separate fill
loops. In columnar conversion loops (ConvertImpl) this data-dependent
branch mispredicts on mixed-sign data and prevents vectorization even
when it predicts well.
Select the extension value (0 or -1) branchlessly instead and use a
single fill loop.
Standalone kernel benchmark of the columnar conversion loop (clang 21,
-march=x86-64-v3, 64k-element blocks, GB/s of processed data):
conversion before after
Int64 -> Int256, random sign 10.3 73.9
Int64 -> Int256, positive only 31.3 73.3
Int128 -> Int256, random sign 12.2 68.0
Int64 -> Int128 61.5 62.2
UInt64 -> UInt128/UInt256 unchanged
The same effect is visible at the query level: with the official
26.6.1 binary, `toInt256(toInt64(cityHash64(number)))` over 100M rows
takes 6.8x the time of the equivalent unsigned conversion in a single
thread, and 3.5x the time of the same conversion with positive-only
values.
Result equivalence with the previous implementation was verified over
36 million checks: all builtin integral types (int8/16/32/64, unsigned
variants) converted to UInt128/Int128/UInt256/Int256 including type
boundary values, 128-to-256-bit widening for all signedness
combinations, narrowing, and constexpr evaluation.
Co-Authored-By: Claude Fable 5 <[email protected]>
Contains general C / C++ utilities used in the project.
Per project:
base/: Many small utilities, including basic type definition. They are independent and unrelated to the database logic.glibc-compatibility/: Wrappers around glibc to keep compatibility with old glibc versions. The aim for the future is to remove this and depend on a static musl build instead (see https://github.com/ClickHouse/ClickHouse/issues/75847).harmful/: A library that traps whenever harmful functions from libc are called. Used to detect dangerous libc function usage in CI.pcg-random/: External library to generate random numbers.poco/: A set of C++ libraries to build programs. This is forked and modified from the original. We aim to slowly remove it from the project completely with our own code or STD when possible, not update it or add new features to it.readpassphrase/: External library to read passwords via terminal prompt.widechar_width/: External library to implementwcwidthwhich measures the width of unicode strings.