The transform fast path in `FunctionsStringSearch::executeImpl` was gated only
on `rows > number_of_enum_members`, but `buildEnumHaystackDictionary` first
zero-fills a dense `value_to_index` array of `max_value - min_value + 1` slots.
For a sparse wide `Enum16` such as `Enum16('a' = -30000, 'b' = 30000)` (span
60001, 2 names), a small block (e.g. 3 rows) passed the old check and cleared
60001 slots just to replace three `castColumn` searches - a regression, even
though `EnumValues` itself avoids a dense direct lookup for `Enum16` spans above
`DIRECT_LOOKUP_THRESHOLD` (1024) for the same reason.
Compute the value span before building the dictionary and take the fast path
only when it provably saves work: there must be more rows than distinct names,
and the `O(span)` dense-array fill must not exceed the per-row work the
`castColumn` path would do anyway (`span <= rows`). The decision is per block,
so a wide enum still takes the fast path once a block is large enough to
amortize the fill; a small block over a wide sparse enum keeps the unchanged
`castColumn` path and therefore cannot regress relative to before the change.
Extend `04539_enum_string_search_dictionary` to cover the guard boundary: a
narrow-span `Enum16` (transform path), a wide sparse `Enum16` with a small
block (falls back to `castColumn`) and with a large block (transform path), all
checked against `toString(...)`. Add a wide sparse `Enum16` section to
`tests/performance/enum_string_operation.xml` (large forced blocks) to benchmark
that the optimization remains a speed-up for wide sparse enums when rows dominate.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The `iso_3166_1_alpha_2` `Enum8` declares the contiguous ranges `[-128, -7]`
and `[0, 127]`; the codes `-6` through `-1` are not declared. The fill query
`INSERT ... SELECT (rand(number) % 256) - 128` generated the full `[-128, 127]`
range, so about 2.3% of the rows carried an undefined code. The numeric-to-`Enum`
cast on `INSERT` validates each code and raises `UNKNOWN_ELEMENT_OF_ENUM`, so the
fill (and therefore the whole performance test) failed before any query ran. This
was the cause of the `Performance Comparison` shard failures.
Map a random value in `[0, 249]` onto the 250 valid codes instead, so the fill
inserts only declared enum values while still covering every distinct name.
This defect was carried over from the original PR #73772.
For an `Enum` haystack and a constant needle, functions such as `like`,
`position`, `match`, `countSubstrings` and `hasToken` previously materialized
the whole `Enum` column into a `String` column (`castColumn`) and searched
every row. This runs the search once per row even though there are only as
many distinct values as there are enum members.
This change runs the search over the distinct enum names only and maps the
results back per row through a plain array indexed by the enum value, as
requested in the issue. A plain array is used instead of a hash map because
`Enum` values are at most `Int16`. The optimization is applied only when the
needle and start position are the same for every row (constant needle, no
per-row start position); otherwise the previous `castColumn` path is used, so
non-constant needles and per-row start positions remain correct.
This supersedes #73772 by @zhanglistar (fork `bigo-sg/ClickHouse`, maintainer
edits disabled so the branch could not be updated in place). The original PR
implemented the same idea in `executeImpl` with a per-row
`std::unordered_map<Int32, Int32>`; this version replaces that with the plain
array the issue asked for and scopes the transform to the constant-needle case,
which also fixes a latent correctness issue in the original for non-constant
needles and per-row start positions.
Closes: https://github.com/ClickHouse/ClickHouse/issues/73114
Related: https://github.com/ClickHouse/ClickHouse/pull/73772
Co-authored-by: zhanglistar <[email protected]>
