mirror of
https://github.com/hanzoai/datastore.git
synced 2026-09-26 13:55:07 +00:00
Alexey MilovidovandClaude Opus 4.8
886119a07f
The performance comparison framework could only run SQL queries over the native protocol. Extend it to also accept shell scripts, marked with `<query type="shell">`, which are run with `bash` and timed end to end. This covers measurements the native protocol cannot express: HTTP latency, response compression, and tool startup time. `perf.py` runs each shell-script query once per server, building per-server `$CLICKHOUSE_*` environment variables (`CLICKHOUSE_BINARY`, host, TCP/HTTP ports, `CLICKHOUSE_CLIENT`, `CLICKHOUSE_LOCAL`, `CLICKHOUSE_CURL`, `CLICKHOUSE_URL`) that mirror the stateless tests in `shell_config.sh`, so the reference and the patched build are measured each on their own. The timing flows into the report as the `client_time` metric, exactly like a SQL query. Parameter substitutions and `<settings>` are not applied to shell scripts, and the server-side profiler runs are skipped for them. Both run paths pass the binary paths and HTTP ports to `perf.py`: `ci/jobs/performance_tests.py` (`CHServer`) for CI and `ci/jobs/scripts/perf/compare.sh` for local runs. The perf-comparison config removes `<http_port>`, so it is re-enabled per server on the command line (a documented config override) with distinct ports for the reference and the patched server. Fix a latent bug in `analyze_queries`: `version` and `query_id` were taken from the non-preserved (left) side of the `RIGHT JOIN` against `query_log`. For runs without a `query_log` row (shell scripts) they defaulted to `0`/`''` and collapsed both servers' runs onto version `0`. Qualify them to `query_runs`. SQL queries are unaffected because their `query_log` row always matches. Add two shell-script performance tests: - `clickhouse_local_startup.xml`: `clickhouse-local --query "SELECT 1"` startup - `http_read_gzip.xml`: reading ~1 MB over HTTP, with and without gzip Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
34 lines
1.5 KiB
XML
34 lines
1.5 KiB
XML
<test>
|
|
<!--
|
|
End-to-end latency of reading data over the HTTP interface, with and
|
|
without gzip response compression.
|
|
|
|
`SELECT number FROM numbers(500000) FORMAT TSV` produces about 3.2 MB of
|
|
text, which gzip compresses to about 1 MB. Comparing the plain and the
|
|
gzip variant, and the reference (LEFT) against the patched (RIGHT)
|
|
binary, shows the cost of the HTTP layer and of the gzip codec.
|
|
|
|
These are shell-script queries (type="shell"): they are run with bash and
|
|
timed end to end, talking to the server over HTTP via $CLICKHOUSE_CURL /
|
|
$CLICKHOUSE_URL, which are prepared per-server (the LEFT and the RIGHT
|
|
server listen on different HTTP ports). The compressed bytes are written
|
|
to /dev/null; client-side decompression is left out on purpose, as it is
|
|
the same for both builds and would only add noise.
|
|
-->
|
|
|
|
<!-- Plain HTTP response, no compression (baseline). -->
|
|
<query type="shell"><![CDATA[
|
|
${CLICKHOUSE_CURL} "${CLICKHOUSE_URL}" \
|
|
--data-binary "SELECT number FROM numbers(500000) FORMAT TSV" \
|
|
-o /dev/null
|
|
]]></query>
|
|
|
|
<!-- gzip-compressed HTTP response (~1 MB transferred). -->
|
|
<query type="shell"><![CDATA[
|
|
${CLICKHOUSE_CURL} -H 'Accept-Encoding: gzip' \
|
|
"${CLICKHOUSE_URL}?enable_http_compression=1" \
|
|
--data-binary "SELECT number FROM numbers(500000) FORMAT TSV" \
|
|
-o /dev/null
|
|
]]></query>
|
|
</test>
|