mirror of
https://github.com/hanzoai/datastore.git
synced 2026-09-26 13:55:07 +00:00
Alexey MilovidovandClaude Opus 4.8
886119a07f
The performance comparison framework could only run SQL queries over the native protocol. Extend it to also accept shell scripts, marked with `<query type="shell">`, which are run with `bash` and timed end to end. This covers measurements the native protocol cannot express: HTTP latency, response compression, and tool startup time. `perf.py` runs each shell-script query once per server, building per-server `$CLICKHOUSE_*` environment variables (`CLICKHOUSE_BINARY`, host, TCP/HTTP ports, `CLICKHOUSE_CLIENT`, `CLICKHOUSE_LOCAL`, `CLICKHOUSE_CURL`, `CLICKHOUSE_URL`) that mirror the stateless tests in `shell_config.sh`, so the reference and the patched build are measured each on their own. The timing flows into the report as the `client_time` metric, exactly like a SQL query. Parameter substitutions and `<settings>` are not applied to shell scripts, and the server-side profiler runs are skipped for them. Both run paths pass the binary paths and HTTP ports to `perf.py`: `ci/jobs/performance_tests.py` (`CHServer`) for CI and `ci/jobs/scripts/perf/compare.sh` for local runs. The perf-comparison config removes `<http_port>`, so it is re-enabled per server on the command line (a documented config override) with distinct ports for the reference and the patched server. Fix a latent bug in `analyze_queries`: `version` and `query_id` were taken from the non-preserved (left) side of the `RIGHT JOIN` against `query_log`. For runs without a `query_log` row (shell scripts) they defaulted to `0`/`''` and collapsed both servers' runs onto version `0`. Qualify them to `query_runs`. SQL queries are unaffected because their `query_log` row always matches. Add two shell-script performance tests: - `clickhouse_local_startup.xml`: `clickhouse-local --query "SELECT 1"` startup - `http_read_gzip.xml`: reading ~1 MB over HTTP, with and without gzip Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
17 lines
736 B
XML
17 lines
736 B
XML
<test>
|
|
<!--
|
|
Measure how fast `clickhouse-local` starts up. The query is trivial
|
|
(`SELECT 1`), so the measured wall-clock time is dominated by process
|
|
startup: dynamic linking, static initializers, context setup, etc.
|
|
|
|
This is a shell-script query (type="shell"): it is run with bash and
|
|
timed end to end, instead of being sent over the native protocol. The
|
|
$CLICKHOUSE_* environment variables are prepared per-server, so the
|
|
reference (LEFT) and the patched (RIGHT) binary are measured each on
|
|
their own, exactly like the SQL performance tests.
|
|
-->
|
|
<query type="shell"><![CDATA[
|
|
$CLICKHOUSE_LOCAL --query "SELECT 1" > /dev/null
|
|
]]></query>
|
|
</test>
|