The mirror override was only applied to a subset of images. Every other
arm64 CI image built directly `FROM ubuntu` still resolved through
`ports.ubuntu.com` and was subject to the same failure.
Add a generic `build_args` field to `Docker.Config` and pass the
in-region AWS mirror (`apt_archive` / `apt_ports_archive`, set literally in
`ci/defs/defs.py`) to the Ubuntu-based CI images on the command line.
`ci/praktika/docker.py` stays generic - it just forwards `build_args` and
knows nothing about apt or the region. Images that don't declare the args
ignore them.
Dockerfile defaults stay canonical so local builds are unchanged, and the
remaining direct-`ubuntu` arm64 images (`style-test`, `install-deb-test`,
`sqlancer-test`, `mysql_dotnet_client`, `postgresql-java-client`) get a
`ports`- and deb822-aware rewrite so the passed mirror reaches arm64.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The NightlySQLancer / SQLancerPP job had three independent problems, surfaced
while investigating run
https://github.com/ClickHouse/ClickHouse/actions/runs/28698784006 (both jobs
were green on GitHub while the praktika report was actually FAIL/ERROR).
1. SQLancer++ WHERE and QUERY_PARTITIONING oracles always failed with
`java.lang.AssertionError: null`. When the WHERE-stripped baseline query hits
an expected/whitelisted ClickHouse error, `executeAndGet` returns null and
`ComparatorHelper` throws `IgnoreMeException` (a "skip this iteration"
signal). `GeneralQueryPartitioningWhere.check` catches it and, for simple
queries, re-throws it as an `AssertionError` - a guaranteed false positive
against ClickHouse (QUERY_PARTITIONING is a composite that contains this
oracle). Patch the oracle in `swap-clickhouse-jdbc.py` (the existing build-time
patch mechanism, alongside the NoREC fix) to let `IgnoreMeException`
propagate; genuine unexpected errors are unaffected.
2. `sqlancer_pp_job.sh` wrote its result to `result.json` instead of
`result_<normalized_job_name>.json`, so praktika never read it and reported
`ERROR "Job killed or terminated, no Result provided"`, discarding every
oracle status and attached log. It also emitted `null` start_time/duration,
which the CIDB inserter rejects. Both fixed to mirror `sqlancer_job.sh`.
3. On failure, both jobs now archive SQLancer's per-database reproducer logs
(`logs/<dbms>/databaseN-cur.log`, written with --log-each-select) as a
downloadable artifact, so the exact CREATE/INSERT/.../SELECT sequence that
triggered a bug is one click away instead of requiring reconstruction from
the interleaved multi-threaded stdout.
Run the SQLancer++ job in the same clickhouse/sqlancer-test image the SQLancer
job uses, instead of a dedicated clickhouse/sqlancer-pp-test image:
- Build SQLancer++ into ci/docker/sqlancer-test/Dockerfile alongside the
SQLancer fork, under Temurin JDK 25, with the clickhouse-jdbc 0.9.8 (client-v2)
swap. jacoco is skipped (0.8.8 can't read JDK 25 class files; tests are
skipped anyway). SQLancer++ targets Java 11 bytecode, which JDK 25 compiles.
- Move swap-clickhouse-jdbc.py into the sqlancer-test build context.
- Remove ci/docker/sqlancer-pp-test/ and its Docker.Config; point the
sqlancer_pp job at run_in_docker=clickhouse/sqlancer-test and track that
image in its digest.
This also fixes the PR CI failure where the brand-new sqlancer-pp-test
multi-arch image could not complete the manifest merge (its per-arch tags were
never in the registry). Reusing the existing image sidesteps that entirely.
Validated locally (amd): the combined image builds both jars under JDK 25, and
all four SQLancer++ oracles (WHERE, NoREC, QUERY_PARTITIONING, FUZZING) run
multi-threaded via the client-v2 clickhouse-jdbc 0.9.8 driver at 67-78%
statement success with zero assertions.
- The `clickhouse/sqlancer-test` image is built for both arm and amd
(`platforms=arm_amd`), but `JAVA_HOME` was hardcoded to the amd64 Temurin
path, which would break the arm64 image build (the path is `-arm64` there,
and `mvn package` runs during the build). Symlink the arch-specific Temurin
dir to a stable `/usr/lib/jvm/temurin-25-jdk` and point `JAVA_HOME` at it.
- Correct the config-override comment in the job script: the merged provider
ships `system_logs_disabled.xml` (full `remove`, not a TTL) and
`alter_mutation_sync.xml`. Verified empirically that with the binary's
embedded config (`users_config` == `config.xml`), `<profiles>` overrides in
`config.d/` reach the default profile (`async_insert=0`, `alter_sync=2`,
`mutations_sync=2`), so all overrides belong in `config.d/` -- the packaged
server's `users.d/` split would silently drop the mutation-sync settings.
Bump the SQLancer provider to ClickHouse/sqlancer `main` after PR #4
(https://github.com/ClickHouse/sqlancer/pull/4): the wrong-result oracle
suite with 31 oracles, a recursive type system, and the
`com.clickhouse:client-v2` (RowBinary) transport replacing `clickhouse-jdbc`.
Docker image (`ci/docker/sqlancer-test/Dockerfile`):
- Bump the cloned commit to `a62ea509ee53862b5dd37f452e8c2edce8038951`.
- The new `pom.xml` targets Java 25, which Ubuntu 22.04 `default-jdk`/`maven`
cannot build, so install Temurin JDK 25 (Adoptium) and Maven 3.9.9, and add
`-Djacoco.skip=true` to the build (jacoco 0.8.12 rejects Java 25 classfiles).
Job script (`ci/jobs/sqlancer_job.sh`):
- Apply the provider's server-side config overrides from
`.claude/clickhouse-config/` (logger to `warning`, remove `system.trace_log`,
TTL the system-log tables, pin `async_insert` off) by starting the server
from a directory containing `config.d/`; the embedded default `config.xml`
still merges `config.d/*.xml` relative to the server CWD.
- Replace the per-oracle loop with a single combined run of all 25 oracles
bounded by one wall-clock timeout, matching the upstream
`.claude/run-sqlancer.sh` reference, and enable `--random-session-settings`.
`-DskipTests` only skips test execution but still compiles tests.
The test code references `ClickHouseDataType` which no longer exists
after the JDBC driver upgrade, causing a compilation failure.
Use `-Dmaven.test.skip=true` instead, which skips both test
compilation and execution.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Pin to ClickHouse/sqlancer commit 308cb3b that upgrades the JDBC driver
from ru.yandex.clickhouse:clickhouse-jdbc:0.3.2 to
com.clickhouse:clickhouse-jdbc:0.9.6.
The old driver failed to decompress error responses from the server,
throwing "Magic is not correct" IOException instead of the actual error.
This caused all SQLancer tests to fail with 0 queries executed.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
SQLancer was disabled after https://github.com/ClickHouse/ClickHouse/pull/81835
which forbids creating tables without insertable columns (e.g., with only
`MATERIALIZED` or `ALIAS` columns). SQLancer's random table generator could
produce such definitions, causing an `AssertionError`.
The fix adds `EMPTY_LIST_OF_COLUMNS_PASSED` to SQLancer's expected errors.
The table creation retry loop in `ClickHouseProvider.generateDatabase` already
handles expected errors by regenerating the table definition.
The fix was submitted upstream: https://github.com/sqlancer/sqlancer/pull/1296
and applied to the ClickHouse fork: https://github.com/ClickHouse/sqlancer
This PR switches CI to use the ClickHouse fork (pinned to a specific commit)
and re-enables the SQLancer job in the master workflow.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
