hanzoai/dev
2
0
Fork 0
mirror of https://github.com/hanzoai/dev.git synced 2026-10-17 15:34:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(release): carry compiled crates between runs through sccache and the org's store

Browse Source
This commit is contained in:
zeekay
2026-10-09 05:41:34 -07:00
parent b32b3122d2
commit 2fa7f5e3c4
2 changed files with 154 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/release.yml
+131
View File
@@ -24,6 +24,12 @@
# platform packages, then @hanzo/dev), pushes the Homebrew formula, and lands
# the version-bump commit on main. The forge mirrors this repository, tags
# included, so nothing here pushes to it.
#
# A runner lives for one job, so each build would compile all ~1,000 crates
# from nothing. sccache wraps every rustc call and keys each object by the hash
# of what went into it, and hanzoai/ci's bin/cache carries those objects and the
# crate downloads to the next run through this org's store (/v1/s3, the IAM
# identity KMS mints, no bucket credential).
name: Release
on:
push:
@@ -44,6 +50,12 @@ env:
XWIN_CRATE_VERSION: '0.23.1'
MACOS_SDK_URL: 'https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX11.3.sdk.tar.xz'
MACOS_SDK_SHA256: 'cd4f08a75577145b8f05245a2975f7c81401d75e9535dcffbb879ee1deefcbf4'
# The musl release tarball, by version and by its sha256 per runner arch.
SCCACHE_VERSION: '0.18.0'
SCCACHE_SHA256_X86_64: '45f1447fbe231e3037bde351ef70677dd212216c8d62ae7ca409fecc4d6acc89'
SCCACHE_SHA256_AARCH64: '2b3284d5da3b46a47dc4229e75bb7b88ac4aa99c8d754fb7d2f84997e5a4354a'
# hanzoai/ci at the commit whose bin/cache this release runs.
CI_SHA: '386b7a26942215254a1964f107ad5e8715e8349e'
jobs:
version:
if: "!contains(github.event.head_commit.message, '[skip ci]')"
@@ -180,6 +192,90 @@ jobs:
CH=$(sed -n 's/^channel = "\(.*\)"/\1/p' rust-toolchain.toml)
rustup toolchain install "$CH" --profile minimal
rustup target add --toolchain "$CH" ${{ matrix.target.rust }}
- name: sccache, and the org's cache tool
# sccache keys an object by the hash of what went into it — source
# bytes, flags, the compiler, the CARGO_* environment — never by mtime,
# so the upstream files `make prepare` rewrites on every run still hit
# when their bytes did not move. The wrapper is set for every later
# step: cargo-zigbuild and cargo-xwin both run rustc through it, and
# cc-rs routes the C the crates carry through it too.
#
# An accelerator, not a dependency: a download that fails or does not
# match its pin leaves RUSTC_WRAPPER unset and the build runs cold.
run: |
set +e # GitHub runs this under bash -e; nothing in this step may fail the build
set -uo pipefail
case "$(uname -m)" in
x86_64) arch=x86_64; want=$SCCACHE_SHA256_X86_64 ;;
aarch64) arch=aarch64; want=$SCCACHE_SHA256_AARCH64 ;;
*) echo "::notice::no sccache is pinned for $(uname -m), so this build runs cold"; exit 0 ;;
esac
n="sccache-v${SCCACHE_VERSION}-${arch}-unknown-linux-musl"
d="$RUNNER_TEMP/sccache-bin"; mkdir -p "$d"
curl -fsSL -m 120 "https://github.com/mozilla/sccache/releases/download/v${SCCACHE_VERSION}/$n.tar.gz" -o "$d/s.tgz" \
|| { echo "::notice::sccache did not download, so this build runs cold"; exit 0; }
echo "$want $d/s.tgz" | sha256sum -c --quiet - \
|| { echo "::warning::$n.tar.gz does not match its pinned sha256; it is not run, and this build runs cold"; exit 0; }
if ! { tar -xzf "$d/s.tgz" -C "$d" --strip-components=1 "$n/sccache" && "$d/sccache" --version; }; then
echo "::notice::sccache did not unpack, so this build runs cold"; exit 0
fi
# No idle timeout: one crate on a crowded node can compile for longer
# than the default ten minutes, and a server that left mid-build would
# split the statistics printed at the end.
{
echo "RUSTC_WRAPPER=$d/sccache"
echo "SCCACHE_DIR=$RUNNER_TEMP/sccache"
echo "SCCACHE_CACHE_SIZE=15G"
echo "SCCACHE_IDLE_TIMEOUT=0"
} >> "$GITHUB_ENV"
echo "$d" >> "$GITHUB_PATH"
if curl -fsSL -m 60 "https://raw.githubusercontent.com/hanzoai/ci/$CI_SHA/bin/cache" -o "$RUNNER_TEMP/cache"; then
chmod +x "$RUNNER_TEMP/cache"
else
echo "::notice::hanzoai/ci@$CI_SHA bin/cache did not download, so nothing is carried between runs"
fi
- name: Restore the Rust cache (this org's store)
# Before the version stamp, because the key hashes the tree as it was
# committed: the stamp rewrites Cargo.lock and patches/upstream.json on
# every run, and a key taken after it would never be seen twice. And
# before Zig, so `cargo install` starts warm too.
#
# The name is what an archive is only valid for: the compiler, the host
# it runs on, sccache's own version, and the target. The key is what
# decides its contents: the lock, the toolchain pin, the owned edits,
# and every flag and cross-toolchain pin in the environment, which
# sccache hashes too. A key that misses restores the newest one saved
# for that name, so a lock bump still reuses every crate that did not
# move.
#
# The IAM token is minted here and handed to bin/cache alone, never to
# GITHUB_ENV: the Build step runs every crate's build script and proc
# macro, and none of them needs this org's identity.
id: rustcache
if: env.RUSTC_WRAPPER != ''
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
TARGET: ${{ matrix.target.rust }}
run: |
set +e # GitHub runs this under bash -e; nothing in this step may fail the build
set -uo pipefail
[ -x "$RUNNER_TEMP/cache" ] || { echo "::notice::no bin/cache, so this build starts cold"; exit 0; }
name="$({ rustc -vV; sccache --version; } | sha256sum | cut -c1-16)-$TARGET"
key=$({ cat Cargo.lock rust-toolchain.toml patches/*.json
env | grep -E '^(RUSTFLAGS|CARGO_[A-Z0-9_]+|ZIG[A-Z0-9_]*|LLVM_[A-Z0-9_]+|XWIN_[A-Z0-9_]+|MACOS_SDK_[A-Z0-9_]+)=' | sort
} | sha256sum | cut -c1-32)
echo "name=$name" >> "$GITHUB_OUTPUT"; echo "key=$key" >> "$GITHUB_OUTPUT"
T=$(printf '{"clientId":"%s","clientSecret":"%s"}' "${KMS_CLIENT_ID:-}" "${KMS_CLIENT_SECRET:-}" \
| curl -sS -m 30 "${KMS_ENDPOINT:-https://api.hanzo.ai}/v1/kms/auth/login" -H 'Content-Type: application/json' -d @- \
| jq -r '.accessToken // empty' 2>/dev/null)
if [ -n "$T" ]; then echo "::add-mask::$T"; else echo "::notice::KMS gave no IAM token, so the org store is not used"; fi
h=${CARGO_HOME:-$HOME/.cargo}
HANZO_API_TOKEN=$T "$RUNNER_TEMP/cache" restore "$name" "$key" \
sccache="$SCCACHE_DIR" index="$h/registry/index" crates="$h/registry/cache" git="$h/git/db"
# What this build reads or writes from here on is its working set;
# everything older is pruned before the save.
touch "$RUNNER_TEMP/sccache.since"
- name: Zig (the cross-linker)
if: ${{ !matrix.target.msvc }}
run: |
@@ -239,10 +335,45 @@ jobs:
run: |
set -euo pipefail
ulimit -n 65536 2>/dev/null || ulimit -n 10240 2>/dev/null || true
# Hits, misses and why a call was not cacheable, on the log, whether or not the build passes.
# Guarded: under -e a failing trap would turn a passing build red.
trap '[ -z "${RUSTC_WRAPPER:-}" ] || sccache --show-stats || true' EXIT
case "${{ matrix.target.rust }}" in
*apple-darwin|*linux-musl) cargo zigbuild --release --locked --target ${{ matrix.target.rust }} --bin dev ;;
*windows-msvc) cargo xwin build --release --locked --target ${{ matrix.target.rust }} --bin dev ;;
esac
- name: Save the Rust cache (this org's store)
# After a green build, so what is saved is what a passing build
# compiled. bin/cache writes only from main or a tag, and only a key the
# store does not hold yet.
#
# The server stops first, so every object is on disk. Then whatever
# this build neither read nor wrote goes: sccache stamps an object's
# mtime when it hits, and the registry keeps the crates this build
# unpacked. So an archive is one build's working set and never grows
# with the builds before it.
if: steps.rustcache.outputs.key != '' && steps.rustcache.outputs.hit != 'exact'
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
NAME: ${{ steps.rustcache.outputs.name }}
KEY: ${{ steps.rustcache.outputs.key }}
run: |
set +e # GitHub runs this under bash -e; nothing in this step may fail the build
set -uo pipefail
sccache --stop-server >/dev/null 2>&1 || true
find "$SCCACHE_DIR" -type f ! -newer "$RUNNER_TEMP/sccache.since" -delete 2>/dev/null
h=${CARGO_HOME:-$HOME/.cargo}
for c in "$h"/registry/cache/*/; do
i=$(basename "$c")
for f in "$c"*.crate; do [ -d "$h/registry/src/$i/$(basename "$f" .crate)" ] || rm -f "$f"; done
done
T=$(printf '{"clientId":"%s","clientSecret":"%s"}' "${KMS_CLIENT_ID:-}" "${KMS_CLIENT_SECRET:-}" \
| curl -sS -m 30 "${KMS_ENDPOINT:-https://api.hanzo.ai}/v1/kms/auth/login" -H 'Content-Type: application/json' -d @- \
| jq -r '.accessToken // empty' 2>/dev/null)
if [ -n "$T" ]; then echo "::add-mask::$T"; else echo "::notice::KMS gave no IAM token, so the org store is not used"; fi
HANZO_API_TOKEN=$T "$RUNNER_TEMP/cache" save "$NAME" "$KEY" \
sccache="$SCCACHE_DIR" index="$h/registry/index" crates="$h/registry/cache" git="$h/git/db"
- name: Package
run: |
set -euo pipefail
LLM.md
+23
View File
@@ -76,6 +76,29 @@ v1 carried, including the defects it was already carrying; see
`make build` is the check that must pass. Fix warnings as well as errors.
## The release cache
Each release build runs rustc through sccache, and hanzoai/ci's `bin/cache` (pinned
by commit as `CI_SHA` in `release.yml`) carries the objects, the registry's crate
files and the git dependencies to the next run through the org's `ci-cache` bucket
in cloud's `/v1/s3`. The IAM token comes from the KMS client pair inside the restore
and save steps and is never exported, so no build script sees it.
- Name `<sha16 of rustc -vV + sccache --version>-<target>`, key `<sha32 of Cargo.lock,
rust-toolchain.toml, patches/*.json and the RUSTFLAGS/CARGO_*/ZIG*/LLVM_*/XWIN_*/
MACOS_SDK_* env>`, both computed before the version stamp. A key that misses restores
`latest` for that name.
- Only main and tags write, and only a key the store does not hold. Before a save the
directory is cut to what that build touched: sccache bumps an object's mtime on a
hit, and a crate file stays only if cargo unpacked it.
- What compiles every release anyway: every first-party crate (upstream's and ours),
because the stamp changes their version, which is in both `-C metadata` and
`CARGO_PKG_VERSION`; proc-macro and build-script crates, which sccache cannot
cache; and the final link. On a quiet node the first-party chain from
codex-protocol to the link is ~46 of a ~58-minute cold build, so that chain, not
the dependencies, is what to shorten next.
- `sccache --show-stats` prints at the end of every Build step, pass or fail.
## House rules
- Hanzo service APIs use `https://api.hanzo.ai/v1/` exclusively.