mirror of
https://github.com/hanzoai/papers.git
synced 2026-10-22 17:20:17 +00:00
ci: restore the papers build and image on git.hanzo.ai
Two commits removed this repo's GitHub workflows without writing the native replacement they named, so nothing built the papers or the site image and no run failed to say so. papers.hanzo.ai has served a frozen Pages build since. Restore both into .hanzo/workflows, which the forge reads: - ci.yml from 8185428^ build-papers.yml. Compiles with `make -k all` so one broken paper does not hide the rest, then asserts every expected PDF exists. The predecessor could not fail: every Makefile rule ends in `|| true` and prints FAIL instead of exiting. Drops the v<run_number> Release and the commit-PDFs-back step, which wrote to what is now a mirror. - deploy.yml from ad31900^ deploy-site.yml. Builds and pushes ghcr.io/hanzoai/papers:<sha> and does not deploy; the universe CR names the tag that is live. Authenticates to ghcr with REGISTRY_TOKEN, since a forge-issued GITHUB_TOKEN authenticates nothing there. Makefile gains `manifest`, the expected PDFs one per line, so the gate reads ALL_PDFS rather than restating which papers exist. .github/workflows/sync.yml becomes the canonical push-refs workflow from universe infra/ci/sync.yml, replacing a mirror-sync nudge that documented GitHub as upstream and asked the forge to pull. Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Hanzo Dev
+87
-45
@@ -1,60 +1,102 @@
|
||||
name: Sync to git.hanzo.ai
|
||||
# DIRECTION: this repo on git.hanzo.ai is a PULL MIRROR fetching github.com/hanzoai/papers.
|
||||
# GitHub is UPSTREAM, git.hanzo.ai is DOWNSTREAM. The giveaway is what this file does:
|
||||
# mirror-sync tells Gitea to PULL, which only makes sense if GitHub is the source.
|
||||
name: sync
|
||||
|
||||
# Carries pushed refs to git.hanzo.ai, the canonical forge, and does nothing
|
||||
# else: build, test and release run natively there.
|
||||
#
|
||||
# This matters beyond labeling: a pull mirror force-updates its refs from upstream on
|
||||
# every fetch, so anything committed directly on git.hanzo.ai is DISCARDED on the next
|
||||
# sync. Do not treat the mirror as writable, and do not reword this into a "native on
|
||||
# git.hanzo.ai" claim — that is the intended end state, not today's.
|
||||
#
|
||||
# What IS true: CI/CD runs on git.hanzo.ai — arcd executes .hanzo/workflows/ on the
|
||||
# Hanzo git-runner. Making git.hanzo.ai canonical means converting the repo
|
||||
# pull-mirror -> native and flipping the write path; an ops migration, not a config edit.
|
||||
#
|
||||
# So this nudge is a LATENCY fix, not a canonicality one: it asks Gitea to pull now
|
||||
# instead of waiting out the ~10-minute mirror interval, so arcd sees the new commit
|
||||
# sooner. Without this file the repo still mirrors — just later.
|
||||
#
|
||||
# IDEMPOTENT: mirror-sync just tells Gitea to pull HEAD, so N nudges == 1 (same latest
|
||||
# state); concurrency coalesces a burst of pushes to a single in-flight sync. FAST: no
|
||||
# checkout, one bounded curl, hard timeouts. FAIL-SOFT: a missing token or hiccup never
|
||||
# fails the push (the App webhook still mirrors). Set HANZO_GIT_TOKEN to enable the nudge.
|
||||
# Repo-agnostic. owner/name come from ${{ github.repository }}, so this file is
|
||||
# dropped into every repo unchanged as .github/workflows/sync.yml and is the
|
||||
# only workflow GitHub is permitted to run.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
# '**' matches slashes; '*' does not, and would skip feature/foo.
|
||||
branches: ['**']
|
||||
tags: ['**']
|
||||
workflow_dispatch:
|
||||
|
||||
# Only the newest push's sync needs to run — cancel any older in-flight one (it would
|
||||
# pull a now-stale HEAD). This is what makes rapid pushes idempotent + cheap.
|
||||
# Serialize per ref. Never cancel: a cancelled run drops that ref and nothing
|
||||
# reconciles it afterwards.
|
||||
concurrency:
|
||||
group: sync-git-hanzo-ai
|
||||
cancel-in-progress: true
|
||||
group: sync-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
# `.github/workflows` is also on Gitea's WORKFLOW_DIRS, so with Actions enabled on
|
||||
# the mirror this file runs THERE too on every sync — where asking git.hanzo.ai to
|
||||
# pull is circular. github.server_url is setting.AppURL on Gitea
|
||||
# (services/actions/context.go:89), so this is the exact test for "am I upstream".
|
||||
if: ${{ github.server_url == 'https://github.com' }}
|
||||
# .github/workflows is also on the forge's own workflow search path, so this
|
||||
# file executes there too in any repo without .hanzo/workflows, where it
|
||||
# would push the forge to itself. server_url is the forge's AppURL, which
|
||||
# makes this the test for "running on GitHub".
|
||||
if: github.server_url == 'https://github.com'
|
||||
# GitHub-hosted on purpose. This is the one workflow that must run ON GitHub,
|
||||
# and the self-hosted pool that used to serve it is retired — a job asking for
|
||||
# it now queues forever against a label nothing answers. Carrying refs is a
|
||||
# push, not a build, so it does not belong on the build fabric anyway: builds
|
||||
# run natively on git.hanzo.ai, which is the whole point of this file.
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
timeout-minutes: 15
|
||||
|
||||
steps:
|
||||
- name: Nudge git.hanzo.ai to pull HEAD (mirror-sync)
|
||||
continue-on-error: true
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
fetch-tags: true
|
||||
persist-credentials: false
|
||||
|
||||
- name: Push refs to git.hanzo.ai
|
||||
env:
|
||||
HANZO_GIT_TOKEN: ${{ secrets.HANZO_GIT_TOKEN }}
|
||||
FORGE: https://git.hanzo.ai
|
||||
GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
|
||||
run: |
|
||||
set -u
|
||||
if [ -z "${HANZO_GIT_TOKEN:-}" ]; then
|
||||
echo "HANZO_GIT_TOKEN not set — relying on the mirror App webhook. Skipping nudge."
|
||||
exit 0
|
||||
set -euo pipefail
|
||||
|
||||
if [ -z "${GIT_TOKEN}" ]; then
|
||||
echo "::error::GIT_TOKEN is unset, so this push never reached the canonical forge. Add the org secret."
|
||||
exit 1
|
||||
fi
|
||||
if curl -fsS --max-time 20 --retry 2 --retry-delay 2 -X POST \
|
||||
-H "Authorization: token ${HANZO_GIT_TOKEN}" \
|
||||
"https://git.hanzo.ai/api/v1/repos/hanzoai/papers/mirror-sync"; then
|
||||
echo "git.hanzo.ai mirror-sync triggered"
|
||||
else
|
||||
echo "mirror-sync nudge failed (non-fatal) — the App webhook still mirrors"
|
||||
|
||||
REMOTE="${FORGE}/${GITHUB_REPOSITORY}.git"
|
||||
|
||||
# The forge reads the token from the basic-auth password and ignores
|
||||
# the username. base64 of a secret is not masked automatically.
|
||||
AUTH="$(printf 'x-access-token:%s' "${GIT_TOKEN}" | base64 -w0)"
|
||||
echo "::add-mask::${AUTH}"
|
||||
# Host-scoped, so the forge token is never sent to github.com.
|
||||
git config --local "http.${FORGE}/.extraheader" "Authorization: Basic ${AUTH}"
|
||||
|
||||
# Reachability is the only transient failure here, so it is the only
|
||||
# thing retried. ls-remote exits 0 on an existing empty repo and
|
||||
# non-zero when the forge is down or the repo is absent.
|
||||
ERR="$(mktemp)"
|
||||
REACHED=""
|
||||
for attempt in 1 2 3; do
|
||||
if git ls-remote "${REMOTE}" >/dev/null 2>"${ERR}"; then
|
||||
REACHED=1
|
||||
break
|
||||
fi
|
||||
echo "forge probe failed (attempt ${attempt}/3): $(cat "${ERR}")"
|
||||
if [ "${attempt}" -lt 3 ]; then
|
||||
sleep $((attempt * 5))
|
||||
fi
|
||||
done
|
||||
if [ -z "${REACHED}" ]; then
|
||||
echo "::error::git.hanzo.ai is unreachable or ${GITHUB_REPOSITORY} does not exist there. $(cat "${ERR}")"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Additive and fast-forward-only: no leading '+', no --force, no
|
||||
# --prune, no --mirror. Tags go every run so a tag that missed its own
|
||||
# event still lands; git refuses to move one that already differs.
|
||||
REFS=('refs/tags/*:refs/tags/*')
|
||||
case "${GITHUB_REF}" in
|
||||
refs/heads/*) REFS+=("${GITHUB_REF}:${GITHUB_REF}") ;;
|
||||
esac
|
||||
|
||||
# Not --atomic: a diverged ref must not hold back the refs that are
|
||||
# clean. Rejections still fail the job.
|
||||
if ! git push "${REMOTE}" "${REFS[@]}"; then
|
||||
echo "::error::git.hanzo.ai rejected a ref for ${GITHUB_REPOSITORY}. The forge is canonical and is never force-updated from here; reconcile the diverged branch or tag by hand."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
name: ci
|
||||
|
||||
# Compiles every paper and fails when one does not come out. Restored from the
|
||||
# deleted .github/workflows/build-papers.yml, minus the two steps that wrote to
|
||||
# GitHub: a v<run_number> Release and a commit of the PDFs back to the branch.
|
||||
# GitHub is a mirror now, so neither belongs here — the PDFs are the run's
|
||||
# artifact and the site image bakes its own copies.
|
||||
#
|
||||
# The predecessor could not fail. Every rule in the Makefile ends in `|| true`
|
||||
# and prints "FAIL" rather than exiting, so `make all` returned 0 with any
|
||||
# number of papers missing. The compile is therefore run with -k (attempt every
|
||||
# paper, do not stop at the first break) and the result is checked against
|
||||
# `make manifest`, which is the Makefile's own ALL_PDFS — one definition of the
|
||||
# paper set, read rather than restated.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ['**.tex', '**.bib', 'Makefile', '.hanzo/workflows/ci.yml']
|
||||
pull_request:
|
||||
paths: ['**.tex', '**.bib', 'Makefile', '.hanzo/workflows/ci.yml']
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
papers:
|
||||
runs-on: hanzo-build-linux-amd64
|
||||
timeout-minutes: 120
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: install latex
|
||||
run: |
|
||||
set -euo pipefail
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y texlive-full
|
||||
|
||||
- name: compile
|
||||
run: make -k all
|
||||
|
||||
- name: compile defense papers
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p pdfs/defense
|
||||
cd defense
|
||||
rc=0
|
||||
for tex in *.tex; do
|
||||
case "$tex" in
|
||||
hanzo-defense.sty|logo-convert.tex) continue ;;
|
||||
esac
|
||||
name="${tex%.tex}"
|
||||
echo "compiling defense/$tex"
|
||||
pdflatex -interaction=nonstopmode "$tex" >/dev/null 2>&1 || true
|
||||
bibtex "$name" >/dev/null 2>&1 || true
|
||||
pdflatex -interaction=nonstopmode "$tex" >/dev/null 2>&1 || true
|
||||
pdflatex -interaction=nonstopmode "$tex" >/dev/null 2>&1 || true
|
||||
if [ -f "${name}.pdf" ]; then
|
||||
mv "${name}.pdf" ../pdfs/defense/
|
||||
echo " ok defense/${name}.pdf"
|
||||
else
|
||||
echo " fail defense/${name}.pdf"
|
||||
rc=1
|
||||
fi
|
||||
rm -f ./*.aux ./*.log ./*.bbl ./*.blg ./*.out ./*.toc
|
||||
done
|
||||
exit "$rc"
|
||||
|
||||
- name: check every paper came out
|
||||
run: |
|
||||
set -euo pipefail
|
||||
missing=0
|
||||
while read -r pdf; do
|
||||
[ -n "$pdf" ] || continue
|
||||
if [ ! -f "$pdf" ]; then
|
||||
echo "::error file=${pdf%.pdf}::did not compile: $pdf"
|
||||
missing=$((missing + 1))
|
||||
fi
|
||||
done < <(make --no-print-directory manifest)
|
||||
echo "missing: ${missing}"
|
||||
[ "$missing" -eq 0 ]
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: papers
|
||||
path: pdfs/**/*.pdf
|
||||
retention-days: 90
|
||||
@@ -0,0 +1,41 @@
|
||||
name: deploy
|
||||
|
||||
# Builds papers.hanzo.ai and publishes the image the operator runs. Restored
|
||||
# from the deleted .github/workflows/deploy-site.yml, which was the sole
|
||||
# producer of ghcr.io/hanzoai/papers and went away with no replacement — the
|
||||
# host has been frozen at its last build since, with no failing run to say so.
|
||||
#
|
||||
# The image tag is the run's commit, and the operator CR in hanzoai/universe
|
||||
# names which tag is live — a build never deploys itself.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ['site/**', '.hanzo/workflows/deploy.yml']
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: deploy-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
image:
|
||||
runs-on: hanzo-build-linux-amd64
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: build and push
|
||||
env:
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${REGISTRY_TOKEN:-}" ]; then
|
||||
echo "::error::REGISTRY_TOKEN is unset, so nothing was published. Add the org secret."
|
||||
exit 1
|
||||
fi
|
||||
tag="$(git rev-parse --short HEAD)"
|
||||
image="ghcr.io/hanzoai/papers:${tag}"
|
||||
echo "${REGISTRY_TOKEN}" | docker login ghcr.io -u hanzo-dev --password-stdin
|
||||
docker build -t "${image}" ./site
|
||||
docker push "${image}"
|
||||
@@ -1,3 +1,76 @@
|
||||
# papers — AI Assistant Context
|
||||
# Papers
|
||||
|
||||
See repository README.md for project documentation.
|
||||
Hanzo's research papers: LaTeX sources at the repo root and in `zen/` and
|
||||
`defense/`, compiled PDFs under `pdfs/`, indexed by `INDEX.md`, and the site at
|
||||
https://papers.hanzo.ai built from `site/`.
|
||||
|
||||
## How this ships
|
||||
|
||||
One way, and it runs on our own stack:
|
||||
|
||||
push -> github.com/hanzoai/papers (a mirror)
|
||||
.github/workflows/sync.yml carries refs onward
|
||||
-> git.hanzo.ai/hanzoai/papers CANONICAL
|
||||
.hanzo/workflows/ci.yml compiles every paper
|
||||
.hanzo/workflows/deploy.yml builds ghcr.io/hanzoai/papers
|
||||
-> hanzoai/universe crs/papers.yaml names the tag that is live
|
||||
-> hanzoai/operator reconciles the App
|
||||
-> hanzoai/static behind hanzoai/ingress serves papers.hanzo.ai
|
||||
|
||||
**git.hanzo.ai is canonical; GitHub is a mirror.** `.github/workflows/` holds
|
||||
exactly one file, `sync.yml`, and its only job is getting refs to the forge. Every
|
||||
build, check and deploy is a workflow under `.hanzo/workflows/`, which the forge
|
||||
reads. `.hanzo/workflows` uses GitHub Actions syntax, so a workflow moves between
|
||||
the two by changing directory and nothing else.
|
||||
|
||||
No GitHub Pages and no Cloudflare Pages. The site is an image the operator runs,
|
||||
like every other Hanzo surface.
|
||||
|
||||
## Why both workflows had to be restored
|
||||
|
||||
Neither existed. Two commits both titled *"ci: remove GitHub CI — native
|
||||
git.hanzo.ai + cd.hanzo.ai only"* deleted them without writing the native
|
||||
replacement they named:
|
||||
|
||||
- `65f58e9` removed `build-papers.yml`, the only thing that compiled the papers.
|
||||
- `9b417a3` removed `deploy-site.yml`, the only producer of
|
||||
`ghcr.io/hanzoai/papers`. Its last successful build was 2026-07-19.
|
||||
|
||||
So `papers.hanzo.ai` served a frozen Cloudflare Pages build and nothing in the
|
||||
repo could update it — and because the workflows were gone rather than broken,
|
||||
there was no failing run anywhere to say so. Deleting a workflow before its
|
||||
replacement exists buys silent staleness; moving one costs nothing. Both files
|
||||
here are those two, restored from `9b417a3^` and `65f58e9^` into the native
|
||||
directory.
|
||||
|
||||
## What ci.yml checks
|
||||
|
||||
`make -k all` compiles all 131 discovered papers, `-k` so one broken paper does
|
||||
not hide the rest, then the gate asserts every expected PDF exists.
|
||||
|
||||
The predecessor could not fail. Every rule in the `Makefile` ends in `|| true`
|
||||
and prints `FAIL` instead of exiting, so `make all` returned 0 no matter how many
|
||||
papers were missing. The gate reads `make manifest` — the Makefile's own
|
||||
`ALL_PDFS` — so the paper set has one definition and the check cannot drift from
|
||||
the build.
|
||||
|
||||
Adding a paper therefore requires it to compile: drop `foo.tex` at the root or in
|
||||
`zen/`, and `pdfs/foo.pdf` is expected from then on.
|
||||
|
||||
`ci.yml` does not write to the repo. The predecessor pushed a `v<run_number>`
|
||||
GitHub Release and committed the PDFs back to `main`; both wrote to what is now a
|
||||
mirror. The PDFs are the run's artifact, and the site image bakes its own.
|
||||
|
||||
## Deploying the site
|
||||
|
||||
A build never deploys itself. `deploy.yml` publishes
|
||||
`ghcr.io/hanzoai/papers:<sha>` from `site/Dockerfile`; a human sets
|
||||
`spec.image.tag` in `hanzoai/universe`
|
||||
`infra/k8s/operator/crs/papers.yaml` and adds `- papers.yaml` to that directory's
|
||||
`kustomization.yaml`. The CR is inert until both are done, which is deliberate:
|
||||
an App promoted with an empty tag takes the host down instead of leaving it
|
||||
alone.
|
||||
|
||||
Order: publish an image -> set the tag -> add the line -> confirm the pod is
|
||||
Running -> only then repoint `papers.hanzo.ai` off Cloudflare Pages, which today
|
||||
still answers from `infra/cf-zones/hanzo-ai.yaml` and `site/public/CNAME`.
|
||||
|
||||
@@ -63,6 +63,13 @@ clean-all:
|
||||
@rm -rf pdfs
|
||||
@echo "Done"
|
||||
|
||||
# The expected PDFs, one per line, for a reader that is not a human: the gate in
|
||||
# .hanzo/workflows/ci.yml checks this list rather than restating which papers
|
||||
# exist, so ALL_PDFS above stays the single definition of the paper set.
|
||||
.PHONY: manifest
|
||||
manifest:
|
||||
@printf '%s\n' $(ALL_PDFS)
|
||||
|
||||
# List papers
|
||||
.PHONY: list
|
||||
list:
|
||||
|
||||
Reference in New Issue
Block a user