mirror of
https://github.com/hanzoai/ci.git
synced 2026-10-20 15:28:05 +00:00
Several sites in the estate began as output from a hosted "prompt to React app" generator. It does not sign its work in a comment you would notice; it signs it in the shape of the repo, four ways at once: an asset folder named after the vendor, a build plugin in package.json, an injected third-party <script>, and the commit messages, author identities and Co-authored-by trailers of its bot. A sweep of 1,065 repositories found the mark in five shipped sites. One served the vendor's upload folder as its FAVICON, so the vendor's path was in the <head> of every page and in every browser tab. Three have been cleared; that took a history rewrite and a force-push of every branch. This gate is so that is never needed a second time. The fourth place is the one nothing else looked at and the one that survives any amount of tidying: a repo can render a perfect page and still say in `git log` who really wrote it. A commit-msg hook on one workstation already strips these trailers, but that hook does not run on a runner, on a teammate's laptop, or on a commit made through the web UI. This does. Content is judged narrowly and paths are not. The vendor's name is also an ordinary English adjective, and the naive gate is unshippable: across those same 1,065 repos the bare word appears innocently in tokenizer vocabularies, word lists, ML corpora, packed public-suffix data, a disposable-email blocklist and an upstream Clerk bundle listing other people's dev hosts. A gate that reds a fifth of the estate is a gate someone switches off. So the content rules are only strings that cannot occur in prose, and two candidate rules were measured and dropped for redding that Clerk bundle. Paths, identities and trailers are refused on the bare word, because there no third party can put it and no tidying reaches it. Measured: red on exactly 4 of 1,219 checkouts, every one a true positive. 17/17 in the suite, which pins the allowances as well as the refusals. .vendormark-allow exists for the one honest case — the history-scrubber rule files in hanzoai/.github, which cannot do their work without spelling the string they remove. Co-authored-by: Hanzo Dev <[email protected]>
143 lines
5.9 KiB
Bash
Executable File
143 lines
5.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# vendormark_test.sh — the refusals AND the allowances, both pinned.
|
|
#
|
|
# A gate is only worth having if it is exact in both directions. A false
|
|
# negative lets the vendor's mark back into a shipped site; a false positive
|
|
# reds a repo that was always fine, and a gate that reds honest repos is a gate
|
|
# someone switches off. So this suite asserts BOTH halves, and the allowance
|
|
# half is the larger one on purpose: the vendor's name is also an ordinary
|
|
# English adjective, and it occurs innocently in tokenizer vocabularies, word
|
|
# lists, ML corpora, packed public-suffix data and third-party host lists all
|
|
# over the estate.
|
|
#
|
|
# Offline and deterministic: temp repos, no network.
|
|
set -uo pipefail
|
|
BIN=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/vendormark
|
|
pass=0; fail=0
|
|
V=$(printf 'lo%s' 'vable') # not spelled literally, so this file does not
|
|
# itself trip the gate it is testing
|
|
|
|
newrepo() {
|
|
d=$(mktemp -d)
|
|
git -C "$d" init -q
|
|
git -C "$d" config user.email [email protected]
|
|
git -C "$d" config user.name "Hanzo Dev"
|
|
printf 'x\n' > "$d/README.md"
|
|
git -C "$d" add -A
|
|
git -C "$d" commit -qm "initial"
|
|
echo "$d"
|
|
}
|
|
check() { # name expected_rc dir [range]
|
|
local name=$1 want=$2 dir=$3 range=${4:-}
|
|
"$BIN" "$dir" $range >/dev/null 2>&1; local got=$?
|
|
if [ "$got" = "$want" ]; then pass=$((pass+1)); echo " ok $name"
|
|
else fail=$((fail+1)); echo " FAIL $name (want rc=$want, got rc=$got)"; fi
|
|
}
|
|
|
|
echo "REFUSALS — the mark in each of the four places it hides"
|
|
|
|
d=$(newrepo)
|
|
check "clean repo is green" 0 "$d"
|
|
mkdir -p "$d/public/$V-uploads"
|
|
printf 'PNG\n' > "$d/public/$V-uploads/28d53ec4.png"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "add an image"
|
|
check "MUTATION: tracked path named for the vendor" 1 "$d"
|
|
git -C "$d" rm -rq "public/$V-uploads" && git -C "$d" commit -qm "move the image"
|
|
check "MUTATION REVERTED: green again" 0 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf '<img src="/%s-uploads/28d53ec4.png" />\n' "$V" > "$d/index.html"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
|
check "content: asset folder referenced in markup" 1 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf '{"devDependencies":{"%s-tagger":"^1.1.3"}}\n' "$V" > "$d/package.json"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "deps"
|
|
check "content: the vendor's build plugin in package.json" 1 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf '<script src="https://cdn.gpteng.co/gptengineer.js"></script>\n' > "$d/index.html"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
|
check "content: the vendor's injected script tag" 1 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf '<meta name="generator" content="Lovable" />\n' > "$d/index.html"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "a page"
|
|
check "content: the generator meta tag" 1 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
|
git -C "$d" commit -qm "Update $V project template"
|
|
check "MUTATION: the vendor named in a commit message" 1 "$d"
|
|
git -C "$d" commit -q --amend -m "update the project template"
|
|
check "MUTATION REVERTED: reworded message is green" 0 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
|
# --no-verify ON PURPOSE. A commit-msg hook on the workstation
|
|
# (~/.githooks/commit-msg) already strips vendor co-author trailers, and with it
|
|
# enabled this case cannot be constructed — the hook silently rewrites the
|
|
# trailer to ours and the assertion passes for the wrong reason. That hook is
|
|
# per-machine: it does not run on a runner, on a teammate's laptop, or on a
|
|
# commit made through the GitHub web UI. This gate is the layer that does. So
|
|
# the test bypasses the hook to prove the GATE catches what the hook would have.
|
|
git -C "$d" commit -q --no-verify -m "a change
|
|
|
|
Co-authored-by: $V bot <bot@$V.dev>"
|
|
check "MUTATION: Co-authored-by credits the vendor" 1 "$d"
|
|
git -C "$d" commit -q --amend --no-verify -m "a change"
|
|
check "MUTATION REVERTED: trailer dropped is green" 0 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A
|
|
git -C "$d" -c user.name="$V" -c user.email="bot@$V.dev" commit -qm "a change"
|
|
check "MUTATION: the vendor as author identity" 1 "$d"
|
|
rm -rf "$d"
|
|
|
|
d=$(newrepo)
|
|
printf 'y\n' > "$d/a.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean subject"
|
|
printf 'z\n' > "$d/b.txt"; git -C "$d" add -A; git -C "$d" commit -qm "Visual edit in $V"
|
|
printf 'w\n' > "$d/c.txt"; git -C "$d" add -A; git -C "$d" commit -qm "clean again"
|
|
check "default scope (HEAD only) misses an older bad message" 0 "$d"
|
|
check "explicit range catches it" 1 "$d" "HEAD~3..HEAD"
|
|
rm -rf "$d"
|
|
|
|
echo
|
|
echo "ALLOWANCES — the same word, innocently, as it really occurs in the estate"
|
|
|
|
d=$(newrepo)
|
|
printf ' "%s</w>": 38565,\n' "$V" > "$d/vocab.json"
|
|
printf '%s\n' "$V" > "$d/words_alpha.txt"
|
|
printf 'Tell me a story about a %s character.\n' "$V" > "$d/harmless.txt"
|
|
printf 'caseStudy: %s and v0 activate users by turning a prompt into an app\n' "${V^}" > "$d/guide.yaml"
|
|
printf 'mail2%s.com\n' "$V" > "$d/generic_emails.txt"
|
|
printf 'let e=[".%s.app",".%sproject.com",".webcontainer-api.io"];\n' "$V" "$V" > "$d/clerk-bundle.js"
|
|
printf 'A collection of UI components. Integrate them in v0, %s, Bolt.\n' "${V^}" > "$d/registries.json"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "corpora, word lists and third-party bundles"
|
|
check "tokenizer vocab / word list / ML corpus / marketing prose / blocklist / upstream host list" 0 "$d"
|
|
rm -rf "$d"
|
|
|
|
echo
|
|
echo "ESCAPE HATCH — a repo whose job is to name the vendor"
|
|
|
|
d=$(newrepo)
|
|
mkdir -p "$d/tools/rules"
|
|
printf 'Hanzo Dev <[email protected]> <bot@%s.dev>\n' "$V" > "$d/tools/rules/mailmap.txt"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "scrubber rules"
|
|
check "scrubber rule file is refused by default" 1 "$d"
|
|
printf 'tools/rules/*\n' > "$d/.vendormark-allow"
|
|
git -C "$d" add -A && git -C "$d" commit -qm "declare the rule files"
|
|
check "...and allowed once declared in .vendormark-allow" 0 "$d"
|
|
rm -rf "$d"
|
|
|
|
echo
|
|
echo "vendormark_test: $pass passed, $fail failed"
|
|
[ "$fail" -eq 0 ]
|