hanzo-research 0.2.1: the org and project the server files under, and what a digest pins

A review of 0.2.0 found the client reading identity and scope other
than the server does. The acting org is now the server's: a person's
home is the first of its signed orgs, not owner (the org of the app it
signed in through); a machine token acts in its owner alone; an API key
acts in its own org and project default. A switch the server would not
honour is refused before the request, a write answered under another
project raises, and a bearer token goes over http to this machine only.

A run id is unique only within a project, so get_run reads the client's
project and revise and retract refuse a record from another one; revise
carries the run's grants to the new version, which lands private; and
both refuse a version the server already holds. A claim needs a score,
and post_run needs confirm=True, nothing merely truthy.

capture() pins the contents of untracked files and records unknown, not
clean, when git fails; a folder with SHA256SUMS is checked against it;
listings escape names as sha256sum does; a remote keeps no userinfo. A
torn last WAL line is cut on open. The CLI's public read is --public.

LLM.md

@@ -424,7 +424,7 @@ hanzo-inc/cloud `apps/research` serves (one write, `record(benchmarks=, studies=
 papers=)`; reads `benchmarks`, `runs`, `studies`, `papers`, `get_run`, `compare`; `grant`,
 `revise`, `retract`), `Arena` every route of `apps/benchmark`,
 `provenance.capture()` reads harness commit (and the diff's SHA-256 when dirty), model,
-weights SHA-256 (a file, a directory by its SHA256SUMS, or a value), host, CPU, GPU,
+weights SHA-256 (a file, a directory checked against its SHA256SUMS, or a value), host, CPU, GPU,
 device, dtype, Python and packages, and `Provenance.apply(run)` folds them into a `Run`.
 Standard library only, Python ≥ 3.10. Tests: `uv run --no-project --with pytest pytest
 pkg/hanzo-research` over an in-process fake of both apps (`tests/fake.py`); `-m integration`
@@ -439,10 +439,19 @@ What the server does that a client has to know (read in `research.go`, `record.g
 - **`GET /v1/research/runs?org=X` is a public read only for a caller with NO org.** A
   credentialled caller gets its own org's runs whatever `org` says, so `runs(org=...)`
   sends no credential.
-- **The acting org is `X-Org-Id`, honoured only when the token's signed `orgs` names it**
-  (or the token is a SuperAdmin's); otherwise the server silently acts in the home org.
-  The client refuses such an org before the request. The project is the token's `project`
-  claim (`X-Project-Id` is minted from it), so a client asking another project is refused.
+- **The acting org is the server's reading of the token, never `owner`.** A person's home
+  is the first of its signed `orgs` (`owner` is the org of the app it signed in through);
+  `X-Org-Id` switches only to another org that set names, any org for a SuperAdmin (home
+  `admin`); a machine token (no `orgs`, or `type: application`) acts in its `owner` and
+  never switches; an `sk-` key acts in the org it was minted in. A switch the server would
+  not honour silently files under the home org, so the client refuses it before the
+  request. The project is the token's `project` claim, `default` for a key or a token
+  naming none; a client asking another is refused, and a write the server answers under
+  another project raises.
+- **A run id is unique only within a project**, and a listing with no `project` spans the
+  org: `get_run` (and so `retract`) reads the client's project, and `revise`/`retract`
+  refuse a record from another project. A new version lands private, so `revise` carries
+  the run's grants over to it.
 - **Typed ops refuse in RFC 7807** (`{"detail", "status", "title", "type"}`); the blob
   route and the gateway answer `{"error"}`. Read `detail` first.
 - The leaderboard's claims are org `admin`'s public ones only; `claims` shows every org's

docs/content/docs/research.mdx

@@ -19,10 +19,12 @@ pip install hanzo-research        # or: uv run --with hanzo-research python meas
 ```
 
 The credential is `HANZO_API_KEY`, else what `hanzo auth token` prints (a CLI token is
-fetched again when it expires). Records land in the token's org (its `owner`) and project
-(its `project` claim, else `default`). `Research(org="lux")` acts in another org the
-token belongs to (`X-Org-Id`); an org it does not belong to is refused before any request,
-since the server would file under the home org instead.
+fetched again when it expires). Records land in the org the server reads from the token
+(a person's home org, the first of its signed `orgs`; a machine token's `owner`; an API
+key's own org) and in the token's project (its `project` claim, else `default`).
+`Research(org="lux")` acts in another org a person's token belongs to (`X-Org-Id`); a
+switch the server would not honour, where it would file under the home org instead, is
+refused before any request, as is a credential sent over plain http to another machine.
 
 ## Worked example
 
@@ -79,10 +81,12 @@ print(back.completion, back.measure("all", "p50_ms").value, back.commit)
 `completion` is the server's: `complete` only when `answered == questions` and `ended` is
 set; `partial`, `inconsistent` or `unknown` otherwise. A model run names its weights:
 `capture(model="kai-1.3", weights="/models/kai-1.3", device="cuda", dtype="bf16")` hashes a
-file, or a directory by its `SHA256SUMS` (else every file under it), and `apply` sets
-`system` to the model and `version` to `<model>@<sha256>`. A dirty harness tree is recorded
-with the SHA-256 of `git diff HEAD` and the untracked files, so the number stays pinned to
-the code that made it.
+file, or a directory: one holding `SHA256SUMS` is checked against it (every file listed and
+matching, none unlisted) and named by that file's digest, as a Kai release is pinned; any
+other is digested as its `sha256sum` listing. `apply` sets `system` to the model and
+`version` to `<model>@<sha256>`. A dirty harness tree is recorded with the SHA-256 of
+`git diff HEAD` and the contents of the untracked files, so the number stays pinned to the
+code that made it; a git that fails records `unknown`, never clean.
 
 ## The research plane: `Research`
 
@@ -91,11 +95,12 @@ the code that made it.
 | `record(benchmarks=, studies=, runs=, papers=)` | `POST /v1/research/{benchmarks,studies,runs,papers}`, one batch a non-empty kind, in that order: `{"runs": [...]}` |
 | `benchmarks(id=, project=)`, `get_benchmark(id)` | `GET /v1/research/benchmarks` |
 | `runs(id=, benchmark=, system=, study=, completion=, since=, until=, project=)` | `GET /v1/research/runs`: your org's runs, private ones included, with the completion census and the fields runs leave unstated |
+| `get_run(id, project=)` | one canonical run in a project of your org (default the client's: an id is unique only within a project) |
 | `runs(org=...)`, `get_run(id, org=...)` | the same route with `?org=` and no credential: that org's public runs |
 | `studies(...)`, `papers(..., cites=)` | `GET /v1/research/{studies,papers}` |
 | `compare(a, b)` | `GET /v1/research/compare`: the reasons the two runs may not be compared, the config they differ in, the deltas |
 | `grant(id, visibility=, publishable=, trainable=)`, `grant(sha256=, visibility=)` | `POST /v1/research/grants`: on every version of the id; a write is always private until this |
-| `revise(record)` | a new version marked `corrected` |
+| `revise(record)` | a new version marked `corrected`, the run's grants carried to it (a new version lands private) |
 | `retract(id, why)` | a new version marked `retracted`: the id leaves every listing, earlier versions stay |
 | `ingest`, `query`, `totals`, `projects`, `experiment(...)` | the experiment plane: `/v1/research/{experiments,totals,projects}` |
 | `upload_artifact`, `upload_json_artifact`, `artifacts`, `get_artifact` | `/v1/research/artifacts[/<sha256>]`: `snapshot` or `report` bytes, addressed by the server's hash |
@@ -129,8 +134,8 @@ hanzo-research post runs.jsonl --grant public --publishable   # a run per line
 hanzo-research post plan.json                                 # {"benchmarks": [...], "runs": [...]}
 hanzo-research post plan.json --dry                           # print the bodies, file nothing
 hanzo-research list runs --benchmark demo.sha256
-hanzo-research list runs --org hanzo                          # hanzo's public runs, no credential
-hanzo-research get demo.sha256-16mib-cpu-dgx --org hanzo
+hanzo-research list runs --public hanzo                       # hanzo's public runs, no credential
+hanzo-research get demo.sha256-16mib-cpu-dgx --public hanzo
 hanzo-research grant RUN_ID ... --visibility public --publishable
 hanzo-research retract RUN_ID --why "read on the wrong split"
 hanzo-research compare RUN_A RUN_B

pkg/hanzo-research/README.md

@@ -12,10 +12,12 @@ pip install hanzo-research        # or: uv run --with hanzo-research python meas
 ```
 
 The credential is `HANZO_API_KEY`, else what `hanzo auth token` prints (a CLI token is
-fetched again when it expires). Records land in the token's org (its `owner`) and project
-(its `project` claim, else `default`). `Research(org="lux")` acts in another org the
-token belongs to (`X-Org-Id`); an org it does not belong to is refused before any request,
-since the server would file under the home org instead.
+fetched again when it expires). Records land in the org the server reads from the token
+(a person's home org, the first of its signed `orgs`; a machine token's `owner`; an API
+key's own org) and in the token's project (its `project` claim, else `default`).
+`Research(org="lux")` acts in another org a person's token belongs to (`X-Org-Id`); a
+switch the server would not honour, where it would file under the home org instead, is
+refused before any request, as is a credential sent over plain http to another machine.
 
 ## Worked example
 
@@ -72,10 +74,12 @@ print(back.completion, back.measure("all", "p50_ms").value, back.commit)
 `completion` is the server's: `complete` only when `answered == questions` and `ended` is
 set; `partial`, `inconsistent` or `unknown` otherwise. A model run names its weights:
 `capture(model="kai-1.3", weights="/models/kai-1.3", device="cuda", dtype="bf16")` hashes a
-file, or a directory by its `SHA256SUMS` (else every file under it), and `apply` sets
-`system` to the model and `version` to `<model>@<sha256>`. A dirty harness tree is recorded
-with the SHA-256 of `git diff HEAD` and the untracked files, so the number stays pinned to
-the code that made it.
+file, or a directory: one holding `SHA256SUMS` is checked against it (every file listed and
+matching, none unlisted) and named by that file's digest, as a Kai release is pinned; any
+other is digested as its `sha256sum` listing. `apply` sets `system` to the model and
+`version` to `<model>@<sha256>`. A dirty harness tree is recorded with the SHA-256 of
+`git diff HEAD` and the contents of the untracked files, so the number stays pinned to the
+code that made it; a git that fails records `unknown`, never clean.
 
 ## The research plane: `Research`
 
@@ -84,11 +88,12 @@ the code that made it.
 | `record(benchmarks=, studies=, runs=, papers=)` | `POST /v1/research/{benchmarks,studies,runs,papers}`, one batch a non-empty kind, in that order: `{"runs": [...]}` |
 | `benchmarks(id=, project=)`, `get_benchmark(id)` | `GET /v1/research/benchmarks` |
 | `runs(id=, benchmark=, system=, study=, completion=, since=, until=, project=)` | `GET /v1/research/runs`: your org's runs, private ones included, with the completion census and the fields runs leave unstated |
+| `get_run(id, project=)` | one canonical run in a project of your org (default the client's: an id is unique only within a project) |
 | `runs(org=...)`, `get_run(id, org=...)` | the same route with `?org=` and no credential: that org's public runs |
 | `studies(...)`, `papers(..., cites=)` | `GET /v1/research/{studies,papers}` |
 | `compare(a, b)` | `GET /v1/research/compare`: the reasons the two runs may not be compared, the config they differ in, the deltas |
 | `grant(id, visibility=, publishable=, trainable=)`, `grant(sha256=, visibility=)` | `POST /v1/research/grants`: on every version of the id; a write is always private until this |
-| `revise(record)` | a new version marked `corrected` |
+| `revise(record)` | a new version marked `corrected`, the run's grants carried to it (a new version lands private) |
 | `retract(id, why)` | a new version marked `retracted`: the id leaves every listing, earlier versions stay |
 | `ingest`, `query`, `totals`, `projects`, `experiment(...)` | the experiment plane: `/v1/research/{experiments,totals,projects}` |
 | `upload_artifact`, `upload_json_artifact`, `artifacts`, `get_artifact` | `/v1/research/artifacts[/<sha256>]`: `snapshot` or `report` bytes, addressed by the server's hash |
@@ -122,8 +127,8 @@ hanzo-research post runs.jsonl --grant public --publishable   # a run per line
 hanzo-research post plan.json                                 # {"benchmarks": [...], "runs": [...]}
 hanzo-research post plan.json --dry                           # print the bodies, file nothing
 hanzo-research list runs --benchmark demo.sha256
-hanzo-research list runs --org hanzo                          # hanzo's public runs, no credential
-hanzo-research get demo.sha256-16mib-cpu-dgx --org hanzo
+hanzo-research list runs --public hanzo                       # hanzo's public runs, no credential
+hanzo-research get demo.sha256-16mib-cpu-dgx --public hanzo
 hanzo-research grant RUN_ID ... --visibility public --publishable
 hanzo-research retract RUN_ID --why "read on the wrong split"
 hanzo-research compare RUN_A RUN_B

pkg/hanzo-research/hanzo_research/benchmark.py

@@ -20,6 +20,7 @@ What reads where:
 
 from __future__ import annotations
 
+import math
 from typing import Any, Mapping, Iterable
 from dataclasses import field, dataclass
 
@@ -242,9 +243,13 @@ class Arena:
         body = []
         for i, r in enumerate(rows):
             d = dump(r)
-            row = {k: d.get(k, 0.0 if k == "score" else "") for k in _CLAIM}
+            row = {k: d.get(k, "") for k in _CLAIM}
             if not str(row["protocol"]).strip() or not str(row["source"]).strip():
                 raise Invalid(f"claim row {i} needs a protocol and a source")
+            score = row["score"]
+            if isinstance(score, bool) or not isinstance(score, (int, float)) or not math.isfinite(score):
+                # absent or null would be filed as 0
+                raise Invalid(f"claim row {i} needs a score, a number")
             body.append(row)
         return Filed.load(self.http.post("/v1/benchmark/claims", {"data": body, "visibility": visibility}))
 
@@ -264,7 +269,7 @@ class Arena:
         confirm: bool = False,
     ) -> Admission:
         """Queue the arena's harness on a model or your endpoint. It spends money: `confirm=True` or it is refused."""
-        if not confirm:
+        if confirm is not True:
             raise Invalid("POST /v1/benchmark/runs queues a paid execution; pass confirm=True to queue it")
         body: dict[str, Any] = {"benchmarks": benchmarks}
         for k, v in (("model", model), ("endpoint", endpoint), ("attempts", attempts)):

pkg/hanzo-research/hanzo_research/cli.py

@@ -3,8 +3,8 @@
     hanzo-research post runs.jsonl --grant public --publishable
     hanzo-research post plan.json                 # {"benchmarks": [...], "runs": [...], ...}
     hanzo-research list runs --benchmark decision.latency
-    hanzo-research list runs --org hanzo          # an org's public runs, no credential
-    hanzo-research get decision.latency-loopback-dgx.cpu-kai --org hanzo
+    hanzo-research list runs --public hanzo       # an org's public runs, no credential
+    hanzo-research get decision.latency-loopback-dgx.cpu-kai --public hanzo
     hanzo-research grant RUN_ID ... --visibility public --publishable
     hanzo-research retract RUN_ID --why "read on the wrong split"
     hanzo-research compare RUN_A RUN_B
@@ -73,13 +73,13 @@ def main(argv: Iterable[str] | None = None) -> int:
     )
     for f in ("id", "benchmark", "system", "study", "completion", "since", "until", "kind", "run", "cites"):
         s.add_argument(f"--{f}")
-    s.add_argument("--org", dest="public", help="runs: that org's public runs, no credential")
+    s.add_argument("--public", metavar="ORG", help="runs: that org's public runs, read with no credential")
     s.add_argument("--in", dest="scope", help="narrow to one project of the org")
 
     s = sub.add_parser("get", help="one record's canonical version")
     s.add_argument("id")
     s.add_argument("--kind", choices=KINDS, default="run")
-    s.add_argument("--org", dest="public", help="read that org's public runs, no credential")
+    s.add_argument("--public", metavar="ORG", help="a run of that org's public runs, read with no credential")
 
     s = sub.add_parser("grant", help="visibility and consent for runs, or an artifact by --sha256")
     s.add_argument("ids", nargs="*")
@@ -172,6 +172,8 @@ def run(a: argparse.Namespace) -> Any:
             return r.projects()
         return r.totals(project=a.scope)
     if a.cmd == "get":
+        if a.public and a.kind != "run":
+            raise ValueError("--public reads runs only")
         if a.kind == "run":
             return r.get_run(a.id, org=a.public)
         if a.kind == "benchmark":
@@ -181,6 +183,8 @@ def run(a: argparse.Namespace) -> Any:
             raise Error(f"no canonical {a.kind} {a.id!r}", 404)
         return rows[0]
     if a.cmd == "grant":
+        if a.visibility is None and a.publishable is None and a.trainable is None:
+            raise ValueError("grant needs --visibility, --publishable or --trainable")
         if a.sha256:
             return {a.sha256: r.grant(sha256=a.sha256, visibility=a.visibility)}
         if not a.ids:

pkg/hanzo-research/hanzo_research/client.py

@@ -19,7 +19,7 @@ import base64
 import hashlib
 from typing import Any, Mapping, TypeVar, Iterable
 
-from . import canonical, provenance
+from . import wire, canonical, provenance
 from .wire import (
     Ack,
     Run,
@@ -36,7 +36,7 @@ from .wire import (
     Comparison,
     dump,
 )
-from .errors import Invalid, Missing
+from .errors import Error, Invalid, Missing
 from .transport import Transport
 
 __all__ = ["Research", "Experiment", "DEFAULT_LIBS"]
@@ -131,18 +131,42 @@ class Research:
         for key, rows in batches.items():
             body = [dump(r) for r in rows]
             if body:
-                out[key] = Batch.load(self.http.post(f"/v1/research/{key}", {key: body}))
+                batch = Batch.load(self.http.post(f"/v1/research/{key}", {key: body}))
+                self._filed(batch.project, key)
+                out[key] = batch
         return out
 
+    def _filed(self, project: str, what: str) -> None:
+        # the crate's `filed`: the server files under the credential's project, whatever was asked
+        if wire.scope(project) != self.project:
+            raise Error(f"{what} were committed under project {project}, not {self.project}")
+
+    def _again(self, rec: Any, revision: str) -> Batch:
+        """File `rec` as its next version, marked `revision`, in the project it lives in."""
+        if rec.project and wire.scope(rec.project) != self.project:
+            raise Invalid(f"{rec.id} is in project {rec.project}; this client files under {self.project}")
+        rec.revision = revision
+        key = _KEYS[type(rec)]
+        batch = self._write({key: [rec]})[key]
+        if batch.recorded == 0:
+            raise Invalid(f"the server already holds this version of {rec.id}: nothing was filed")
+        if isinstance(rec, Run) and (rec.visibility not in ("", "private") or rec.trainable or rec.publishable):
+            # a new version lands private; the run's grants carry over to it
+            self.grant(rec.id, visibility=rec.visibility or None, trainable=rec.trainable, publishable=rec.publishable)
+        return batch
+
     def revise(self, record: R, notes: str | None = None) -> Batch:
-        """File a correction: a new version of the record under its id, marked corrected."""
-        record.revision = "corrected"
+        """File a correction: the record's next version, marked corrected, keeping the run's grants.
+
+        A record read from this client's project (`get_run`) or built here; one from another
+        project is refused, since the server would file the version beside it, not over it.
+        """
         if notes is not None and isinstance(record, Run):
             record.notes = notes
-        return self._write({_KEYS[type(record)]: [record]})[_KEYS[type(record)]]
+        return self._again(record, "corrected")
 
     def retract(self, record: str | R, why: str) -> Batch:
-        """Withdraw a record: a version marked retracted. A run id reads its canonical run first.
+        """Withdraw a record: its next version, marked retracted. A run id reads its canonical run first.
 
         A retracted latest version takes the id out of every listing; the earlier versions stay
         retained. `why` becomes the version's notes for a run (the only kind with notes).
@@ -150,10 +174,9 @@ class Research:
         if not why.strip():
             raise Invalid("a retraction says why")
         rec: Any = self.get_run(record) if isinstance(record, str) else record
-        rec.revision = "retracted"
         if isinstance(rec, Run):
             rec.notes = why
-        return self._write({_KEYS[type(rec)]: [rec]})[_KEYS[type(rec)]]
+        return self._again(rec, "retracted")
 
     def grant(
         self,
@@ -187,8 +210,9 @@ class Research:
         out = self.http.get("/v1/research/benchmarks", {"id": id, "project": project})
         return [Benchmark.load(x) for x in out.get("data") or []]
 
-    def get_benchmark(self, id: str) -> Benchmark:
-        return _one(self.benchmarks(id=id), "benchmark", id)
+    def get_benchmark(self, id: str, project: str | None = None) -> Benchmark:
+        """One benchmark's canonical version in a project (default this client's), or `Missing`."""
+        return _one(self.benchmarks(id=id, project=project or self.project), "benchmark", id)
 
     def runs(
         self,
@@ -223,9 +247,11 @@ class Research:
         out.data = [Run.load(x) for x in out.data]
         return out
 
-    def get_run(self, id: str, org: str | None = None) -> Run:
-        """One run's canonical version, or `Missing`."""
-        return _one(self.runs(id=id, org=org).data, "run", id)
+    def get_run(self, id: str, org: str | None = None, project: str | None = None) -> Run:
+        """One run's canonical version, or `Missing`: in a project of your org (default this
+        client's; an id is unique only within a project), or with `org`, of that org's public runs."""
+        project = project or (None if org else self.project)
+        return _one(self.runs(id=id, org=org, project=project).data, "run", id)
 
     def studies(self, id: str | None = None, project: str | None = None) -> list[Study]:
         out = self.http.get("/v1/research/studies", {"id": id, "project": project})
@@ -247,9 +273,11 @@ class Research:
         self, experiments: list[Mapping[str, Any]] | None = None, attempts: list[Mapping[str, Any]] | None = None
     ) -> Ack:
         """File experiment and attempt versions in one batch, idempotent by content."""
-        return Ack.load(
+        ack = Ack.load(
             self.http.post("/v1/research/experiments", {"experiments": experiments or [], "attempts": attempts or []})
         )
+        self._filed(ack.project, "experiments and attempts")
+        return ack
 
     def query(self, project: str | None = None, kind: str | None = None) -> list[Record]:
         """Canonical experiments of a project (default the client's), narrowed to one kind."""

pkg/hanzo-research/hanzo_research/lineage.py

@@ -3,7 +3,9 @@
 A child's lineage only ever narrows: its purpose is the most restrictive of its own and its
 parents', and derivation is allowed only when every parent allows it and the purpose is not
 an evaluation one. A record's `digest()` is the SHA-256 of its RFC 8785 bytes, the same
-bytes the crate hashes, so a Rust harness and a Python one agree on every address.
+bytes the crate hashes, so a Rust harness and a Python one agree on every address whose
+numbers are integers within 2**53 or floats from 1e-6 to 1e21 in magnitude (outside that,
+the crate writes Rust's `Display`, not RFC 8785; see `canonical`).
 """
 
 from __future__ import annotations

pkg/hanzo-research/hanzo_research/provenance.py

@@ -45,6 +45,7 @@ __all__ = [
 
 PACKAGES = ("hanzo-research", "numpy", "torch", "transformers", "datasets", "vllm", "jax", "hanzo-kai")
 _HEX = re.compile(r"^[0-9a-fA-F]{1,64}$")
+BS = chr(92)  # a backslash, spelled once
 _SHA = re.compile(r"^(sha256:)?([0-9a-f]{64})$")
 
 
@@ -60,6 +61,15 @@ def _git(repo: str, *args: str) -> str:
     return _run("git", "-C", repo, *args)
 
 
+def _bytes(repo: str, *args: str) -> bytes | None:
+    """`git -C repo args` stdout as bytes; None when git fails or takes past 120 s, never "" for an error."""
+    try:
+        out = subprocess.run(["git", "-C", repo, *args], capture_output=True, timeout=120)
+    except (OSError, subprocess.SubprocessError):
+        return None
+    return out.stdout if out.returncode == 0 else None
+
+
 def git_state(repo: str) -> dict[str, Any]:
     """The producing repo's commit sha, branch, and whether the tree was dirty."""
     return {
@@ -125,20 +135,68 @@ def digest_file(path: str) -> str:
 
 
 def digest_tree(path: str) -> str:
-    """`sha256:<hex>` of a directory: of its SHA256SUMS when it holds one, else of the
-    `sha256sum` listing of every file under it, sorted by path, written as that file would be.
+    """`sha256:<hex>` of a directory, as its SHA256SUMS addresses it.
+
+    A directory holding SHA256SUMS is checked against it, every file listed and matching
+    and none unlisted, and its digest is that file's (the address a Kai release is pinned
+    by); a mismatch is a ValueError. A directory without one is digested as the SHA256SUMS
+    `sha256sum` would write for it, files sorted by path, names escaped as sha256sum does.
+    A directory symlink inside the tree is refused: what it points at is not the tree.
     """
-    sums = os.path.join(path, "SHA256SUMS")
-    if os.path.isfile(sums):
-        return digest_file(sums)
-    lines = []
+    have = {}
     for root, dirs, files in os.walk(path):
-        dirs.sort()
-        for name in sorted(files):
+        for d in dirs:
+            if os.path.islink(os.path.join(root, d)):
+                raise ValueError(f"{os.path.join(root, d)} is a directory symlink inside {path}")
+        for name in files:
             full = os.path.join(root, name)
-            rel = os.path.relpath(full, path).replace(os.sep, "/")
-            lines.append(f"{digest_file(full)[7:]}  {rel}\n")
-    return "sha256:" + hashlib.sha256("".join(sorted(lines, key=lambda s: s[66:])).encode()).hexdigest()
+            have[os.path.relpath(full, path).replace(os.sep, "/")] = full
+    sums = have.pop("SHA256SUMS", None)
+    listing = "".join(_line(digest_file(have[rel])[7:], rel) for rel in sorted(have))
+    if sums is None:
+        return "sha256:" + hashlib.sha256(listing.encode()).hexdigest()
+    want = {}
+    for line in open(sums, encoding="utf-8").read().splitlines():
+        if line.strip():
+            hexd, name = _parse(line)
+            want[name] = hexd
+    got = {rel: digest_file(full)[7:] for rel, full in have.items()}
+    for rel in sorted(set(want) | set(got)):
+        if want.get(rel) != got.get(rel):
+            what = (
+                "is not listed in"
+                if rel not in want
+                else "is missing but listed in"
+                if rel not in got
+                else "does not match"
+            )
+            raise ValueError(f"{rel} {what} {sums}")
+    return digest_file(sums)
+
+
+def _line(hexd: str, name: str) -> str:
+    # sha256sum's escaping: a name holding a backslash or a newline is escaped and the line marked
+    if BS in name or "\n" in name:
+        return BS + hexd + "  " + name.replace(BS, BS + BS).replace("\n", BS + "n") + "\n"
+    return f"{hexd}  {name}\n"
+
+
+def _parse(line: str) -> tuple[str, str]:
+    """A SHA256SUMS line as (hex, path): `hex  name` or `hex *name`, escaped when it opens with a backslash."""
+    escaped = line.startswith(BS)
+    body = line[1:] if escaped else line
+    hexd, name = body[:64].lower(), body[66:]
+    if escaped:
+        out, k = [], 0
+        while k < len(name):
+            if name[k] == BS and k + 1 < len(name):
+                out.append("\n" if name[k + 1] == "n" else name[k + 1])
+                k += 2
+            else:
+                out.append(name[k])
+                k += 1
+        name = "".join(out)
+    return hexd, name.removeprefix("./")
 
 
 def weights(value: str) -> str:
@@ -283,18 +341,15 @@ def capture(
     `git diff HEAD`, so the uncommitted state the number came from is pinned too.
     """
     top = find_repo(repo) if repo is None or os.path.isdir(repo) else repo
-    state = git_state(top)
-    diff = ""
-    if state["git_dirty"]:
-        listing = _git(top, "diff", "HEAD", "--binary") + "\n" + _git(top, "ls-files", "--others", "--exclude-standard")
-        diff = "sha256:" + hashlib.sha256(listing.encode()).hexdigest()
+    commit = _git(top, "rev-parse", "HEAD")
+    dirty, diff = _tree(top) if commit else (False, "")
     names = list(packages) if packages is not None else list(PACKAGES)
     return Provenance(
         repo=top,
         remote=_remote(top),
-        commit=state["git_sha"],
-        branch=state["git_branch"],
-        dirty=state["git_dirty"],
+        commit=commit,
+        branch=_git(top, "rev-parse", "--abbrev-ref", "HEAD"),
+        dirty=dirty,
         diff=diff,
         model=model,
         weights=_weights(weights) if weights else "",
@@ -315,10 +370,34 @@ def capture(
 _weights = weights  # capture() takes a parameter of that name
 
 
+def _tree(repo: str) -> tuple[bool, str]:
+    """(dirty, `sha256:` of what differs from HEAD): the diff and every untracked file's
+    contents. A git call that fails or times out reads dirty with the digest `unknown`,
+    never clean."""
+    status = _bytes(repo, "status", "--porcelain")
+    if status is None:
+        return True, "unknown"
+    if not status.strip():
+        return False, ""
+    diff = _bytes(repo, "diff", "HEAD", "--binary")
+    others = _bytes(repo, "ls-files", "--others", "--exclude-standard", "-z")
+    if diff is None or others is None:
+        return True, "unknown"
+    h = hashlib.sha256(diff)
+    for name in sorted(n for n in others.split(b"\0") if n):
+        full = os.path.join(repo, os.fsdecode(name))
+        h.update(b"\0" + name + b"\0" + (bytes.fromhex(digest_file(full)[7:]) if os.path.isfile(full) else b""))
+    return True, "sha256:" + h.hexdigest()
+
+
 def _remote(repo: str) -> str:
     url = _git(repo, "remote", "get-url", "origin")
-    # never a credential: https://user:token@host/... keeps only the host and path
-    return re.sub(r"^(\w+://)[^@/]+@", r"\1", url)
+    if "://" not in url:
+        return url  # scp-like git@host:path names a user, never a secret
+    # never a credential: everything up to the last @ goes, even where an unencoded / or @
+    # in the userinfo hides where the authority ends (a remote's path holds no @)
+    scheme, rest = url.split("://", 1)
+    return f"{scheme}://{rest.rpartition('@')[2]}"
 
 
 def _cpu() -> str:

pkg/hanzo-research/hanzo_research/transport.py

@@ -29,6 +29,7 @@ MAX_RESPONSE_BYTES = 64 << 20
 # 503 is the research store's failover answer ("retry"): asked again after 2, 4 and 8 s.
 WAITS = (2, 4, 8)
 ADMIN = "admin"
+LOOPBACK = ("127.0.0.1", "localhost", "::1")
 
 
 class _NoRedirect(urllib.request.HTTPRedirectHandler):
@@ -40,10 +41,13 @@ class Transport:
     """Base URL, credential, acting org and project, shared by `Research` and `Arena`.
 
     `token` None resolves `HANZO_API_KEY`, else `hanzo auth token`, on the first call that
-    needs one; "" sends no credential. `org` selects the org a call acts in (`X-Org-Id`);
-    the server honours it only when the token's signed `orgs` names it, so a JWT that does
-    not is refused here instead of filing under its home org. `project` must be the
-    token's own (`project` claim, else `default`): the server files under the claim.
+    needs one; "" sends no credential. The org a call acts in is the server's rule, read
+    from the token: a person's home org is the first of its signed `orgs`, and `org`
+    switches to another of them (`X-Org-Id`), any org for a SuperAdmin (home `admin`); a
+    machine token acts in its `owner` alone; an API key in the org it was minted in. A
+    switch the server would not honour, where it would act in the home org instead, is
+    refused here. The project is the token's `project` claim, `default` for a key or a
+    token naming none, and `project` (or RESEARCH_PROJECT) asking another is refused.
     """
 
     def __init__(
@@ -55,13 +59,15 @@ class Transport:
         timeout: float = 120.0,
     ):
         self.base = (base or os.environ.get("RESEARCH_BASE") or DEFAULT_BASE).rstrip("/")
-        if urllib.parse.urlsplit(self.base).scheme not in ("http", "https"):
-            raise ValueError(f"base {self.base!r} is not an http(s) URL")
+        parts = urllib.parse.urlsplit(self.base)
+        if parts.scheme != "https" and not (parts.scheme == "http" and parts.hostname in LOOPBACK):
+            # a bearer token never crosses a network in the clear
+            raise ValueError(f"base {self.base!r} is neither https nor http on this machine")
         self.timeout = timeout
         self._token = token
         self._cli = False
         self._org = (org or "").strip() or None
-        self._project = project
+        self._project = project or os.environ.get("RESEARCH_PROJECT") or None
         self._checked = False
         self._opener = urllib.request.build_opener(_NoRedirect)
 
@@ -79,34 +85,34 @@ class Transport:
 
     @property
     def project(self) -> str:
-        """The project records are filed under: the token's claim, else the one asked for, else `default`."""
+        """The project records are filed under: the token's claim, `default` for a key or none."""
         tok = self.token
-        c = wire.claimed(tok) if tok else None
-        return c or wire.scope(self._project or os.environ.get("RESEARCH_PROJECT", ""))
+        if tok:
+            return wire.claimed(tok) or "default"
+        return wire.scope(self._project or "")
 
     @property
     def org(self) -> str | None:
-        """The org calls act in: the one asked for, else the token's home org (`owner`); None when unknown."""
-        if self._org:
-            return self._org
-        c = wire.claims(self.token) if self.token else None
-        return str(c.get("owner")) if c and c.get("owner") else None
+        """The org calls act in: the one asked for, else the token's home org; None for a key or no token."""
+        return self._org or (_identity(self.token)[0] if self.token else None)
 
     def _check(self) -> None:
         tok = self._token or ""
-        claimed = wire.claimed(tok) if tok else None
-        if claimed is not None and self._project and wire.scope(self._project) != claimed:
-            raise Error(
-                f"the token's project is {claimed}, not {self._project}: records would be filed under {claimed}"
-            )
-        c = wire.claims(tok) if tok else None
-        if c is not None and self._org:
-            orgs = [str(m.get("org")) for m in c.get("orgs") or [] if isinstance(m, dict)]
-            home = orgs[0] if orgs else str(c.get("owner") or "")
+        if any(not 32 < ord(ch) < 127 for ch in tok):
+            # never echoed: the value is a secret
+            raise Denied("the credential holds characters an HTTP header cannot carry")
+        have = wire.claimed(tok) or "default"
+        if tok and self._project and wire.scope(self._project) != have:
+            raise Error(f"the token's project is {have}, not {self._project}: records would be filed under {have}")
+        if tok and self._org:
+            home, orgs, machine = _identity(tok)
+            if home is None:
+                raise Denied("an API key acts in the org it was minted in; org= cannot move it")
+            if machine and self._org != home:
+                raise Denied(f"a machine token acts in its own org, {home}, never {self._org}")
             if home != ADMIN and self._org != home and (self._org == ADMIN or self._org not in orgs):
                 raise Denied(
-                    f"this token cannot act in org {self._org} (its orgs: {', '.join(orgs) or home}); "
-                    f"the server would act in {home}"
+                    f"this token cannot act in org {self._org} (its orgs: {', '.join(orgs)}); the server would act in {home}"
                 )
         self._checked = True
 
@@ -122,22 +128,22 @@ class Transport:
         """One request; its raw answer on 2xx, else the typed error. 503 is asked again after 2, 4 and 8 s."""
         url = self.base + path + _query(query)
         data = None if body is None else json.dumps(body, allow_nan=False).encode()
+        waits = list(WAITS)
         refreshed = False
-        for wait in (*WAITS, None):
-            # the base is http(s), checked in __init__
+        while True:
+            # the base is https, or http to this machine, checked in __init__
             req = urllib.request.Request(url, data=data, method=method, headers=self._headers(auth, data is not None))  # noqa: S310
             try:
                 with self._opener.open(req, timeout=self.timeout) as r:
                     return _read(r, method, path)
             except urllib.error.HTTPError as e:
                 raw = _read(e, method, path) if e.fp else b""
-                if e.code == 503 and wait is not None:
-                    time.sleep(wait)
+                if e.code == 503 and waits:
+                    time.sleep(waits.pop(0))
                     continue
                 if e.code == 401 and auth and self._cli and not refreshed:
                     # a CLI token lives about an hour: fetch the current one and ask once more
-                    refreshed = True
-                    self._token = wire.token()
+                    refreshed, self._token, self._checked = True, wire.token(), False
                     continue
                 err = refusal(e.code, raw, method, path, _retry(e.headers))
                 if isinstance(err, Denied) and auth and not self._token:
@@ -148,7 +154,6 @@ class Transport:
                 raise
             except (urllib.error.URLError, TimeoutError, ConnectionError, OSError) as e:
                 raise Unavailable(f"no answer: {getattr(e, 'reason', e)}", 0, method, path) from None
-        raise AssertionError("unreachable")
 
     def _headers(self, auth: bool, body: bool) -> dict[str, str]:
         h = {"Accept": "application/json", "User-Agent": f"hanzo-research/{VERSION} (python)"}
@@ -164,6 +169,16 @@ class Transport:
         return h
 
 
+def _identity(tok: str) -> tuple[str | None, list[str], bool]:
+    """(home org, signed orgs, machine) of a JWT as the server reads them; home None for a non-JWT key."""
+    c = wire.claims(tok)
+    if c is None:
+        return None, [], False
+    orgs = [str(m.get("org")) for m in c.get("orgs") or [] if isinstance(m, dict) and m.get("org")]
+    machine = c.get("type") == "application" or not orgs
+    return (str(c.get("owner") or "") if machine else orgs[0]), orgs, machine
+
+
 def _query(q: Mapping[str, Any] | None) -> str:
     pairs = []
     for k, v in (q or {}).items():

pkg/hanzo-research/hanzo_research/wal.py

@@ -2,7 +2,10 @@
 
 Each line is one `ResearchEvent`; its `previous_event_hash` is the digest of the line before
 (64 zeros for the first), so editing, dropping or reordering a line breaks the chain, which
-`verify` finds. A log written by the Rust crate reads here, and the reverse.
+`verify` finds. A log written by the Rust crate reads and verifies here, and the reverse,
+wherever their canonical bytes agree: every payload of strings, integers within 2**53 and
+floats from 1e-6 to 1e21 in magnitude. A number outside that range hashes differently,
+since the crate writes it with Rust's `Display` rather than RFC 8785.
 """
 
 from __future__ import annotations
@@ -32,6 +35,7 @@ class ResearchWal:
         os.makedirs(run_dir, exist_ok=True)
         name = "wal.jsonl" if worker_id in ("", "worker-0") else f"wal-{worker_id}.jsonl"
         path = os.path.join(run_dir, name)
+        _mend(path)
         sequence, last = 0, GENESIS
         for event in read(path):
             sequence, last = event.sequence, event.digest()
@@ -63,8 +67,38 @@ def read(path: str) -> list[ResearchEvent]:
     """Every event of a log, in file order; a missing file is an empty log, a blank line is skipped."""
     if not os.path.exists(path):
         return []
+    out = []
     with open(path, encoding="utf-8") as f:
-        return [ResearchEvent.load(json.loads(line)) for line in f if line.strip()]
+        for n, line in enumerate(f, 1):
+            if line.strip():
+                try:
+                    out.append(ResearchEvent.load(json.loads(line)))
+                except ValueError:
+                    raise ValueError(f"{path}: line {n} is not an event") from None
+    return out
+
+
+def _mend(path: str) -> None:
+    """Mend the last line a crash left: a torn one (not JSON) is cut, a whole one missing its
+    newline gets it, so the next append starts on its own line. A torn line anywhere else
+    is corruption, which `read` reports."""
+    if not os.path.exists(path):
+        return
+    with open(path, "rb") as f:
+        data = f.read()
+    if not data or data.endswith(b"\n"):
+        return
+    end = data.rfind(b"\n") + 1
+    try:
+        json.loads(data[end:])
+    except ValueError:
+        with open(path, "r+b") as f:
+            f.truncate(end)
+            os.fsync(f.fileno())
+        return
+    with open(path, "ab") as f:
+        f.write(b"\n")
+        os.fsync(f.fileno())
 
 
 def verify(path: str) -> str:

pkg/hanzo-research/hanzo_research/wire.py

@@ -1,7 +1,9 @@
 """Wire types of `/v1/research`: the records hanzo-inc/cloud `apps/research` stores and returns.
 
-Field names, order and JSON are the Rust crate's (`wire.rs`, `lib.rs`), which are the
-server's (`record.go`, `research.go`). Reads are tolerant as the crate's are: an unknown
+Field names, order and JSON are the server's (`record.go`, `research.go`) and the Rust
+crate's (`wire.rs`, `lib.rs`) where the crate agrees with the server: `Comparison` carries
+`a`, `b` and `config` (the crate reads `differences`, which the server never writes) and
+`Stored` carries `ref`. Reads are tolerant as the crate's are: an unknown
 key is ignored, a missing one takes its default, and a null list reads as empty. A
 pointer the server declares (`k`, `temperature`, `questions`, a measure's `lo`) stays
 `None` when absent, never 0: absence is the record declining to say.

pkg/hanzo-research/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "hanzo-research"
-version = "0.2.0"
+version = "0.2.1"
 description = "File measurements to Hanzo's /v1/research and /v1/benchmark with their provenance: harness commit, model weights, host, device."
 readme = "README.md"
 license = "MIT"

pkg/hanzo-research/tests/fake.py

@@ -98,6 +98,7 @@ class Fake:
         self.requests: list[dict[str, Any]] = []
         self.unavailable = 0  # answer the next N requests 503
         self.expired: set[str] = set()  # bearer tokens answered 401
+        self.project = ""  # file every write under this project instead of the caller's
         self.seq = 0
         self.lock = threading.Lock()
 
@@ -111,14 +112,19 @@ class Fake:
         if len(parts) != 3:
             return None
         c = json.loads(base64.urlsafe_b64decode(parts[1] + "=" * (-len(parts[1]) % 4)))
-        orgs = [m["org"] for m in c.get("orgs") or []] or [c.get("owner", "")]
-        home = orgs[0]
+        # the server's rule (auth_identity.go homeOrg, Acts): a person's home is the first of
+        # its signed orgs and it may switch among them, any org for a SuperAdmin (home admin);
+        # a machine (no orgs, or type application) is its owner and never switches
+        orgs = [m["org"] for m in c.get("orgs") or []]
+        machine = c.get("type") == "application" or not orgs
+        home = c.get("owner", "") if machine else orgs[0]
+        sudo = home == "admin" and not machine
         want = headers.get("X-Org-Id") or ""
-        org = want if want and (home == "admin" or (want != "admin" and want in orgs)) else home
+        org = want if want and not machine and (sudo or (want != "admin" and want in orgs)) else home
         return {
             "org": org,
             "home": home,
-            "super": home == "admin",
+            "super": sudo,
             "user": c.get("sub", ""),
             "project": c.get("project") or "default",
         }
@@ -176,7 +182,11 @@ class Fake:
                 }
             )
             added += 1
-        return {"project": who["project"], "recorded": added, "canonical": len(self.canonical(who["org"], kind))}
+        return {
+            "project": self.project or who["project"],
+            "recorded": added,
+            "canonical": len(self.canonical(who["org"], kind)),
+        }
 
     def runs(self, who: dict[str, Any] | None, q: dict[str, str]) -> dict[str, Any]:
         if who is None:

pkg/hanzo-research/tests/test_arena.py

@@ -88,3 +88,19 @@ def test_presets_echo(server):
 def test_research_hands_its_credential_to_the_arena(server):
     r = Research(base=server.url, api_key=HANZO, org="lux", repo=".")
     assert r.arena.post_claims([claim()]).org == "lux"
+
+
+def test_a_claim_names_its_score(server):
+    a = arena(server)
+    n = len(server.fake.requests)
+    for bad in ({"score": None}, {}, {"score": "91"}, {"score": True}, {"score": float("nan")}):
+        row = {"benchmark": "gpqa_diamond", "model": "m", "protocol": "p", "source": "s", **bad}
+        with pytest.raises(Invalid, match="needs a score"):
+            a.post_claims([row])
+    assert len(server.fake.requests) == n
+
+
+def test_confirm_is_true_or_nothing(server):
+    for truthy in ("0", 1, "yes"):
+        with pytest.raises(Invalid, match="paid execution"):
+            arena(server).post_run(["gpqa_diamond"], model="m", confirm=truthy)

pkg/hanzo-research/tests/test_cli.py

@@ -40,7 +40,7 @@ def test_post_a_jsonl_file_and_grant(cli, server, tmp_path):
     code, out, _ = cli("post", str(f), "--grant", "public", "--publishable")
     assert code == 0 and out["runs"]["recorded"] == 3
     assert out["granted"] == {f"demo-dev-c-s{i}": 1 for i in range(3)}
-    code, out, _ = cli("list", "runs", "--org", "hanzo")
+    code, out, _ = cli("list", "runs", "--public", "hanzo")
     assert out["total"] == 3 and out["completion"]["complete"] == 3
 
 

pkg/hanzo-research/tests/test_client.py

@@ -6,6 +6,7 @@ import json
 import base64
 
 import pytest
+from fake import jwt
 from conftest import ATLAS, HANZO, SUPER
 from hanzo_research import (
     Run,
@@ -302,3 +303,71 @@ def test_the_default_client(server, monkeypatch):
     assert research.benchmarks(id="b")[0].id == "b" and research.studies() == [] and research.papers() == []
     c = research.compare("r1", "r2")
     assert c.a.id == "r1" and c.b.id == "r2" and server.fake.requests[-1]["query"] == {"a": "r1", "b": "r2"}
+
+
+def test_a_batch_filed_under_another_project_is_refused(server):
+    server.fake.project = "elsewhere"
+    with pytest.raises(Error, match="runs were committed under project elsewhere, not default"):
+        client(server).record(runs=[run()])
+
+
+def test_a_lookup_stays_in_the_clients_project(server):
+    atlas, home = client(server, token=ATLAS), client(server)
+    atlas.record(runs=[run("r1")])
+    atlas.grant("r1", visibility="public")
+    with pytest.raises(Missing):
+        home.retract("r1", "wrong split")
+    assert server.fake.requests[-1]["query"] == {"id": "r1", "project": "default"}
+    theirs = atlas.get_run("r1")
+    with pytest.raises(Invalid, match="r1 is in project atlas; this client files under default"):
+        home.revise(theirs)
+    assert home.runs(org="hanzo").data[0].id == "r1"
+
+
+def test_revise_keeps_the_runs_grants_and_refuses_no_change(server):
+    r = client(server)
+    r.record(runs=[run()])
+    r.grant("demo.bench-dev-a-sys", visibility="public", publishable=True)
+    got = r.get_run("demo.bench-dev-a-sys")
+    got.measures[0].value = 0.5
+    r.revise(got, notes="rescored")
+    public = r.get_run("demo.bench-dev-a-sys", org="hanzo")
+    assert public.revision == "corrected" and public.visibility == "public" and public.publishable
+    with pytest.raises(Invalid, match="already holds this version"):
+        r.revise(r.get_run("demo.bench-dev-a-sys"))
+
+
+def test_identity_is_read_as_the_server_reads_it(server):
+    acme = jwt({"sub": "u", "owner": "hanzo", "orgs": [{"org": "acme"}, {"org": "lux"}], "exp": 4102444800})
+    r = client(server, token=acme)
+    assert r.org == "acme" and r.url("x").endswith("?org=acme&id=x")  # owner is the app's org, not the home
+    machine = jwt({"owner": "admin", "type": "application", "exp": 4102444800})
+    with pytest.raises(Denied, match="a machine token acts in its own org, admin, never hanzo"):
+        client(server, token=machine, org="hanzo").runs()
+    assert client(server, token=machine).org == "admin"
+    with pytest.raises(Denied, match="an API key acts in the org it was minted in"):
+        client(server, token="sk-opaque", org="hanzo").runs()
+    with pytest.raises(Error, match="the token's project is default, not kai"):
+        client(server, token="sk-opaque", project="kai").runs()
+    assert client(server, token="sk-opaque").project == "default"
+
+
+def test_a_credential_a_header_cannot_carry_is_never_echoed(server):
+    with pytest.raises(Denied) as e:
+        client(server, token="sk-SECRET-ONE\nSECRET-TWO").runs()
+    assert "SECRET" not in str(e.value) and server.fake.requests == []
+
+
+def test_a_401_after_the_last_wait_refreshes_once(server, monkeypatch):
+    server.fake.expired.add("old")
+    server.fake.unavailable = 3
+    fresh = iter(["old", HANZO])
+    monkeypatch.setattr(wire, "token", lambda: next(fresh))
+    assert Research(base=server.url, repo=".").runs().total == 0
+    assert len(server.fake.requests) == 5
+
+
+def test_a_bearer_token_never_crosses_a_network_in_the_clear():
+    with pytest.raises(ValueError, match="neither https nor http on this machine"):
+        Research(base="http://api.example.com", api_key=HANZO)
+    assert Research(base="http://localhost:8080", api_key=HANZO).base == "http://localhost:8080"

pkg/hanzo-research/tests/test_provenance.py

@@ -98,3 +98,64 @@ def test_the_narrative_takes_only_an_object_id(repo):
     assert provenance.git_state(str(repo))["git_branch"] == "main"
     assert provenance.find_repo(str(repo / ".")) == os.path.realpath(repo)
     assert provenance.host()["hostname"]
+
+
+def test_the_dirty_digest_pins_untracked_contents(repo):
+    (repo / "eval.yaml").write_text("split: test\n")
+    a = provenance.capture(repo=str(repo), gpu=False).diff
+    (repo / "eval.yaml").write_text("split: train\n")
+    b = provenance.capture(repo=str(repo), gpu=False).diff
+    assert a.startswith("sha256:") and a != b
+    (repo / "bench.py").write_bytes(b"print(\xff\xfe)\n")  # not UTF-8
+    assert provenance.capture(repo=str(repo), gpu=False).dirty
+
+
+def test_a_git_that_fails_is_never_clean(repo, monkeypatch):
+    monkeypatch.setattr(provenance, "_bytes", lambda *a: None)
+    p = provenance.capture(repo=str(repo), gpu=False)
+    assert p.dirty and p.diff == "unknown"
+
+
+def test_a_folder_is_checked_against_its_sha256sums(tmp_path):
+    d = tmp_path / "release"
+    d.mkdir()
+    (d / "model.safetensors").write_bytes(b"weights")
+    (d / "kai.json").write_text("{}")
+    sums = "".join(
+        f"{hashlib.sha256(x).hexdigest()}  {n}\n" for n, x in (("kai.json", b"{}"), ("model.safetensors", b"weights"))
+    )
+    (d / "SHA256SUMS").write_text(sums)
+    assert provenance.weights(str(d)) == "sha256:" + hashlib.sha256(sums.encode()).hexdigest()
+    (d / "model.safetensors").write_bytes(b"other weights")
+    with pytest.raises(ValueError, match="model.safetensors does not match"):
+        provenance.weights(str(d))
+    (d / "model.safetensors").write_bytes(b"weights")
+    (d / "extra.bin").write_bytes(b"x")
+    with pytest.raises(ValueError, match="extra.bin is not listed"):
+        provenance.weights(str(d))
+
+
+def test_a_name_cannot_forge_a_listing(tmp_path):
+    one, two = tmp_path / "one", tmp_path / "two"
+    one.mkdir()
+    two.mkdir()
+    z = hashlib.sha256(b"Z").hexdigest()
+    (one / f"a\n{z}  b").write_bytes(b"X")
+    (two / "a").write_bytes(b"X")
+    (two / "b").write_bytes(b"Z")
+    assert provenance.weights(str(one)) != provenance.weights(str(two))
+    (two / "linked").symlink_to(one, target_is_directory=True)
+    with pytest.raises(ValueError, match="directory symlink"):
+        provenance.weights(str(two))
+
+
+def test_no_credential_survives_in_a_remote(monkeypatch):
+    # git's own insteadOf rewrites differ by machine, so the url is given, not configured
+    for url, want in (
+        ("https://user:[email protected]/hanzoai/x.git", "https://github.com/hanzoai/x.git"),
+        ("https://user:tok/[email protected]/hanzoai/x.git", "https://github.com/hanzoai/x.git"),
+        ("https://user:p@[email protected]/hanzoai/x.git", "https://github.com/hanzoai/x.git"),
+        ("[email protected]:hanzoai/x.git", "[email protected]:hanzoai/x.git"),
+    ):
+        monkeypatch.setattr(provenance, "_git", lambda repo, *args, u=url: u)
+        assert provenance._remote("repo") == want

pkg/hanzo-research/tests/test_wal.py

@@ -62,3 +62,20 @@ def test_workers_merge_in_one_order(tmp_path):
 def test_a_payload_with_no_json_form_is_refused(tmp_path):
     with pytest.raises(ValueError):
         ResearchWal.open(str(tmp_path), "run").append("step", {"loss": float("nan")})
+
+
+def test_a_torn_last_line_is_cut_and_a_whole_one_kept(tmp_path):
+    w = ResearchWal.open(str(tmp_path), "run")
+    w.append("step", {"i": 1})
+    w.append("step", {"i": 2})
+    with open(w.path, "a") as f:
+        f.write('{"run_id":"run","seq')
+    again = ResearchWal.open(str(tmp_path), "run")
+    assert again.sequence == 2
+    again.append("step", {"i": 3})
+    assert wal.verify(again.path) == again.last_event_hash
+    text = open(again.path).read().rstrip("\n")
+    open(again.path, "w").write(text)
+    last = ResearchWal.open(str(tmp_path), "run")
+    assert last.sequence == 3 and last.append("step", {"i": 4}).sequence == 4
+    assert wal.verify(last.path) == last.last_event_hash