sites: the edge asks the app that owns the store, because it is never in this process

Every published site served the console SPA. The host fix in 8b729f8ef was
necessary and not sufficient: with the host resolving correctly the edge still
found no site, because sites.SetResolver writes a PACKAGE-LEVEL registry inside
`projects` and the edge middleware reads it inside whichever process fronts
:8000. The pod boots ~25 single-app processes ("enabled":["<one>"], 25 distinct,
zero multi-app — measured on the live pod), so those are never the same process
and the registry is always nil where it is consulted.

A nil registry is a clean MISS, not a fault. So every lookup failed silently,
every request fell through to the API pipeline, and no error was logged anywhere
because nothing had failed. Proven at the pod with the ingress bypassed:

  wget --header="Host: app.maxpower.hanzo.app" http://127.0.0.1:8000/
    -> <title>Hanzo Cloud Console

for a site that is genuinely published and has its own Ingress.

The fix is the seam this repo already uses for exactly this shape:
FinanceScopeRules is on the plane, in its own words, because "the READER is a
cloud EDGE middleware" and the fact belongs to another app. projects now
publishes sites_resolve / sites_resolve_org from the one process that owns the
store, and the edge falls back to them. Co-residence still wins with no hop —
currentResolver prefers the in-process registry and only then asks.

Not-found stays a clean 404; a failure to ASK stays an error, so the edge can
render 503. Collapsing those would serve 404s for live customer sites during any
transient failure of the owning app, which is indistinguishable from deletion.

The comment on SetResolver said "until it is set, every site request is an honest
404 (the projects subsystem is not mounted)". That premise was the bug: projects
IS mounted, just elsewhere, and 404 is not honest when the site exists.

Negative-controlled: removing the fallback fails the new test with the
fall-through this commit is named for. The second test pins that a co-resident
store is still used without the hop.

apps/projects/projects.go

@@ -272,6 +272,13 @@ func Mount(app cloud.Router, deps cloud.Deps) error {
 	// reads it per request.
 	sites.SetResolver(siteResolver{store: store})
 
+	// ...and publish the SAME resolver on the internal plane, because in
+	// production the edge is never in this process (the pod boots ~25 single-app
+	// processes, so the registry above is nil wherever it is read). Keep both:
+	// co-resident takes the in-process answer with no hop, split takes the plane.
+	setResolverForPlane(siteResolver{store: store})
+	exposeSites()
+
 	// Register the store as a project-ownership resolver for the identity trust
 	// boundary (cloud.SanitizeIdentity), so a forged cross-org X-Project-Id is
 	// refused before any subsystem reads it. Same inversion as sites.SetResolver —

apps/projects/sites_rpc.go

@@ -0,0 +1,105 @@
+package projects
+
+import (
+	"context"
+
+	"github.com/zap-proto/zip"
+
+	cloud "github.com/hanzoai/cloud"
+	"github.com/hanzoai/cloud/apps/sites"
+	"github.com/hanzoai/cloud/plane"
+)
+
+// The site edge asks projects which published site a host belongs to.
+//
+// sites.SetResolver already installs an in-process resolver at Mount, and that
+// stays — when the edge and projects happen to share a process it is the right
+// answer, with no hop. But the pod boots ~25 SINGLE-app processes, so in
+// production they never do: the registry was set inside `projects` and read
+// inside whichever process fronts :8000, where it is nil. A nil registry is a
+// clean miss rather than a fault, so every published site resolved as not-found
+// with no error anywhere and fell through to the API pipeline — every
+// <slug>.hanzo.app served the console SPA, and the whole cloud API answered on
+// the customer's own hostname. Measured at the pod with the ingress bypassed.
+//
+// This is the same seam FinanceScopeRules already uses and for the same stated
+// reason: the READER is a cloud edge middleware and the fact belongs to another
+// app. The store read stays in the one process that owns the store.
+//
+// No org is taken from the caller on the multi-tenant path. The host IS the
+// tenant key here, so accepting an org would let a caller name someone else's
+// project; ResolveOrg pins an org only for the first-party path, which is
+// exactly what it is for.
+func exposeSites() {
+	zip.Post[plane.SiteIn, plane.Site](cloud.Plane(), "/sites/resolve", planeResolveSite,
+		zip.WithOperationID(plane.SitesResolve),
+		zip.WithSummary("Resolve a published site by host label"))
+
+	zip.Post[plane.SiteIn, plane.Site](cloud.Plane(), "/sites/resolve-org", planeResolveSiteOrg,
+		zip.WithOperationID(plane.SitesResolveOrg),
+		zip.WithSummary("Resolve a published site pinned to one org"))
+}
+
+// planeResolveSite answers the multi-tenant product URL (<slug>.hanzo.app) and
+// bound custom domains. Not-found is `Found:false`, never an error: the edge
+// turns that into an honest 404, and an error into a 503. Collapsing the two
+// would serve 404s for real live sites during a transient failure.
+func planeResolveSite(ctx context.Context, in *plane.SiteIn) (*plane.Site, error) {
+	r, err := currentResolver()
+	if err != nil {
+		return nil, err
+	}
+	s, ok, err := r.Resolve(ctx, in.Slug)
+	if err != nil {
+		return nil, err
+	}
+	return wireSite(s, ok), nil
+}
+
+// planeResolveSiteOrg is the first-party path: it NEVER falls back to
+// unique-across-orgs, so an internal host is served only by our own project and
+// never a customer's same-named one.
+func planeResolveSiteOrg(ctx context.Context, in *plane.SiteIn) (*plane.Site, error) {
+	r, err := currentResolver()
+	if err != nil {
+		return nil, err
+	}
+	s, ok, err := r.ResolveOrg(ctx, in.Org, in.Slug)
+	if err != nil {
+		return nil, err
+	}
+	return wireSite(s, ok), nil
+}
+
+// The resolver this process serves plane answers from. Set at Mount beside
+// sites.SetResolver, so the two can never name different stores.
+var planeResolver siteResolver
+
+func setResolverForPlane(r siteResolver) { planeResolver = r }
+
+// currentResolver refuses rather than answering not-found when the store is
+// absent. A process that has not mounted projects cannot know whether a site
+// exists, and saying "no" would take every live site off the air with a 404.
+func currentResolver() (siteResolver, error) {
+	if planeResolver.store == nil {
+		return siteResolver{}, zip.ErrInternal("sites: this process does not own the project store")
+	}
+	return planeResolver, nil
+}
+
+// wireSite projects a resolved Site onto the wire shape, carrying found-ness
+// explicitly so the edge can tell "no such site" from "could not ask".
+func wireSite(s sites.Site, ok bool) *plane.Site {
+	if !ok {
+		return &plane.Site{Found: false}
+	}
+	return &plane.Site{
+		Found:                true,
+		Org:                  s.Org,
+		Slug:                 s.Slug,
+		Bucket:               s.Bucket,
+		Prefix:               s.Prefix,
+		Status:               s.Status,
+		CrossOriginIsolation: s.CrossOriginIsolation,
+	}
+}

apps/sites/sites.go

@@ -84,22 +84,49 @@ type Resolver interface {
 var (
 	resolverMu sync.RWMutex
 	resolver   Resolver
+	fallback   Resolver
 )
 
 // SetResolver installs the slug→Site resolver. projects.Mount calls this once
-// with its store. Until it is set, every site request is an honest 404 (the
-// projects subsystem is not mounted), never a crash.
+// with its store — the no-hop answer when the edge and projects share a process.
 func SetResolver(r Resolver) {
 	resolverMu.Lock()
 	resolver = r
 	resolverMu.Unlock()
 }
 
+// SetFallbackResolver installs the resolver used when projects is NOT in this
+// process. cloud's composition root sets it to a plane-backed client.
+//
+// It exists because in production they are never in the same process: the pod
+// boots ~25 single-app processes, so the registry above was written inside
+// `projects` and read inside whichever process fronts :8000, where it is nil. A
+// nil registry is a clean miss, not a fault — so every published site resolved
+// as not-found with no error anywhere, fell through to the API pipeline, and
+// <slug>.hanzo.app served the console SPA with the whole cloud API answering on
+// the customer's own hostname. Measured at the pod, ingress bypassed.
+//
+// The old comment here read "until it is set, every site request is an honest
+// 404 (the projects subsystem is not mounted)". That premise was the bug:
+// projects IS mounted, just somewhere else, and 404 is not honest when the site
+// exists.
+func SetFallbackResolver(r Resolver) {
+	resolverMu.Lock()
+	fallback = r
+	resolverMu.Unlock()
+}
+
+// currentResolver prefers the in-process store and falls back to the plane. A
+// process that owns the store never pays for a hop; one that does not can still
+// answer, instead of silently serving the API for every customer's site.
 func currentResolver() Resolver {
 	resolverMu.RLock()
-	r := resolver
+	r, fb := resolver, fallback
 	resolverMu.RUnlock()
-	return r
+	if r != nil {
+		return r
+	}
+	return fb
 }
 
 // Config configures the site host-router. Apex is the zone whose subdomains are

apps/sites/sites_test.go

@@ -652,3 +652,52 @@ func TestMiddlewareForwardedHostNeverOverridesARealHost(t *testing.T) {
 		t.Fatalf("resolver called with %v, want exactly [victim] — the header must not override a real host", got)
 	}
 }
+
+// The edge must resolve a site when projects is in ANOTHER process.
+//
+// This is the production shape and it is the defect this fallback exists for:
+// the pod boots ~25 single-app processes, so the in-process registry is nil at
+// the edge. A nil resolver is a clean miss, so every published site fell through
+// to the API pipeline and <slug>.hanzo.app served the console SPA — with no
+// error logged anywhere, because nothing had failed.
+func TestFallbackResolverServesWhenProjectsIsElsewhere(t *testing.T) {
+	SetResolver(nil) // projects is NOT in this process — the production case.
+	fb := &fakeResolver{found: false}
+	SetFallbackResolver(fb)
+	defer SetFallbackResolver(nil)
+	app := newTestApp(testServer())
+
+	req := httptest.NewRequest("GET", "http://quest.hanzo.app/index.html", nil)
+	resp, err := app.Fiber().Test(req)
+	if err != nil {
+		t.Fatalf("test: %v", err)
+	}
+	if resp.Header.Get("X-Sentinel") == "hit" {
+		t.Fatal("fell through to the API pipeline — this is the console-instead-of-site defect")
+	}
+	if got := fb.slugs(); len(got) != 1 || got[0] != "quest" {
+		t.Fatalf("fallback called with %v, want exactly [quest]", got)
+	}
+}
+
+// A co-resident store still answers WITHOUT the hop: the in-process resolver
+// wins whenever it is set, so sharing a process costs nothing.
+func TestInProcessResolverWinsOverTheFallback(t *testing.T) {
+	inproc := &fakeResolver{found: false}
+	fb := &fakeResolver{found: false}
+	SetResolver(inproc)
+	SetFallbackResolver(fb)
+	defer func() { SetResolver(nil); SetFallbackResolver(nil) }()
+	app := newTestApp(testServer())
+
+	req := httptest.NewRequest("GET", "http://quest.hanzo.app/index.html", nil)
+	if _, err := app.Fiber().Test(req); err != nil {
+		t.Fatalf("test: %v", err)
+	}
+	if len(inproc.slugs()) != 1 {
+		t.Errorf("in-process resolver was not used: %v", inproc.slugs())
+	}
+	if n := len(fb.slugs()); n != 0 {
+		t.Errorf("fallback was consulted %d times; the in-process store must win", n)
+	}
+}

plane/plane.go

@@ -47,6 +47,20 @@ import (
 // The token is the op's operationId, which is also its OpenAPI operation, its
 // MCP tool name and its CLI command — one identity across every projection.
 const (
+	// SitesResolve / SitesResolveOrg answer "which published site is this host?"
+	// for the site EDGE, which is the same reason FinanceScopeRules is here: the
+	// reader is a cloud edge middleware and the owner of the fact is another app.
+	//
+	// It is on the plane because it HAS to be. The edge middleware and projects
+	// (which owns the project store, and called sites.SetResolver at its Mount)
+	// run in DIFFERENT processes — the pod boots ~25 single-app processes — so a
+	// package-level registry is nil wherever it is consulted. Every published
+	// site therefore resolved as not-found and fell through to the API pipeline,
+	// and <slug>.hanzo.app served the console SPA. Measured at the pod, ingress
+	// bypassed, 2026-08-03.
+	SitesResolve    = "sites_resolve"
+	SitesResolveOrg = "sites_resolve_org"
+
 	FinanceAuthorize = "finance_authorize" // the prepaid gate
 	FinanceBalance   = "finance_balance"
 	FinanceRecord    = "finance_record" // the meter
@@ -705,3 +719,27 @@ func BindRuntimeDir() string {
 	_ = os.Setenv("ZIP_RUNTIME_DIR", dir)
 	return dir
 }
+
+// SiteIn names a published site to resolve: the host label for the multi-tenant
+// product URL, or a bound custom domain. Org is set ONLY by the first-party
+// path (ResolveOrg), which pins the lookup to one org so an internal host is
+// never served by a customer's same-named project.
+type SiteIn struct {
+	Slug string `json:"slug"`
+	Org  string `json:"org,omitempty"`
+}
+
+// Site is a published site's serving facts. Found is explicit: a site that does
+// not exist is a clean answer, not an error, and the edge must be able to tell
+// "no such site" (honest 404) from "the owner could not be reached" (503) —
+// collapsing them is how a transient failure would start serving 404s for real
+// customers' live sites.
+type Site struct {
+	Found                bool   `json:"found"`
+	Org                  string `json:"org"`
+	Slug                 string `json:"slug"`
+	Bucket               string `json:"bucket"`
+	Prefix               string `json:"prefix"`
+	Status               string `json:"status"`
+	CrossOriginIsolation bool   `json:"crossOriginIsolation"`
+}

serve.go

@@ -287,6 +287,11 @@ func Listen(plugins []Plugin, enable []string) error {
 	// injected at its Mount via sites.SetResolver; until then a site host 404s
 	// honestly. Org isolation (org+prefix come only from the store keyed by the
 	// validated slug; object keys are rooted-clean) lives in clients/sites.
+	// The edge asks the app that owns the store when it is not in this process,
+	// which in production is always: the pod boots ~25 single-app processes, so
+	// the registry projects.Mount writes is nil here. Co-resident still wins with
+	// no hop — currentResolver prefers the in-process one.
+	sites.SetFallbackResolver(planeSites{})
 	app.Use(sites.New(sites.Config{Apex: cfg.SitesApex, Reserved: cfg.SitesReserved, SelfDomains: cfg.SitesSelfDomains, FirstPartyApex: cfg.SitesFirstPartyApex, FirstPartySites: cfg.SitesFirstPartySites, FirstPartyOrg: cfg.SitesFirstPartyOrg}, deps.Logger).Middleware())
 
 	// Edge policy — the "gateway role" cloud absorbs to serve the public

sites_plane.go

@@ -0,0 +1,65 @@
+package cloud
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/hanzoai/cloud/apps/sites"
+	"github.com/hanzoai/cloud/plane"
+)
+
+// planeSites resolves a published site by asking the app that owns the project
+// store, over the internal plane.
+//
+// The site edge middleware runs at the compose root, in whatever process fronts
+// the public port; the store belongs to `projects`. In production those are
+// never the same process — the pod boots ~25 single-app processes — so the
+// package-level registry projects.Mount writes is nil where the edge reads it.
+// That is why every <slug>.hanzo.app served the console SPA: a nil resolver is a
+// clean miss, so the request fell through to the API pipeline with no error
+// anywhere to notice.
+//
+// Same seam, same reason, as the balance and scope-rule reads that already cross
+// this plane: the reader is an edge middleware and the fact belongs elsewhere.
+type planeSites struct{}
+
+// Resolve answers the multi-tenant product URL and bound custom domains.
+//
+// A site that does not exist comes back Found:false and becomes an honest 404.
+// A failure to ASK is an error, and stays one — the edge renders 503 for that.
+// Collapsing the two would serve 404s for real, live customer sites during any
+// transient failure of the owning app, which looks exactly like the site being
+// deleted.
+func (planeSites) Resolve(ctx context.Context, slug string) (sites.Site, bool, error) {
+	return askSite(ctx, &plane.SiteIn{Slug: slug}, plane.SitesResolve)
+}
+
+// ResolveOrg is the first-party path, pinned to one org so an internal host is
+// never served by a customer's same-named project.
+func (planeSites) ResolveOrg(ctx context.Context, org, slug string) (sites.Site, bool, error) {
+	return askSite(ctx, &plane.SiteIn{Slug: slug, Org: org}, plane.SitesResolveOrg)
+}
+
+func askSite(ctx context.Context, in *plane.SiteIn, op string) (sites.Site, bool, error) {
+	// The site plane read is org-less by construction: the HOST is the tenant
+	// key, and the answer names the org. Passing one in would let a caller point
+	// at someone else's project.
+	out, err := Ask[plane.SiteIn, plane.Site](For(ctx, ""), "projects", op, in)
+	if err != nil {
+		return sites.Site{}, false, fmt.Errorf("sites: ask projects: %w", err)
+	}
+	if out == nil {
+		return sites.Site{}, false, fmt.Errorf("sites: projects answered nothing")
+	}
+	if !out.Found {
+		return sites.Site{}, false, nil
+	}
+	return sites.Site{
+		Org:                  out.Org,
+		Slug:                 out.Slug,
+		Bucket:               out.Bucket,
+		Prefix:               out.Prefix,
+		Status:               out.Status,
+		CrossOriginIsolation: out.CrossOriginIsolation,
+	}, true, nil
+}