Files
ai/object/cache_bus_integration_test.go
55da8c8a68 fix(router): explore honors true servability + 5s org-settings TTL & Valkey invalidation bus (#112)
* fix(router): explore/heuristic honor true servability (served ∩ enabled), not just the org allowlist

The per-org policy override replaced the process's real servability predicate
(c.Known) with the org's Enabled allowlist, so the exploration floor and the
servable-first heuristic could pick an ENABLED-but-UNSERVED model (e.g. one still
allowlisted after its provider disconnected) — which 404s downstream. With
ec.Allow left unset the last-resort also relaxed to ALL models, ignoring disabled.

Intersect instead of replace: ec.Known = served ∩ enabled (explore + servable-first
heuristic only pick actually-served models); set ec.Allow = rp.Enabled as the HARD
allowlist so the last-resort may relax servability to a listed model (beats an empty
id) but never routes outside the allowlist. Adds TestExploreStaysServableUnderPolicy
+ TestExploreSamplesOnlyServedAlternatives.

* feat(router): 5s org-settings cache TTL + opt-in Valkey cross-pod invalidation bus

Reconcile the org-settings (router policy / savings↔quality dial) cache TTL from
60s to 5s so a console change converges fleet-wide within 5s (overridable via
ROUTER_SETTINGS_CACHE_TTL_SECONDS). Add an opt-in cross-pod invalidation bus:
when KV_URL is set, every org-settings write broadcasts over Valkey pub/sub so all
pods drop the stale entry sub-second, with the TTL as the fail-safe backstop; a
no-op without KV_URL. Mirrors InitDatastore/InitTelemetry; wired from Bootstrap.

---------

Co-authored-by: hanzo-dev <[email protected]>
2026-07-22 15:51:35 -07:00

58 lines
1.9 KiB
Go

package object
import (
"context"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
redis "github.com/hanzoai/kv-go/v9"
)
// TestCacheBus_CrossPodInvalidation proves the FULL pub/sub loop end-to-end against
// a real Redis-protocol broker (miniredis, in-process): a DIFFERENT pod's write,
// published to the invalidation channel, reaches THIS pod's subscriber and flushes
// its local org-settings cache — the sub-second cross-pod convergence KV_URL buys.
func TestCacheBus_CrossPodInvalidation(t *testing.T) {
mr := miniredis.RunT(t) // in-process Redis w/ pub-sub; auto-closed by t.Cleanup
// Bring up THIS pod's bus (subscriber) pointed at the broker.
t.Setenv("KV_URL", "redis://"+mr.Addr())
InitCacheBus()
t.Cleanup(func() {
cacheBusMu.Lock()
if cacheBusClient != nil {
_ = cacheBusClient.Close()
cacheBusClient = nil
}
cacheBusMu.Unlock()
})
// Seed this pod's cache as if it had served a read.
invalidateOrgSettingsCacheLocal()
orgSettingsCacheMu.Lock()
orgSettingsCache["acme"] = &orgSettingsCacheEntry{settings: &OrgSettings{Owner: "acme"}, fetchedAt: time.Now()}
orgSettingsCacheMu.Unlock()
// A SEPARATE pod publishes an org_settings write. Retry until it lands — pub/sub
// is lossy until this pod's subscriber has attached — or time out. A live pod
// subscribes at boot, long before any write, so it needs no retry in production.
pub := redis.NewClient(&redis.Options{Addr: mr.Addr()})
defer pub.Close()
deadline := time.Now().Add(5 * time.Second)
for {
_ = pub.Publish(context.Background(), cacheBusChannel, "org_settings").Err()
orgSettingsCacheMu.RLock()
n := len(orgSettingsCache)
orgSettingsCacheMu.RUnlock()
if n == 0 {
return // subscriber received the broadcast and flushed — loop proven
}
if time.Now().After(deadline) {
t.Fatal("cross-pod broadcast did not flush the local cache within 5s")
}
time.Sleep(20 * time.Millisecond)
}
}