mirror of
https://github.com/hanzoai/ai.git
synced 2026-11-21 02:36:49 +00:00
* fix(router): explore/heuristic honor true servability (served ∩ enabled), not just the org allowlist The per-org policy override replaced the process's real servability predicate (c.Known) with the org's Enabled allowlist, so the exploration floor and the servable-first heuristic could pick an ENABLED-but-UNSERVED model (e.g. one still allowlisted after its provider disconnected) — which 404s downstream. With ec.Allow left unset the last-resort also relaxed to ALL models, ignoring disabled. Intersect instead of replace: ec.Known = served ∩ enabled (explore + servable-first heuristic only pick actually-served models); set ec.Allow = rp.Enabled as the HARD allowlist so the last-resort may relax servability to a listed model (beats an empty id) but never routes outside the allowlist. Adds TestExploreStaysServableUnderPolicy + TestExploreSamplesOnlyServedAlternatives. * feat(router): 5s org-settings cache TTL + opt-in Valkey cross-pod invalidation bus Reconcile the org-settings (router policy / savings↔quality dial) cache TTL from 60s to 5s so a console change converges fleet-wide within 5s (overridable via ROUTER_SETTINGS_CACHE_TTL_SECONDS). Add an opt-in cross-pod invalidation bus: when KV_URL is set, every org-settings write broadcasts over Valkey pub/sub so all pods drop the stale entry sub-second, with the TTL as the fail-safe backstop; a no-op without KV_URL. Mirrors InitDatastore/InitTelemetry; wired from Bootstrap. --------- Co-authored-by: hanzo-dev <[email protected]>
58 lines
1.9 KiB
Go
58 lines
1.9 KiB
Go
package object
|
|
|
|
import (
|
|
"context"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/alicebob/miniredis/v2"
|
|
redis "github.com/hanzoai/kv-go/v9"
|
|
)
|
|
|
|
// TestCacheBus_CrossPodInvalidation proves the FULL pub/sub loop end-to-end against
|
|
// a real Redis-protocol broker (miniredis, in-process): a DIFFERENT pod's write,
|
|
// published to the invalidation channel, reaches THIS pod's subscriber and flushes
|
|
// its local org-settings cache — the sub-second cross-pod convergence KV_URL buys.
|
|
func TestCacheBus_CrossPodInvalidation(t *testing.T) {
|
|
mr := miniredis.RunT(t) // in-process Redis w/ pub-sub; auto-closed by t.Cleanup
|
|
|
|
// Bring up THIS pod's bus (subscriber) pointed at the broker.
|
|
t.Setenv("KV_URL", "redis://"+mr.Addr())
|
|
InitCacheBus()
|
|
t.Cleanup(func() {
|
|
cacheBusMu.Lock()
|
|
if cacheBusClient != nil {
|
|
_ = cacheBusClient.Close()
|
|
cacheBusClient = nil
|
|
}
|
|
cacheBusMu.Unlock()
|
|
})
|
|
|
|
// Seed this pod's cache as if it had served a read.
|
|
invalidateOrgSettingsCacheLocal()
|
|
orgSettingsCacheMu.Lock()
|
|
orgSettingsCache["acme"] = &orgSettingsCacheEntry{settings: &OrgSettings{Owner: "acme"}, fetchedAt: time.Now()}
|
|
orgSettingsCacheMu.Unlock()
|
|
|
|
// A SEPARATE pod publishes an org_settings write. Retry until it lands — pub/sub
|
|
// is lossy until this pod's subscriber has attached — or time out. A live pod
|
|
// subscribes at boot, long before any write, so it needs no retry in production.
|
|
pub := redis.NewClient(&redis.Options{Addr: mr.Addr()})
|
|
defer pub.Close()
|
|
|
|
deadline := time.Now().Add(5 * time.Second)
|
|
for {
|
|
_ = pub.Publish(context.Background(), cacheBusChannel, "org_settings").Err()
|
|
orgSettingsCacheMu.RLock()
|
|
n := len(orgSettingsCache)
|
|
orgSettingsCacheMu.RUnlock()
|
|
if n == 0 {
|
|
return // subscriber received the broadcast and flushed — loop proven
|
|
}
|
|
if time.Now().After(deadline) {
|
|
t.Fatal("cross-pod broadcast did not flush the local cache within 5s")
|
|
}
|
|
time.Sleep(20 * time.Millisecond)
|
|
}
|
|
}
|