Files
ai/object/router_training_log_test.go
627946e6ac feat(router): improvement-history endpoint + append-only retrain log + regional task-mix (#104)
The investor/customer flywheel surface for world.hanzo.ai — the router getting
smarter OVER TIME, and what customers in each region are DOING. Aggregates only,
honest-empty (the ledger is nearly empty today, so every series starts at zero and
GROWS with real data — never seeded, never faked).

- GET /v1/router/history (public ?scope=platform, auth+balance-exempt like
  /v1/router/stats): a PURE, unit-tested computeRouterHistory folds the RoutingEvent
  ledger into a UTC-daily series — events/day (adoption), reward_rate, cost_saved_index
  + cumulative (routed vs the whole-window premium baseline, the same proxy as
  router/stats), engine_share, and per-day by_task — plus the retrain timeline and a
  totals roll-up. Empty days emit explicit zero rows (honest gaps → a line that starts
  flat and climbs). Privacy: the platform scope exposes the task mix (safe) but NEVER
  model ids.

- Append-only retrain log (object.RouterTrainingLog): RouterArtifactMeta is upsert-only
  (latest per scope), so the TIMELINE needs its own immutable table. PublishRouterArtifact
  Meta now appends one row per 4:20am fit; history reads it for the version markers.
  holdout_accuracy is surfaced ONLY when the gate metric is a holdout accuracy — a
  reward/AIQ gate is never relabeled as accuracy.

- Regional task-mix on the globe: object/traffic.go refactored to a nested per-geo
  {count, byService, byTask} bucket; new GlobalTraffic.RecordTask(country, region, task)
  enriches an EXISTING geo group's byTask only (a subset of count, never a phantom
  point, never a total change). resolveAutoModel now returns the classified task and the
  chat controller records it with the edge geo headers — geo stays out of the routing
  logic, and there is still NO IP anywhere (RecordTask has no IP parameter). TrafficPoint
  gains byTask (omitempty).

Tests: daily bucketing + cost-saved/cumulative + reward + task mix + honest-empty +
days-clamp + the accuracy-only holdout rule; append-only log (distinct ids, chronology,
since, limit, per-scope isolation); byTask aggregation + RecordTask-leaves-totals +
no-IP-still-holds + normTask. gofmt/gofumpt/vet clean; build + -race green.


Co-authored-by: hanzo-dev <[email protected]>
2026-07-16 20:07:52 -07:00

64 lines
2.4 KiB
Go

// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
package object
import "testing"
// TestRouterTrainingLog exercises the append-only retrain timeline: immutable rows
// (distinct ids + logged times), chronological listing by trained_time, the since
// filter, the most-recent limit, and per-scope isolation.
func TestRouterTrainingLog(t *testing.T) {
restore, err := UseMemoryDB("file:router_training_log_test?mode=memory&cache=shared", &RouterTrainingLog{})
if err != nil {
t.Fatalf("UseMemoryDB: %v", err)
}
defer restore()
rows := []*RouterTrainingLog{
{Owner: "*", Version: "v1", TrainedTime: "2026-07-14T04:20:00Z", Events: 100, GatePassed: true, Published: true, GateMetric: "holdout_accuracy", GateValue: 0.71},
{Owner: "*", Version: "v2", TrainedTime: "2026-07-15T04:20:00Z", Events: 220, GatePassed: true, Published: true, GateMetric: "holdout_accuracy", GateValue: 0.73},
{Owner: "acme", Version: "o1", TrainedTime: "2026-07-15T04:20:00Z", Events: 40, GatePassed: false, Published: false, GateMetric: "holdout_reward", GateValue: 0.31},
}
for _, r := range rows {
if err := AppendRouterTrainingLog(r); err != nil {
t.Fatalf("append: %v", err)
}
}
// Shared-base scope: two rows, oldest first, with unique append ids + logged times.
base, err := ListRouterTrainingLog("*", "", 0)
if err != nil {
t.Fatalf("list: %v", err)
}
if len(base) != 2 || base[0].Version != "v1" || base[1].Version != "v2" {
t.Fatalf("shared-base log wrong (want v1,v2 chronological): %+v", base)
}
if base[0].Id == "" || base[0].Id == base[1].Id {
t.Error("append-only rows must have distinct non-empty ids")
}
if base[0].LoggedTime == "" {
t.Error("append must stamp LoggedTime")
}
// since filter (trained_time >= since).
recent, _ := ListRouterTrainingLog("*", "2026-07-15T00:00:00Z", 0)
if len(recent) != 1 || recent[0].Version != "v2" {
t.Errorf("since filter wrong: %+v", recent)
}
// limit keeps the MOST RECENT, still chronological.
capped, _ := ListRouterTrainingLog("*", "", 1)
if len(capped) != 1 || capped[0].Version != "v2" {
t.Errorf("limit should keep the most recent: %+v", capped)
}
// Per-scope isolation: an org sees only its own rows.
org, _ := ListRouterTrainingLog("acme", "", 0)
if len(org) != 1 || org[0].Version != "o1" {
t.Errorf("org scope not isolated: %+v", org)
}
}