- BaseHook: a no-op base class for subclassing, exported at the top level next to the Hook protocol. - laya.hooks gains a process-wide registry: default_hooks, set_default_hooks, add_default_hook, clear_default_hooks, and compose_hooks. Defaults run before installed and per-call hooks, and cover the Router lifecycle events too. - Tests and docs updated (api, index, patterns, examples).
6.2 KiB
Prediction Hooks
Hooks let you observe or shape every decision Laya makes, without forking it.
They are the extension seam for the things every real deployment needs: audit logging, PII
redaction before inference, caching, metrics, confidence gating, routing overrides, and
forwarding a decision to an external service. They are opt-in: with no hooks configured the
behaviour of Agent, Router and ONNXAgent is unchanged.
This folder is the full reference. Start here, then dive into the page you need:
| page | what is in it |
|---|---|
| API reference | every class, field, parameter and default |
| Lifecycle | exactly when each hook runs, with flowcharts |
| Errors | hooks_raise, on_error, exception chaining, failure matrix |
| Patterns and anti-patterns | what to do, what to avoid, and why |
| Examples | copy-paste recipes for every use case |
| Tracing | run_id, span correlation, OpenTelemetry |
Quick start
import laya
def log(ctx):
print(ctx.model, ctx.results[0]["answers"], ctx.elapsed_ms)
agent = laya.load("convaiinnovations/laya", on_predict_end=log)
agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})
An object can implement any subset of the lifecycle events:
class Audit:
def on_predict_start(self, ctx):
print("start", ctx.run_id)
def on_predict_end(self, ctx):
print("end", ctx.run_id, ctx.usage, ctx.elapsed_ms)
def on_error(self, ctx):
print("failed", ctx.run_id, ctx.error)
laya.load("convaiinnovations/laya", hooks=[Audit()])
Hooks can also be added later or scoped to a block:
agent.add_hook(tracer) # attach at runtime
with agent.hooks_installed(debug): # installed for the block, removed on exit
agent.system_one(state, questions)
See runtime registration. For a hook that should apply everywhere without threading it through every call, register it once with process-wide defaults:
from laya import hooks
hooks.set_default_hooks(hooks=[Tracer()])
The mental model
There are three ideas.
-
A hook is a callable or an object. A plain function is convenient for one event; an object is convenient for several. Both are passed to
hooks=/on_predict_start=/on_predict_end=. -
Every hook of one call shares one mutable
PredictContext. It carries the states, questions, results, routing decision, model name, usage, timing and any error. Because it is mutable, a hook can shape the call, not only watch it: redact the state, rewrite the questions, replace the result, or skip inference with a cached answer. -
There are two scopes.
Agenthooks wrap a forward pass;Routerhooks wrap routing plus inference and can also see model lifecycle (on_route,on_load,on_evict). This mirrors the "run hooks" vs "agent hooks" split in other agent frameworks.
Router.predict(state, questions)
┌──────────────────────────────────────────────────────────────────────────┐
│ route() │
│ ├─ detect language / workflow │
│ └─ on_route ctx.decision (a hook may replace it) │
│ │
│ load(decision.model) │
│ ├─ build checkpoint on first use ──► on_load ctx.model, ctx.agent │
│ └─ evict LRU checkpoint ───────────► on_evict ctx.model │
│ │
│ on_predict_start ctx.states, ctx.questions, ctx.decision │
│ │ │
│ ├── ctx.skip(results)? ──► skip the forward pass │
│ │ │
│ └── Agent.system_one(...) ──► Agent-level hooks run here │
│ on_predict_start ─► forward ─► on_predict_end │
│ │
│ result["routing"] = decision │
│ on_predict_end ctx.results, ctx.usage, ctx.elapsed_ms │
└──────────────────────────────────────────────────────────────────────────┘
any failure on the way ──► on_error, then on_predict_end
Scope at a glance
Agent / ONNXAgent |
Router |
|
|---|---|---|
on_predict_start |
yes | yes |
on_predict_end |
yes | yes |
on_error |
yes | yes |
on_route |
no | yes |
on_load |
no | yes |
on_evict |
no | yes |
laya.serve and the MCP server call Router.predict, so Router hooks fire for them
automatically. Agent hooks fire whenever the Router runs an attached or built agent.
Compatibility
- No hooks configured means no behavioural change. The unset path is regression-tested.
- All hook parameters are keyword arguments with defaults, so existing calls keep working.
laya/hooks.pyis pure Python:import layadoes not pull in torch because of it.- Hooks are synchronous. Keep them fast and non-blocking; see
errors and patterns for the consequences on
laya.serve.
See also
examples/hooks/: runnable audit, redact, cache and metrics hooks.tests/test_hooks.py: the behaviour spec.tests/test_hooks_api.py: the API-stability guard.laya/hooks.py: the implementation.