6.1 KiB
laya-ts
TypeScript inference for Laya (Agent.predict, Router, lang, email, presets, shortlist, hooks) on Node and the browser via split ONNX (encoder.onnx + head.onnx). ESM-only ("type": "module"); no CJS build — import from ESM or bundle.
Export weights (once per checkpoint)
python laya-ts/scripts/export_onnx.py --model-dir <ckpt> --out-dir ./model
# writes encoder.onnx, head.onnx + copies tokenizer.json, rl_agent_config.json
# verifies torch vs ONNX match within 1e-4 (skip with --no-verify)
Node (CPU/CUDA)
import { Agent, Router } from "laya-ts";
const agent = await Agent.load("./model"); // local dir, or ("convaiinnovations/laya", { subfolder: "multilingual" })
const router = new Router();
router.attach("english", agent);
const out = await router.predict({ body: "charged twice, refund please" }, {
intent: { type: "choice", instructions: "What does the customer want?", criteria: { refund: "money back", other: "anything else" } },
});
console.log(out.answers.intent);
CUDA: Agent.load("./model", { device: "cuda" }) (falls back to CPU with a warning).
Browser (WebGPU → WASM fallback)
import { Agent } from "laya-ts";
const agent = await Agent.load("https://example.com/models/laya"); // serves encoder.onnx, head.onnx, tokenizer.json, rl_agent_config.json
const out = await agent.predict("charged twice", {
d: { type: "choice", instructions: "pick", criteria: { refund: "money back", other: "rest" } },
});
onnxruntime-node / onnxruntime-web are optional peer deps, imported lazily behind the provider you use.
Hooks (observe or shape every decision)
Port of the Python laya.hooks lifecycle. A hook is a (ctx) => void for onPredictStart /
onPredictEnd, or an object implementing any subset of onPredictStart, onPredictEnd,
onRoute, onLoad, onEvict, onError:
const tracer = {
onPredictStart(ctx) { console.time(ctx.runId); },
onPredictEnd(ctx) { console.timeEnd(ctx.runId); console.log(ctx.model, ctx.usage, ctx.elapsedMs); },
};
const router = new Router({ hooks: [tracer], hooksRaise: false }); // telemetry must not fail a request
await router.withHooks([auditHook], () => router.predict(state, questions)); // scoped install
// a start hook may rewrite ctx.states / ctx.questions, or serve a cached result:
const cache = { onPredictStart(ctx) { const hit = lookup(ctx.states[0]); if (hit) ctx.skip([hit]); } };
// an onRoute hook may replace ctx.decision (e.g. pin a checkpoint)
// subclass BaseHook to override only the events you need:
class MetricsHook extends BaseHook {
onPredictEnd(ctx) { record(ctx.usage); }
}
// process-wide defaults run before installed and per-call hooks for every Agent/Router,
// so a tracer or metrics hook does not have to be threaded through every construction:
setDefaultHooks([new MetricsHook()]); // addDefaultHook(...) appends; clearDefaultHooks() resets
Structured decisions (decide)
Turn a JSON schema into typed values in one call — the port of Python's laya.structured
(#280). Enum properties become choice questions, booleans become noul, bounded integers
become scores; anything the fixed-option model cannot answer (free strings, arrays, nested
objects, $ref) is rejected with a SchemaError naming the path:
import { Agent, decide } from "laya-ts";
const agent = await Agent.load("./dist/laya");
const values = await agent.decide(ticketText, {
type: "object",
properties: {
department: { type: "string", enum: ["billing", "support", "sales"] },
urgency: { type: "integer", minimum: 0, maximum: 2 },
needs_human: { type: "boolean" },
},
});
// { department: "billing", urgency: 2, needs_human: false }
router.decide(...) works the same way (routing options are forwarded to predict), and the
free decide(runner, state, schema, opts) accepts anything with a predict method. Pass
{ returnDetails: true } for per-field confidence and probabilities, or { questions }
instead of a schema to get raw answers. Zod/TypeBox users can pass z.toJSONSchema(Model) —
any object with a toJSONSchema() method is accepted. planFromJsonSchema,
questionsFromJsonSchema and answersToJson expose the planning and projection steps.
Shortlist (many labels)
Shortlist (many labels)
import { shortlistChoice, predictShortlist, embedFnFromAgent } from "laya-ts";
const keep = await shortlistChoice(state, bigCriteriaDict, embedFn, 20);
const out = await predictShortlist(agent, state, questions, embedFn, 20);
// out.shortlist[qid] = { labels, scores, k, n, passthrough }
// embedFnFromAgent(agent) mean-pools the loaded encoder; a dedicated bi-encoder usually shortlists better.
Per-language calibration (lang_temperatures)
Port of the Python Agent(lang_temperatures=...) knob. A language override replaces the
checkpoint's temperature for matching requests — keys normalise to the base subtag
(de-AT → de), an omitted temperature inherits the base one, and
temperature_by_options works per option-count bucket as usual:
const agent = await Agent.load("convaiinnovations/laya", {
lang_temperatures: {
de: { temperature: [1.2, 1.2, 1.2] }, // fitted on German evals
ja: { temperature_by_options: { "choice:11+": 1.4 } }, // buckets only, base temperature kept
},
});
await agent.systemOne(state, questions, { lang: "de" }); // uses the German temperature
await router.predict(state, questions); // Router forwards the detected language
Router.predict forwards an explicit lang verbatim and otherwise the detected language
(never "en" — matching Python, where detection only names non-English languages), so an
override applies exactly to the requests it was fitted on.
Example (repo root)
node laya-ts/examples/try-ml.mjs # needs ./model-ml from the export step
node laya-ts/examples/snake.mjs --ticks 50 # autonomous snake demo, headless smoke (live TUI without --ticks)
Packaging
ponytail: CJS/browser-field dual build + tsconfig tests-include deferred — Task 7 verified ESM-only; CJS needs second tsc config + export-map change, untested. Add when a CJS consumer or browser-field swap is requested.