Files
NandakishorandGitHub d4cd60c187 Merge pull request #339 from aashish254/feat-ts-structured
feat(ts): port structured decisions (decide) to laya-ts
2026-09-24 21:56:31 +05:30

6.1 KiB

laya-ts

TypeScript inference for Laya (Agent.predict, Router, lang, email, presets, shortlist, hooks) on Node and the browser via split ONNX (encoder.onnx + head.onnx). ESM-only ("type": "module"); no CJS build — import from ESM or bundle.

Export weights (once per checkpoint)

python laya-ts/scripts/export_onnx.py --model-dir <ckpt> --out-dir ./model
# writes encoder.onnx, head.onnx + copies tokenizer.json, rl_agent_config.json
# verifies torch vs ONNX match within 1e-4 (skip with --no-verify)

Node (CPU/CUDA)

import { Agent, Router } from "laya-ts";

const agent = await Agent.load("./model"); // local dir, or ("convaiinnovations/laya", { subfolder: "multilingual" })
const router = new Router();
router.attach("english", agent);
const out = await router.predict({ body: "charged twice, refund please" }, {
  intent: { type: "choice", instructions: "What does the customer want?", criteria: { refund: "money back", other: "anything else" } },
});
console.log(out.answers.intent);

CUDA: Agent.load("./model", { device: "cuda" }) (falls back to CPU with a warning).

Browser (WebGPU → WASM fallback)

import { Agent } from "laya-ts";

const agent = await Agent.load("https://example.com/models/laya"); // serves encoder.onnx, head.onnx, tokenizer.json, rl_agent_config.json
const out = await agent.predict("charged twice", {
  d: { type: "choice", instructions: "pick", criteria: { refund: "money back", other: "rest" } },
});

onnxruntime-node / onnxruntime-web are optional peer deps, imported lazily behind the provider you use.

Hooks (observe or shape every decision)

Port of the Python laya.hooks lifecycle. A hook is a (ctx) => void for onPredictStart / onPredictEnd, or an object implementing any subset of onPredictStart, onPredictEnd, onRoute, onLoad, onEvict, onError:

const tracer = {
  onPredictStart(ctx) { console.time(ctx.runId); },
  onPredictEnd(ctx) { console.timeEnd(ctx.runId); console.log(ctx.model, ctx.usage, ctx.elapsedMs); },
};
const router = new Router({ hooks: [tracer], hooksRaise: false }); // telemetry must not fail a request
await router.withHooks([auditHook], () => router.predict(state, questions)); // scoped install

// a start hook may rewrite ctx.states / ctx.questions, or serve a cached result:
const cache = { onPredictStart(ctx) { const hit = lookup(ctx.states[0]); if (hit) ctx.skip([hit]); } };
// an onRoute hook may replace ctx.decision (e.g. pin a checkpoint)

// subclass BaseHook to override only the events you need:
class MetricsHook extends BaseHook {
  onPredictEnd(ctx) { record(ctx.usage); }
}

// process-wide defaults run before installed and per-call hooks for every Agent/Router,
// so a tracer or metrics hook does not have to be threaded through every construction:
setDefaultHooks([new MetricsHook()]);  // addDefaultHook(...) appends; clearDefaultHooks() resets

Structured decisions (decide)

Turn a JSON schema into typed values in one call — the port of Python's laya.structured (#280). Enum properties become choice questions, booleans become noul, bounded integers become scores; anything the fixed-option model cannot answer (free strings, arrays, nested objects, $ref) is rejected with a SchemaError naming the path:

import { Agent, decide } from "laya-ts";

const agent = await Agent.load("./dist/laya");
const values = await agent.decide(ticketText, {
  type: "object",
  properties: {
    department: { type: "string", enum: ["billing", "support", "sales"] },
    urgency: { type: "integer", minimum: 0, maximum: 2 },
    needs_human: { type: "boolean" },
  },
});
// { department: "billing", urgency: 2, needs_human: false }

router.decide(...) works the same way (routing options are forwarded to predict), and the free decide(runner, state, schema, opts) accepts anything with a predict method. Pass { returnDetails: true } for per-field confidence and probabilities, or { questions } instead of a schema to get raw answers. Zod/TypeBox users can pass z.toJSONSchema(Model) — any object with a toJSONSchema() method is accepted. planFromJsonSchema, questionsFromJsonSchema and answersToJson expose the planning and projection steps.

Shortlist (many labels)

Shortlist (many labels)

import { shortlistChoice, predictShortlist, embedFnFromAgent } from "laya-ts";

const keep = await shortlistChoice(state, bigCriteriaDict, embedFn, 20);
const out = await predictShortlist(agent, state, questions, embedFn, 20);
// out.shortlist[qid] = { labels, scores, k, n, passthrough }
// embedFnFromAgent(agent) mean-pools the loaded encoder; a dedicated bi-encoder usually shortlists better.

Per-language calibration (lang_temperatures)

Port of the Python Agent(lang_temperatures=...) knob. A language override replaces the checkpoint's temperature for matching requests — keys normalise to the base subtag (de-AT → de), an omitted temperature inherits the base one, and temperature_by_options works per option-count bucket as usual:

const agent = await Agent.load("convaiinnovations/laya", {
  lang_temperatures: {
    de: { temperature: [1.2, 1.2, 1.2] },                 // fitted on German evals
    ja: { temperature_by_options: { "choice:11+": 1.4 } }, // buckets only, base temperature kept
  },
});
await agent.systemOne(state, questions, { lang: "de" });   // uses the German temperature
await router.predict(state, questions);                    // Router forwards the detected language

Router.predict forwards an explicit lang verbatim and otherwise the detected language (never "en" — matching Python, where detection only names non-English languages), so an override applies exactly to the requests it was fitted on.

Example (repo root)

node laya-ts/examples/try-ml.mjs   # needs ./model-ml from the export step
node laya-ts/examples/snake.mjs --ticks 50   # autonomous snake demo, headless smoke (live TUI without --ticks)

Packaging

ponytail: CJS/browser-field dual build + tsconfig tests-include deferred — Task 7 verified ESM-only; CJS needs second tsc config + export-map change, untested. Add when a CJS consumer or browser-field swap is requested.