Files
NandakishorandClaude Opus 5.5 970dc8c5f6 test(ts): warm up and take the best of five in the identifier ratio test
On a shared CI runner one descheduled run pushed the 4x-input ratio to 19 on
Node 22 (it passed on the rerun and on Node 20). One untimed call per size keeps
JIT tiering out of the measurement and five samples keep a single slow run from
deciding it. The absolute ceilings above still carry the quadratic guarantee.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-09-24 22:52:50 +05:30
..

laya-ts

TypeScript inference for Laya (Agent.predict, Router, lang, email, presets, shortlist, hooks) on Node and the browser via split ONNX (encoder.onnx + head.onnx). ESM-only ("type": "module"); no CJS build — import from ESM or bundle.

Export weights (once per checkpoint)

python laya-ts/scripts/export_onnx.py --model-dir <ckpt> --out-dir ./model
# writes encoder.onnx, head.onnx + copies tokenizer.json, rl_agent_config.json
# verifies torch vs ONNX match within 1e-4 (skip with --no-verify)

Node (CPU/CUDA)

import { Agent, Router } from "laya-ts";

const agent = await Agent.load("./model"); // local dir, or ("convaiinnovations/laya", { subfolder: "multilingual" })
const router = new Router();
router.attach("english", agent);
const out = await router.predict({ body: "charged twice, refund please" }, {
  intent: { type: "choice", instructions: "What does the customer want?", criteria: { refund: "money back", other: "anything else" } },
});
console.log(out.answers.intent);

CUDA: Agent.load("./model", { device: "cuda" }) (falls back to CPU with a warning).

Browser (WebGPU → WASM fallback)

import { Agent } from "laya-ts";

const agent = await Agent.load("https://example.com/models/laya"); // serves encoder.onnx, head.onnx, tokenizer.json, rl_agent_config.json
const out = await agent.predict("charged twice", {
  d: { type: "choice", instructions: "pick", criteria: { refund: "money back", other: "rest" } },
});

onnxruntime-node / onnxruntime-web are optional peer deps, imported lazily behind the provider you use.

Hooks (observe or shape every decision)

Port of the Python laya.hooks lifecycle. A hook is a (ctx) => void for onPredictStart / onPredictEnd, or an object implementing any subset of onPredictStart, onPredictEnd, onRoute, onLoad, onEvict, onError:

const tracer = {
  onPredictStart(ctx) { console.time(ctx.runId); },
  onPredictEnd(ctx) { console.timeEnd(ctx.runId); console.log(ctx.model, ctx.usage, ctx.elapsedMs); },
};
const router = new Router({ hooks: [tracer], hooksRaise: false }); // telemetry must not fail a request
await router.withHooks([auditHook], () => router.predict(state, questions)); // scoped install

// a start hook may rewrite ctx.states / ctx.questions, or serve a cached result:
const cache = { onPredictStart(ctx) { const hit = lookup(ctx.states[0]); if (hit) ctx.skip([hit]); } };
// an onRoute hook may replace ctx.decision (e.g. pin a checkpoint)

// subclass BaseHook to override only the events you need:
class MetricsHook extends BaseHook {
  onPredictEnd(ctx) { record(ctx.usage); }
}

// process-wide defaults run before installed and per-call hooks for every Agent/Router,
// so a tracer or metrics hook does not have to be threaded through every construction:
setDefaultHooks([new MetricsHook()]);  // addDefaultHook(...) appends; clearDefaultHooks() resets

Structured decisions (decide)

Turn a JSON schema into typed values in one call — the port of Python's laya.structured (#280). Enum properties become choice questions, booleans become noul, bounded integers become scores; anything the fixed-option model cannot answer (free strings, arrays, nested objects, $ref) is rejected with a SchemaError naming the path:

import { Agent, decide } from "laya-ts";

const agent = await Agent.load("./dist/laya");
const values = await agent.decide(ticketText, {
  type: "object",
  properties: {
    department: { type: "string", enum: ["billing", "support", "sales"] },
    urgency: { type: "integer", minimum: 0, maximum: 2 },
    needs_human: { type: "boolean" },
  },
});
// { department: "billing", urgency: 2, needs_human: false }

router.decide(...) works the same way (routing options are forwarded to predict), and the free decide(runner, state, schema, opts) accepts anything with a predict method. Pass { returnDetails: true } for per-field confidence and probabilities, or { questions } instead of a schema to get raw answers. Zod/TypeBox users can pass z.toJSONSchema(Model) — any object with a toJSONSchema() method is accepted. planFromJsonSchema, questionsFromJsonSchema and answersToJson expose the planning and projection steps.

Shortlist (many labels)

Shortlist (many labels)

import { shortlistChoice, predictShortlist, embedFnFromAgent } from "laya-ts";

const keep = await shortlistChoice(state, bigCriteriaDict, embedFn, 20);
const out = await predictShortlist(agent, state, questions, embedFn, 20);
// out.shortlist[qid] = { labels, scores, k, n, passthrough }
// embedFnFromAgent(agent) mean-pools the loaded encoder; a dedicated bi-encoder usually shortlists better.

Per-language calibration (lang_temperatures)

Port of the Python Agent(lang_temperatures=...) knob. A language override replaces the checkpoint's temperature for matching requests — keys normalise to the base subtag (de-AT → de), an omitted temperature inherits the base one, and temperature_by_options works per option-count bucket as usual:

const agent = await Agent.load("convaiinnovations/laya", {
  lang_temperatures: {
    de: { temperature: [1.2, 1.2, 1.2] },                 // fitted on German evals
    ja: { temperature_by_options: { "choice:11+": 1.4 } }, // buckets only, base temperature kept
  },
});
await agent.systemOne(state, questions, { lang: "de" });   // uses the German temperature
await router.predict(state, questions);                    // Router forwards the detected language

Router.predict forwards an explicit lang verbatim and otherwise the detected language (never "en" — matching Python, where detection only names non-English languages), so an override applies exactly to the requests it was fitted on.

Example (repo root)

node laya-ts/examples/try-ml.mjs   # needs ./model-ml from the export step
node laya-ts/examples/snake.mjs --ticks 50   # autonomous snake demo, headless smoke (live TUI without --ticks)

Packaging

ponytail: CJS/browser-field dual build + tsconfig tests-include deferred — Task 7 verified ESM-only; CJS needs second tsc config + export-map change, untested. Add when a CJS consumer or browser-field swap is requested.