mirror of
https://github.com/hanzoai/laya.git
synced 2026-10-30 19:33:31 +00:00
* fix(router): score reordered-but-equal question schemas separately Grouping by dict equality can merge two requests whose questions are equal but keyed in a different insertion order; the group then scores the second caller with the first caller's option order. Options are positional in render_options, so that request's batched answers would not match its own single-request answer -- the same failure mode as serialising questions to JSON with sort_keys=True and reading them back. Group by an order-sensitive schema signature instead (json.dumps with sort_keys=False and default=str, matching render_criterion's tolerance), with a deterministic fake-agent regression test. * bench: consistency harness + benchmarks for the merged predict_batch Point the harness at the merged Router.predict_batch (#138): * tests/test_predict_batch.py -- tiny-checkpoint tests (CPU): agent-level batch==single, router-level mixed-language routing/order preservation, and the reordered-schema regression at the router layer * benchmarks/bench_predict_batch.py -- --router now speaks the requests-dict API and adds router_consistency(): predict_batch vs one predict per request on 50/50 mixed EN/Hindi, the merge bar on the workload that regroups --consistency also writes its JSON now * plot_results.py renders bench-rtx4070.png from the three result files RTX 4070, laya 0.3.11, torch 2.6.0+cu124: agent layer 1.5-1.8x (<=64 states, 4 questions), router layer up to 3.4x (32 requests, 93 -> 313 states/s); consistency 1800 agent answers + 600 router answers, 0 flips, max|dp| 2.2%.