mirror of
https://github.com/hanzoai/laya.git
synced 2026-10-30 19:33:31 +00:00
`laya/serve.py` answers 413 to a request carrying more than MAX_QUESTIONS
(64) questions or a state over MAX_STATE_CHARS (50 000), because Laya
encodes the state once per question -- cost is questions x state size,
collated into one tensor.
`examples/server.py` enforced neither. The asymmetry sat inside one
class: `states` was bounded to 64, `questions` beside it was not, and the
state was only checked for emptiness.
states: Field(..., min_length=1, max_length=64)
questions: Field(..., min_length=1) # no upper bound
Same payloads against the two surfaces:
examples/server.py laya/serve.py
20 000 questions accepted 413
5 MB state accepted 413
README points at this server as the way to try Laya without writing code,
and its module docstring offers `--host 0.0.0.0` as a usage example. The
default bind is loopback, which is what keeps this moderate.
Worth noting for #335: its Agent-level MAX_TOTAL_OPTIONS (512) does not
cover this. The total sums only `choice` and `score` criteria, so a
`noul` question contributes zero options -- 20 000 noul questions pass
that budget while still producing 20 000 sequences. The count is what has
to be bounded, and the test covers that case.
Checked in the handler rather than declared as `Field(max_length=...)`,
for two reasons. A Field violation is reported 422 where laya.serve
answers 413, so a client that keys on 413 to shrink and retry would not
retry. And FastAPI's validation-error body includes the offending
`input`: rejecting a 5 MB state that way returns a 5 MB response, turning
a size limit into an amplifier. Measured on the handler-side version, a
rejected 5 MB state answers 413 in 52 bytes, and the test asserts the
response stays under 1 kB.
The bounds are read from `laya.serve` rather than restated so the demo
cannot drift, via `getattr` with the 0.3.10 defaults so `python
examples/server.py` still runs against an older installed laya. State
length is measured exactly as `_check_request_limits` measures it --
`len(v)` for a string, `len(str(v))` for a dict or list.
Deliberately out of scope: `Question.instructions` and `criteria` carry
unbounded text that no per-field bound can see, and capping the request
body is the backstop for those. A body cap here cannot simply reuse
laya.serve's 2 MiB, because /predict/batch legitimately accepts 64 states
of MAX_STATE_CHARS each, about 3.2 MB. That needs its own discussion
rather than a number picked in this patch.
ci.yml installs the `serve` extra. Without it fastapi is absent, so this
test -- and `tests/test_serve.py` and `tests/test_server_example.py`,
neither of which any workflow currently runs -- skip themselves silently
and pass. #376 makes that same install change while fixing #374; if it
lands first, drop that hunk from here and keep only the added test line,
which is appended at the end of the suite list rather than inserted
mid-list, where it is least likely to collide with another branch.
tests/test_example_server_limits.py: 19 assertions over HTTP, no weights
loaded. Seven fail against the current code, including the noul flood,
the oversized dict state and the no-echo assertion. The rest are the
boundary guards -- exactly 64 questions, a state of exactly 50 000
characters, a list state, a 64-state batch, and the four pre-existing
422s, which stay 422.
Checks: ruff and compileall clean; test_hooks_api 115; test_serve 22;
test_server_example 23; the rest of the suite unchanged against main.