Count a form's separators in one pass

A url-encoded form is refused when it carries 10000 or more '&', so
more than 10000 pieces whether fields or empty, by one bytes.Count
before anything is split. Counting only non-empty pieces split the
body one bytes.Cut per separator, so 4 KiB of gzip that decoded to
4 MiB of '&' cost about 60 ms of CPU before fasthttp parsed it again.
This commit is contained in:
2026-10-09 12:35:59 -07:00
parent 1dc336e5e2
commit cba7649b6b
3 changed files with 22 additions and 23 deletions
+1 -1
View File
@@ -73,7 +73,7 @@ Patch tags, one above the highest `v*` on GitHub. Push the commit to main, read
- The `/docs` page loads Swagger UI from cdn.jsdelivr.net. Embedding swagger-ui-dist 5 would add about 1.74 MB (bundle 1,552,209 bytes, CSS 185,784 bytes, Apache-2.0) to every binary that links zip.
- A manifest op may say `"stream": "sse"`, `"bytes"` or `"socket"` (the answer is written as produced, or the connection upgrades; `ProjectOpenAPI` publishes `text/event-stream` with `x-events`, binary media, or `101` with `x-socket`) and `"raw": true` (the request body is bytes, bound to no type; the Rust front end's). A stream beside an `out` means either. `Check` refuses a socket beside an `out` and a raw body beside an `in`.
- Every wire is a typed op. A request body that is not JSON is a `zip.Body` field (bytes and Content-Type as sent), a `Parser` In (decodes itself; the document keeps In's schema), or a form: `*zip.File` parts make multipart, and `form:` fields bind only when the op `Consumes` a form media type (otherwise the tags are another binder's and JSON binds as before). A `cookie:` field is an `in: cookie` parameter set by the request's cookie and nothing else: the body, an argument object and the URL never set it, and no schema lists it as a body field. An answer that is not one JSON value is a `Body` (bytes, filename, its own status and headers; a `Reader` streams chunked; no bytes and no Type sends no Content-Type), `Verbatim[T]` (a relay's bytes documented as T; the upstream's 4xx/5xx pass undeclared), `Sse[T]` (`Send(ctx, emit)` on its own goroutine; ctx ends with the stream), `Socket[M]` (GET only; the handshake is decided before the handler runs: 501 over ZAP, 426 without an upgrade or for another version, 400 without a key, 403 for a browser origin the op does not admit — its own, plus `zip.Origins(...)`, `"*"` for any), `Redirect` or a `CookieCoder`; a type embedding `Body` or `Redirect` is that answer, so one with a `Cookies` method sets them beside it. A JSON answer keeps fiber's `application/json; charset=utf-8` unless the op `Produces` another. Every declared status carries the Out's schema, except one a union's alternative claims with its value-receiver `StatusCode()`. A union is a type whose `OneOf()` result types are its alternatives; `Or[A,B]` is the stock one. Over MCP bytes are base64 (in) and tool content (out); bytes, a Reader and a stream (as JSON lines) are bounded at `toolBound` and a Reader or stream at `toolWait`, and a socket op is no tool (`_meta.refused`). 204, 304 and a redirecting op's 3xx carry no body in the document; any other declared status carries the Out and its example. The call plane writes an `Sse` as the REST door does, refuses a `Socket` before its handler, and carries a union that holds a value as the union. A header field is a CLI flag under its field's name, as before; there are no header flags. An op that takes the body itself (a `Body` field, a form, a `Parser`) reads its content through `content`: each Content-Encoding (every header line) undone and held to BodyLimit, 415 for a coding it does not undo or a chain of more than three, 413 past the limit, 400 for one that does not decode, all before the handler; a multipart form is parsed in memory from that content, a url-encoded one of more than 10000 fields is 400 before any is decoded, and a form keeps only the keys In binds. Over HTTP fasthttp has read a multipart body as a form before routing, so a `Body` field holds that form re-encoded (what `c.Body()` gives a raw handler), not the bytes as sent; ZAP carries them as sent. A JSON op still reads fiber's `Body` as it always has. An answer refused after its handler ran closes its `Reader`. The graph (`/.well-known/graph`) and `Here` carry only plain ops (`registeredOp.plain`: one JSON value in, one out); a `cookie:` field is never a graph argument, and `Here` refuses a socket op before its handler.
- Every wire is a typed op. A request body that is not JSON is a `zip.Body` field (bytes and Content-Type as sent), a `Parser` In (decodes itself; the document keeps In's schema), or a form: `*zip.File` parts make multipart, and `form:` fields bind only when the op `Consumes` a form media type (otherwise the tags are another binder's and JSON binds as before). A `cookie:` field is an `in: cookie` parameter set by the request's cookie and nothing else: the body, an argument object and the URL never set it, and no schema lists it as a body field. An answer that is not one JSON value is a `Body` (bytes, filename, its own status and headers; a `Reader` streams chunked; no bytes and no Type sends no Content-Type), `Verbatim[T]` (a relay's bytes documented as T; the upstream's 4xx/5xx pass undeclared), `Sse[T]` (`Send(ctx, emit)` on its own goroutine; ctx ends with the stream), `Socket[M]` (GET only; the handshake is decided before the handler runs: 501 over ZAP, 426 without an upgrade or for another version, 400 without a key, 403 for a browser origin the op does not admit — its own, plus `zip.Origins(...)`, `"*"` for any), `Redirect` or a `CookieCoder`; a type embedding `Body` or `Redirect` is that answer, so one with a `Cookies` method sets them beside it. A JSON answer keeps fiber's `application/json; charset=utf-8` unless the op `Produces` another. Every declared status carries the Out's schema, except one a union's alternative claims with its value-receiver `StatusCode()`. A union is a type whose `OneOf()` result types are its alternatives; `Or[A,B]` is the stock one. Over MCP bytes are base64 (in) and tool content (out); bytes, a Reader and a stream (as JSON lines) are bounded at `toolBound` and a Reader or stream at `toolWait`, and a socket op is no tool (`_meta.refused`). 204, 304 and a redirecting op's 3xx carry no body in the document; any other declared status carries the Out and its example. The call plane writes an `Sse` as the REST door does, refuses a `Socket` before its handler, and carries a union that holds a value as the union. A header field is a CLI flag under its field's name, as before; there are no header flags. An op that takes the body itself (a `Body` field, a form, a `Parser`) reads its content through `content`: each Content-Encoding (every header line) undone and held to BodyLimit, 415 for a coding it does not undo or a chain of more than three, 413 past the limit, 400 for one that does not decode, all before the handler; a multipart form is parsed in memory from that content, a url-encoded one of 10000 or more `&` (more than 10000 pieces, fields or empty) is 400 before any is decoded, and a form keeps only the keys In binds. Over HTTP fasthttp has read a multipart body as a form before routing, so a `Body` field holds that form re-encoded (what `c.Body()` gives a raw handler), not the bytes as sent; ZAP carries them as sent. A JSON op still reads fiber's `Body` as it always has. An answer refused after its handler ran closes its `Reader`. The graph (`/.well-known/graph`) and `Here` carry only plain ops (`registeredOp.plain`: one JSON value in, one out); a `cookie:` field is never a graph argument, and `Here` refuses a socket op before its handler.
## Rust
+8 -21
View File
@@ -695,38 +695,25 @@ func undo(coding string, b []byte, limit int) ([]byte, error) {
return out, nil
}
// maxFormFields is the most fields a url-encoded form may carry: the bound Go's
// own query parser holds (net/url, urlmaxqueryparams). A multipart form is held
// to mime/multipart's own bounds on parts and part headers.
// maxFormFields is the most pieces a url-encoded form may carry between its
// '&'s, a field or empty: the bound Go's own query parser holds (net/url,
// urlmaxqueryparams). Counting separators rather than fields keeps the check
// one pass over the bytes, so a form of nothing but '&' costs no more to refuse
// than any other. A multipart form is held to mime/multipart's own bounds on
// parts and part headers.
const maxFormFields = 10000
// fewFields reports whether a url-encoded form holds at most maxFormFields
// fields, counting the non-empty pieces between its '&'s without decoding any.
func fewFields(form []byte) bool {
n := 0
for len(form) > 0 {
var field []byte
field, form, _ = bytes.Cut(form, []byte("&"))
if len(field) > 0 {
if n++; n > maxFormFields {
return false
}
}
}
return true
}
// readForm reads a REST request's form from its content: url-encoded fields,
// or a multipart form's fields and parts, each part read whole, keeping only
// the keys the op binds (names). The content is already decoded and held to the
// body limit, and a multipart form is parsed from it in memory, so no part is
// larger than the body that carried it. A url-encoded form is decoded as fiber
// decodes one, and one of more than [maxFormFields] fields is refused before
// decodes one, and one of more than [maxFormFields] pieces is refused before
// any is decoded.
func readForm(c fiber.Ctx, media string, body []byte, names map[string]bool) (formData, error) {
fd := formData{values: map[string][]string{}, files: map[string][]File{}}
if m, _, _ := mime.ParseMediaType(media); m != mimeMultipart {
if !fewFields(body) {
if bytes.Count(body, []byte("&")) >= maxFormFields {
return fd, fmt.Errorf("a form carries more than %d fields", maxFormFields)
}
var args fasthttp.Args
+13 -1
View File
@@ -294,7 +294,7 @@ type sessionOut struct {
}
// A url-encoded form binds the fields the op declares and no others, and one
// of more fields than Go's own query parser takes is refused before any is
// of more pieces than Go's own query parser takes is refused before any is
// decoded or the handler runs. The refusal is cheap: a few kilobytes that
// decode to millions of fields cost the decode and nothing per field.
func TestContent_AFormKeepsItsFieldsAndIsBounded(t *testing.T) {
@@ -333,6 +333,18 @@ func TestContent_AFormKeepsItsFieldsAndIsBounded(t *testing.T) {
if alloc := after.TotalAlloc - before.TotalAlloc; alloc > 64<<20 {
t.Errorf("refusing a %d-byte flood allocated %d MiB; it should cost the decode, not a value per field", len(flood), alloc>>20)
}
// Empty pieces count: a form of nothing but separators is refused in the
// same one pass, not split piece by piece.
ran.Store(0)
seps := gzipped(t, bytes.Repeat([]byte("&"), 4<<20-16))
if code, body = send(t, big, "/v1/form", form, "gzip", seps); code != 400 || ran.Load() != 0 {
t.Errorf("a %d-byte flood of separators: %d %s, want 400 and no run", len(seps), code, body)
}
padded := strings.Repeat("a=1&&", 5000) + "note=padded"
if code, body = send(t, big, "/v1/form", form, "", []byte(padded)); code != 400 {
t.Errorf("5001 fields padded past 10000 pieces: %d %s, want 400", code, body)
}
}
// Every Content-Encoding line is a coding, as one comma-separated line is, and