Files
zeekay 999e2153b3 Type every wire: bytes, forms, streams, sockets, redirects, unions
A typed op could only take one JSON value and answer one, so uploads,
signed webhooks, form posts, YAML and merge-patch bodies, PDFs, SSE,
WebSockets, OAuth redirects and upstream relays stayed untyped routes
with no schema, tool, command or SDK method. Each is now a typed op and
its REST wire is unchanged.

Requests: a zip.Body field takes the body and Content-Type exactly as
sent; an In implementing Parser decodes itself; form: fields and
*zip.File parts bind url-encoded and multipart forms; cookie: binds a
request cookie as an in: cookie parameter. Consumes names the media.

Answers: zip.Body (bytes, filename, its own status and headers, a
Reader streams chunked), Verbatim[T] (a relay documented as T), Sse[T]
(events with keep-alive), Socket[M] (an upgrade after the contract
runs; 426 without one, 501 over ZAP), Redirect, CookieCoder, Produces,
and unions: a type whose OneOf method's results are the alternatives,
with Or[A,B] as the stock one filing each under the status it states.

Projections follow from the manifest: the document publishes binary
media, form schemas, x-events, x-socket with 101, 3xx with Location
and oneOf; MCP takes bytes as base64 and returns bytes, text or
collected events as tool content, listing a socket op in _meta as no
tool; the CLI reads --body @file|-, --field name=@file and --out,
prints events as they arrive and bridges a socket to stdin and stdout.
tools/list is ProjectMCP over the manifest, the projection MCPTools
already was.

Prose reaches promoted fields (filed under the declaring struct), a
type defined over another struct, and the type arguments of zip's
kinds and OneOf results; an instantiated generic is published under
its name followed by its argument names instead of an illegal
component key. The unused reflect-only document and tool builders go.
2026-10-09 09:42:00 -07:00

886 lines
26 KiB
Go

// Package zapenc encodes a Go value as a ZAP message and reads one back.
//
// It is what makes the op-call plane ZAP end to end. An op's In and Out are
// ordinary Go structs; this derives their wire layout from the type itself, so
// a service-to-service call carries the same bytes in memory that it puts on
// the socket, with no second encoding in the middle. JSON stays where it
// belongs — at the boundary, for browsers — and never inside the binary
// protocol.
//
// # The layout is the type
//
// Fields take slots in declaration order, each aligned to its own width, which
// is the rule zap's own schema builder applies. Nothing is named on the wire:
// a field IS its offset, exactly as a hand-written ZAP codec would spell it,
// and the type declaration is the schema both ends read. That is also the whole
// compatibility rule — REORDERING, INSERTING OR RETYPING A FIELD CHANGES THE
// WIRE. Append at the end, and only at the end.
//
// # What crosses
//
// bool, every sized int/uint, float32/64, string, []byte, nested structs,
// pointers to any of those, and slices. A slice writes each element
// length-prefixed, which is the variable-element list layout zap.List.BytesAt
// and ObjectAt already read; a slice of structs therefore carries one complete
// ZAP message per element.
//
// # A pointer is present or absent, whatever it points at
//
// A pointer to a scalar, a string or a []byte crosses as a list of at most one
// element: nil is the empty list, anything else is that one element. A scalar
// slot has no null — the zero value and an absent value are the same bytes — so
// a pointer written into one came back nil whenever it pointed at zero. enforce
// pointing at false arrived unset, and a cap the caller made soft was created
// hard; a spend of 0 arrived as "unknown". A list has its own empty, so the
// element is there or it is not.
//
// A pointer to a struct needs none of this: its slot holds a complete message,
// which is never empty, so present is "the slot is not null".
//
// Anything else — a map, an interface, a channel, a FIXED-SIZE ARRAY — is
// refused at encode rather than dropped. A field that silently does not cross is
// the failure this package exists to make impossible. A field that says so
// itself, with the tag zap:"-", is the exception: it does not cross, and it was
// never meant to.
//
// # A type that owns its wire is not reflected over
//
// Deriving the layout is cached; COPYING THE FIELDS IS NOT. Every call runs
// reflect.ValueOf, sizes a builder by guess (lay.size+256) and walks the fields
// again, so the cost is paid per call for a fact that is fixed at compile time.
//
// So a type may state its own wire form — [zip.Wire], MarshalZAP/UnmarshalZAP —
// and both entry points below take that answer BEFORE they reach for reflect.
// Generated code emits those two methods with the offsets as constants, which is
// what a hand-written ZAP codec is, so the derivation moves to build time and the
// call path holds none of it.
//
// It is also the only way to carry what the derivation cannot express. A
// bytes_fixed[N] field — an ids.ID is [32]byte — is refused here and written
// inline by generated code through zap.Object.BytesFixed, so a round trip that
// SUCCEEDS for such a type is proof the reflective path was not taken.
package zapenc
import (
"encoding/binary"
"fmt"
"math"
"reflect"
"strconv"
"strings"
"sync"
zap "github.com/zap-proto/go"
)
// wire is [zip.Wire] restated where it is consumed, so this package depends on
// nothing to recognise it. Both methods are required together: a type that
// carried only one would encode from generated offsets and decode by reflection,
// which is two answers to where its layout lives.
type wire interface {
MarshalZAP() ([]byte, error)
UnmarshalZAP([]byte) error
}
// Marshal encodes v as a ZAP message. v must be a struct or a pointer to one:
// the message's root is an object, and there is nothing for a bare scalar to be
// the root of.
//
// A v that states its own wire form answers for itself and nothing here reflects.
func Marshal(v any) ([]byte, error) {
if w, ok := v.(wire); ok {
return w.MarshalZAP()
}
rv := reflect.ValueOf(v)
for rv.Kind() == reflect.Pointer {
if rv.IsNil() {
return nil, nil // a nil input is an absent body, not an empty object
}
rv = rv.Elem()
}
if rv.Kind() != reflect.Struct {
return nil, fmt.Errorf("zapenc: %s is not a struct", rv.Type())
}
lay, err := layoutOf(rv.Type())
if err != nil {
return nil, err
}
b := zap.NewBuilder(lay.size + 256)
if err := writeStruct(b, lay, rv, true); err != nil {
return nil, err
}
return b.Finish(), nil
}
// Unmarshal reads a ZAP message into v, which must be a non-nil pointer to a
// struct. An empty message leaves v untouched — a void reply is an absence, not
// a zero value that a caller might mistake for an answer.
func Unmarshal(data []byte, v any) error {
if len(data) == 0 {
return nil
}
if w, ok := v.(wire); ok {
return w.UnmarshalZAP(data)
}
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Pointer || rv.IsNil() {
return fmt.Errorf("zapenc: Unmarshal needs a non-nil pointer, got %T", v)
}
rv = rv.Elem()
if rv.Kind() != reflect.Struct {
return fmt.Errorf("zapenc: %s is not a struct", rv.Type())
}
msg, err := zap.Parse(data)
if err != nil {
return fmt.Errorf("zapenc: %w", err)
}
lay, err := layoutOf(rv.Type())
if err != nil {
return err
}
return readStruct(msg.Root(), lay, rv)
}
// ---- layout ---------------------------------------------------------------
type kind uint8
const (
kBool kind = iota
kInt8
kInt16
kInt32
kInt64
kUint8
kUint16
kUint32
kUint64
kFloat32
kFloat64
kText
kBytes
kStruct
kSlice
// kFixed is bytes_fixed[N] — N bytes written INLINE, aligned to 1. It has a
// layout and no reflective codec: see [Layout] and writeStruct's refusal.
kFixed
// kOpt is a pointer to a scalar, a string or a []byte: a list of at most one
// element, so absent and zero are different bytes. slice says how the one
// element is encoded.
kOpt
)
type field struct {
index int
offset int
kind kind
n int // kFixed: the array length
ptr bool // the Go field is a pointer to the encoded type
elem *layout // kStruct: the nested layout
slice *sliceElem // kSlice, kOpt: how one element is encoded
typ reflect.Type // the Go type at this field, minus any pointer
}
// width is the field's slot in the fixed area. Everything variable — text,
// bytes, a list, a nested value — carries an offset and a length in 8 bytes;
// bytes_fixed[N] is its N bytes, inline.
func (f field) width() int {
if f.kind == kFixed {
return f.n
}
return width(f.kind)
}
// align is where the slot may begin. It is the slot's own width, except for
// bytes_fixed: those bytes are inline and align to 1, so a 32-byte id follows a
// u32 at offset 4 rather than at 32 — which is what a .zap schema writes.
func (f field) align() int {
if f.kind == kFixed {
return 1
}
return width(f.kind)
}
type sliceElem struct {
kind kind
elem *layout // when the element is a struct
typ reflect.Type // the element type, minus any pointer
ptr bool
// n is kFixed's array length. A field carries it and an element has to
// carry it too: without it the schema states bytes_fixed[0], which is a
// width nothing writes and a generated codec would read zero bytes for.
n int
}
type layout struct {
fields []field
size int
}
var layouts sync.Map // reflect.Type -> *layout
// derive guards the COLD path: the set of types whose layout is being computed,
// and the lock that lets several goroutines reach it.
//
// A type that contains itself — directly, or through a slice or a pointer —
// asks for its own layout while computing it, and recurses until the stack
// ends. Four fleet apps did exactly that, and a stack overflow is a poor way to
// learn a type has no wire form.
//
// The answer is not a deeper stack. A ZAP field IS an offset and a width, so a
// type containing itself has no bounded width and no layout exists to derive.
// It is refused with the type named, which is what this package already does
// with a map or an interface.
//
// The lock is only ever taken when the answer is not cached, so a served call
// does not reach it: layouts.Load answers first, and every type is derived
// once.
var derive struct {
sync.Mutex
building map[reflect.Type]bool
}
func layoutOf(t reflect.Type) (*layout, error) {
if v, ok := layouts.Load(t); ok {
return v.(*layout), nil
}
derive.Lock()
defer derive.Unlock()
return layoutLocked(t)
}
// layoutLocked derives one layout with derive held. buildLayout re-enters
// through fieldOf, which is where the cycle is caught.
func layoutLocked(t reflect.Type) (*layout, error) {
if v, ok := layouts.Load(t); ok {
return v.(*layout), nil
}
if derive.building[t] {
return nil, fmt.Errorf("zapenc: %s contains itself; a field is an offset and a width, so a recursive type has no wire form", t)
}
if derive.building == nil {
derive.building = map[reflect.Type]bool{}
}
derive.building[t] = true
lay, err := buildLayout(t)
delete(derive.building, t)
if err != nil {
return nil, err
}
layouts.Store(t, lay)
return lay, nil
}
func buildLayout(t reflect.Type) (*layout, error) {
lay := &layout{}
off := 0
for i := 0; i < t.NumField(); i++ {
sf := t.Field(i)
if !sf.IsExported() || Skipped(sf) {
// An unexported field cannot be read or written by reflection, so it
// cannot cross. Skipping it silently is right — it was never part of
// the contract — but it must not consume a slot either. A field
// tagged zap:"-" says the same about itself: it is part of the value
// in this process and not of the message.
continue
}
f, err := fieldOf(sf.Type)
if err != nil {
return nil, fmt.Errorf("%s.%s: %w", t.Name(), sf.Name, err)
}
f.index = i
off = align(off, f.align())
f.offset = off
off += f.width()
lay.fields = append(lay.fields, f)
}
lay.size = align(off, 8)
return lay, nil
}
func fieldOf(t reflect.Type) (field, error) {
f := field{}
if t.Kind() == reflect.Pointer {
f.ptr = true
t = t.Elem()
}
f.typ = t
switch t.Kind() {
case reflect.Bool:
f.kind = kBool
case reflect.Int8:
f.kind = kInt8
case reflect.Int16:
f.kind = kInt16
case reflect.Int32:
f.kind = kInt32
case reflect.Int, reflect.Int64:
f.kind = kInt64
case reflect.Uint8:
f.kind = kUint8
case reflect.Uint16:
f.kind = kUint16
case reflect.Uint32:
f.kind = kUint32
case reflect.Uint, reflect.Uint64:
f.kind = kUint64
case reflect.Float32:
f.kind = kFloat32
case reflect.Float64:
f.kind = kFloat64
case reflect.String:
f.kind = kText
case reflect.Struct:
f.kind = kStruct
lay, err := layoutLocked(t)
if err != nil {
return f, err
}
f.elem = lay
case reflect.Array:
// bytes_fixed[N]. Only a byte array: an array of anything else has no
// inline wire form, and a list is how a sequence crosses.
if t.Elem().Kind() != reflect.Uint8 {
return f, fmt.Errorf("zapenc: an array of %s has no wire form; use a slice", t.Elem())
}
f.kind, f.n = kFixed, t.Len()
case reflect.Slice:
if t.Elem().Kind() == reflect.Uint8 {
f.kind = kBytes
break
}
f.kind = kSlice
se, err := sliceElemOf(t.Elem())
if err != nil {
return f, err
}
f.slice = se
default:
return f, fmt.Errorf("zapenc: %s cannot cross the plane; give it a type that can", t.Kind())
}
if f.ptr && f.kind != kStruct && f.kind != kSlice && f.kind != kFixed {
f.slice = &sliceElem{kind: f.kind, typ: t}
f.kind = kOpt
}
return f, nil
}
func sliceElemOf(t reflect.Type) (*sliceElem, error) {
se := &sliceElem{}
if t.Kind() == reflect.Pointer {
se.ptr = true
t = t.Elem()
}
se.typ = t
f, err := fieldOf(t)
if err != nil {
return nil, err
}
se.kind = f.kind
se.elem = f.elem
se.n = f.n
if se.kind == kSlice {
return nil, fmt.Errorf("zapenc: a slice of slices has no wire form; wrap the inner one in a struct")
}
return se, nil
}
// width is the fixed-area size of one field. Everything variable — text, bytes,
// slices and nested structs — carries an offset and a length.
func width(k kind) int {
switch k {
case kBool, kInt8, kUint8:
return 1
case kInt16, kUint16:
return 2
case kInt32, kUint32, kFloat32:
return 4
case kInt64, kUint64, kFloat64:
return 8
case kText, kBytes, kSlice, kStruct, kOpt:
return 8
}
return 8
}
func align(off, n int) int { return (off + n - 1) &^ (n - 1) }
// ---- encode ---------------------------------------------------------------
func writeStruct(b *zap.Builder, lay *layout, rv reflect.Value, asRoot bool) error {
// Nested values and list elements must be written BEFORE the object that
// points at them, because SetObject/SetList take an offset that already
// exists. Collect them first, then fill the fixed area.
type pending struct {
f field
list int
length int
}
var pends []pending
for _, f := range lay.fields {
fv := rv.Field(f.index)
if f.ptr {
if fv.IsNil() {
continue
}
fv = fv.Elem()
}
switch f.kind {
case kSlice:
if fv.Len() == 0 {
continue
}
off, n, err := writeList(b, f.slice, fv)
if err != nil {
return err
}
pends = append(pends, pending{f: f, list: off, length: n})
case kOpt:
enc, err := marshalElem(f.slice, fv)
if err != nil {
return err
}
blob := binary.LittleEndian.AppendUint32(nil, uint32(len(enc)))
pends = append(pends, pending{f: f, list: b.WriteBytes(append(blob, enc...)), length: 1})
}
}
ob := b.StartObject(lay.size)
for _, f := range lay.fields {
fv := rv.Field(f.index)
if f.ptr {
if fv.IsNil() {
continue // an absent pointer writes nothing: a null, not a zero
}
fv = fv.Elem()
}
switch f.kind {
case kBool:
ob.SetBool(f.offset, fv.Bool())
case kInt8:
ob.SetInt8(f.offset, int8(fv.Int()))
case kInt16:
ob.SetInt16(f.offset, int16(fv.Int()))
case kInt32:
ob.SetInt32(f.offset, int32(fv.Int()))
case kInt64:
ob.SetInt64(f.offset, fv.Int())
case kUint8:
ob.SetUint8(f.offset, uint8(fv.Uint()))
case kUint16:
ob.SetUint16(f.offset, uint16(fv.Uint()))
case kUint32:
ob.SetUint32(f.offset, uint32(fv.Uint()))
case kUint64:
ob.SetUint64(f.offset, fv.Uint())
case kFloat32:
ob.SetFloat32(f.offset, float32(fv.Float()))
case kFloat64:
ob.SetFloat64(f.offset, fv.Float())
case kText:
ob.SetText(f.offset, fv.String())
case kBytes:
ob.SetBytes(f.offset, fv.Bytes())
case kStruct:
// A nested value is a complete ZAP message stored as bytes, so a reader
// Parses it on its own — the same rule list elements follow, rather
// than a second one for the singular case.
inner, err := marshalStruct(f.elem, fv)
if err != nil {
return err
}
ob.SetBytes(f.offset, inner)
case kFixed:
return fixedRefusal(fv.Type())
}
}
for _, p := range pends {
ob.SetList(p.f.offset, p.list, p.length)
}
if asRoot {
ob.FinishAsRoot()
} else {
ob.Finish()
}
return nil
}
// writeList lays out a variable-element list: each element length-prefixed with
// a little-endian uint32, which is the layout zap.List.BytesAt and ObjectAt
// read. The returned length is the ELEMENT COUNT, never a byte count.
func writeList(b *zap.Builder, se *sliceElem, rv reflect.Value) (int, int, error) {
var blob []byte
for i := 0; i < rv.Len(); i++ {
ev := rv.Index(i)
if se.ptr {
if ev.IsNil() {
ev = reflect.Zero(se.typ)
} else {
ev = ev.Elem()
}
}
enc, err := marshalElem(se, ev)
if err != nil {
return 0, 0, err
}
var n [4]byte
binary.LittleEndian.PutUint32(n[:], uint32(len(enc)))
blob = append(blob, n[:]...)
blob = append(blob, enc...)
}
return b.WriteBytes(blob), rv.Len(), nil
}
// marshalElem encodes one list element. A struct element is a complete ZAP
// message; a scalar element is its own little-endian bytes, and text is its
// UTF-8 — so an element is always self-describing given the field's type.
func marshalElem(se *sliceElem, ev reflect.Value) ([]byte, error) {
switch se.kind {
case kStruct:
return marshalStruct(se.elem, ev)
case kText:
return []byte(ev.String()), nil
case kBytes:
return ev.Bytes(), nil
case kBool:
if ev.Bool() {
return []byte{1}, nil
}
return []byte{0}, nil
case kInt8, kInt16, kInt32, kInt64:
return leInt(uint64(ev.Int()), width(se.kind)), nil
case kUint8, kUint16, kUint32, kUint64:
return leInt(ev.Uint(), width(se.kind)), nil
case kFloat32:
return leInt(uint64(f32bits(float32(ev.Float()))), 4), nil
case kFloat64:
return leInt(f64bits(ev.Float()), 8), nil
case kFixed:
return nil, fixedRefusal(se.typ)
}
return nil, fmt.Errorf("zapenc: list element of kind %d has no wire form", se.kind)
}
func marshalStruct(lay *layout, rv reflect.Value) ([]byte, error) {
b := zap.NewBuilder(lay.size + 128)
if err := writeStruct(b, lay, rv, true); err != nil {
return nil, err
}
return b.Finish(), nil
}
func leInt(v uint64, n int) []byte {
var buf [8]byte
binary.LittleEndian.PutUint64(buf[:], v)
return append([]byte(nil), buf[:n]...)
}
// ---- decode ---------------------------------------------------------------
func readStruct(o zap.Object, lay *layout, rv reflect.Value) error {
if o.IsNull() {
return nil // nothing to read; every accessor would dereference a nil message
}
for _, f := range lay.fields {
fv := rv.Field(f.index)
if f.ptr {
// Allocated only when the wire says the value is THERE — the one
// element of an optional, a nested message — never when it is merely
// non-zero: a pointer to false is a pointer, and nil is not false.
if err := readPointer(o, f, fv); err != nil {
return err
}
continue
}
if err := readField(o, f, fv); err != nil {
return err
}
}
return nil
}
func readPointer(o zap.Object, f field, fv reflect.Value) error {
nv := reflect.New(f.typ)
switch f.kind {
case kOpt:
l := o.List(f.offset)
if l.Len() == 0 {
return nil
}
if err := readElem(l.BytesAt(0), f.slice, nv.Elem()); err != nil {
return err
}
case kStruct:
if len(o.Bytes(f.offset)) == 0 {
return nil
}
if err := readField(o, f, nv.Elem()); err != nil {
return err
}
default:
// A pointer to a list: an empty list and an absent one are both null.
if err := readField(o, f, nv.Elem()); err != nil {
return err
}
if nv.Elem().IsZero() {
return nil
}
}
fv.Set(nv)
return nil
}
func readField(o zap.Object, f field, target reflect.Value) error {
switch f.kind {
case kBool:
target.SetBool(o.Bool(f.offset))
case kInt8:
target.SetInt(int64(o.Int8(f.offset)))
case kInt16:
target.SetInt(int64(o.Int16(f.offset)))
case kInt32:
target.SetInt(int64(o.Int32(f.offset)))
case kInt64:
target.SetInt(o.Int64(f.offset))
case kUint8:
target.SetUint(uint64(o.Uint8(f.offset)))
case kUint16:
target.SetUint(uint64(o.Uint16(f.offset)))
case kUint32:
target.SetUint(uint64(o.Uint32(f.offset)))
case kUint64:
target.SetUint(o.Uint64(f.offset))
case kFloat32:
target.SetFloat(float64(o.Float32(f.offset)))
case kFloat64:
target.SetFloat(o.Float64(f.offset))
case kText:
// zap reads text zero-copy, over the frame that arrived, and the caller's
// transport reuses that frame once the call returns. The decoded value
// outlives the call, so it is copied here, as the generated codecs copy.
target.SetString(strings.Clone(o.Text(f.offset)))
case kBytes:
if b := o.Bytes(f.offset); len(b) > 0 {
target.SetBytes(append([]byte(nil), b...))
}
case kStruct:
raw := o.Bytes(f.offset)
if len(raw) == 0 {
return nil // absent nested value: leave the zero struct
}
msg, err := zap.Parse(raw)
if err != nil {
return fmt.Errorf("zapenc: nested: %w", err)
}
return readStruct(msg.Root(), f.elem, target)
case kSlice:
return readList(o, f, target)
case kFixed:
return fixedRefusal(f.typ)
}
return nil
}
// fixedRefusal is what the reflective path answers for bytes_fixed[N].
//
// The LAYOUT knows the shape — [Layout] gives it an offset and a width, which is
// what a schema and a code generator need — and the reflective codec deliberately
// does not carry it. Reflection is what this whole seam exists to leave, so making
// it more capable is the wrong direction: an id is exactly the case that should
// force a type to declare its own wire, and a type that has done so never reaches
// this line.
func fixedRefusal(t reflect.Type) error {
return fmt.Errorf("zapenc: %s is bytes_fixed[%d], which the reflective codec does not carry; "+
"declare the type's wire (MarshalZAP/UnmarshalZAP) and it crosses inline", t, t.Len())
}
// readList walks the list once. Element i by index re-walks the stream from its
// start, so decoding a list that way is quadratic in its length — a 160KB body
// was over a second of CPU. The slice grows from the elements that are actually
// there rather than being allocated from the count the message claims.
func readList(o zap.Object, f field, target reflect.Value) error {
l := o.List(f.offset)
if l.Len() == 0 {
return nil
}
out := reflect.MakeSlice(target.Type(), 0, 0)
var err error
l.EachBytes(func(_ int, raw []byte) bool {
ev := reflect.New(f.slice.typ)
if err = readElem(raw, f.slice, ev.Elem()); err != nil {
return false
}
if f.slice.ptr {
out = reflect.Append(out, ev)
} else {
out = reflect.Append(out, ev.Elem())
}
return true
})
if err != nil {
return err
}
target.Set(out)
return nil
}
// readElem decodes one element from its own bytes. A struct element is a
// complete message; one that does not parse is an error, never a nil object read
// as if it were one — a single malformed element used to take the process down.
func readElem(raw []byte, se *sliceElem, target reflect.Value) error {
if se.kind == kStruct {
if len(raw) == 0 {
return nil // an empty element is the zero value, as the generated codec reads it
}
msg, err := zap.Parse(raw)
if err != nil {
return fmt.Errorf("zapenc: list element: %w", err)
}
return readStruct(msg.Root(), se.elem, target)
}
switch se.kind {
case kText:
target.SetString(string(raw))
case kBytes:
target.SetBytes(append([]byte(nil), raw...))
case kBool:
target.SetBool(len(raw) > 0 && raw[0] != 0)
case kInt8, kInt16, kInt32, kInt64:
target.SetInt(signed(raw, width(se.kind)))
case kUint8, kUint16, kUint32, kUint64:
target.SetUint(unsigned(raw))
case kFloat32:
target.SetFloat(float64(f32from(uint32(unsigned(raw)))))
case kFloat64:
target.SetFloat(f64from(unsigned(raw)))
case kFixed:
return fixedRefusal(se.typ)
}
return nil
}
func unsigned(b []byte) uint64 {
var buf [8]byte
copy(buf[:], b)
return binary.LittleEndian.Uint64(buf[:])
}
func signed(b []byte, n int) int64 {
v := unsigned(b)
shift := uint(64 - 8*n)
return int64(v<<shift) >> shift // sign-extend from the field's own width
}
func f32bits(f float32) uint32 { return math.Float32bits(f) }
func f64bits(f float64) uint64 { return math.Float64bits(f) }
func f32from(u uint32) float32 { return math.Float32frombits(u) }
func f64from(u uint64) float64 { return math.Float64frombits(u) }
// Skipped reports whether a field opts out of the message with zap:"-": a value
// that lives only in the process holding it, such as a reader an answer
// streams from. It takes no slot and is not a loss.
func Skipped(f reflect.StructField) bool { return f.Tag.Get("zap") == "-" }
// ---- the layout, for whoever needs to write it down -------------------------
// Slot is where one field of a message sits on the wire, and what it is.
//
// It is what a .zap schema states as `Name type @Offset` and what a generated
// codec states as a Go constant. Both are projections of the SAME derivation the
// reflective codec encodes against, which is the point: the schema, the generated
// code and the running encoder cannot describe three wires.
type Slot struct {
Name string // the Go field name
Offset int
Width int
Type string // the .zap type: u64, text, bytes, bytes_fixed[32], list<…>, struct
N int // bytes_fixed[N]: the length. Otherwise 0.
Ptr bool // the Go field is a pointer; to a scalar, text or bytes it is list<T> of at most one
Elem string // list<…>: the element's .zap type
}
// Shape is a whole message: its slots in declaration order, and its size.
type Shape struct {
Slots []Slot
Size int
}
// LayoutOf derives the wire layout of a struct type.
//
// A field is aligned to its own width and takes that many bytes; the size is
// rounded up to 8. bytes_fixed[N] is the exception and the reason alignment is
// stated apart from width: it is N bytes INLINE, aligned to 1, so an id sits
// immediately after the u32 before it.
//
// This is the derivation Marshal encodes against, so a schema or a generator
// reading it describes the wire that is actually spoken. Deriving it a second
// time — packing the offsets, or giving a nested value four bytes instead of the
// eight it takes as bytes — produces a document about a wire nobody speaks.
func LayoutOf(t reflect.Type) (Shape, error) {
for t.Kind() == reflect.Pointer {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return Shape{}, fmt.Errorf("zapenc: %s is not a struct", t)
}
lay, err := layoutOf(t)
if err != nil {
return Shape{}, err
}
sh := Shape{Size: lay.size}
for _, f := range lay.fields {
s := Slot{
Name: t.Field(f.index).Name,
Offset: f.offset,
Width: f.width(),
Type: spell(f.kind, f.n),
N: f.n,
Ptr: f.ptr,
}
if (f.kind == kSlice || f.kind == kOpt) && f.slice != nil {
s.Elem = spell(f.slice.kind, f.slice.n)
s.Type = "list<" + s.Elem + ">"
}
sh.Slots = append(sh.Slots, s)
}
return sh, nil
}
// spell is the .zap name of a kind. A nested value is "bytes" and not "struct",
// because that is what it IS on this wire: writeStruct carries it through
// SetBytes as a complete ZAP message in an 8-byte {offset,length} slot, never
// through SetObject's 4-byte relative offset.
func spell(k kind, n int) string {
switch k {
case kBool:
return "bool"
case kInt8:
return "i8"
case kInt16:
return "i16"
case kInt32:
return "i32"
case kInt64:
return "i64"
case kUint8:
return "u8"
case kUint16:
return "u16"
case kUint32:
return "u32"
case kUint64:
return "u64"
case kFloat32:
return "f32"
case kFloat64:
return "f64"
case kText:
return "text"
case kBytes, kStruct:
return "bytes"
case kFixed:
return "bytes_fixed[" + strconv.Itoa(n) + "]"
}
return "bytes"
}