Reading a ZAP result writes every field its layout states, the file payload
included, so files attached before decoding were overwritten with none. A
frame whose header is not ZAP is refused at the magic rather than read as a
length.
Co-authored-by: Hanzo Dev <[email protected]>
encoding/json walked every payload on every call. ZAP reads a field at a
constant offset out of the buffer that arrived, and //tools/zap has
already stated those offsets for every payload, so the transport now asks
the payload for its own bytes and the parse step is gone.
The envelope collapses with it. clientCall and serverCall existed only
because JSON needed one type to hold an any and another to hold a
RawMessage; bytes are bytes, so one Call serves both ends. Framing comes
from the ZAP header, which declares its own length.
A payload that is nothing -- the *struct{} a method takes when it has no
argument or no result -- carries nothing. Anything else without a layout
fails the call naming the type, so a payload that would have arrived
silently empty does not arrive at all.
The methods that took a bare string or returned a bare int now take and
return named payloads, because a scalar has no layout to state and the
rest of the API was already shaped this way.
Co-authored-by: Hanzo Dev <[email protected]>
A prometheus Data held its labels in a map, and control.ExecArgs held the
live kernel objects an exec runs against -- a mount namespace, an fd
table, a limit set. Neither has offsets, so neither payload could state
a layout.
Labels become a list, which also retires the O(n^2) walk that existed to
read a map in sorted order. The kernel objects move to control.Env, which
is what the sentry supplies and what a call arriving on the wire cannot:
a namespace is a reference, not a value.
Every urpc payload now states its own wire.
Co-authored-by: Hanzo Dev <[email protected]>
urpc encodes with encoding/json, which walks the payload on every call: a
realistic control.ExecArgs costs 7084ns and 10 allocations to cross. ZAP
reads a field at a constant offset out of the buffer that arrived, so the
same payload is 740ns and one allocation -- but only if the offsets are
stated, and a stated offset that is wrong is silent corruption rather than
a compile error. So state them from the types themselves.
//tools/zap reflects over the payloads at build time and writes the wire
as Go source with the offsets as constants. It lives inside Bazel because
pkg/sentry/control cannot be imported from outside it: the *_go_proto
packages it depends on have no source in the tree.
Six payloads had no layout to state. A map, an interface and a type from
another module have no offsets, so each is answered in the type:
SaveOpts.Metadata and StartContainerArgs.InitialCgroups/Limits cross as
lists of pairs; boot.Route.Destination restates net.IPNet as Subnet;
OpenMountArgs restates the OCI mount fields the call actually uses;
StartArgs.Spec and SinkConfig.Config carry their documents as the bytes
they arrived as; MetricsRegistrationResponse carries the protocol
buffer the same way; and seccheck's session options become a type
instead of a map of any.
ExecArgs and MetricsExportData still have none: ExecArgs holds live
kernel objects behind an interface, and MetricsExportData reaches a
prometheus snapshot whose labels are a map.
Co-authored-by: Hanzo Dev <[email protected]>
A sandbox that answers "-gvisor" is upstream's binary. Telling the two apart
by config is a guess — the RuntimeClass, the containerd entry and the shim
options can all name our binary while the pod runs something else, which is
exactly what happened when an inline BinaryName was accepted by containerd and
never read by the shim. The guest's own uname is the one answer that cannot be
wrong about which kernel is running it.
Co-authored-by: Hanzo Dev <[email protected]>
createdState.State and runningState.State transitioned the shim's init
process to stoppedState whenever a single `runsc state` call answered
"stopped". That transition bypasses SetExited: waitBlock is never
closed, no TaskExit event is published, and handleStoppedKill silently
swallows all subsequent kills, so containerd waits forever and the
container can never be torn down.
This behaviour comes from commit 0b39816a3, which tried to fix the
opposite problem of containerd never seeing that a container died due
to query errors. However, this part was never essential, because the
real fixes had to do with state queries on dead or deleted containers
erroring out. In light of https://github.com/google/gvisor/pull/14645,
there is now a concrete example of a buggy runsc not being an
authoritative and irreversible source for the stopped state, so kills
can be accepted but dropped, and the pod was stuck terminating forever.
Fix a panic when writing to cgroup v2 control files from outside the sandbox
(e.g., via Cgroups.WriteControlFiles / WriteControl), where writes occur
without an open in-sandbox VFS file descriptor (fd == nil). cgroup.procs now
safely falls back to context credentials with unconfined namespace migration,
and checkNSDelegateWrite skips namespace delegation checks. Add an integration
test for runsc cgroup v2 control file writes.
Also:
- Export cgroup2fs.InitHierarchy to initialize sandbox-wide cgroup2 state
and root controller enablement.
- Plumb CgroupNamespace through sentry lifecycle and task context to derive
initial container cgroup2 placements.
PiperOrigin-RevId: 978087817
If we don't register the newly created sysmsg thread _before_ we start executing
contexts on it, we are liable to race with any interrupts that may arrive to
contexts it starts executing, upon which point we won't find the newly created
thread.
Reported-by: [email protected]
PiperOrigin-RevId: 977899577
Mark seccomp as partially supported on amd64 and arm64 and document the accepted operation and flags.
Fixes#14627
Co-authored-by: Hanzo Dev <[email protected]>
To kill the subprocess, we kill the syscall thread, which in turn kills all
stub threads by virtue of being their parent. This means we have two critical
sections that need to be protected:
1. Creating new sysmsg threads -- the function expects that it's new baby
will stay alive at least while it's being created;
2. Using the syscall thread itself.
We also kick out any currently sleeping task goroutines, and put guard-checks
on wait loops and possible entrances to subprocess functionality so that tasks
don't come back to a dead subprocess.
Additionally, killed subprocesses were not properly decref'ed before, this CL
also fixes that.
PiperOrigin-RevId: 977651945
OrderedChildren.Lookup returns the inode with a ref that the caller is
expected to release (normally by handing it to a dentry via Init). Five
cgroup2fs call sites that look an inode up for a transient purpose
(permission checks, control-file read/write, interface-file removal)
never released that ref.
This is NOT a bug fix: every inode these sites can return
(cgroupInterfaceFile and eventFile, both via kernfs.DynamicBytesFile,
and cgroup directories themselves) embeds kernfs.InodeNoopRefCount, so
IncRef/DecRef are no-ops and nothing leaks today. The change only makes
the ref discipline explicit and correct, so these sites stay safe if a
lookup ever returns a genuinely refcounted inode.
PiperOrigin-RevId: 976563179
SCMSender.beforeSave disconnects the endpoint on save: it shuts down and
closes the host fd, leaving fd = -1. Send already handles this state via
its IsSendClosed check (beforeSave sets sendShutdown), so writes fail
with EPIPE, but Recv had no guard and passed -1 to recvmsg(2), leaking
EBADF to applications for an fd that is still valid in their fd table.
Return ErrClosedForReceive instead, which the socket layer already
surfaces as read() == 0, matching Linux EOF semantics for a socket whose
read side is shut down and making a severed host endpoint behave like
any socket whose peer has gone away.
The guard keys on fd < 0 rather than IsRecvClosed: after a local SHUT_RD
on a live fd, the host socket may still hold queued data that must be
drained before EOF, and host recvmsg(2) handles that correctly.
PiperOrigin-RevId: 976063849
- In HostSender.HostReadiness, return hangup and readable/writeable events
((ReadableEvents | WriteableEvents | EventHUp | EventRdHUp) & mask) when
the host FD is closed (c.fd < 0), ensuring pollers (epoll/poll) are notified
of EOF and disconnection instead of masking the closed state.
- In SCMSender.beforeSave, return early if e.fd < 0 to make beforeSave
idempotent and prevent nil-pointer dereferences in fdnotifier.RemoveFD(-1)
on already-closed endpoints.
PiperOrigin-RevId: 975898094
renameat2(2) support was added in 61f761343f ("Implement renameat2."),
but only for the gofer filesystem; tmpfs and overlayfs still return
EINVAL for RENAME_EXCHANGE.
Implement RENAME_EXCHANGE in tmpfs by swapping the two dentries between
their parent directories. When the exchanged files differ in type, the
parents' link counts (for "..") and the exchanged directory's reference
on its parent are transferred.
Implement RENAME_EXCHANGE in overlayfs by copying up the destination
file (and its descendants, if it is a directory) in addition to the
source, and then exchanging both files on the upper layer. Unlike a
regular rename, no whiteout is created at the source location, since
both locations remain occupied. Exchanged directories are marked opaque
at their new locations to prevent merging with lower layer directories
there.
RENAME_EXCHANGE permits the two files to differ in type and does not
require a directory being exchanged to be empty, but exchanging a file
with an ancestor directory must fail with EINVAL; see Linux's
fs/namei.c:__start_renaming(). The gofer filesystem previously required
both files to have the same type; lift that restriction, and update
parent link counts when files of different types are exchanged across
directories.
The RENAME_EXCHANGE syscall tests no longer accept EINVAL; they are
skipped where the flag remains unsupported, i.e. on FUSE and on gVisor
kernels other than runsc. New tests cover mixed-type exchanges and
exchanges with ancestor directories.
Fixes#7895
github.com/google/gvisor/pull/14531 exposed a race window with stub threads
setting ThreadID and the sentry reading it. If the sentry attempted to interrupt
a stub that in the process of switching a context back, it could have read
an invalid thread ID along with a context state that indicated it was still
being worked on. This window has existed before #14531, but was relatively
benign; with that CL, the window became punishable by subprocess death.
The (mostly generated) test seems to reliably test the race-window in case
things get reordered in some future refactor.
Fixes#14531.
PiperOrigin-RevId: 974868738
The cpu controller tracks each task's CPU via a per-instance baseline
snapshot plus a cumulative total for exited tasks, but it never walked
the whole accounting domain when reporting cpu.stat and never reconciled
that state when a controller was disabled or when a task migrated within
a single controller's domain. This caused three accounting errors, each
covered by a new test:
- Live tasks in descendant cgroups that had not enabled their own cpu
controller were absent from the ancestor's cpu.stat until they
exited, even though Linux aggregates descendant usage into the
ancestor continuously.
- Migrating a task between two cgroups sharing the same nearest cpu
controller reset its baseline, discarding the pre-migration CPU.
- Disabling +cpu discarded the controller's accumulated usage instead
of reparenting it (Linux folds an offlined css's stats into parent).
Sum the whole domain (the controller's cgroup plus every descendant
without its own cpu controller, recursing into descendants that do) when
reading cpu.stat; leave the baseline untouched on same-instance
migration; and on disable, migrate the domain's tasks to the parent's
nearest cpu controller and fold the accumulated total into it.
PiperOrigin-RevId: 974821312
Enabling +pids in cgroup.subtree_control creates child pids controllers
with committed = 0, ignoring tasks already present in those cgroups.
pids.current then underreports, and migrating such a task away drives
it negative, since attach() applies deltas unguarded.
Charge a newly instantiated pids controller for the tasks already in
its subtree, both when a parent enables it (rebuildCtrlsLocked) and
when it is returned to the root (returnController). Adds a regression
test.
PiperOrigin-RevId: 974763124
updatePopulated is called when a cgroup's own tasksCount crosses the
0<->1 boundary, but propagated the change to ancestor
nrPopulatedChildren counters even when the cgroup stayed populated
through a child. Draining a mid-level cgroup then made ancestors report
"populated 0" with live tasks below, and the child's later drain
decremented ancestors a second time, leaving the counter permanently
negative.
Skip the notification/propagation walk when the cgroup still has
populated children. Adds a regression test covering both the false
"populated 0" and the stuck-negative counter.
PiperOrigin-RevId: 974662615
Initialize FPU and XCR0 in assembly before entering Go code, switch %rip
to the lower half before calling kernel syscall and exception
trampolines.
In recent Go toolchains, the compiler-generated ABI0 wrappers that bridge
assembly and internal ABI Go code zero X15. The wrapper performs a
RIP-relative load of runtime.x86HasAVX to see if AVX is available on the
host, and executes VXORPS.
This introduces two requirements in ring0:
1. When booting a vCPU, XCR0 only enables x87 state by default. Executing
VXORPS in startGo.abi0 triggers a #UD fault if AVX is not enabled in
XCR0. Moving fninit and xsetbv into assembly before calling startGo
ensures AVX state is enabled before the wrapper runs.
2. The syscall and exception entrypoints execute in the upper half of the
virtual address space where data pages are not mapped. The RIP-relative
load of runtime.x86HasAVX faults unless we transition %rip to the lower
half via jumpToUser before calling into Go trampolines.
PiperOrigin-RevId: 974188468
Scope the signal mutex to just the tg.tty clear in CreateSession, matching how
CreateProcessGroup, JoinProcessGroup, and the exit path already call
handleOrphan without holding a signal mutex.
CreateSession held tg.signalHandlers.mu across the entire process group
manipulation, but it only needs that mutex to clear tg.tty. When setsid(2)
orphans the caller's old process group, decRefWithParent calls handleOrphan,
which locks the signal mutex of every thread group remaining in the group. If
the caller itself, or a sibling thread group that shares the caller's
SignalHandlers (created via clone(CLONE_VM|CLONE_SIGHAND) without CLONE_THREAD),
is still a member, handleOrphan re-locks the already-held, non-reentrant mutex
and the sentry deadlocks while holding TaskSet.mu, freezing the whole sandbox.
Regression test hangs prior to the fix.
Non-hint PAC instructions will generate a SIGILL even if PAC has never been
enabled in the guest. This is circumvented by enabling PAC in the guest, but
setting all of the PAC keys to zero which effectively makes all PAC instructions
no-ops, as was originally the intention with disabling PAC in KVM in the
first place.
PiperOrigin-RevId: 974001522
A systrap task could wait in sleepOnState forever when its stub stopped
responding: one interrupt was sent, then the loop only logged warnings.
Container teardown then blocked indefinitely in WaitExited, wedging the
sandbox and its control RPCs (pods stuck Terminating for days, see #14408
and #14405).
A single interrupt can be lost (#14405), so resend it on every checkup
wakeup. If the stub stays unresponsive through the 30s stuck deadline, or
is already gone, dump all goroutine stacks and kill only the stuck
subprocess through the path NotifyInterrupt already uses (mark dead,
ContextStateUnexpectedDeath, kill the syscall thread). The task goroutine
returns instead of blocking Kernel.Pause() forever, and healthy
subprocesses are unaffected.
Fixes#14408
Co-authored-by: Hanzo Dev <[email protected]>
The peer payload is sliced at the size of a struct ifinfomsg before anything measures it, so 4 bytes panic the Sentry and take the sandbox with it. Any user reaches it after unshare -Ur -n.
Check the length instead of re-testing a lookup that already succeeded. A short payload now answers EINVAL.
Fixes#14516
gVisor returned `ENOSYS` for `copy_file_range(2)`, causing toolchains and
runtimes without fallback paths (such as Zig during linker output generation)
to fail inside sandboxes.
Implement `copy_file_range` as a generic buffered copy in the Sentry to
maintain backing filesystem independence and avoid widening the Sentry seccomp
filter. Argument validation and error handling mirror Linux kernel
`fs/read_write.c` semantics, including offset range checks, cross-device copy
support, and same-file overlap detection.
Add `copy_file_range_test` covering offset handling, short copies, overlap
detection, and Linux errno parity across runsc platforms.
PiperOrigin-RevId: 973465473
It's just a matter of exposing the option via sysfs. Capped to same max as
Linux, to follow convention of what we're already doing with nr_open.
PiperOrigin-RevId: 972901017
In the case of a partly umounted mount, we should only drop the reference on
the parent, not the mount point itself.
Keep the mount point reference since the umounted mount still refers to its
mount point.
PiperOrigin-RevId: 972771131
As of Linux commit 5d39580f68e6, SO_ATTACH_FILTER requires
CAP_NET_ADMIN for TCP sockets.
Skip in tests if we don't have it, and update Netstack to require it
too.
This change fixes a mount leak, where a failed bind mount clone would still
remain as part of the mount peergroup. This clone would be found via as the
next entry in mnt.sharedEntry for a new MS_SLAVE mount. The fix is to simply
set the new clone's propagation to MS_PRIVATE, which makes it remove itself
from the peer ring.
This error is demonstrated pretty clearly in the included unit test
(which is mostly LLM-generated, but is relatively straighforward).
Reported-by: [email protected]
PiperOrigin-RevId: 972688916
Removes nsfs/cgroupfs's exemptions from umounted checks when bind mounting, and
exempts them from mount namespace checks only for non-recursive binds.
Documents the invariant that umounted children should never have mounted
parents more clearly with a checkInvariants panic in cloneMountTree.
PiperOrigin-RevId: 972149358
zeekayandHanzo Dev