Fix a panic when writing to cgroup v2 control files from outside the sandbox
(e.g., via Cgroups.WriteControlFiles / WriteControl), where writes occur
without an open in-sandbox VFS file descriptor (fd == nil). cgroup.procs now
safely falls back to context credentials with unconfined namespace migration,
and checkNSDelegateWrite skips namespace delegation checks. Add an integration
test for runsc cgroup v2 control file writes.
Also:
- Export cgroup2fs.InitHierarchy to initialize sandbox-wide cgroup2 state
and root controller enablement.
- Plumb CgroupNamespace through sentry lifecycle and task context to derive
initial container cgroup2 placements.
PiperOrigin-RevId: 978087817
OrderedChildren.Lookup returns the inode with a ref that the caller is
expected to release (normally by handing it to a dentry via Init). Five
cgroup2fs call sites that look an inode up for a transient purpose
(permission checks, control-file read/write, interface-file removal)
never released that ref.
This is NOT a bug fix: every inode these sites can return
(cgroupInterfaceFile and eventFile, both via kernfs.DynamicBytesFile,
and cgroup directories themselves) embeds kernfs.InodeNoopRefCount, so
IncRef/DecRef are no-ops and nothing leaks today. The change only makes
the ref discipline explicit and correct, so these sites stay safe if a
lookup ever returns a genuinely refcounted inode.
PiperOrigin-RevId: 976563179
renameat2(2) support was added in 61f761343f ("Implement renameat2."),
but only for the gofer filesystem; tmpfs and overlayfs still return
EINVAL for RENAME_EXCHANGE.
Implement RENAME_EXCHANGE in tmpfs by swapping the two dentries between
their parent directories. When the exchanged files differ in type, the
parents' link counts (for "..") and the exchanged directory's reference
on its parent are transferred.
Implement RENAME_EXCHANGE in overlayfs by copying up the destination
file (and its descendants, if it is a directory) in addition to the
source, and then exchanging both files on the upper layer. Unlike a
regular rename, no whiteout is created at the source location, since
both locations remain occupied. Exchanged directories are marked opaque
at their new locations to prevent merging with lower layer directories
there.
RENAME_EXCHANGE permits the two files to differ in type and does not
require a directory being exchanged to be empty, but exchanging a file
with an ancestor directory must fail with EINVAL; see Linux's
fs/namei.c:__start_renaming(). The gofer filesystem previously required
both files to have the same type; lift that restriction, and update
parent link counts when files of different types are exchanged across
directories.
The RENAME_EXCHANGE syscall tests no longer accept EINVAL; they are
skipped where the flag remains unsupported, i.e. on FUSE and on gVisor
kernels other than runsc. New tests cover mixed-type exchanges and
exchanges with ancestor directories.
Fixes#7895
The cpu controller tracks each task's CPU via a per-instance baseline
snapshot plus a cumulative total for exited tasks, but it never walked
the whole accounting domain when reporting cpu.stat and never reconciled
that state when a controller was disabled or when a task migrated within
a single controller's domain. This caused three accounting errors, each
covered by a new test:
- Live tasks in descendant cgroups that had not enabled their own cpu
controller were absent from the ancestor's cpu.stat until they
exited, even though Linux aggregates descendant usage into the
ancestor continuously.
- Migrating a task between two cgroups sharing the same nearest cpu
controller reset its baseline, discarding the pre-migration CPU.
- Disabling +cpu discarded the controller's accumulated usage instead
of reparenting it (Linux folds an offlined css's stats into parent).
Sum the whole domain (the controller's cgroup plus every descendant
without its own cpu controller, recursing into descendants that do) when
reading cpu.stat; leave the baseline untouched on same-instance
migration; and on disable, migrate the domain's tasks to the parent's
nearest cpu controller and fold the accumulated total into it.
PiperOrigin-RevId: 974821312
Enabling +pids in cgroup.subtree_control creates child pids controllers
with committed = 0, ignoring tasks already present in those cgroups.
pids.current then underreports, and migrating such a task away drives
it negative, since attach() applies deltas unguarded.
Charge a newly instantiated pids controller for the tasks already in
its subtree, both when a parent enables it (rebuildCtrlsLocked) and
when it is returned to the root (returnController). Adds a regression
test.
PiperOrigin-RevId: 974763124
updatePopulated is called when a cgroup's own tasksCount crosses the
0<->1 boundary, but propagated the change to ancestor
nrPopulatedChildren counters even when the cgroup stayed populated
through a child. Draining a mid-level cgroup then made ancestors report
"populated 0" with live tasks below, and the child's later drain
decremented ancestors a second time, leaving the counter permanently
negative.
Skip the notification/propagation walk when the cgroup still has
populated children. Adds a regression test covering both the false
"populated 0" and the stuck-negative counter.
PiperOrigin-RevId: 974662615
It's just a matter of exposing the option via sysfs. Capped to same max as
Linux, to follow convention of what we're already doing with nr_open.
PiperOrigin-RevId: 972901017
Read SACK and ephemeral port ranges from the owning stack instead of
per-file caches. Separate proc mounts can cache different values, and a
rejected SACK write can change its cache. The saved stack already
preserves these settings.
Keep the last successfully written forwarding value, since individual
interfaces may have different forwarding settings. Guard it with a mutex
and publish updates only after the stack setter succeeds, leaving failed
writes out of subsequent readback.
Also make the existing TCP buffer-size helper contracts checkable. Their
mutex still serializes requests through the same byte source.
Cover shared readback, rejected SACK and forwarding writes, and
concurrent forwarding readback in the existing proc test target.
Co-authored-by: Hanzo Dev <[email protected]>
Procfs mounts used a per-mount major number for /proc/gvisor/checkpoint.
Since Linux reserves a limited number of dynamic major numbers, running
many isolated container actions concurrently would exhaust the pool and
crash the sentry with EBUSY.
This adds VFS.GetSharedDynamicCharDevMajor(key) to allocate a major on
first use and share it across all subsequent mounts for that key.
The sentry now consumes exactly two majors for these devices regardless
of how many procfs instances are mounted.
PiperOrigin-RevId: 967985107
Three nearly identical names covered three unrelated concepts
(ConnectedEndpoint, connectedEndpoint, connectionedEndpoint), and the
notification callbacks were named for the opposite of what they do
(RecvNotify woke writers, SendNotify woke readers).
- ConnectedEndpoint -> Sender, the write half of a connection
(implementations queueSender, HostSender, SCMSender); the
baseEndpoint field becomes connected -> peer: the type is named for
what you do with it, the field for its role to the holder.
- SendNotify -> NotifyDataReady, RecvNotify -> NotifyWriteSpace,
CloseNotify -> NotifyStateChange; each named for the wakeup it
delivers. NotifyStateChange must wake hangup-only pollers.
- queue's embedded waiter queues become readerWaiters/writerWaiters,
and baseEndpoint's embedded *waiter.Queue becomes a named field,
WaitQueue, so wakeup call sites name their target.
- writeShutdown and wrShutdown/rdShutdown ->
sendShutdown/recvShutdown, the shutdown(2) directions they track.
- A package doc block describes how the pieces fit together.
No behavior change.
PiperOrigin-RevId: 966732452
When an AF_UNIX socket is bound to a path, the resulting BoundEndpoint is
stored on the gofer inode, and other processes reach it by path: they walk
to the dentry, then pull the endpoint off its inode and connect. Crucially,
a bound socket stays connectable even when no file descriptor is holding
that path open.
The gofer refcounting invariant is: "An additional reference is held on all
synthetic dentries, and on all dentries for which endpoint is non-nil, until
they are unlinked or invalidated."
This purpose of this invariant is for a host-backed bound UDS to stay
resolvable by path even when no FD holds it. A gofer dentry at refs == 0 is
eligible for reclaim; reclaiming it tears down the inode and drops the
HostBoundEndpoint (and its host FD). So the filesystem pins every endpoint-
bearing dentry with one extra ref, held from bind until the name goes away.
This extra ref is dropped unconditionally, guarded only by `endpoint != nil`
in many places: unlinkAt, RenameAt, revalidation/invalidation, filesystem
teardown etc. (These drop paths assume that the invariant holds.)
To establish the invariant the extra ref is taken in the bind paths sure, but
we were missing a reference increment for a dentry that later comes to point
at the same endpoint-bearing inode. This can happen either due to a hard link
or another walk that lands on the same inode.
PiperOrigin-RevId: 966116353
Add an efaproxy driver plug-in for AWS Elastic Fabric Adapter (EFA) devices
bound to the host `efa` driver. EFA's driver-private uverbs ABI carries no
userspace buffer pointers on CQ/QP CREATE: queue memory is kernel-allocated
and handed back via mmap keys in the CREATE response, which the core already
forwards to the host FD verbatim. PrepareCreateDMA is therefore a no-op; the
plug-in exists to register the "efa" driver name and to model EFA's MR_QUERY
driver-namespace method (used by libfabric to fetch RDMA read/write
interconnect IDs).
Model the standard AH object's destroy method (UVERBS_OBJECT_AH): EFA (SRD)
creates address handles via the legacy write path and destroys them through
the modern object, which the core must allow.
Extend the virtual RDMA sysfs for libfabric/hwloc topology discovery:
- device/driver symlink and /sys/bus/pci/drivers/<driver> tree, which
libfabric's EFA provider resolves during device discovery.
- Mirror each PCI node's raw config space ("config" file). hwloc (used by
aws-ofi-nccl to build the NCCL topology) reads it to recover the PCI
bridge bus-number registers; without it hwloc cannot reconstruct the
bridge hierarchy and aws-ofi-nccl aborts its topology write.
- Serve the per-port lid_mask_count attribute alongside the other port
attributes libibverbs reads from sysfs; ib_core creates it on every
port.
The sysfs integration tests cover the new driver tree and config mirroring.
Advertise a >= 5.12 kernel release in RDMA-enabled sandboxes so aws-ofi-nccl
enables dmabuf-based GPU memory registration (ibv_reg_dmabuf_mr), which the
proxy handles via the existing REG_DMABUF_MR path. Sandboxes without RDMA
keep the current 4.19 release to limit the bump's blast radius; a later
change can make the newer release the default.
Co-authored-by: Alessio Ricci Toniolo <[email protected]>
It's my understanding that Linux exposes `/proc/modules` empty in most
configurations (it's possible to not expose modules but that seems rare; only if the kernel is compile with that configuration).
Some applications like NVIDIA's GPUDirect Storage libraries (libcufile)
will scan `/proc/modules`. However, under gVisor, the scan never
succeeds and will hang forever.
GPU profilers (Nsight Systems/CUPTI) build their trace timeline from
CLOCK_MONOTONIC_RAW. gVisor serves CLOCK_MONOTONIC_RAW as an alias of
CLOCK_MONOTONIC, this will drift from CLOCK_MONOTONIC_RAW breaking profiling.
When nvproxy is enabled with `capProfiling`, this change exposes
CLOCK_MONOTONIC_RAW as a clock distinct from CLOCK_MONOTONIC that tracks the
host's CLOCK_MONOTONIC_RAW (absolute, unadjusted). CLOCK_MONOTONIC is
unchanged. When profiling is disabled, CLOCK_MONOTONIC_RAW continues to alias
CLOCK_MONOTONIC as before.
This change extends the split filesystem checkpoint/restore feature to support
saving and restoring only a subset of filesystems (specified by paths). This
supports a "combined restore" mechanism where some memory files (tmpfs backings)
are restored from the filesystem checkpoint, while others are restored from the
main sentry checkpoint.
- Added `SplitFSCheckpointPaths` to `SaveOpts` to allow specifying paths for
partial FS checkpoints.
- Updated `createPrivateMemoryFile` to return whether it successfully loaded
state from the filesystem checkpoint.
- Track which memory files were loaded from the FS checkpoint during restore
(`fsCheckpointedMfs`) and pass this information to the kernel via context
(`CtxFSCheckpointedMemoryFiles`).
- In `loadPrivateMemoryFiles` (during sentry restore), only load memory files
from the sentry checkpoint if they were not already loaded from the FS
checkpoint.
- Added validation to ensure each memory file is restored from exactly one
source (no duplicates, and none are missed).
PiperOrigin-RevId: 964518735
Mirrors fsimpl/gofer's caching model: each dentry holds one ref on its
parent; childMap is a weak map. refs == 0 means cache-eligible; refs
== -1 means destroyed. Cached children still hold their parent's ref,
so a parent only enters the cache after all its children are gone.
Per-dentry cachingMu serializes caching state; fs.cacheMu is held only
for LRU list mutations. Eviction re-checks under parent.dirMu, severs
from childMap, then destroys, cascading checkCaching up the tree.
Cache limit is 1000.
Co-authored-by: Ayush Ranjan <[email protected]>
This change implements split filesystem checkpoint and restore, allowing the
filesystem state to be saved and restored separately from the sentry state.
- Added `--split-fscheckpoint` flag to the `checkpoint` command to trigger
saving filesystem state separately. Added `--split-fsrestore` and
`--fs-restore-image-path` flags to `restore` and `create` commands to support
restoring from split files.
- Added `FSSave` and restore code for split filesystems.
- Implemented saving/restoring tmpfs state (MemoryFile pages and directory
structure) to/from a tar-like format.
- Refactored tar read/write path to support merging filesystem state in tar.go
- Implemented donating filesystem checkpoint FDs to the sandbox during
creation/restore.
- Added test to verify split checkpoint and restore.
PiperOrigin-RevId: 964380388
Regular file reads copy data out of the image's shared mapping. When the
image is backed by slow storage such as a FUSE mount, walking the mapping
faults it in one page at a time, turning a large read into many small,
largely serialized requests to the FUSE server. This makes erofs much
slower than fsimpl/gofer for large reads on such backends.
On those filesystems, read the data directly from the image file
descriptor with preadv2(2) instead, so the host serves a large read with a
single syscall and its own readahead, matching how fsimpl/gofer reads from
host FDs. The mapping copy is kept everywhere else, since it avoids a
syscall and is faster when the page cache is warm. The backing filesystem
is detected at mount time via fstatfs(2); currently only FUSE opts into FD
reads.
Inode.Data, which returned the data as a mapping-backed BlockSeq, is
replaced by Inode.DataRanges, which returns the image byte ranges that
hold the inode's data.
A tty inode was kept alive by a single ownership chain: the stdio file
description's reference on the import-time dentry. Everything else that used
the inode referred to it uncounted: a file description opened via /dev/tty
pins only the devtmpfs dentry, and the controlling-terminal association
(ThreadGroup.tty) is a bare pointer.
PiperOrigin-RevId: 962479941
On Linux, udev maintains /dev/char/<major>:<minor> and
/dev/block/<major>:<minor> symlinks to all device nodes it manages.
Applications open devices by device number through these symlinks; e.g.
rdma-core uses /dev/char to locate uverbs device nodes from the device
numbers advertised in sysfs.
There is no udev in the sandbox, so create these symlinks when device
files are created in devtmpfs. This covers both sentry-registered
devices and devices injected via the OCI runtime spec.
When --mount-cgroup-v2 is enabled, a container whose spec includes a
cgroup mount is now given its own cgroup "/<cid>" in the cgroup2
hierarchy, with its init process placed there.
Cgroup namespaces from the spec are now honored, mirroring PID
namespace policy: no entry means the root namespace, an entry with a
path joins the container that declared the same path, and an entry
without a path gets a new namespace rooted at the container's cgroup.
Since a fresh cgroup2 mount is rooted at the mounting namespace's
root, a private-namespace container's /sys/fs/cgroup shows only its
own subtree, as on Linux, with no need for bind mounts.
At boot the sentry now enables nsdelegate and delegates controllers
to per-container cgroups via the root cgroup.subtree_control. Exec
sessions join the container's cgroup and cgroup namespace, like
runc exec. Per-container "runsc events" and read-control queries
now resolve for any container with a cgroup mount.
Note two differences from the existing v1 support. In v1 we bind mount:
the per-container dir "/$cid" of each controller hierarchy appears as
/sys/fs/cgroup/<controller>. With v2 no bind mounts are needed: a fresh
cgroup2 mount is rooted at the mounting namespace's root, so honoring
the spec's cgroupns request yields the scoped view naturally. (Binding
a subtree a la v1 was not an option anyway: systemd explicitly rejects
a sub-hierarchy mounted without a matching cgroup namespace.) Second,
on container destroy we now make a best-effort, recursive attempt to
remove the container's cgroup subtree; v1 leaves its per-container
dirs behind.
The root container is not special-cased. As with v1, where even the
root container gets container-id-stamped cgroup dirs and the stats
queries they enable, it gets its own "/$root_cid", and its cgroup
namespace comes from the spec like everyone else's. Docker and
Kubernetes both request a private cgroup namespace for every container
on cgroup2 hosts, so in practice the root container sees its cgroup as
a pleasing "/" rather than "/$root_cid"; a spec with no cgroupns entry
(e.g. docker run --cgroupns=host) instead sees the full sandbox
hierarchy, as on Linux. One consequence of the private default: in a
multi-container sandbox the root container cannot see the other
containers' cgroups (namespace roots are not hierarchical). I don't
expect that to be problematic, and a spec without a cgroupns entry
regains the full view if ever needed.
PiperOrigin-RevId: 960153510
The goal is to differentiate O_DSYNC and O_SYNC, O_DSYNC behaves same
as O_SYNC today in gVisor.
assisted-by: agy
Fix#1897
PiperOrigin-RevId: 958610974
Adds initial scaffolding for eBPF in gVisor. It implements the bpf(2) syscall
with support for BPF_PROG_LOAD, BPF_PROG_QUERY, and BPF_PROG_ATTACH.
Necessary for the cgroups v2 flag to be enabled by default, as this fixes
Docker-in-gVisor with cgroups v2.
Containers with /dev/infiniband/uverbs* devices need a sysfs surface for
device discovery: libibverbs enumerates /sys/class/infiniband_verbs, and
NCCL builds its hardware topology by resolving /sys/class symlinks with
realpath() and walking the PCI hierarchy.
This adds a --rdmaproxy flag which will gate all RDMA support. For now it
only wires up the sysfs topology: runsc snapshots the host sysfs closure
derived from the spec's uverbs devices (ibdevs, netdevs, PCI ancestor
chains, GPU functions, NUMA nodes) before pivot_root, and the sentry
constructs a path-identical virtual tree: one canonical /sys/devices/pci...
subtree per device, with /sys/class/* and /sys/bus/pci/devices as symlink
farms matching the kernel layout. Per-port dynamic state (RoCE GID tables,
link state, counters) is served live through targeted read-only bind mounts
of the per-ibdev ports subtrees; identity and PCI attributes are a static
snapshot, since the collecting process and the constructed sandbox tree do
not observe the same sysfs mount (the RDMA netdevs move into the sandbox
network namespace only after the sandbox is created).
Each PCI function directory also gets a subsystem symlink into a
synthesized /sys/bus/pci, and every node's bus gets a pci_bus entry: NCCL
classifies a directory as a PCI device by following its subsystem symlink,
and aborts topology discovery without it.
The NUMA topology is not mirrored: the sentry advertises a single node
holding every sandbox CPU, consistent with its single-node memory model
(set_mempolicy's maxNodes == 1) and its single-socket /proc/cpuinfo view.
Host node cpumasks name CPUs the sandbox does not have, which breaks
libnuma/hwloc consumers whenever the sandbox CPU count is below the
host's (numactl node binding, which NUMA-aware stacks exec in front of
worker subprocesses, kills the child). The PCI numa_node, local_cpus and
local_cpulist attributes are rewritten to match the single-node view.
The verbs device proxy that makes RDMA usable end-to-end is not part of
this change; --rdmaproxy is a work in progress and RDMA is not yet
functional.
Co-authored-by: Alessio Ricci Toniolo <[email protected]>
Implements POSIX ACLs for gVisor's overlayfs.
- Adds plumbing to keep access and default ACLs in sync with the underlying filesystems, simliar to mode
- Moves permission/filetype checking into helpers in posix_acl.go, used by both tmpfs and overlayfs
- Updates the POSIX ACL tests to be parametrized by filesystem. The existing acl test suite runs both on overlayfs and on tmpfs
This fixes journald not working for systemd user services.
Linux skips the mount-level write check when opening a character device, block device, FIFO, or socket, so write-opening /dev/null on a read-only bind mount succeeds.
Mark the remaining FileDescriptions backed by special-file inodes. Also skip overlay copy-up for writable opens of special files; Linux overlayfs applies the same special_file() exception before deciding whether copy-up is required.
Add regression coverage for character devices and FIFOs through read-only bind mounts.
This mount option turns cgroup namespaces into delegation boundaries,
which results in the following two behaviors:
First, resource control. A cgroup's resource-control files carve out
the parent's resources, so they belong to the delegater, not to the
subtree's occupants. Accordingly, tasks in a non-init cgroup namespace
may no longer write to interface files of their own namespace root.
The delegatable files (cgroup.procs, cgroup.subtree_control) are exempt.
Subtree tasks cannot change what share of resources the subtree receives,
but remain free to organize processes and distribute that share within it.
Second, containment. Migration (including via CLONE_INTO_CGROUP) now
requires that both the source and destination cgroups be descendants of
the cgroupns root of the task performing the migration, failing with
ENOENT otherwise (actually it is the cgroupns of the task that opened the
fd for the migration file: this detail forces me to snapshot the cgroupns
against the fd, increasing the size of this patch).
PiperOrigin-RevId: 954892289
This change fixes a bug in FUSE getdents where directory entries that straddle
the readdir boundary could be dropped or parsed incorrectly. It adds validation
to reject empty-name directory entries with EIO and implements the Seek method
for FUSE directory file descriptors.
Fixes#13721
PiperOrigin-RevId: 952998757
