GPU profilers (Nsight Systems/CUPTI) build their trace timeline from
CLOCK_MONOTONIC_RAW. gVisor serves CLOCK_MONOTONIC_RAW as an alias of
CLOCK_MONOTONIC, this will drift from CLOCK_MONOTONIC_RAW breaking profiling.
When nvproxy is enabled with `capProfiling`, this change exposes
CLOCK_MONOTONIC_RAW as a clock distinct from CLOCK_MONOTONIC that tracks the
host's CLOCK_MONOTONIC_RAW (absolute, unadjusted). CLOCK_MONOTONIC is
unchanged. When profiling is disabled, CLOCK_MONOTONIC_RAW continues to alias
CLOCK_MONOTONIC as before.
cgroup namespaces are non hierarchical and they are defined primarily by a
"root" cgroup node. The init cgroup namespaces's root node is the true root of
the (only) cgroup2fs tree. The root node of other cgroup namespaces is the
cgroup node of the task at the time of namespace creation.
Residing in a cgroup namespace affects tasks this way:
1) The views into /proc/[pid]/cgroup and the "root" field under
/proc/[pid]/mountinfo are virtualized: they are rendered relative to the
reading task's cgroupns.
2) mount()ing a cgroup2 fs furnishes only a subtree of the full cgroup2fs
tree, rooted at the "root" cgroup of the hosting cgroupns.
Fixes#11989
PiperOrigin-RevId: 948882249
The cgroup v2 fs is initialized early in the boot process, so that every task
in the sandbox resides in at least the root v2 cgroup. A new runsc flag
--mount-cgroup-v2 has been added: this controls whether runsc mounts a
cgroupv2 filesystem at /sys/fs/cgroup automatically (currently it always
mounts a v1 hierarchy at this path). This flag also gates whether userspace
can mount a cgroup2 fs. When this flag is false, which is the default, all
tasks in the sandbox reside forever in the root cgroup.
Like in Linux, the v2 hierarchy is deemed the default owner of a controller,
that is, until the corresponding v1 hierarchy is mounted: then, the v1 tree
steals ownership of that controller and it remains this way till as long as
a v1 mount exists. In other words, hybrid mode is supported, even if such
workloads are rare these days.
The pids controller is the only controller implemented in full: it serves
as validation for the unified hierarchy. The memory controller can be used
to learn the usage stats of a subtree: but like the current v1 implementation,
and unlike Linux, it doesn't achieve this through charging up the hierarchy
during allocation, instead it queries pgalloc for every active controller's
id. And it does no enforcing. The cpu and cpuset controllers are mere stubs,
like in the current v1 implementation.
The following are supported:
* General matters of topology: cgroup.subtree_control, cgroup.controllers,
cgroup.procs etc
* CLONE_INTO_CGROUP
* cgroup.kill
* cgroup.events
* A working pids controller and its events files
* A minimal memory controller that just reports mem usage
* Stubbed cpu and cpuset controllers
The following are NOT supported:
* threaded cgroups
* cgroup namespaces
* BPF_PROG_TYPE_CGROUP_DEVICE
* cgroup.freeze
Updates #13007
PiperOrigin-RevId: 943519748
User namespace entries under `/proc/[pid]/ns` currently render as fake
namespace symlinks. They look like the other namespace files, but opening
them does not produce an `nsfs` file that `setns(2)` can use. Rootless
container tools such as `buildah` and `podman` rely on that file when they
re-enter the pause process user namespace, so the second lifecycle command
fails with `EINVAL`.
Make `UserNamespace` implement `vfs.Namespace` and give each user namespace
an `nsfs` inode when it is created. `/proc/[pid]/ns/user` now uses the
regular namespace symlink path, so opening it returns a joinable namespace
file instead of a fake link target.
`Setns` now accepts `CLONE_NEWUSER` from both `nsfd`s and `pidfd`s. It
follows the Linux restrictions for user namespace joins by rejecting the
caller's current user namespace, requiring `CAP_SYS_ADMIN` in the target
user namespace, rejecting multithreaded callers, and rejecting callers with
`fs` state shared outside the thread group. The capability checks for any
other namespaces in the same `setns` call use the credentials the caller
would have after joining the user namespace.
Add a syscall regression test that creates a child user namespace, opens
`/proc/<pid>/ns/user`, and verifies that `setns(CLONE_NEWUSER)` succeeds.
Fixes https://github.com/google/gvisor/issues/13314
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/gvisor/pull/13323 from shayonj:issue-13314-userns-setns 8060b5f91ba1f2e9c7b6746da233d644728be2e9
PiperOrigin-RevId: 926855389
The net-disconnect-ok flag was previously used only to cause a panic in the
beforeSave() method of TCP and unix connections if it was set to false.
Since this flag is now set to true by default, and this default setting causes
TCP and unix connections to always be aborted during the save operation, the
flag no longer serves its original purpose of conditionally preventing
connection termination. So, the flag's usage is removed.
PiperOrigin-RevId: 840433518
First and foremost, this flag is meant to be a debugging tool for
Systrap-related issues; very often the first step to debugging Systrap errors is
to ask users to disable syscall patching, and this makes that easier. Secondly,
it is also meant to be a temporary stop-gap to fix workloads that don't work
with syscall patching until the universal solution is implemented to roll-back
existing patches to their original state at runtime.
Updates #11649.
PiperOrigin-RevId: 756520044
This CL addresses the following major issues:
- When an application releases memory to the sentry, the sentry unconditionally
releases that memory to the host, rather than allowing it to be reused for
future allocations, in order to ensure that new allocations are uniformly
decommitted (use no memory): cl/145016083. In most cases, this should have
relatively little performance impact; since releasing memory from the
application to the OS is expensive even outside of gVisor, application memory
allocators optimizing for performance already limit the rate at which they
release memory to the OS. However, in applications that involve frequent
process creation and exit (e.g. build systems), this practice prevents reuse
of memory deallocated by exiting processes for memory allocated by new
processes, resulting in both performance degradation and a spike in memory
usage (since the sentry may not have released all deallocated memory to the
host by the time new allocations occur).
- gVisor's historical approach to application THP is based on THP being enabled
on a per-memfd basis, using the MFD_HUGEPAGE flag not merged into the
upstream Linux kernel
(https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/).
Thus, on vanilla Linux kernels, gVisor cannot use THP for application memory
without requiring the system to enable THP for all tmpfs files and memfds (by
setting /sys/kernel/mm/transparent_hugepage/shmem_enabled to "always" or
"force").
- Both MM and the application page allocator (pgalloc) are agnostic as to
whether the underlying memory file will be THP-backed. Instead, both attempt
to align hugepage-sized and larger allocations to hugepage boundaries, such
that if the memory file happens to support THP then such allocations will be
appropriately aligned to use THP. This is suboptimal since many allocations
do not benefit from THP, resulting in memory underutilization.
These issues are especially relevant to platforms based on hardware
virtualization, where acquiring memory from the host is significantly more
expensive due to EPT/NPT fault overhead; when effective, THP reduces the
frequency with which said cost is incurred by a factor of 512, and page reuse
avoids incurring it at all.
Thus:
- Instead of inferring whether THP use is desired from allocation size,
indicate this explicitly as AllocOpts.Huge, and only set it to true for
allocations for non-stack private anonymous mappings.
- Add AllocateCallerIndirectCommit, a new possible value for AllocOpts.Mode
that indicates that the caller will commit all pages in the allocation. In
such cases, pgalloc can reuse deallocated pages without risking increased
memory usage, internally referred to as "recycling".
AllocateCallerIndirectCommit is used primarily for page faults on a
THP-backed region. (It is also used for single-page allocations on non-THP
backed regions, but due to expansion of faults to mm.privateAllocUnit-aligned
ranges, this is relatively uncommon.)
- Allow different chunks in pgalloc.MemoryFile's backing file to have varying
THP-ness, indicated to the host using MADV_HUGEPAGE/NOHUGEPAGE.
- Split pgalloc.MemoryFile's existing page metadata set into two sets tracking
deallocated pages for small/huge-page-backed regions respectively; two sets
tracking in-use pages for small/huge-page-backed regions respectively; and a
fifth set tracking memory accounting state.
- Add MemoryFileOpts.DisableMemoryAccounting; this is primarily intended for
pgalloc tests, but may also be applicable to disk-backed MemoryFiles.
Cleanup:
- Remove MemoryFile.usageSwapped; the UpdateUsage() optimization it enabled,
described in updateUsageLocked(), was based on the condition that
MemoryFile.mu would be locked throughout the call to updateUsageLocked(),
which was invalidated by cl/337865250.
- Remove MemoryFileOpts.ManualZeroing, which is unused.
- Rename "reclaiming" to "releasing"; the former is confusing since "reclaim"
in Linux has a significantly different meaning (essentially "eviction" in
pgalloc), and the latter seems to be conventional in user-mode memory
allocators.
Using THP for application memory requires setting
/sys/kernel/mm/transparent_hugepage/shmem_enabled to "advise", in order to
allow runsc to request THP from the kernel.
After this CL, pgalloc.MemoryFile still releases memory to the host as fast as
possible, limiting the effectiveness of page recycling. A following CL adds
optional memory release throttling to improve this.
Performance outcomes vary by workload and platform. (In all of the below,
"baseline" is without this CL, "expt" is with this CL, and "expt2" is with this
CL + reclaim throttling (cl/575046398).)
For systrap in GKE: As noted, this change is required to enable application THP
without forcing it on all host shmem users. In conjunction with recycling
(which has a relatively small effect on systrap since it does not use hardware
virtualization), THP use slightly improves performance, although whether this
is measurable is case-dependent. On an idle VM, with shmem_enabled = "advise":
```
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.80GHz
│ baseline │ expt │ expt2 │
│ sec/op │ sec/op vs base │ sec/op vs base │
BuildABSL/page_cache.clean/filesystem.bindfs-16 39.09 ± 4% 38.84 ± 5% ~ (p=0.947 n=30) 38.84 ± 3% ~ (p=0.854 n=30)
BuildABSL/page_cache.dirty/filesystem.bindfs-16 37.83 ± 3% 36.58 ± 4% ~ (p=0.057 n=30) 36.83 ± 5% ~ (p=0.314 n=30)
BuildABSL/page_cache.clean/filesystem.tmpfs-16 39.34 ± 3% 38.59 ± 4% ~ (p=0.350 n=30) 38.58 ± 4% ~ (p=0.300 n=30)
BuildABSL/page_cache.dirty/filesystem.tmpfs-16 37.83 ± 3% 36.08 ± 4% -4.64% (p=0.026 n=30) 36.58 ± 4% ~ (p=0.123 n=30)
BuildABSL/page_cache.clean/filesystem.rootfs-16 39.59 ± 4% 38.83 ± 3% ~ (p=0.485 n=30) 40.09 ± 5% ~ (p=0.971 n=30)
BuildABSL/page_cache.dirty/filesystem.rootfs-16 36.83 ± 3% 38.08 ± 5% ~ (p=0.307 n=30) 38.08 ± 1% ~ (p=0.242 n=30)
BuildABSL/page_cache.clean/filesystem.fusefs-16 38.34 ± 3% 37.59 ± 5% ~ (p=0.752 n=30) 38.59 ± 3% ~ (p=0.982 n=30)
BuildABSL/page_cache.dirty/filesystem.fusefs-16 37.58 ± 4% 38.08 ± 5% ~ (p=0.708 n=30) 36.08 ± 6% ~ (p=0.127 n=30)
BuildGRPC/page_cache.clean/filesystem.bindfs-16 212.7 ± 2% 211.0 ± 1% ~ (p=0.138 n=30) 211.2 ± 1% ~ (p=0.458 n=30)
BuildGRPC/page_cache.dirty/filesystem.bindfs-16 210.0 ± 1% 210.0 ± 1% ~ (p=0.542 n=30) 209.7 ± 1% ~ (p=0.665 n=30)
BuildGRPC/page_cache.clean/filesystem.rootfs-16 210.5 ± 1% 210.0 ± 1% ~ (p=0.423 n=30) 210.0 ± 1% ~ (p=0.142 n=30)
BuildGRPC/page_cache.dirty/filesystem.rootfs-16 210.2 ± 1% 209.0 ± 1% ~ (p=0.219 n=30) 209.5 ± 1% ~ (p=0.230 n=30)
geomean 67.62 66.97 -0.96% 67.12 -0.74%
```
The KVM platform benefits significantly from reduced nested page faults due to
huge pages, and to a lesser extent due to recycling:
```
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz
│ baseline │ expt │ expt2 │
│ sec/op │ sec/op vs base │ sec/op vs base │
BuildABSL/page_cache.clean/filesystem.bindfs-12 43.11 ± 2% 39.35 ± 3% -8.71% (p=0.000 n=20) 38.10 ± 4% -11.63% (p=0.000 n=20+19)
BuildABSL/page_cache.dirty/filesystem.bindfs-12 42.35 ± 3% 39.09 ± 4% -7.69% (p=0.000 n=20+19) 39.09 ± 5% -7.69% (p=0.000 n=20+19)
BuildABSL/page_cache.clean/filesystem.tmpfs-12 42.35 ± 3% 38.34 ± 5% -9.46% (p=0.000 n=20) 38.59 ± 3% -8.87% (p=0.000 n=20+19)
BuildABSL/page_cache.dirty/filesystem.tmpfs-12 42.09 ± 1% 37.59 ± 4% -10.70% (p=0.000 n=20) 38.09 ± 4% -9.51% (p=0.000 n=20+19)
BuildABSL/page_cache.clean/filesystem.rootfs-12 42.85 ± 3% 38.84 ± 3% -9.35% (p=0.000 n=20) 39.09 ± 3% -8.77% (p=0.000 n=20+17)
BuildABSL/page_cache.dirty/filesystem.rootfs-12 41.85 ± 2% 39.59 ± 6% -5.40% (p=0.000 n=20+19) 38.09 ± 3% -9.00% (p=0.000 n=20+19)
BuildABSL/page_cache.clean/filesystem.fusefs-12 42.60 ± 2% 38.34 ± 2% -10.00% (p=0.000 n=20) 39.59 ± 3% -7.06% (p=0.000 n=20+19)
BuildABSL/page_cache.dirty/filesystem.fusefs-12 42.09 ± 4% 39.09 ± 3% -7.13% (p=0.000 n=20) 38.09 ± 3% -9.52% (p=0.000 n=20+19)
BuildGRPC/page_cache.clean/filesystem.bindfs-12 207.7 ± 1% 206.4 ± 0% -0.60% (p=0.018 n=20) 205.9 ± 1% -0.85% (p=0.001 n=20+19)
BuildGRPC/page_cache.dirty/filesystem.bindfs-12 206.9 ± 1% 206.9 ± 1% ~ (p=0.121 n=20) 204.4 ± 1% -1.22% (p=0.004 n=20+19)
BuildGRPC/page_cache.clean/filesystem.rootfs-12 207.7 ± 1% 204.9 ± 1% -1.33% (p=0.004 n=20) 203.9 ± 0% -1.81% (p=0.000 n=20+19)
BuildGRPC/page_cache.dirty/filesystem.rootfs-12 206.9 ± 1% 204.9 ± 0% -0.97% (p=0.004 n=20+19) 203.9 ± 0% -1.45% (p=0.000 n=20+19)
geomean 71.97 67.63 -6.03% 67.28 -6.52%
```
PiperOrigin-RevId: 647771821
We want to decouple the netstack from the kernel. This coupling is
causing bugs in restore because netstack needs to be created before the Kernel
is restored. So right now, netstack ends up using a "temporary" Kernel which is
later destroyed in the restore sequence, but netstack keeps referencing it.
Before this change, netstack was being initialized with a `kernel.TimeKeeper`.
This `Timekeeper` was being initialized with `VDSOParamPage`, which references
the MemoryFile of the kernel. So as a result, on restore the netstack ends up
referencing the destroyed kernel's MemoryFile.
So instead move out VDSOParamPage from TimeKeeper altogether. The callers of
`TimeKeeper.SetClocks()` and `TimeKeeper.ResumeUpdates()` pass VDSOParamPage
from the correct kernel being used currently.
PiperOrigin-RevId: 647168588
The work done in a9af2d309b ("Plumb restore context to afterLoad()") makes
pgalloc.MemoryFileProvider redundant as structs can now easily restore
pgalloc.MemoryFile in stateify's afterLoad() method. This allows structs to
have a pgalloc.MemoryFile field and use that directly, instead of going through
the provided interface.
This cleans up a lot of code and also should be more performant (avoids an
interface method call on many hot paths).
PiperOrigin-RevId: 615258927
It is an idea of running codespell as part of our presubmit checks.
Before enabling it for new changes, let's fix what it has found.
Signed-off-by: Andrei Vagin <[email protected]>
This moves the initialization of `cpuid.hostFeatureSet` out of package-level
`init` and instead moves it to an explicit `cpuid.Initialize()` function.
On AMD64, this saves about 512KiB of heap memory that would otherwise always
be live.
PiperOrigin-RevId: 514896323
This moves the initialization of `seccheck.Points` out of package-level
`init` and instead moves it to an explicit `seccheck.Initialize()` function.
On AMD64, this saves about 614KiB of heap memory that would otherwise always
be live.
PiperOrigin-RevId: 514813332
This allows overriding the path used to access the kernel's KVM device file,
typically at `/dev/kvm`, with a flag-controlled path instead.
PiperOrigin-RevId: 431073099
Weirdness metric contains fields to track the number of clock fallback,
partial result and vsyscalls. This metric will avoid the overhead of
having three different metrics (fallbackMetric, partialResultMetric,
vsyscallCount).
PiperOrigin-RevId: 369970218
These are primarily simplification and lint mistakes. However, minor
fixes are also included and tests added where appropriate.
PiperOrigin-RevId: 351425971
This reduces confusion with context.Context (which is also relevant to
kernel.Tasks) and is consistent with existing function kernel.LoadTaskImage().
PiperOrigin-RevId: 342167298
TCP/IP will work with netstack networking. hostinet doesn't work, and sockets
will have the same behavior as it is now.
Before the userspace is able to create device, the default loopback device can
be used to test.
/proc/net and /sys/net will still be connected to the root network stack; this
is the same behavior now.
Issue #1833
PiperOrigin-RevId: 296309389
- Added fsbridge package with interface that can be used to open
and read from VFS1 and VFS2 files.
- Converted ELF loader to use fsbridge
- Added VFS2 types to FSContext
- Added vfs.MountNamespace to ThreadGroup
Updates #1623
PiperOrigin-RevId: 295183950
Because the abi will depend on the core types for marshalling (usermem,
context, safemem, safecopy), these need to be flattened from the sentry
directory. These packages contain no sentry-specific details.
PiperOrigin-RevId: 291811289
Luis Capelo