- `runsc fscheckpoint` saves a filesystem checkpoint, analogous to `runsc
checkpoint`. `-direct` and `-leave-running` flags are supported for
compatibility with `runsc checkpoint`. `-save-restore-exec*` is currently
not supported due to lack of a use case. `-compression` support is not
planned (specific files in filesystem checkpoint images may use compression
as appropriate, but this is an implementation detail).
`-exclude-committed-zero-pages` is inapplicable.
- `runsc create -fs-restore-image-path` restores a filesystem checkpoint. The
`-fs-restore-direct` flag has the same effect as `runsc restore -direct`.
The behavior of `runsc restore -background` is always in effect for
filesystem checkpoints.
PiperOrigin-RevId: 885914969
... and use it everywhere in preference to its string. This is to be able to
reliably inspect pgalloc.MemoryFile.ResourceID().Path in cl/879147587.
PiperOrigin-RevId: 883274090
Add PageSize checks in Kernel.SaveTo() and Kernel.LoadFrom() to return
a clean error when save/restore is attempted with non-4K page sizes.
This is similar to the checks added for KVM and Systrap platforms.
Save/restore serialization has not been adapted for 64K pages yet,
so this gives a clear error message instead of a cryptic failure.
Previously, external connections were terminated during both restore and
resume operations, causing a reconnection after every resume. By limiting
termination to the restore only, connections now remain active during resumes
as intended.
Added tests to verify that resume does not terminate the connection.
PiperOrigin-RevId: 868830577
The correctness of save/restore depends on the kernel object graph remaining
constant while it is being saved/restored. Allowing fdnotifier to send host FD
readiness notifications during save/restore violates this requirement.
PiperOrigin-RevId: 856472612
- Introduce Credentials.HasRootCapability().
- Rename Credentials.HasCapability() to Credentials.HasSelfCapability().
- Rename Task.HasCapability() to Task.HasSelfCapability().
- Introduce inet.Namespace.HasCapability(ctx).
- CapError(), invoked by unimplemented syscalls' stubs, now checks
against the root userns.
All towards making it less easy to evaluate creds relative to self userns
instead of root userns in the future.
PiperOrigin-RevId: 853942212
This change adds RSEQ support for platform/kvm. RSEQ is disabled by default.
To support RSEQ, the KVM platform must provide two key capabilities:
(1) A stable, unique CPU ID.
(2) A way to detect when a thread as been preempted.
This implementation provides the necessary support as follows:
CPU ID:
- platform/kvm now advertises the KVM vCPU ID as the cpu_id
Preemption Detection:
- Compares the last context the CPU ran against the current context being
scheduled.
- Compares the context's rseqCPU and CPU ID retrieved by the platform
To facilitate this, several new methods are introduced to the platform
interface and implemented by platform/kvm:
- HasCpuNumbers()
- NumCPUs()
- DetectsCPUPreemption()
- PreemptCpu()
- PreemptAllCpus()
A new user-facing KVM specific option, use_cpu_numbers, is introduced to
effectively feature guard RSEQ. When use_cpu_numbers is true, vCPU IDs are
exposed to the guest, enabling RSEQ support. Setting it to false reverts to
the previous behavior and effectively disables RSEQ for applications that
rely on it.
PiperOrigin-RevId: 844846226
The net-disconnect-ok flag was previously used only to cause a panic in the
beforeSave() method of TCP and unix connections if it was set to false.
Since this flag is now set to true by default, and this default setting causes
TCP and unix connections to always be aborted during the save operation, the
flag no longer serves its original purpose of conditionally preventing
connection termination. So, the flag's usage is removed.
PiperOrigin-RevId: 840433518
Before this change, the processors were closed in beforeSave which caused the
packet processing for established connections to stop even in the case of
save/resume. Do not close in beforeSave, instead close these processors in
afterLoad for restore.
PiperOrigin-RevId: 821714849
Aside from improving checkpointing performance by using AIO to better utilize
disk bandwidth, this makes it possible to parallelize writing pages with
scanning for zero pages (in a following CL).
As part of this, update and document ownership semantics for the state file,
pages metadata file, and pages file, with the following goals:
- Since io.WriteCloser.Close() may flush buffers, it must return a nil error
before write success is reported
- The pages metadata file is closed while async page saving is in progress (to
hide the latency of the former)
PiperOrigin-RevId: 814315485
During restore there will be new NICs and routes created based on the network
config. Close these old NICs and routes in beforeSave() only for save/restore.
For save/resume, do not close them.
As new link endpoints are created during restore, there is no need to start
the associated old processor goroutines in afterLoad. Start them only for
save/resume.
PiperOrigin-RevId: 804994281
There are two issues being fixed: 1) checkAndPreventSharingOutsideTG() was
accessing t.fsContext without grabbing t.mu and thus causing a data race with
a concurrent unshare and 2) the concurrent unshare might present an incomplete
view of the shared fscontext and cause execve to incorrectly conclude that it
was shared outside the thread group.
Reported-by: [email protected]
Reported-by: [email protected]
PiperOrigin-RevId: 800327461
Currently, on certain error paths during task creation (e.g., when
"runsc exec" fails), we do not properly handle the refcount of the
mntns/FSContext, leading to a refcount leak. This patch adds cleanup
that would be released before the reference ownership is transferred
to callee, ensuring that the refcount is correctly decremented.
Fixes#12054
Signed-off-by: Tianyu Zhou <[email protected]>
There is already some support for privileged executables before this change:
file capabilities were taken into account when creating new processes inside
the sandbox from external instigation (but not during execve(2)). set-user-id
and set-group-id bits were never taken into account, and the Sentry behaved
as if NO_NEW_PRIVS is always true. After this change, execve(2) and the
external Exec entry point both take the set-user(group)-id bits and file
capabilities into account.
Because NO_NEW_PRIVS can no longer be assumed to be always set, a couple of
extra checks to prevent privilege elevation during execve(2) are needed:
the presence of a ptracer and the use of a shared fsContext. Moreover,
execve had to be serialized with seccomp(2) when it is syncing filters to
all threads in a group. Serializing execve with PTRACE_ATTACH and seccomp
required some Task runState gymnastics.
Task dumpability is reset when privilege elevation takes place and privilege
elevation is denied for binaries living on nosuid mounts. To fix an old bug
where some aux vectors were populated using data from the old pre-execve
creds, the computation of the (file-privilege-augmented) creds during execve
now takes place before the stack for the new program is set up.
gvisor users who thus far have exec'd into the sandbox with UID as (sandbox)
root will lose the caps an all-powerful namespace root'd have if: 1) the binary
is a suid/guid binary 2) the owner of the binary is host root and there is
no mapping for host root in the host user namespace of the sandbox. To avoid
this surprise, the suid/sgid bits are ignored by default. Users can opt-in
via the --allow-suid flag.
One result of this change is that sudo now works inside the sandbox.
PiperOrigin-RevId: 794414101
When using async page loading (the checkpoint image has compression=none), the
kernel and memory files are loaded in parallel. A step in kernel loading tries
writing to the main MemoryFile. Ensure that step happens after the main
MemoryFile has started being loaded (i.e. mainMF.LoadFrom() method has returned
successfully). MemoryFile.asyncPageLoad field needs to be initialized before
it can be written to.
Fixes#11842
PiperOrigin-RevId: 779995419
- During restore if endpoints' states are not loaded before starting
dispatchers, then the incoming packets will be queued and the dispatcher will
try to send the packets, which will result in a panic as the endpoint might
still be in an invalid state (eg: initial).
- To avoid this, wait for all the endpoints to be loaded completely before
starting the dispatchers by calling AsycnLoading.Wait() in stack.Restore() and
remove this from kernel.LoadFrom()
- Added a test to check save restore of multiple(100) listening sockets.
PiperOrigin-RevId: 762010964
All the following changes are compatible with what Linux does.
- Do not attempt to apply file capabilities when it is not present.
- If the rootID specified in v3 file capability struct does not own the
current userns, then file capabilities are not applied.
- While applying file capabilities, EPERM is only returned if
VFS_CAP_FLAGS_EFFECTIVE is set.
- Re-raise capabilities of root user as per capabilities(7). Compare
security/commoncap.c:handle_privileged_root().
PiperOrigin-RevId: 761725759
This allows us to use the generated `Marshal/Unmarshal` functions and get rid
of the manual binary encoding/decoding.
Also fixed up vfs.VfsCapDataOf() to emulate Linux behavior more accurately.
It now returns linuxerr errors instead of fmt.Errorf()s, because in future
changes this code will be used on syscall paths.
PiperOrigin-RevId: 754101576
This change does not change behavior, but improves readability.
- The PagesFileLoader interface methods were over-fitted to some of their
implementations. For instance, KickoffPrivate() was very specific to the
implementation of separatePagesFileLoader and the timing of its creation.
In PagesFileLoader.Load(), mainMf parameter was only needed by
singleStateFilePFL implementation and background parameter was only needed
by separatePagesFileLoader implementation. It is tough to build a clean
interface around the 2 kinds of page loading we support without leaking
implementation details to the API.
- It was not readable/obvious why Kernel.LoadFrom() was calling
pfl.KickoffPrivate() in the beginning of the function. That call depends on
the fact that main MemoryFile load was already started in the background
before the call to Kernel.LoadFrom(). Then the call to pfl.Load() was
confusing, especially because it came after the call to KickoffPrivate().
- singleStateFilePFL did not own the reader (state file FD) while
separatePagesFileLoader owned and closed the FDs it used to restore the
MemoryFiles. This created another asymmetry between the two. The FD
ownership details could not be captured in the interface.
- Removed the interface. Replaced it with concrete type AsyncMFLoader. This
has the added benefit of avoiding interface method dispatch cost. Only when
AsyncMFLoader == nil (when MemoryFile contents are in the state file) does
Kernel.LoadFrom() try loading the memory files using the conventional method
of loading them from the state file synchronously. This makes sense and is
readable because all consumption from the state file FD happens in one place
in a transparent manner.
- separatePagesFileLoader.KickoffPrivate() was being called multiple times due
to separatePagesFileLoader.Load() also calling it for completeness. Now
runsc restorer calls it explicitly only for AsyncMFLoader and only calls it
once. It allows us to get rid of loadPrivateMFs sync.Once.
- AsyncMFLoader provides clear WaitMetadata() and WaitPages() functions. These
are more readable.
Future work: I think all of MemoryFile saving/loading logic should be moved to
the pgalloc package. It has not dependency on Kernel save/load. Kernel and
MemoryFile save/load happen independently in discrete steps irrespective of
save/load configurations.
PiperOrigin-RevId: 745187478
This change modifies the sandbox restore process such that if the memory
pages are stored as separate files, their contents are optimistically read
from these files as soon as possible during the sandbox restore process,
as opposed to the previous behavior where they were read only after the last
container of the sandbox had been restored.
When memory page data is stored as separate files, the process of restoring
multiple `MemoryFile`s is a bit more intertwined than it appears, for two
reasons:
- The pages metadata file stores data for all `MemoryFile`s one after the
other. So we must also plumb the pages metadata file reader that is used
for reading the main `MemoryFile` metadata to the code that also loads the
private `MemoryFile`s, and ensure that these operations are correctly
serialized, since the primary `MemoryFile` metadata comes earlier in the
file.
- The `MemoryFile` page contents are all in the same file, and the byte offset
within this file at which each each `MemoryFile` begins must be known before
kicking off async loading of that MemoryFile.
So this change introduces a `PagesFileLoader`, which is meant to be a
thread-safe struct that keeps track of this process. It owns the FDs for the
pages and page metadata files and handles the serialization of the operations
executed using these FDs.
Support for the previous checkpoint/restore format (with all data in a single
file) is maintained using a synchronous alternative that implements the same
interface but simply loads everything from the shared state file reader when
the time to do so comes around.
PiperOrigin-RevId: 743727664
This CL restores the listening connections when netstack s/r is enabled.
The changes include:
- New method as a workaround to replace the new routes and nics to the loaded
stack after restore.
- New Restore() for transport layer protocols to restore the protocol level
background workers.
- Adds afterLoad() method for fdbased processors.
- Adds a test to verify listening connection is restored after checkpointing
with netstack s/r enabled.
- Few other changes to save restore fields to enable netstack s/r.
PiperOrigin-RevId: 698453124
- Move ktime.VariableTimer to kernel.timekeeperTcpipTimer, its only use case.
This allows timekeeperTcpipTimer to use concrete types kernel.timekeeperClock
and ktime.SampledTimer instead of ktime.Clock and ktime.Timer, saving a tiny
amount of memory (interface values consist of two pointers) and CPU (for
interface method calls).
- Fix a bug where timekeeperTcpipTimer expiration can cancel a racing call to
timekeeperTcpipTimer.Reset() (see use of new field
timekeeperTcpipTimer.resets).
- Define Listener.NotifyTimer directly on timekeeperTcpipTimer (dropping
ktime.functionNotifier), and move goroutine spawning from the anonymous
function in ktime.AfterFunc() into timekeeperTcpipTimer.NotifyTimer(). This
slightly simplifies the control flow and saves an allocation for the
anonymous function object.
- Use monotonicClock rather than realtimeClock. It doesn't make sense for
time-of-day clock adjustments to affect netstack timeouts, and this is
consistent with tcpip.stdClock => time.AfterFunc => runtime.timer.
PiperOrigin-RevId: 695504159
gVisor currently implements CPU clocks as follows:
- A per-sentry "CPU clock ticker goroutine"
(task_sched.go:Kernel.runCPUClockTicker()) periodically advances
Kernel.cpuClock, causing it to serve as a very coarse but inexpensive
monotonic wall clock (that happens to be suspended when no tasks are
running).
- Task goroutines observe the most recent value of Kernel.cpuClock when
changing state (Task.gosched.Timestamp), and use it to compute the number of
CPU clock ticks that have elapsed in a given state. Thus, task CPU clocks are
approximately based on the wall time during which they were marked as
running.
- ITIMER_VIRTUAL, ITIMER_PROF, and RLIMIT_CPU are checked by the CPU clock
ticker goroutine after advancing Kernel.cpuClock. POSIX interval timers and
timerfds check CPU clocks (taskClock/tgClock) in ktime.SampledTimer
goroutines.
This has three major problems:
- ktime.SampledTimer goroutines for CPU clock timers run concurrently with the
CPU clock ticker, and are not informed as to when corresponding tasks start
or stop running (due to overhead on the task execution critical path), so
they can't determine when CPU clocks have/will advance; instead, they simply
poll CPU clocks on a period equal to that of the represented timer, resulting
in significant overhead for CPU-clock-based POSIX interval timers and
timerfds.
- For the same reason, CPU clock interval timers and timerfds may expire much
later than when the CPU clock is actually incremented; in the interval timer
case, this can result in notification signals being sent long after tasks
have stopped running. (This is the same problem as in b/116538398, which
motivated the special-casing of ITIMER_VIRTUAL and ITIMER_PROF described
above, but applied to POSIX interval timers.)
- The sentry does not impose a limit on the number of tasks that may be
concurrently marked running, so if more tasks are marked running than the
number of CPUs advertised to applications, application CPU utilization can
appear to exceed 100%.
This CL fixes these problems by introducing explicit per-Task and ThreadGroup
CPU clocks, directly advancing (up to Kernel.applicationCores of) them in the
CPU clock ticker, and directly expiring CPU timers when doing so. Itimer and
RLIMIT_CPU timers lose their special-casing and instead behave like other CPU
timers (see task_acct.go). Kernel.cpuClock is still required, but only for the
sentry watchdog.
Minor cleanup changes:
- Gather all stateify hooks in kernel_state.go.
- Replace kernel.randInt31n() with math/rand/v2, which fixes the same problem
(https://go.dev/blog/randv2#problem.rand).
Test workload:
```
#include <err.h>
#include <signal.h>
#include <time.h>
#include <chrono>
#include <thread>
constexpr int kNumTimers = 1000;
constexpr long kTimerPeriodNS = 10000000;
int main(int argc, char** argv) {
for (int i = 0; i < kNumTimers; i++) {
struct sigevent sev = {.sigev_notify = SIGEV_NONE};
timer_t timerid;
if (timer_create(CLOCK_THREAD_CPUTIME_ID, &sev, &timerid) < 0) {
err(1, "timer_create failed");
}
struct itimerspec it = {
.it_interval = {0, kTimerPeriodNS},
.it_value = {0, kTimerPeriodNS},
};
if (timer_settime(timerid, 0, &it, nullptr) < 0) {
err(1, "timer_settime failed");
}
}
std::this_thread::sleep_for(std::chrono::seconds(5));
return 0;
}
```
Before this CL:
```
# /usr/bin/time ./runsc --ignore-cgroups --platform kvm --network none do $(pwd)/workloads/threadcputimers
1.50user 0.17system 0:05.25elapsed 31%CPU (0avgtext+0avgdata 35792maxresident)k
0inputs+184outputs (10major+20889minor)pagefaults 0swaps
```
After this CL:
```
# /usr/bin/time ./runsc --ignore-cgroups --platform kvm --network none do $(pwd)/workloads/threadcputimers
0.10user 0.12system 0:05.22elapsed 4%CPU (0avgtext+0avgdata 34040maxresident)k
0inputs+192outputs (6major+20929minor)pagefaults 0swaps
```
PiperOrigin-RevId: 695198313
- Rename Timer to SampledTimer.
- Move all Clock methods except Now to new interface SampledClock.
- Move SampledTimer's exported methods (except SetClock) to new interface
Timer. Combine Swap and SwapAnd into Set to reduce the number of redundant
methods that must be implemented.
- Add interface method Clock.NewTimer.
This is in preparation for cl/693856539, which adds a second Timer
implementation.
PiperOrigin-RevId: 694299679
We used to track the foreground process group & session on the
TTYFileOperation, but these are already tracked in kernel.TTY.ThreadGroup.
So remove TTYFileOperations.fgProcessGroup and .session, and replace them with
a kernel.TTY.
This is analogous to how sentry-internal tty's already work.
Updates #10925
PiperOrigin-RevId: 681957240
This allows "remote" locking of ThreadGroup.signalHandlers.mu without needing
to lock TaskSet.mu, analogously to Linux's lock_task_sighand().
This reveals a bug: kernel.Task.sendSignal[Timer]Locked() unintentionally
requires TaskSet.mu to be locked since it reads Task.exitState. To fix this,
use atomic memory operations on Task.exitState when required.
PiperOrigin-RevId: 681128063
When a pages file is provided to `runsc restore`, reads from that file are
asynchronous (via statefile.AsyncReader) in order to maximize throughput.
However, all such reads must complete before Kernel.LoadFrom() returns, so
applications cannot execute before MemoryFile loading is complete. The main
objective of this CL is to allow reads to continue after Kernel.LoadFrom()
returns, allowing applications to execute while MemoryFile loading is still in
progress. This behavior is user-visible: it affects whether deleting the pages
file frees disk space immediately on POSIX filesystems, may affect whether
deletion is possible on non-POSIX filesystems, and prevents unmounting
regardless. Thus it is flag-guarded as `runsc restore --background`.
MemoryFile ranges that have yet to be loaded, but that are being waited-for by
applications, should be prioritized over ranges for which no application is
waiting. This requires that application requests for data (calls to
MemoryFile.(memmap.File).DataFD/MapInternal()) are able to determine which
ranges have not yet been loaded, request reads for such ranges with elevated
priority, and wait for only those reads to be completed; none of these are
supported by the existing statefile.AsyncReader.
Thus:
- Add //pkg/sentry/pgalloc/aio, which provides an async I/O API that is
designed to be easily implementable using a goroutine pool, Linux native AIO,
or io_uring, though only includes a goroutine pool implementation. (io_uring
is widely disabled due to security vulnerabilities. In my testing, Linux
native AIO is slower than the goroutine pool, but this may change with lower
GOMAXPROCS which needs further testing.)
- Move I/O scheduling into pgalloc: introduce an async page loader goroutine
that is started by MemoryFile.LoadFrom() when async page loading is requested
(implicitly, via the existence of a pages file), which is responsible for
driving submission of read requests and handling their completions.
PiperOrigin-RevId: 679321884
- Adds a new flag which will enable netstack s/r. When the flag is not enabled,
there is no change in the existing behavior. The flag will be enabled only in
tests to verify the s/r functionality of netstack.
- Some additional fields in netstack were causing panic when netstack is
save/restored. Such fields are marked as 'save'/'nosave' accordingly to resolve
the panic.
PiperOrigin-RevId: 668566657
This compliments f282f9775d ("Load pgalloc.MemoryFile and kernel parallely
with compression=none mode.")
This is a performance optimization. Kernel and MemoryFile are saved
independently. The save can be done in parallel when using --compression=none
because the kernel and MemoryFile are being saved in different files.
PiperOrigin-RevId: 668128205
