- Adds methods to Enter, Leave and Migrate the tasks for memory controller.
- Update the memory cgroup id when the task enters and leaves the cgroups.
PiperOrigin-RevId: 555568875
task.netns is always changed from a task goroutine under task.mu.
It means that we can access it without any locks from a task goroutine
we don't need to increment a reference counter in such cases.
In all other cases, we need to take task.mu.
PiperOrigin-RevId: 552913323
This change implements only the basic functions of mount namespaces.
All features that depends on user namespaces will be implemented separately.
PiperOrigin-RevId: 552673896
This change introduces the nsfs file system. Each new namespace allocates
a new nsfs inode.
Here are reasons why we need these inodes:
* each namespace has to have an unique id.
* proc/pid/ns/ contains one entry for each namespace. Bind mounting one of
the files in this directory to somewhere else in the filesystem keeps the
corresponding namespace alive even if all processes currently in
the namespace terminate.
* setns() allows the calling process to join an existing namespace specified
by a file descriptor.
PiperOrigin-RevId: 550694515
Syzkaller came up with workloads that fallocate(2) 1 TB in /tmp. The host
mlock(2) or madvise(2) syscalls on memfd(2) files end up hanging for multiple
minutes in such situations causing the watchdog to mark the calling goroutine
as stuck. memfd(2) files have not size limits.
Linux fails such fallocate(2) attempts in /tmp with ENOSPC.
In Linux tmpfs (shmem), when size= mount option is not specified, the default
size limit for the mount is set to 50% of physical RAM size. But in gVisor, it
is set to MaxInt64. Which is why Linux fails with ENOSPC and gVisor doesn't.
In runsc, the physcial RAM size is already exposed to the containerized
application via `/proc/meminfo` which uses `usage.*TotalMemoryBytes`. These
fields are configured using the `MemTotal:` field from host `/proc/meminfo`.
So use that information to set the default size limit correctly.
Reported-by: [email protected]
PiperOrigin-RevId: 548252095
For memory accounting per task, the memory cgroup id of the task is required.
This CL retrieves the cgroup id from the task context and stores it in the
AllocOpts struct which will later be used to account for memory usage.
PiperOrigin-RevId: 546996195
This CL makes changes to read all the fields from FAsync struct at once to
avoid inconsistent results when FileAsync is changed.
PiperOrigin-RevId: 543528321
This is to aid debugging. The application logs make more sense wrt caller pidns
because the application uses PID numbers from its pidns.
Logs like `stat(/pid/PID/status)` are understandable.
PiperOrigin-RevId: 542884517
t.netns is an atomic pointer so it should not be a problem for readers.
As for writers, only task can change its network namespace.
Reported-by: [email protected]
Reported-by: [email protected]
Reported-by: [email protected]
PiperOrigin-RevId: 542384765
Currently there's no implementation to enable the container to be
initialized to it's cgroups and hence the `EnterInitialCgroups` would
inherit all the root's cgroups when the container would start.
With this implementation we make sure that when the container starts
it enter's into the cgroups that is passed down it to from the sandbox.
PiperOrigin-RevId: 540650868
This is necessary in order to avoid a heap allocation when calling
`unimplementedSyscallCounter.Increment`. Prior to this CL, this code actually
allocated a 1-element slice, because the `Increment` method takes in a
variadic number of arguments.
Somehow this had not tripped up the `+checkescape` check at submission time,
but running `nogo` manually confirms the problem. This CL fixes it.
PiperOrigin-RevId: 539737956
FDTable.descriptorTable is a slice of unsafe.Pointer-s and its maximum length
is MaxInt32. It requires up to 16GB of memory. A process can use just a few
descriptors but sets one or more of them to high numbers. In this case,
FDTable.descriptorTable is extended to the maximum size.
The problem here is that go-runtime zeros memory regions when they are reused.
In the case of fdtable, the memory region is 16GB, so it is a time consuming
operation. Second, it forces the kernel to allocate physical pages to
the entire region.
This change adds another level to descriptorTable, so the first level is
a slice of buckets where each bucket is a slice of descriptors. The bucket
size is fixed to 512 entries to fit one page.
Before:
BenchmarkFDLookupAndDecRef-12 50834290 23.70 ns/op
BenchmarkCreateWithMaxFD-12 2 7194873988 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 23775555 49.68 ns/op
BenchmarkTableLookup-12 412888780 2.835 ns/op
BenchmarkTableMapLookup-12 87944782 12.84 ns/op
After:
BenchmarkFDLookupAndDecRef-12 46229940 25.03 ns/op
BenchmarkCreateWithMaxFD-12 13 82573899 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 21889380 54.13 ns/op
BenchmarkTableLookup-12 415851230 2.821 ns/op
BenchmarkTableMapLookup-12 97236267 11.89 ns/op
Reported-by: [email protected]
PiperOrigin-RevId: 539138632
This cl reverts cl/537942317 and cl/538645255. We need to lock the mutex once
and read all required fields of the FileAsync structure and then release it
right before calling SendSignal.
Right now, getPermAndSignal() and recipient() can return inconsistent results,
because FileAsync can be changed between them.
PiperOrigin-RevId: 538829847
This introduces a `metric.FieldValue` struct type that wraps a string.
All metric interfaces that deal with field values have been updated to use
pointers to this type instead of strings.
The intent of this change is to make it more obvious that field values must
be passed using references. Prior to this change, this was done using string
pointer comparisons. Now this must be done by using a pointer to the same
`metric.FieldValue` struct.
The struct type still externally exposes its string so that it can be referred
to in value function callbacks by "custom" metrics. (Though there are no
current uses of callback metrics with fields.)
PiperOrigin-RevId: 527030738
This optimizes both increments and lookup operations.
It does so using the following:
- For metrics of potential cardinality larger than 48, it will switch to using
a map rather than linear search for mapping field value combinations to the
index that combination corresponds to in the flattened list of values.
For metrics of cardinality 48 or smaller, linear search is still used as it
is still faster (and also still faster than binary search), as determined by
benchmarks.
- All field values are checked for pointer uniqueness (i.e. the pointer to the
start of each field value string must be unique). This is then used for
faster matching: instead of comparing whole strings (and needing to hash
them, in the case of doing map-based lookups), it compares pointer values.
In the context of this metric library, because all field values must be
pre-declared ahead of time, keeping references to these pre-declared strings
should always be possible. This is enforced: it will `panic` if the metric
user does not do this. In practice, this is easy to do by using `const`
strings for all metric values. This change does just that for existing
metrics with string fields.
From benchmarks, this speeds up the time to take a snapshot of existing
metrics by -8.35%. With the unimplemented syscall counter metric, this
optimization reduces the slowdown of adding this metric from +4,250% to
a still-large but much more manageable +290%. A further optimization
(cl/524419591) will reduce this overhead further before re-introducing the
unimplemented syscall counter metric.
PiperOrigin-RevId: 526134756
This catches up the interface to the `EmitUnimplementedEvent` method signature
on `kernel.Kernel`.
Also add build-time test to verify that `kernel.Kernel` implements this
interface, in order to catch such breakages at build time in the future.
PiperOrigin-RevId: 519000411
This will be used to plumb the syscall number through to a counter metric that
exports the number of times an unimplemented syscall has been called.
Plenty of syscall implementations call `EmitUnimplementedEvent` for flags and
settings that are not implemented. With `sysno` available, they will be able
to plumb that bit of information through.
PiperOrigin-RevId: 518635831
There were 2 problems when trying to allocate a high FD value:
- Rlimit is stored as uint64 and could be truncated when converting to int32
to calculate the max value allowed for the FD.
- While trying to double the FD table size, the new length for the table could
end up short due to invalid type convertion again.
Reported-by: [email protected]
PiperOrigin-RevId: 518362257
This is a relatively benign data race that's possible when two of the last
remaining contexts are calling s.unregisterContext at the same time, and both
decrement s.numContexts. One continues to release() the subprocess which puts it
into the available pool, but the other one has yet to read s.released. Then,
since the subprocess is in the available pool it can be retrieved with
fetchAvailable, which did not lock s.mu, which is where the data race happens,
because fetchAvailable flips s.released but one of the exiting contexts has yet
to read it.
The proper fix for this is to start holding references onto the subprocess. We
cannot reuse the subprocess while any systrap.contexts are still holding onto
sysmsg.ThreadContexts, so each context get's a reference. Each subprocess is
held by a MemoryManager as an AddressSpace interface, which calls Release on the
subprocess once.
This CL also removes subprocessPool.active because it is unnecessary at this
moment.
PiperOrigin-RevId: 516272892
This message was supposed to help troubleshoot application that are
killed inside a container. But it can be noisy and doesn't really
help in most cases, e.g. it's expected in many cases that processes
are killed, message refers to PIDs/TIDs which are not actionable
outside the container execution.
PiperOrigin-RevId: 513627281
We no longer store the foreground process directly in the terminal. Instead, we
get it from the terminal TTY's ThreadGroup. Added a new method:
tty.SignalForegroundProcessGroup to simplify this.
Cleaned up some things along the way:
* Terminal had a bunch of methods to get/set foreground process group and
controlling TTY, but those methods were only usable by Ioctl, since they
read/wrote to syscall arguments. I moved that logic to Ioctl, and deleted the
methods from Terminal, which is now a very simple type.
* Fixed a bug in ThreadGroud.SetForegroundProcessGroup where we were
overwriting the ID of an existing process group, rather than setting a new
process group on the session.
* Simplified the construction of lineDiscipline type.
Reported-by: [email protected]
PiperOrigin-RevId: 512330758
