We now make sure that the requested domain is a valid domain based on the
custom domain and project ID settings before redirecting.
PiperOrigin-RevId: 548859553
Attempt to fix flaky port forward test using "netstat" in the container
to check the a process is listening on the port before attempting to
connect.
Also, re-enable the test in a seperate job, but w/ a soft fail. This
will be removed later in favor of adding port-forward to integration
tests to remove the high overhead of starting a new job (scheduling,
compiling runsc, downloading images, etc).
PiperOrigin-RevId: 526135859
Port forward stream mode will be used for k8s. Stream mode takes a
UNIX socket and donates a connected FD to the sandbox in question,
and returns. No proxy process is kept around to do the transfer
and the UDS will be returned to containerd as the address on which
to send i/o to the forwarded port.
PiperOrigin-RevId: 521823250
Add portforward comand so that we can use runsc to forward connections
to container ports. This will eventually be supported in k8s.
PiperOrigin-RevId: 520739913
We plan on making --overlay2=root:self the default for runsc. That will be a
risky change which might need rollbacks. This change is in preparation for
that. We manually set --overlay2=none in places where we don't want the
overlay configuration to impact. This change should be a noop. The intention
for this change is to make the risky change very small and limited to just
flipping a flag.
PiperOrigin-RevId: 513930702
With the latest nogo change, facts are no longer in the build paths. Since full
is only used to speed these paths (and is enabled for full analysis) the full
flag is now redundant and can be removed safely.
PiperOrigin-RevId: 511935530
This script was being run as root. This script clones containerd and builds it.
On some Ubuntu versions, sudo(1) does not change the home directory. On
oldkernel machines from BuildKite, this script was running with
$HOME=/home/agent. As a result, root was writing and creating files in
/home/agent/.cache/go-build/. Other tests that call `go build` that were
scheduled to run on such VMs would fail due to permission issues because
/home/agent/.cache/go-build/ is owned by root.
Run the tools/install_containerd.sh script sudo -H so that the home directory
is set appropriately and all packages are built and installed with root user's
cache in /root/.cache/.
Also fixed lint errors in tools/install_containerd.sh and got rid of redundant
sudo calls. Added the check that the script must be run as root.
Co-authored-by: Andrei Vagin <[email protected]>
Acknowledgements: Andrei Vagin discovered this issue.
PiperOrigin-RevId: 511661564
The systrap platform like the ptrace platform uses stub processes to manage
the user address space. The difference is how they intercept system calls and
other events like memory faults, exceptions, etc.
In case of systrap, all events that have to be handled by the Sentry trigger
signals that are handled by a custom signal handler installed on stub
processes. The signal handler switches control to the Sentry.
Here are a few other optimizations:
* On x86, system calls can be replaced with a function call to remove overhead
of signals.
* For fast interactions of sentry and stub processes, futex wait/wake can
be a bottle neck, so we use a polling mode.
The platform is launched for the purpose of testing and gathering initial
feedback. It is not yet ready for use in production.
PiperOrigin-RevId: 511650064
This medium instructs runsc to place the overlay-backing host file inside the
mount being overlayed itself. The file is placed inside the root directory of
the mount at path `.gvisor.overlay.img.{CID}/filestore-{random}`.
Note that the same volume may be mounted at various places in the same
container. Furthermore, the same volume may be mounted by multiple containers
in the same pod. So we create a filestore directory in the mount which is
unique to each container. This filestore directory hosts all the filestore
files for the mount for the given container. This directory is destroyed along
with the contianer on `runsc delete`.
The self medium based host file is a named file because it needs to be
discoverable via path traversal from the host. For the root mount, this allows
k8s to scan the container's root filesystem and apply local ephemeral storage
limits appropriately. This makes self medium appropriate for k8s usage.
Note that the overlay host file (MemoryFile) is a sparse file. Thankfully k8s
uses stat.st_blocks to perform accounting, not stat.st_size. stat.st_blocks
indicates the number of pages allocated towards a file, which works well with
sparse files.
PiperOrigin-RevId: 507301254
1e2bc64038 ("Create separate overlay filestore for subcontainers.") moved
the ownership of pgalloc.MemoryFile to tmpfs mount (upper layer in overlay).
However, this also means that each tmpfs mount needs its own filestore file.
Currently if there are multiple overlay mounts in a container that are backed
by the same host file. Since they use different MemoryFiles, they will
overwrite each other.
This change creates a separate overlay file per overlay mount per container
and plumbs the FDs to their respective tmpfs mounts. This is similar to how
gofer FDs are dealt with. Added a regression test for this too.
PiperOrigin-RevId: 507289704
These are made obsolete by the new add_fusefs attribute. There is
no reason to keep them around, they were already disabled.
PiperOrigin-RevId: 506440841
This is to make it clear that the host file will be created inside this
directory. This also makes it look cleaner when other medium options are added
later.
PiperOrigin-RevId: 505033408
Linux overlayfs uses the max of all layers' filename length limit. Do the same
in gVisor.
This is needed to get PHP runtime test ext/standard/tests/strings/007.phpt to
pass with overlayfs.
PiperOrigin-RevId: 493942515
--overlay2 flag supersedes --overlay flag. It allows more granular
configuration for overlayfs in runsc. It does so in two ways:
1. Allows to apply overlay on all mounts or only the root mount.
--overlay applies overlay to all mounts.
2. Allows to specify if overlay's upper layer should be backed by
container memory or disk. --overlay always used container memory.
Allowing tmpfs to be backed by a file on disk prevents the container
memory from bloating up. Note that the tmpfs filesystem tree will
still be stored in sentry memory.
Using overlay on the root filesystem, helps avoid expensive
communication with the gofer process. The root filesystem of the
container is not preserved across container lifecycle. So we don't
need to keep updating the host filesystem, which will anyways be
destroyed once the container is destroyed. It is wasted effort.
Instead we keep all the changes to the root filesystem in tmpfs which
is directly accessible by the sentry.
The host file is created as an unnamed file using O_TMPFILE. Support
has been added for sub-containers too. Save/restore support is
still lacking.
Co-authored-by: Andrei Vagin <[email protected]>
PiperOrigin-RevId: 491988485
Serves as a regression test for #8126.
Our unit tests didn't catch the issue because they run runsc with the flag
-TESTONLY-unsafe-nonroot. Docker tests are more e2e, they run tests in Docker
containers.
Fixes#8126
PiperOrigin-RevId: 488945922
Without this, attempting to build or test gVisor on machines where the default
runtime isn't `runc` may fail, as that runtime may not support what we need.
PiperOrigin-RevId: 486205081
These versions are compatible with Go 1.18. There are a lot of OSS bugs
about this issue. Notable discussions:
- golang/go#51285
- containerd/containerd#6586
This was fixed upstream in the main branch in containerd/containerd#6605.
The fix was back-ported to v1.5 and v1.6 in containerd/containerd#6716
and containerd/containerd#6717 respectively.
These backports are available in the following versions:
```shell
/tmp/gopathJJGZf/containerd$ git tag --contains 765df66099eec88d0365eaa1e9a933877a058f0b
v1.6.2
v1.6.3
v1.6.4
v1.6.5
v1.6.6
v1.6.7
v1.6.8
/tmp/gopathJJGZf/containerd$ git tag --contains 86bec213720b54bf9ea2813e2c2a577271af58d3
v1.5.11
v1.5.12
v1.5.13
```
So use the smallest versions out of them.
PiperOrigin-RevId: 480965567
Made some other changes:
- We no longer generate a separate runtime for p9 on `make
dev`.
- We no longer run p9 runtime tests. Instead run lisafs runtime
tests on presubmits.
- All tests are generated with lisafs. Only 1 p9 variant is
generated with default platform.
- Run integration tests only with lisafs.
P9 is now considered deprecated.
Use `--lisafs=false` if you need to disable it. Make sure to
report a bug if you have the need to disable LISAFS.
Fixes#6319
PiperOrigin-RevId: 469548694
This is useful when wanting to run benchmarks in a way where the output can
be captured to a file, such that this file only contains one platform's
specific performance data.
PiperOrigin-RevId: 468061772
This adds flags and env variables for the following settings to runtime tests:
- `per_test_timeout`: A per-test timeout which can be shorter
than the batch timeout. Useful to cap the duration of tests
which flake by getting stuck (as is the case for
`bug60120.phpt`)
- `runs_per_test`: Number of times to run each test (useful to
detect flakes).
- `flaky_is_error`: Controls whether a flaky test is considered
passing or failing for batch error code purposes. Useful when
either diagnosing a flaky test, or diagnosing a
consistently-failing test while bypassing flaky others.
- `flaky_short_circuit`: If a test is found to be flaky, declare
it as such immediately, rather than waiting for the rest of the
`--runs_per_test` to finish. Speeds up bisecting flaky test, at
the cost of flakiness percentage accuracy.
PiperOrigin-RevId: 466791296
When running in COS in a container, things are different when accessing
process or cgroup data.
This CL uses the `HOST_PROCFS_MOUNTPOINT` and `HOST_CGROUPFS_MOUNTPOINT`
environment variables as alternate roots for `/proc` and `/sys/fs/cgroup` in
`root_test` and in other related places.
Also includes some minor refactors:
- Factor out code to get the parent PID of a child PID to its own function
- Fix some typos
PiperOrigin-RevId: 457102457
k8s.gcr.io/busybox:latest doesn't have the arm variant, so
let's use the upstream busybox and create /etc/recolv.conf symlink.
PiperOrigin-RevId: 450059917
nogo:fast is converted to nogo:full with the opposite meaning.
All nogo tests have to be always executed with this option.
Analyzing Go Standard Library takes about 10 minutes on buildkite,
but it is required only to run nogo tests.
PiperOrigin-RevId: 438343203
proctor is running in a sandbox and it executes tests. If it is able to
handle timeouts, we will know that a test sandbox is alive and proctor
will send SIGTERM to test processes and collect all logs.
In addition, these changes contains a few things:
* upload runsc logs with other test artifacts.
* increase log level for java tests.
* call Fatalf instead of Errorf when we want to terminate a test.
PiperOrigin-RevId: 437385756
make run-benchmark should not install a new runtime. Instead, just run the
benchmark with the given RUNTIME.
Move all of the logic to install the runtime to benchmark-platforms.
PiperOrigin-RevId: 434838884
- Generates a lisafs variant of all syscall tests.
- Adds Makefile targets to run runtime tests with lisafs. We run runtime tests
continuously (on every commit to master) and on presubmits iff the commit
message contains "lisafs".
- Updates the docker tests target to run the integration tests with lisafs too.
Fixes#5466
PiperOrigin-RevId: 433782954
This will make deadlocks more apparent in test results. Currently runtime tests
just timeout after 30 mins if there is some deadlock.
PiperOrigin-RevId: 433654750
