Commit Graph
70 Commits
Author SHA1 Message Date
Kevin KrakauerandgVisor bot 11dc870157 Ensure ip6tables module installed
This module isn't always loaded automatically.

PiperOrigin-RevId: 317164471
2020-06-18 13:06:50 -07:00
Kevin KrakauerandgVisor bot fb6b2cfcb1 Invoke bazel query via bash function.
PiperOrigin-RevId: 315514034
2020-06-09 10:48:24 -07:00
Adin ScannellandgVisor bot e08aab967d Move to make for tag release workflow.
This will make tag & release workflows idempotent.

PiperOrigin-RevId: 314154888
2020-06-01 10:31:26 -07:00
Adin ScannellandgVisor bot 798329a10b Fix the APT repository structure.
This change fixes the apt repository structure to avoid emiting warnings on
Ubuntu 18.04 (and potentially other versions). This requires a slight refactor
of the repository generation scripts, since we can no longer copy the same
release files for different "suites".

This should avoid the warning by setting the suite to the distribution:
  https://github.com/Debian/apt/blob/master/apt-pkg/metaindex.cc#L75

This change also moves over to the standardized Makefile entrypoint, which
makes settings clearer and enables local testing.

PiperOrigin-RevId: 313817017
2020-05-29 11:30:02 -07:00
Andrei VaginandgVisor bot 5775135c99 Allow to run kvm syscall tests on the RBE cluster
PiperOrigin-RevId: 309265978
2020-04-30 11:33:07 -07:00
Zach KoopmansandgVisor bot 7d4992e621 VFS2: Get HelloWorld image tests to pass with VFS2
This change includes:
- Modifications to loader_test.go to get TestCreateMountNamespace to
pass with VFS2.
- Changes necessary to get TestHelloWorld in image tests to pass with
VFS2. This means runsc can run the hello-world container with docker
on VSF2.

Note: Containers that use sockets will not run with these changes.
See "//test/image/...". Any tests here with sockets currently fail
(which is all of them but HelloWorld).
PiperOrigin-RevId: 308363072
2020-04-24 18:23:37 -07:00
Adin ScannellandgVisor bot e4cf551982 Standardize all Docker images.
This change moves all Docker images to a standard location, and abstracts the
build process so that they can be maintained in an automated fashion. This also
allows the images to be architecture-independent.

All images will now be referred to by the test framework via the canonical
`gvisor.dev/images/<name>`, where `<name>` is a function of the path within the
source tree.

In a subsequent change, continuous integration will be added so that the images
will always be correct and available locally.

In the end, using `bazel` for Docker containers is simply not possible. Given
that we already have the need to use `make` with the base container (for
Docker), we extend this approach to get more flexibility.

This change also adds a self-documenting and powerful Makefile that is intended
to replace the collection of scripts in scripts. Canonical (self-documenting)
targets can be added here for targets that understand which images need to be
loaded and/or built.

PiperOrigin-RevId: 308322438
2020-04-24 14:11:42 -07:00
Andrei VaginandgVisor bot c171df4aae kokoro: save all files from test.outputs/
If a test fails by timeout, bazel doesn't generate outputs.zip.

PiperOrigin-RevId: 308307815
2020-04-24 12:52:49 -07:00
Adin ScannellandgVisor bot 1b8133d9f9 Simplify Docker test infrastructure.
This change adds a layer of abstraction around the internal Docker APIs,
and eliminates all direct dependencies on Dockerfiles in the infrastructure.

A subsequent change will automated the generation of local images (with
efficient caching). Note that this change drops the use of bazel container
rules, as that experiment does not seem to be viable.

PiperOrigin-RevId: 308095430
2020-04-23 11:33:30 -07:00
Adin ScannellandgVisor bot 092bc1555d Remove unnecessary kokoro configurations.
PiperOrigin-RevId: 307841689
2020-04-22 10:15:10 -07:00
Ting-Yu WangandgVisor bot 8cf99fbc32 Fix release.sh. git commands need to be run in git repo.
PiperOrigin-RevId: 307458938
2020-04-20 12:58:32 -07:00
Adin ScannellandgVisor bot 60c9a488e8 Move runtime_tests.sh to align with other scripts.
PiperOrigin-RevId: 307435879
2020-04-20 11:08:59 -07:00
Andrei VaginandgVisor bot 75032705f3 kokoro: fix handling of apt-get errors
When a command is called as if expression, its error
code can be get only in this if block.

For example, the next script prints 0:

if ( false ); then
  true
fi
echo $?

PiperOrigin-RevId: 305638629
2020-04-09 01:12:27 -07:00
Zach KoopmansandgVisor bot 2002b366f7 Fix typo in benchmarks.sh
PiperOrigin-RevId: 304440599
2020-04-02 11:25:26 -07:00
Zach KoopmansandgVisor bot 9d98effd99 Add "/snap/bin" to PATH.
"gcloud" may be installed as a snap, under "/snap/bin". Make
sure this is in our PATH so that we can use gcloud.

PiperOrigin-RevId: 304297180
2020-04-01 17:40:19 -07:00
Zach KoopmansandgVisor bot 3928ba699a Debug script issues on Kokoro.
PiperOrigin-RevId: 304068950
2020-03-31 16:17:56 -07:00
Zach KoopmansandgVisor bot dc6a5ad433 Internal change.
PiperOrigin-RevId: 303773475
2020-03-30 10:44:55 -07:00
Kevin Krakauer 2f7f08f724 iptable: fix tests timeouts
Tests were run assuming a runtime of "runsc" was present, and did not
have --net-raw enabled.
2020-03-25 20:37:48 -07:00
Eyal SohaandgVisor bot e61032cb76 Add packetimpact tests to presubmit and CI testing
PiperOrigin-RevId: 301872161
2020-03-19 12:33:29 -07:00
Zach KoopmansandgVisor bot 516ab159fd Internal change
PiperOrigin-RevId: 300836270
2020-03-13 15:18:14 -07:00
Zach KoopmansandgVisor bot ee54a29401 Run "startup" benchmark in Kokoro benchmark job.
PiperOrigin-RevId: 300798423
2020-03-13 11:51:50 -07:00
Kevin Krakauer 33481c2876 iptables: ready tests to be enabled in kokoro
Fixed flakes (tested via --runs_per_test=100) and added skips for
not-yet-implemented features. Once submitted, the iptables tests will be
ready to enable in kokoro.
2020-03-11 15:13:58 -07:00
Zach KoopmansandgVisor bot 24b08767d7 Internal change
PiperOrigin-RevId: 300410856
2020-03-11 14:46:17 -07:00
Fabricio VoznikaandgVisor bot 290c03fab8 Fix runsc permissions for "dev.sh --refresh"
PiperOrigin-RevId: 299466906
2020-03-06 17:01:51 -08:00
Adin ScannellandgVisor bot 6c8285cae1 Fix apt-get reliability issues.
This is frequently causing the core build scripts to fail. The core ubuntu
distribution will perform an auto-update at first start, which may cause the
lock file to be held. All apt-get commands may be done in a loop in order to
retry to avoid this issue. We may want to consider retrying other pieces, but
for now this should avoid the most frequent cause of build flakes.

PiperOrigin-RevId: 297704789
2020-02-27 15:36:13 -08:00
Adin ScannellandgVisor bot 4741395298 Use automated release notes, if available.
PiperOrigin-RevId: 297628615
2020-02-27 10:22:41 -08:00
Brad BurlageandgVisor bot 30a1db64bc Internal change.
PiperOrigin-RevId: 296972565
2020-02-24 14:22:32 -08:00
gVisor botandCopybara-Service 0f62ba5caf Internal change.
PiperOrigin-RevId: 296055452
2020-02-19 14:18:11 -08:00
gVisor bot be9266e218 Minor runtime test fixes.
* Allow scripts/common.sh to be sourced from outside the scripts/ directory
* Fix passing empty args to Bazel, which causes the tool to exit with a failure
  even if the command succeeds.

PiperOrigin-RevId: 294785456
2020-02-12 16:27:07 -08:00
Brad BurlageandgVisor bot 0898b61665 Fix undeclared variable error in common_build.sh.
PiperOrigin-RevId: 293413711
2020-02-05 11:34:46 -08:00
Eyal SohaandgVisor bot 0923242686 Add packetdrill tests to presubmit and CI testing
PiperOrigin-RevId: 293409718
2020-02-05 11:18:02 -08:00
Brad BurlageandgVisor bot 61fc256c41 Add 1 Kokoro job per runtime test.
PiperOrigin-RevId: 293019326
2020-02-03 15:56:57 -08:00
Kevin KrakauerandgVisor bot 6be1b02744 Fix iptables tests that were broken by rename.
The name of the runner binary target changed from "runner" to "runner-image",
causing iptables tests to fail.

PiperOrigin-RevId: 292242263
2020-01-29 16:27:12 -08:00
Adin ScannellandgVisor bot 5cd5c2536a Standardize on tools directory.
PiperOrigin-RevId: 291745021
2020-01-27 12:21:00 -08:00
Brad BurlageandgVisor bot 9c29b13a63 Try running kythe build on RBE.
Also add our RBE project/instance to the --config=remote defaults.

PiperOrigin-RevId: 291201426
2020-01-23 11:01:38 -08:00
Adin ScannellandgVisor bot bc875d439d Fix master build case.
Otherwise, this will be built when building a specific
release tag (typically in the past), causing the master
binary to be overwritten with something older.

We can generally assume that tags will be applied after
the commit has been integrated, and therefore that any
builds pointing to tags will use only the tags.

Another way to fix this would be to introduce something
akin to the KOKORO_BUILD_NIGHTLY environment variable,
but it doesn't seem strictly necessary.

PiperOrigin-RevId: 291198171
2020-01-23 10:45:48 -08:00
Tamir DubersteinandgVisor bot 0eed6bd2cc Allow dual stack sockets to operate on AF_INET
Fixes #1490
Fixes #1495

PiperOrigin-RevId: 289523250
2020-01-13 14:47:22 -08:00
Fabricio VoznikaandgVisor bot e67320901b Configure issue reviver to run with Kokoro
PiperOrigin-RevId: 288921032
2020-01-09 10:17:54 -08:00
Kevin KrakauerandgVisor bot 4f801a39f9 Add iptables testing framework.
It would be preferrable to test iptables via syscall tests, but there are some
problems with that approach:

* We're limited to loopback-only, as syscall tests involve only a single
  container. Other link interfaces (e.g. fdbased) should be tested.
* We'd have to shell out to call iptables anyways, as the iptables syscall
  interface itself is too large and complex to work with alone.
* Running the Linux/native version of the syscall test will require root, which
  is a pain to configure, is inherently unsafe, and could leave host iptables
  misconfigured.

Using the go_test target allows there to be no new test runner.

PiperOrigin-RevId: 285274275
2019-12-12 14:42:11 -08:00
Adin ScannellandgVisor bot 004c2a166e Mark runner_test as manual.
Because it is local-only, it should also be marked manual.

PiperOrigin-RevId: 284596186
2019-12-09 11:28:41 -08:00
Fabricio VoznikaandgVisor bot 91c615a440 Update Kokoro image to install Golang 1.13
PiperOrigin-RevId: 284308422
2019-12-06 20:13:09 -08:00
Adin ScannellandgVisor bot 25b09ce49b Add runtime tracing.
This adds meaningful annotations to the trace generated by the runtime/trace
package.

PiperOrigin-RevId: 284290115
2019-12-06 17:00:07 -08:00
Zach KoopmansandgVisor bot 1a370536ab Import benchmark-tools to main repository.
This has adapted for use with bazel from the original commit
a26e93769ebefd82593a43e22fb13a09717cfa6d.

In particular, the style has been made consistent with internal python style
guidelines, and the packages (including the main entrypoint) have been
refactored in order to allow bazel testing targets.

PiperOrigin-RevId: 283484433
2019-12-02 22:53:26 -08:00
Adin ScannellandgVisor bot 51c50b8850 Use a GitHub credential for tagging a release.
PiperOrigin-RevId: 281617882
2019-11-20 15:39:34 -08:00
Andrei VaginandgVisor bot 82608f1bf1 kokoro: correct a path to outputs.zip
PiperOrigin-RevId: 280021914
2019-11-12 12:01:37 -08:00
Andrei VaginandgVisor bot ab6f98051c test: merge log files of all shards for each test suite
This significantly speeds up a process of uploading this files
to sponge and resultstore by kokoro.

PiperOrigin-RevId: 279416349
2019-11-08 16:40:51 -08:00
Adin ScannellandgVisor bot 745a66bcc8 Fix repository build scripts.
This fixes a number of issues with the repository build process:

 * Fix the overall structure of the repository.
 * Fix the debian package description.
 * Fix the broken version number for packages.
 * Update the digest algorithm used for signing the release.

I've validated that installation works from a separate staging bucket.

Updates #852

PiperOrigin-RevId: 278716914
2019-11-05 15:16:04 -08:00
Andrei VaginandgVisor bot bbc4e2c9a1 kokoro: run KVM syscall tests
We don't know how stable they are, so let's start with warning.

PiperOrigin-RevId: 278484186
2019-11-04 16:00:34 -08:00
Brad BurlageandgVisor bot 0dc3688b6b Fix overloaded use of $RUNTIME.
Turns out we use $RUNTIME in scripts/common.sh to give a name to the runsc
runtime used by the tests.

PiperOrigin-RevId: 277764383
2019-10-31 11:28:18 -07:00
Brad BurlageandgVisor bot aad0ede339 Add Kokoro config for new runtime tests
PiperOrigin-RevId: 277607217
2019-10-30 16:16:15 -07:00