The addition of the "Parameterized" version of iperf requires a filter
so that only the original tests run. Otherwise, we get test timeouts.
PiperOrigin-RevId: 506938451
This CL uses container links rather than going over the host network stack.
This avoids relying on that network stack, making the benchmarks more reliable,
and simplifies benchmark setup code.
#codehealth
PiperOrigin-RevId: 486221868
This fixes `iperf` getting stuck on some versions of Docker, and mirrors the
way other multi-container tests do networking.
PiperOrigin-RevId: 468049041
Currently, iperf runs a client that scales by
bytes sent. In practice, this causes b.N to scale
slowly and have several short lived containers.
Instead, scale by KB to more quickly reach required time.
PiperOrigin-RevId: 358244926
Fix issue with iperf where b.N wasn't changing across runs.
Also, if the given runtime is runc/not given, don't run a profile against it.
PiperOrigin-RevId: 357231450
- Tweak the benchmarks to work with b.N where appropriate. In many cases,
b.N was simply being ignored. This creates an implicit dependency in the
user passing a reasonable benchtime (less than or equal to the actual
runtime of the test, or using the X syntax) otherwise the test runs
forever.
- In cases where the above is impossible, explicitly set benchtime from
the test wrapper, to prevent the above behavior (tensorflow).
- Drop the *Reverse variants, which are simply hey benchmarks. We should
just add a hey benchmark. The platforms benchmarks already include a
native platform, and thus these benchmarks are incredibly confusing.
(In other words, BenchmarkNginxReverse has nothing to do with an nginx
benchmark for runsc.)
- Remove the redunant Harness object, which contains no state, in order
to slightly simplify the code.
- Make Block and Heap profiling actually work, but setting appropriate
runtime parameters (and plumbing them through the config).
- Split the profiling into two phases: start and stop, since some will
need to be started early, and others will need to happen at the end.
PiperOrigin-RevId: 349495377
For "hey", requests >= concurrency. b.N can be set by the
--test.benchtime={b.N}x. The previous setting of b.N * c
can be surprisingly slow for larger c.
Set the requests to max(b.N, c) and log to the user if it is c.
PiperOrigin-RevId: 347053675
Add parser and formatting for golang benchmarks for docker benchmarks.
Change adds a library for printing and parsing Test parameters and metrics.
Benchmarks use the library to print parameters in the Benchmark title
(e.g. the name field in b.Run()), and to report CustomMetrics. Parser
uses the library to parse printed data from benchmark output and
put it into BigQuery structs.
PiperOrigin-RevId: 336365628
Adds docs to nginx and refactors both Httpd and Nginx benchmarks.
Key changes:
- Add docs and make nginx tests the same as httpd (reverse, all docs, etc.).
- Make requests scale on c * b.N -> a request per thread. This works well
with both --test.benchtime=10m (do a run that lasts at least 10m) and
--test.benchtime=10x (do b.N = 10).
-- Remove a doc from both tests (1000Kb) as 1024Kb exists.
PiperOrigin-RevId: 329751091
This change adds a new reverse HTTP test where the HTTPD server runs
in a native container but the client runs inside gVisor. It allows
us to test download performance under varying levels of concurrency.
Also tweaks the concurrent request numbers to test for high levels
of concurrency.
PiperOrigin-RevId: 324651203
This change:
- Ports the nginx benchmark.
- Switches the Httpd benchmark to use 'hey' as a client.
- Moves all parsers to their own package 'tools'.
Parsers are moved to their own package because 1) parsing output of a command
is often dependent on the format of the command (e.g. 'fio --json'), 2) to
enable easier reuse, and 3) clean up and simplify actual running benchmarks
(no TestParser functions and ugly sample output in benchmark files).
PiperOrigin-RevId: 324144165
Adds profiling with `runsc debug` or pprof to dockerutil. All
targets using dockerutil should now be able to use profiling.
In addition, modifies existing benchmarks to use profiling.
PiperOrigin-RevId: 323298634
