This is a step forward for the full migration of Automatic Exec Groups (AEGs). This change will be effective once AEGs are enabled.
In this CL I've added a toolchain_type to `toolchain` attribute of ctx.actions.{run, run_shell} since the Execution platform is selected on a toolchain type level, not on a rule level like before AEGs.
PiperOrigin-RevId: 560782417
This adds a command-line tool named `gvisor_k8s_tool` that allows running the
`runsc` installer image as a DaemonSet in a Kubernetes cluster, either through
GKE or through `kubectl` configuration.
Example:
```shell
# Install using default kubectl context:
$ ./gvisor_k8s_tool install --cluster=kube: --image=my-runsc-installer
# Install using custom kubectl config and context:
$ KUBECONFIG=/tmp/myconfig ./gvisor_k8s_tool \
install --cluster=kube:mycontext --image=my-runsc-installer
# Install in a GKE cluster:
$ ./gvisor_k8s_tool install \
--cluster=gke:projects/myproject/locations/us-central1-a/clusters/mylittlecluster \
--image=my-runsc-installer
```
PiperOrigin-RevId: 557945316
This creates a `go_template` that can generate a `go_library` rule which
extracts a compressed binary to a temporary location, then executes it.
This is useful to implement `runsc` subcommands without requiring that these
subcommands' dependencies are linked into the main `runsc` binary.
PiperOrigin-RevId: 557270682
The last remaining !go1.22 build is protecting the definition of
pkg/sync.maptype, which is a copy of runtime.maptype. We need to ensure these
definitions match so we can safely access the hasher field.
At its core, this CL achieves this check by ensuring that
unsafe.Offsetof(maptype{}.Hasher) matches the offset in the runtime version of
the type.
Several things happen along the way to achieve this:
* As of May 2023, runtime.maptype is actually a type alias for
internal/abi.MapType. checkoffset was failing to record the offsets because it
skipped type aliases for no good reason. Simply removing the type alias check
is sufficient to make type aliases work. (This part of the CL is technically
unnecessary because this CL ultimately references internal/abi.MapType
directly in anticipation of removal of the type alias. But there is no reason
not to allow type aliases).
* The checkconst / checkoffset regexp unintentionally does not allow / in
package paths, even though the rest of the package supports /. Fix this.
* checkconst was comparing the literal AST expression string against the
runtime value (i.e., "unsafe.Offsetof(maptype{}.Hasher)" vs "72", which fails
comparison. Switch to getting the resolved constant value from the type
checker.
* nogo/check.importer only loads package facts on direct import (stored in
importer.cache). If a package is not directly imported ImportPackageFact will
not find the facts. Typically packages need to ensure they directly depend on
packages they want facts from (e.g., pkg/sync has a dummy import of runtime in
runtime.go). This doesn't work for internal/abi because we cannot directly
import an internal package. Work around this as a hack by unconditionally
"importing" internal/abi when analyzing any package.
With regard to the last point, not that the nogo/defs.bzl nogo integration only
provides facts from the direct dependencies and the entire stdlib (since the
stdlib is analyzed as one bundle). So this trick only works for a stdlib
package. A bazel package indirect dependency would be missing facts altogether.
PiperOrigin-RevId: 549999084
We don't want these checks in the production binary at all. Even though the
hardware branch predictor will do away with any runtime costs of the if
condition, the code under the "if checkInvariants {...}" could force escapes
to the heap. It could also increase function complexity, making it no longer
inlinable. And these invariant checks lie on hot code paths.
PiperOrigin-RevId: 513398208
With the latest nogo change, facts are no longer in the build paths. Since full
is only used to speed these paths (and is enabled for full analysis) the full
flag is now redundant and can be removed safely.
PiperOrigin-RevId: 511935530
This allows fact information to be validated in the underlying source files,
but requires us to explicitly maintain this in appropriate version-tagged, and
architecture-tagged files. This is more explicit and safer.
This mechanism uses a special regular expression for matching a +checkconst
stanza to validate constant values, sizes and offsets. This applies to both Go
source files and assembly files.
PiperOrigin-RevId: 511867507
This script was being run as root. This script clones containerd and builds it.
On some Ubuntu versions, sudo(1) does not change the home directory. On
oldkernel machines from BuildKite, this script was running with
$HOME=/home/agent. As a result, root was writing and creating files in
/home/agent/.cache/go-build/. Other tests that call `go build` that were
scheduled to run on such VMs would fail due to permission issues because
/home/agent/.cache/go-build/ is owned by root.
Run the tools/install_containerd.sh script sudo -H so that the home directory
is set appropriately and all packages are built and installed with root user's
cache in /root/.cache/.
Also fixed lint errors in tools/install_containerd.sh and got rid of redundant
sudo calls. Added the check that the script must be run as root.
Co-authored-by: Andrei Vagin <[email protected]>
Acknowledgements: Andrei Vagin discovered this issue.
PiperOrigin-RevId: 511661564
The systrap platform like the ptrace platform uses stub processes to manage
the user address space. The difference is how they intercept system calls and
other events like memory faults, exceptions, etc.
In case of systrap, all events that have to be handled by the Sentry trigger
signals that are handled by a custom signal handler installed on stub
processes. The signal handler switches control to the Sentry.
Here are a few other optimizations:
* On x86, system calls can be replaced with a function call to remove overhead
of signals.
* For fast interactions of sentry and stub processes, futex wait/wake can
be a bottle neck, so we use a polling mode.
The platform is launched for the purpose of testing and gathering initial
feedback. It is not yet ready for use in production.
PiperOrigin-RevId: 511650064
After update to Go 1.20 in 3217061ac5 ("Bump go version to 1.20"), the
benchmark jobs have been failing on BuildKite. This seems to be happening
because benchmark test binaries are built in Docker containers but are run on
the host. On some hosts, the glibc isn't compatible with glibc in our docker
container. So dynamically linked binaries may fail. Linking them statically
resolves this glibc compatibility issue.
As a result, this change also adds support for static test binaries to
tools/bazeldefs/go.bzl.
PiperOrigin-RevId: 510455271
* We build all binaries in a build container but then we execute them in
another evironments, so we don't want to have a dependency on glibc.
* runsc is build without cgo, so it looks reasonable to build tests with
the same flags.
* If we build all packeges with the same flags, they all shares the same
cache.
This change adds the race config to build with the race detector:
Unexported constants can end up unaddressable in objectpath and thus their
object facts get lost on fact import. Work around this by adding a
package-level fact which is a map of all constants to their value.
Remove the Value facts because it is confusing to have two ways to access the
same thing (especially when one only works sometimes).
PiperOrigin-RevId: 505153453
Based on cl/504066914, this generates the proper procid offsets at build time.
Package facts needs an update to properly skip underscore imports, which don't
have facts for some reason (because they are unnamed?).
Drop use of runtime.getprocid given that these generated code is pretty solid.
PiperOrigin-RevId: 504926257
This removes the need for ongoing tags.
This change requires some minor updates to remove dependency cycles, since
the goid package is a base library used by many internals (log, sync, etc.).
PiperOrigin-RevId: 504066914
Note that the select_arch macro changed in order to allow for its use within
a rule below arch_genrule. The command line settings were being manipulated in
a way that did not correctly propagate the select_arch select constraints.
PiperOrigin-RevId: 499651425
This CL does the following:
- Add the ability for nested locks to have names.
- Give names to all current uses of nested locks in the codebase.
- Truncate `lockdep` debug stack traces to avoid the clutter from the
`lockdep` code itself
- Simplify `lockdep` to not longer require `classMap`.
PiperOrigin-RevId: 491486620
