The last remaining !go1.22 build is protecting the definition of
pkg/sync.maptype, which is a copy of runtime.maptype. We need to ensure these
definitions match so we can safely access the hasher field.
At its core, this CL achieves this check by ensuring that
unsafe.Offsetof(maptype{}.Hasher) matches the offset in the runtime version of
the type.
Several things happen along the way to achieve this:
* As of May 2023, runtime.maptype is actually a type alias for
internal/abi.MapType. checkoffset was failing to record the offsets because it
skipped type aliases for no good reason. Simply removing the type alias check
is sufficient to make type aliases work. (This part of the CL is technically
unnecessary because this CL ultimately references internal/abi.MapType
directly in anticipation of removal of the type alias. But there is no reason
not to allow type aliases).
* The checkconst / checkoffset regexp unintentionally does not allow / in
package paths, even though the rest of the package supports /. Fix this.
* checkconst was comparing the literal AST expression string against the
runtime value (i.e., "unsafe.Offsetof(maptype{}.Hasher)" vs "72", which fails
comparison. Switch to getting the resolved constant value from the type
checker.
* nogo/check.importer only loads package facts on direct import (stored in
importer.cache). If a package is not directly imported ImportPackageFact will
not find the facts. Typically packages need to ensure they directly depend on
packages they want facts from (e.g., pkg/sync has a dummy import of runtime in
runtime.go). This doesn't work for internal/abi because we cannot directly
import an internal package. Work around this as a hack by unconditionally
"importing" internal/abi when analyzing any package.
With regard to the last point, not that the nogo/defs.bzl nogo integration only
provides facts from the direct dependencies and the entire stdlib (since the
stdlib is analyzed as one bundle). So this trick only works for a stdlib
package. A bazel package indirect dependency would be missing facts altogether.
PiperOrigin-RevId: 549999084
FDTable.descriptorTable is a slice of unsafe.Pointer-s and its maximum length
is MaxInt32. It requires up to 16GB of memory. A process can use just a few
descriptors but sets one or more of them to high numbers. In this case,
FDTable.descriptorTable is extended to the maximum size.
The problem here is that go-runtime zeros memory regions when they are reused.
In the case of fdtable, the memory region is 16GB, so it is a time consuming
operation. Second, it forces the kernel to allocate physical pages to
the entire region.
This change adds another level to descriptorTable, so the first level is
a slice of buckets where each bucket is a slice of descriptors. The bucket
size is fixed to 512 entries to fit one page.
Before:
BenchmarkFDLookupAndDecRef-12 50834290 23.70 ns/op
BenchmarkCreateWithMaxFD-12 2 7194873988 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 23775555 49.68 ns/op
BenchmarkTableLookup-12 412888780 2.835 ns/op
BenchmarkTableMapLookup-12 87944782 12.84 ns/op
After:
BenchmarkFDLookupAndDecRef-12 46229940 25.03 ns/op
BenchmarkCreateWithMaxFD-12 13 82573899 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 21889380 54.13 ns/op
BenchmarkTableLookup-12 415851230 2.821 ns/op
BenchmarkTableMapLookup-12 97236267 11.89 ns/op
Reported-by: [email protected]
PiperOrigin-RevId: 539138632
For all trivial and zero frame-sized functions, we now require an explicit
NOFRAME annotation as the heuristic has changed. See go.dev/cl/466316.
Most of these functions do not *require* NOFRAME, but this change encodes
the existing behavior in order to avoid accidental bugs or regressions.
PiperOrigin-RevId: 513946210
The problem is that it checkes circular dependencies and one then adds the
target lock to the graph.
Reported-by: [email protected]
PiperOrigin-RevId: 512637945
This allows fact information to be validated in the underlying source files,
but requires us to explicitly maintain this in appropriate version-tagged, and
architecture-tagged files. This is more explicit and safer.
This mechanism uses a special regular expression for matching a +checkconst
stanza to validate constant values, sizes and offsets. This applies to both Go
source files and assembly files.
PiperOrigin-RevId: 511867507
cl/504661697 removes this build tag once and for all, but it is blocked on
additional work needed to get it working with OSS bazel builds.
Until that is ready, bump the build tag to unblock testing Go tip (1.21).
PiperOrigin-RevId: 505923005
This removes the need to check the offset every release.
I've also removed the negative build tag from runtime_amd64.go, which less
obviously correct. In theory, we should check that this package's use of
nmspinning is still valid in each release, but there is no way to automate that
and I don't realistically seeing checking happening beyond verifying tests
work. Additionally, the existing use is already suspect. :)
The good news is the misuse is likely to cause scheduling issues, not memory
corruption.
PiperOrigin-RevId: 505114683
Accessing a pointer to the data in a slice can be achieved with
`unsafe.Pointer(&slice[0])`.
347e062d3b motivated using gohacks.SliceHeader in
this way with "we often use SliceHeader to extract pointers from slices in a
way that avoids bounds checking and/or handles nil slices correctly", but this
no longer seems to be the case. None of the remaining uses are obviously
performance sensitive or necessarily include bounds checks, nor get used with
nil slices.
This brings us one step closer to removing gohacks.SliceHeader, which is one
less internal detail to keep in sync with Go.
For #8422
PiperOrigin-RevId: 504408578
This CL does the following:
- Add the ability for nested locks to have names.
- Give names to all current uses of nested locks in the codebase.
- Truncate `lockdep` debug stack traces to avoid the clutter from the
`lockdep` code itself
- Simplify `lockdep` to not longer require `classMap`.
PiperOrigin-RevId: 491486620
Also adds the "// +checkalignedignore" escape hatch for packages to opt out of
checking.
See cl/439349432 for justification (https://github.com/google/gvisor/pull/7376).
PiperOrigin-RevId: 444918125
Upstream Go tip has updated its version to 1.19, so we need to bump these build
tags to allow tip testing.
This CL bumps the minimum version in pkg/sync/runtime_amd64.s from 1.8 to 1.14.
The offset was different prior to Go 1.14.
Ideally we'll get checklinkname (or equivalent) to check these remaining unsafe
uses (like internal struct field offsets) soon.
For golang/go#51445.
PiperOrigin-RevId: 433304124
The fdnotifier goroutine alternates between [host] epoll_wait()ing for I/O
readiness events and propagating those events through the sentry, possibly
waking task goroutines blocked in e.g. [application] epoll_wait(). Since woken
goroutines are queued on the runqueue of the waking P (cf. runtime.ready()),
these goroutines will almost always wait to execute until another OS thread
takes them (either directly via runtime.runqsteal(), or indirectly via sysmon
stealing the P and runtime.handoffp()ing it to another thread). Furthermore,
epoll_wait() is most likely to block (new events are least likely to have
arrived) shortly after a previous call (on the same epoll FD) returns events,
so this behavior utilizes the thread's OS-scheduled time slice poorly. Avoid
both of these problems by runtime.goyield()ing between goroutine wakeup and
epoll_wait().
PiperOrigin-RevId: 430793631
Some synchronization patterns require the ability to simultaneously wake and
sleep a goroutine. For the sleep package, this is the case when a waker must be
asserted when a subsequent fetch is imminent.
Currently, this operation results in significant P churn in the runtime, which
ping-pongs execution between multiple system threads and cores and consumes a
significant amount of host CPU (and because of the context switches, this can
be significant worse with mitigations for side channel vulnerabilities).
The solution is to introduce a dedicated mechanism for a synchronous switch
which does not wake another runtime P (see golang/go#32113). This can be used
by the `AssertAndFetch` API in the sleep package.
The benchmark results for this package are very similiar to raw channel
operations for all cases, with the exception of operations that do not wait.
The primary advantage is more precise control over scheduling. This will be
used in a subsequent change.
```
BenchmarkGoAssertNonWaiting
BenchmarkGoAssertNonWaiting-8 261364384 4.976 ns/op
BenchmarkGoSingleSelect
BenchmarkGoSingleSelect-8 20946358 57.77 ns/op
BenchmarkGoMultiSelect
BenchmarkGoMultiSelect-8 6071697 197.0 ns/op
BenchmarkGoWaitOnSingleSelect
BenchmarkGoWaitOnSingleSelect-8 4978051 235.4 ns/op
BenchmarkGoWaitOnMultiSelect
BenchmarkGoWaitOnMultiSelect-8 2309224 520.2 ns/op
BenchmarkSleeperAssertNonWaiting
BenchmarkSleeperAssertNonWaiting-8 447325033 2.657 ns/op
BenchmarkSleeperSingleSelect
BenchmarkSleeperSingleSelect-8 21488844 55.19 ns/op
BenchmarkSleeperMultiSelect
BenchmarkSleeperMultiSelect-8 21851674 54.89 ns/op
BenchmarkSleeperWaitOnSingleSelect
BenchmarkSleeperWaitOnSingleSelect-8 2860327 416.4 ns/op
BenchmarkSleeperWaitOnSingleSelectSync
BenchmarkSleeperWaitOnSingleSelectSync-8 2741733 427.1 ns/op
BenchmarkSleeperWaitOnMultiSelect
BenchmarkSleeperWaitOnMultiSelect-8 2867484 418.1 ns/op
BenchmarkSleeperWaitOnMultiSelectSync
BenchmarkSleeperWaitOnMultiSelectSync-8 2789158 427.9 ns/op
```
PiperOrigin-RevId: 415581417
- Don't attempt to create directory is controller is not
present in the system
- Ensure that all files being written exist in cgroupfs
- Attempt to delete directories during Uninstall even if
other deletions have failed
Fixes#6446
PiperOrigin-RevId: 402614820
We install seccomp rules so that the SIGSYS signal is generated for
each mmap system call. Then our signal handler executes the real mmap
syscall and if a new regions is created, it maps it to the guest.
Signed-off-by: Andrei Vagin <[email protected]>
Go's dev.typeparams branch already claims to be Go 1.18, so our !go1.18 build
tags breaking testing gVisor with that branch.
Normally I would not want to bump the build tags this early, but I plan to
extend checklinkname to check the assumptions in these files and remove the
build tags ASAP. So we just go ahead and bump the tags until then to unblock
testing.
PiperOrigin-RevId: 389037239
This CL introduces a 'checklinkname' analyzer, which provides rudimentary
type-checking that verifies that function signatures on the local and remote
sides of //go:linkname directives match expected values.
If the Go standard library changes the definitions of any of these function,
checklinkname will flag the change as a finding, providing an error informing
the gVisor team to adapt to the upstream changes. This allows us to eliminate
the majority of gVisor's forward-looking negative build tags, as we can catch
mismatches in testing [1].
The remaining forward-looking negative build tags are covering shared struct
definitions, which I hope to add to checklinkname in a future CL.
[1] Of course, semantics/requirements can change without the signature
changing, so we still must be careful, but this covers the common case.
PiperOrigin-RevId: 387873847
This change makes the checklocks analyzer considerable more powerful, adding:
* The ability to traverse complex structures, e.g. to have multiple nested
fields as part of the annotation.
* The ability to resolve simple anonymous functions and closures, and perform
lock analysis across these invocations. This does not apply to closures that
are passed elsewhere, since it is not possible to know the context in which
they might be invoked.
* The ability to annotate return values in addition to receivers and other
parameters, with the same complex structures noted above.
* Ignoring locking semantics for "fresh" objects, i.e. objects that are
allocated in the local frame (typically a new-style function).
* Sanity checking of locking state across block transitions and returns, to
ensure that no unexpected locks are held.
Note that initially, most of these findings are excluded by a comprehensive
nogo.yaml. The findings that are included are fundamental lock violations.
The changes here should be relatively low risk, minor refactorings to either
include necessary annotations to simplify the code structure (in general
removing closures in favor of methods) so that the analyzer can be easily
track the lock state.
This change additional includes two changes to nogo itself:
* Sanity checking of all types to ensure that the binary and ast-derived
types have a consistent objectpath, to prevent the bug above from occurring
silently (and causing much confusion). This also requires a trick in
order to ensure that serialized facts are consumable downstream. This can
be removed with https://go-review.googlesource.com/c/tools/+/331789 merged.
* A minor refactoring to isolation the objdump settings in its own package.
This was originally used to implement the sanity check above, but this
information is now being passed another way. The minor refactor is preserved
however, since it cleans up the code slightly and is minimal risk.
PiperOrigin-RevId: 382613300
The presence of multiple packages in a single directory sometimes
confuses `go mod`, producing output like:
go: downloading gvisor.dev/gvisor v0.0.0-20210601174640-101a132e9436
$GOMODCACHE/gvisor.dev/[email protected]/pkg/linewriter/linewriter.go:21:2: found packages sync (aliases.go) and seqatomic (generic_atomicptr_unsafe.go) in $GOMODCACHE/gvisor.dev/[email protected]/pkg/sync
imports.go:67:2: found packages tcp (accept.go) and rcv (rcv_test.go) in $GOMODCACHE/gvisor.dev/[email protected]/pkg/tcpip/transport/tcp
PiperOrigin-RevId: 376956213
This validates that struct fields if annotated with "// checklocks:mu" where
"mu" is a mutex field in the same struct then access to the field is only
done with "mu" locked.
All types that are guarded by a mutex must be annotated with
// +checklocks:<mutex field name>
For more details please refer to README.md.
PiperOrigin-RevId: 360729328
- Use atomic add rather than CAS in every Gate method, which is slightly
faster in most cases.
- Implement Close wakeup using gopark/goready to avoid channel allocation.
New benchmarks:
name old time/op new time/op delta
GateEnterLeave-12 16.7ns ± 1% 10.3ns ± 1% -38.44% (p=0.000 n=9+8)
GateClose-12 50.2ns ± 8% 42.4ns ± 6% -15.44% (p=0.000 n=10+10)
GateEnterLeaveAsyncClose-12 972ns ± 2% 640ns ± 7% -34.15% (p=0.000 n=9+10)
PiperOrigin-RevId: 359336344
See https://github.com/golang/go/issues/19367 for rationale. Note that the
upstream decision arrived at in that thread, while useful for some of our use
cases, doesn't account for all of our SliceHeader use cases (we often use
SliceHeader to extract pointers from slices in a way that avoids bounds
checking and/or handles nil slices correctly) and also doesn't exist yet.
PiperOrigin-RevId: 358071574
Per runtime.memmove, pointers are always copied atomically, as this is required
by the GC. (Also, the init() safety check doesn't work because it gets renamed
to <prefix>init() by template instantiation.)
PiperOrigin-RevId: 345800302
