Once a NIC is chosen for a route, we currently just grab the first IP on the
NIC as the source address. We should choose the address with the longest
matching prefix instead.
This is a simple linear search of the route table. That's slow for large
tables, but large tables are exceedingly rare for use with gVisor and we can
always reimplement this with a trie if it becomes necessary.
PiperOrigin-RevId: 557941259
Previously, when a packet was sent from a raw packet socket, the packet
buffer's link header was left unpopulated and the link header was only
found in the packet buffer's payload. This breaks the expectations of
LinkEndpoints which expect the link layer header to always be populated
when the link requires a header.
PiperOrigin-RevId: 542349445
It was possible for `close` and `dispatch` to race, each trying to `DecRef` the
same packet. We now clear the bucket of packets upon `close` so that `DecRef`
occurs only once.
PiperOrigin-RevId: 540663408
The underlying chunks returned from PullUp should not be shared,
since the underlying slice can sometimes be directly modified. This change
also reworks some of the network parsing code so that ownership of
views is more explicit to the reader.
PiperOrigin-RevId: 538230394
Merge clears the other buffer, so rather than cloning and dealing with
refcounts, we can just steal other's view list and adjust sizes.
This increases the speed of Merge 10x, and this was with a benchmark that
didn't do any TrimFront/Truncate (which will induce copying in the old
implementation).
BenchmarkMerge
BenchmarkMerge-33 99412884 12.74 ns/op 0 B/op 0 allocs/op
BenchmarkOldMerge
BenchmarkOldMerge-33 6075222 217.4 ns/op 0 B/op 0 allocs/op
This change uncovered that PacketBuffer.DeepCopyForForwarding wasn't doing a
true deep copy. It seems that, in the ICMP error path, a call to Merge was
forcing a deep copy (maybe because Append was copying bytes), obfuscating the
issue. This change also addresses that issue by having DeepCopyForForwarding
produce a copy that shares no data with other packets.
PiperOrigin-RevId: 537143838
All tcpip.Endpoints use a tcpip.FullAddress to hold addresses associated
with an outgoing or incoming packet, but the struct only had a field for
a tcpip.Address. This field was overloaded to also hold a link address
(which is normally held in a tcpip.LinkAddress). This worked relatively
fine until the change to make addresses opaque types.
commit ef88b30bff required the use of a
workaround to continue passing around link addresses through the
tcpip.Address field. This workaround requires assumptions about the
size of the hardware address and unncessary jumping-through-hoops.
This change just introduces a new field to tcpip.FullAddress with type
tcpip.LinkAddress that can hold link addresses and this new field will
be used by packet endpoints.
The change referenced above is cl/532284732.
PiperOrigin-RevId: 536561363
tcp_benchmark throughput increase 2-3%, but allocations go down (25% in the
download benchmark, only 2% in the upload path).
PiperOrigin-RevId: 532523146
The important change here is in tcpip/tcpip.go, where tcpip.Address is defined.
The rest is updating uses of tcpip.Address.
This is preparation for netip.Addr or []byte based addresses, which should save
us a bunch of allocations. Currently, we allocate every time we want to, say,
get a tcpip.Address from a header. This is because the header is a byte slice,
but Address is a string. Strings are immutable, so Go allocates and copies.
PiperOrigin-RevId: 532284732
This is effectively a rollback of cl/450976957. The original motivation never
panned out, and it's easier to work with the lists. They also are easier to
avoid allocations with.
PiperOrigin-RevId: 531020857
The benefits of PacketBufferPtr never materialized and it makes the type
difficult to work with, e.g. it can't be used with go_generics to make a
(performant) list.
This is effectively a rollback of cl/480518221.
PiperOrigin-RevId: 530954630
nic.networkEndpoints is never modified after the NIC is created so there
is no need to hold a lock while accessing it.
We keep `n.mu` (renamed to `n.enableDisableMu`) to synchronize attempts
to enable and disable the device as interleaving of these operations may
result in inconsistent state.
PiperOrigin-RevId: 525288865
Fuchsia's Netstack component would like to depend on this constant when
setting the buffer size for neighbor events. See https://fxrev.dev/824598.
Fixes#8766
PiperOrigin-RevId: 522337436
DF doesn't need to be set. Linux also allows GRO on packets without DF set.
This fixes an issue where loopback traffic couldn't be GRO'd because gVisor
never sets the DF bit (we don't implement PMTUD yet).
PiperOrigin-RevId: 518684608
GRO would not immediately flush the final ACK in the SYN-SYN/ACK-ACK handshake.
This could lead to a situation where
- Client A calls connect(), which returns once the final ACK of the handshake
is sent and A reaches state ESTABLISHED
- The ACK gets GRO'd, delaying it from reaching the server
- Client B calls non-blocking connect()
- Client B's ACK gets GRO'd as well
- Client B is marked as ESTABLISHED
- The server, with accept queue size 1, is only going to accept one connection,
but two clients are ESTABLISHED.
It now immediately flushes packets with no payload, as they are important to
TCP connection state and management.
PiperOrigin-RevId: 517474519
Also enables GRO in syscall tests. GRO should be totally transparent and should
not affect behavior. This ensures it's tested and fixes the bug it uncovered.
GRO would not immediately flush the final ACK in the SYN-SYN/ACK-ACK handshake.
This could lead to a situation where
- Client A calls connect(), which returns once the final ACK of the handshake
is sent and A reaches state ESTABLISHED
- The ACK gets GRO'd, delaying it from reaching the server
- Client B calls non-blocking connect()
- Client B's ACK gets GRO'd as well
- Client B is marked as ESTABLISHED
- The server, with accept queue size 1, is only going to accept one connection,
but two clients are ESTABLISHED.
It now immediately flushes packets with no payload, as they are important to
TCP connection state and management.
PiperOrigin-RevId: 516870932
Originally Ghanan's cl/494241863.
This prevents goroutines from being unnecessarily scheduled when there
are no packets in the GRO dispatcher, reducing wasted CPU cycles.
With this change, a timer to flush GRO packets is scheduled when a
packet is enqueued and no previous timer exists. The timer is
rescheduled to be fired again if after flushing packets, packets remain
in the GRO dispatcher. If GRO is disabled, and a timer was previously
set, the timer is reset to fire immediately and flush all packets.
Performance:
The big win here is for non-network-heavy workloads. The GRO flush thread no
longer fires when unnecessary. GRO used to add absurd overhead for such
workloads. Now it adds none: e.g. `sleep 10` used <= 10ms without GRO, 7.79s
with it (!!!), and now uses <= 10ms with this change.
This lowers throughput and CPU usage roughly proportionally in networking-heavy
benchmarking. CPU usage seems to decrease slightly more than throughput (which
is good!).
PiperOrigin-RevId: 516720406
Opted for separate code paths rather than over-fitting to interfaces when
there're only 2 supported network protocols and 1 transport protocol.
The performance improvement is more modest than IPv4, I'm guessing due to the
process of parsing extension headers. As with IPv4, we'd benefit from parsing
only once and saving header state.
PiperOrigin-RevId: 513928270
In preparation for IPv6 support. IPv6 doesn't have a total length field.
pkt.Data().Size() is equivalent and also works with IPv6.
PiperOrigin-RevId: 512677616
In preparation for improving ARP handling of gratuitous ARPs, set PktType in
regular inbound flow instead of only when handling packet sockets.
PiperOrigin-RevId: 501562322
Some hosts use iptables and are configured to respond an ICMP destination
unreachable message with a {host/net/admin}prohibited code instead of a RST.
Linux handles these codes correctly by aborting the handshake. This commit
adds handling of all possible codes of an ICMPv4 destination unreachable message
to bring gVisor to parity.
PiperOrigin-RevId: 496703813
AddressDispatcher was being set when adding an address only for new
addresses, resulting in cases such as when an expired addresses is
re-added to lose the AddressDispatcher that is passed.
Added a regression test.
PiperOrigin-RevId: 494220872
Implement this check in netstack integration (in sentry) so that the
core netstack does not prevent an integrator from removing loopback.
PiperOrigin-RevId: 493992707
...before returning from groDispatcher.close.
This is to make sure that when we return from cleanup functions, the
caller can safely assume that resources have been released.
PiperOrigin-RevId: 493714729
Linux GRO simply uses the size of the first packet in a flow as the MSS. So
identically sized packets are coalesced until an oddly-sized one comes along.
See net/core/gro.c:dev_gro_receive and net/ipv4/tcp_offload.c:tcp_gro_receive.
Also fixed clobbering of the groPacket.idx field, which caused memory
corruption.
PiperOrigin-RevId: 490361840
