docs: add RDMA support user guide

Document RDMA (InfiniBand/RoCE) support: what it does, how to enable it
(--rdmaproxy), GPUDirect via dma-buf, and the current limitations
(Mellanox-only, single-container sandboxes, device lifecycle, RoCE shared
namespace mode, no checkpoint).

Co-authored-by: Alessio Ricci Toniolo <[email protected]>

g3doc/user_guide/BUILD

@@ -81,6 +81,14 @@ doc(
     weight = "51",
 )
 
+doc(
+    name = "rdma",
+    src = "rdma.md",
+    category = "User Guide",
+    permalink = "/docs/user_guide/rdma/",
+    weight = "52",
+)
+
 doc(
     name = "runtime_monitoring",
     src = "runtime_monitoring.md",

g3doc/user_guide/rdma.md

@@ -0,0 +1,64 @@
+# RDMA Support
+
+[TOC]
+
+gVisor supports RDMA (Remote Direct Memory Access) networking, allowing
+sandboxed applications to use high-performance InfiniBand/RoCE hardware — for
+example, multi-node NCCL collectives in distributed AI/ML training — while
+keeping the host isolated from the workload.
+
+To achieve this, gVisor implements a proxy driver inside the sandbox,
+henceforth referred to as `rdmaproxy`. `rdmaproxy` proxies the application's
+interactions with the host's RDMA driver, giving the sandboxed application
+access to the RDMA verbs devices (`/dev/infiniband/uverbs*`) declared in its
+OCI spec. The application — and libraries such as `libibverbs` and NCCL — can
+run unmodified inside the sandbox and interact transparently with these
+devices. gVisor also constructs a faithful view of the host's RDMA topology
+under `/sys` so that device-discovery and topology-detection logic behaves the
+same inside the sandbox as on the host.
+
+RDMA support is enabled with the `--rdmaproxy` flag and applies to containers
+whose OCI spec lists one or more `/dev/infiniband/uverbs*` devices.
+
+When combined with GPUs (see [GPU Support](gpu.md)), gVisor supports
+GPUDirect RDMA, allowing the NIC to transfer data directly to and from GPU
+memory.
+
+## Limitations
+
+RDMA support is under active development. The following limitations apply:
+
+*   **Mellanox NICs only.** Only Mellanox ConnectX (`mlx5`) adapters are
+    currently supported. Support for additional vendors is planned.
+
+*   **Host kernel 5.12 or newer.** `rdmaproxy` proxies the modern
+    `RDMA_VERBS_IOCTL` interface only; the legacy `write(2)` command
+    interface is not supported. The newest uverbs ioctl method it relies on,
+    dma-buf memory registration for GPUDirect RDMA, landed in Linux 5.12.
+
+*   **GPUDirect uses dma-buf only.** GPU memory is registered with the NIC
+    through the dma-buf mechanism, which is the modern default. The legacy
+    `nvidia-peermem` kernel-module path is not supported.
+
+*   **Single-container sandboxes only.** The RDMA devices must be declared in
+    the OCI spec of the sandbox's root container. Deployments where the
+    devices appear only in a sub-container's spec — such as a Kubernetes pod
+    where the RDMA devices belong to an application container rather than the
+    pod's root — are not yet supported.
+
+*   **Device lifecycle at sandbox creation.** The RDMA network devices must
+    still reside in the host network namespace when the sandbox is created
+    (`runsc create`), and must not be pre-placed into the sandbox's network
+    namespace beforehand. They should then be moved, fully configured, into
+    the sandbox's network namespace before the application connects RDMA
+    queue pairs — for example via an OCI `createRuntime` hook or between
+    `runsc create` and `runsc start`. Such setup is not unusual; Docker seems
+    to be doing this as well.
+
+*   **RoCE requires shared RDMA namespace mode.** For RoCE (RDMA over
+    Converged Ethernet) devices, the host's RDMA subsystem must be in the
+    shared network-namespace mode (`rdma system set netns shared`), which is
+    the default on most systems.
+
+*   **No checkpoint/restore.** Sandboxes using RDMA devices cannot be
+    checkpointed.

website/BUILD

@@ -171,6 +171,7 @@ docs(
         "//g3doc/user_guide:observability",
         "//g3doc/user_guide:platforms",
         "//g3doc/user_guide:production",
+        "//g3doc/user_guide:rdma",
         "//g3doc/user_guide:rootfs_snapshot",
         "//g3doc/user_guide:rootless",
         "//g3doc/user_guide:runtime_monitoring",