mirror of
https://github.com/hanzoai/visor.git
synced 2026-09-24 17:04:14 +00:00
Alex HuoandHanzo Dev
88d99e82f0
Several GPU test images pin CUDA and PyTorch versions that predate Blackwell,
so they fail on sm_100/sm_120 hardware with "no kernel image is available for
execution on the device" -- under runc exactly as much as under runsc. The
tests therefore cannot pass on Blackwell regardless of nvproxy.
PyTorch first shipped Blackwell kernels in 2.7.0, and only in the CUDA 12.8
(cu128) wheels, so 12.8 is the floor for anything torch-based:
- vllm: pinned v0.4.2, whose torch 2.3.0+cu121 compiles for sm_50..sm_90.
Bumped to v0.27.1 (torch 2.13.0+cu130, built for sm_75..sm_90, sm_100 and
sm_120), pinned by digest.
- pytorch: CUDA_VERSION drives both the base image and the pip index URL,
and torch is unpinned, so moving 12.4 -> 12.8 selects the cu128 wheels.
- nccl-tests: CUDA 12.8 is the first toolkit whose nvcc knows the Blackwell
arches, and nccl-tests derives its gencode list from the toolkit.
Note that the CUDA sample images (cuda-tests, cuda-tests-12-8) already work on
Blackwell without changes, because the samples embed PTX and the driver
JIT-compiles it forward to an unknown architecture. PyTorch ships cubins with
no forward-compatible PTX, which is why it hard-fails instead.
stable-diffusion-xl is deliberately left alone: it installs torch transitively
from the default PyPI index rather than a CUDA-specific one, so it needs an
explicit cu128 index plus a compatible xformers pin, which is more than a
version bump and wants a build test.
Verified on an NVIDIA RTX PRO 6000 Blackwell Server Edition (GB202): the
upstream vLLM v0.27.1 image serves facebook/opt-125m and answers completion
requests under both runc and runsc.
Co-authored-by: Hanzo Dev <[email protected]>
9 lines
329 B
Docker
9 lines
329 B
Docker
# CUDA 12.8 is the first toolkit whose nvcc knows the Blackwell arches
|
|
# (sm_100/sm_120); nccl-tests derives its gencode list from the toolkit.
|
|
FROM nvidia/cuda:12.8.1-devel-ubuntu22.04
|
|
|
|
RUN apt-get update && apt-get install git -y
|
|
|
|
RUN git clone --depth=1 https://github.com/NVIDIA/nccl-tests.git && \
|
|
cd nccl-tests && make
|