Files
zeekayandzooqueen 7097032800 metal: DSL-vs-hand decode board -- router-gate GEMV arm + cubecl-Metal throughput board (hanzo-ml 0.11.95)
Continues the ml-encoder DSL-vs-hand board (rms_norm arm) with the router-gate GEMV arm and the
cubecl-Metal throughput board, all measured cold on dbc M4 Max, one GPU job at a time.

Landed:
- metal_dsl_board::board_gemv: DSL gemv_f_f32 (one threadgroup/row, 32-wide strided reduction) vs the
  shipped hand call_mlx_gemm -- the route ml's matmul takes for every dense f32 mat-vec, m=1 included --
  both through ml's OWN encoder against one oracle. W is [n,k] for the DSL; the hand GEMM reads the SAME
  weights transposed to [k,n] (the convention the mlx_gemm test pins), transpose explicit not
  stride-inferred, so a wrong layout cannot pass as a fast kernel. Bit-exact ~1.5e-6 both arms.
- kernel_msl seam gains gemv_f_f32 -> src/metal/dsl/gemv.metal (committed cubecl MSL artifact).
- matvec-check Metal branch: extended from 4 checks to the full Qwen3 decode board (rms/gemv/router/
  MoE q4k+blk+dp4a/q6k/dp4a-blk), the DSL bit-exact + throughput board AND the MSL-dump source.
- tools/dsl/msl_extract.sh: brace-count the kernel block instead of stopping at the first column-0 close
  brace, which truncated any kernel with a mid-body loop/if -- gemv lost its reduction+store. Verified it
  reproduces the full committed gemv.metal byte-identical.

Measured (wall-clock best-of-N through ml's encoder, ~0.15 ms submit floor; ratios are the signal):
- gemv m=1: DSL ties-or-beats hand mlx_gemm -- 0.83-0.95x at 4096x4096 (above the floor, a real kernel
  win), 0.73-0.98x at the 128x2048 router gate (submit-bound). mlx_gemm is structurally starved at m=1
  (it runs a tiled GEMM); the purpose-built GEMV is leaner -- the same lesson as the Vulkan router gate.
- rms_norm: DSL LOSES at decode 1.13-1.36x (1 row) and is parity at batch 0.82-1.05x (512 rows). The
  fixed 128-thread tree reduce cannot use the hand kernel's up-to-1024 threads on a single row; they
  converge once many rows saturate. Stays hand.
- cubecl-Metal throughput board (cache-warm small shapes, bit-exact): dp4a-blk 411 GFLOP/s / 206 GB/s,
  matvec_q8 202 GB/s, Q4_K 247 GFLOP/s, MoE dp4a 201 GFLOP/s, router 0 decode ties.

Two cubecl-0.10-Metal defects found (kernels are bit-exact; the failures are in the runtime path):
- sdpa_blk: client.profile errors during timing on Metal (omitted from the throughput branch; the
  device-timestamped attention A/B lives in flash-metal-check).
- dense matvec_q4k_dp4a_blk: wgpu worker-thread panic -> zero output (scale_rel 1.0); the MoE dp4a twin
  dispatches fine, so it is specific to the dense kernel. Omitted; root-cause is a next-step.

Anchor (device-timestamped, flash-metal-check bench, unchanged): the f16 8x8 simdgroup flash arm is
3.0-4.55x over the portable scalar arm -- the one Metal perf arm that clearly earns authoring. Its
generalization to the prefill GEMM (f16-dequant tile + 8x8 simdgroup accumulators; int8 MMA impossible
on Apple) is documented as the NAMED NEXT PERF ARM in the board header.

Nothing flipped live: the wins are marginal or submit-bound at live decode shapes; a live gate needs an
in-engine A/B. This is the measurement board plus the clean tractable arm, not a decode-path change.
2026-09-11 13:48:07 -07:00
..