Files
zeekay 1b4f17bb0b Q8_0 tiles take any row count, twelve columns a tile, and quantize reads a view's own elements.
A matmul whose rows were not a multiple of four fell back to one dot product per output, and the
4x4 tile kept four SDOT chains eight deep, each behind a broadcast, near a third of the dotprod
rate. The last row group is quantized with zero rows past m and stored only to m; a row group's
columns go in tiles of twelve (twelve independent chains, SDOT by element) with a four-column
tile for the rest; the activations are quantized with NEON, rounding as BlockQ8_0::from_float
does. The i8mm tiles take a partial row group the same way. repack::tiled says whether this CPU
tiles a quantized matmul, for a caller choosing between quantized and dense weights.

QTensor::quantize read the raw storage from its start, so a narrow at an offset (the up half of
a packed gate-and-up weight) quantized the rows before it; it reads the view's elements now.

Claude-Session: https://claude.ai/code/session_018Gt5TvrE2rxXuZikWSauv3
2026-09-27 04:52:58 -07:00
..
2026-07-28 06:03:51 -07:00
2026-07-28 06:03:51 -07:00
2026-07-28 06:03:51 -07:00
2026-07-28 06:03:51 -07:00