AARCH64 LAB #03 — Picking AArch64 tile sizes without Arm hardware (llvm-mca)

TRACK: aarch64 | DIFFICULTY: 3 | NEEDS: llvm-mca, tools/mca_loop.sh, track 04's sched/gen.sh

What this teaches

Qemu gives you correctness, never speed. Until you have the device, the honest proxy for AArch64 performance is llvm-mca: it runs a loop body through LLVM's scheduling model of a named core (-mcpu=neoverse-v1, cortex-x4, ...) and reports cycles per iteration. No noise, no hardware.

tools/mca_loop.sh FILE.s -mtriple=aarch64 -mcpu=CPU pulls the innermost loop out of an assembly file (LLVM annotates it with Inner Loop Header: Depth=N) and feeds it to llvm-mca. The number to read is Block RThroughput = cycles per iteration at steady state.

The metric for a matmul microkernel: FMA instructions per cycle. Neoverse-V1 can issue 4 NEON fmla per cycle, so 4.0 is the ceiling.

STEP 1 — Tile sweep for NEON (the hoisted schedule from track 04, cross-compiled)

neon() { ../04_optimization/sched/gen.sh T V H TM=$1 TN=$2 > out/n.s.mlir && mlir-opt matmul.mlir --transform-preload-library="transform-library-paths=out/n.s.mlir" --transform-interpreter --test-transform-dialect-erase-schedule | mlir-opt --pass-pipeline="$LOWER_STD" | mlir-translate --mlir-to-llvmir -o out/neon_$1x$2.ll && clang --target=aarch64-linux-gnu -O2 -mcpu=neoverse-v1 -S out/neon_$1x$2.ll -o out/neon_$1x$2.s -Wno-override-module; }
for t in "1 4" "4 4" "4 8" "8 8" "8 16"; do set -- $t; neon $1 $2
  r=$(../tools/mca_loop.sh out/neon_$1x$2.s -mtriple=aarch64 -mcpu=neoverse-v1 | grep -oE 'RThroughput: [0-9.]+')
  n=$(awk '/Inner Loop Header: Depth=3/{p=1} p&&/fmla/{c++} p&&/^\s*b\.[a-z]+\s/{exit} END{print c+0}' out/neon_$1x$2.s)
  echo "tile $1x$2: fmla/iter=$n $r"; done

Real output, plus the column that matters (fmla ÷ cycles):

tile 1x4:  fmla/iter=2   RThroughput: 1.1    -> 1.8 fmla/cycle
tile 4x4:  fmla/iter=4   RThroughput: 1.8    -> 2.2
tile 4x8:  fmla/iter=8   RThroughput: 2.5    -> 3.2
tile 8x8:  fmla/iter=16  RThroughput: 4.3    -> 3.7   <- 93% of the 4.0 peak
tile 8x16: fmla/iter=32  RThroughput: 15.3   -> 2.1   <- spills: 94 instructions for 32 fmla

Compare with x86 (lab 04/01), where 4×16 won: AVX2 has 16 × 256-bit registers, NEON has 32 × 128-bit. An 8×8 f32 tile is 16 accumulator registers: too many for AVX2, comfortable for NEON. The best tile is a property of the register file, which is why tile selection must be per target.

STEP 2 — The SVE loop from lab 02, same model

../tools/mca_loop.sh out/sve.s -mtriple=aarch64 -mcpu=neoverse-v1

# innermost loop (Depth=3), 14 instructions
...
Block RThroughput: 3.5

2 fmla per 3.5 cycles ≈ 0.57 fmla/cycle. Even though each SVE fmla is 256 bits wide on V1 (2 NEON lanes' worth), that's ~1.1 NEON-equivalent/cycle against 3.7 for the hoisted NEON 8×8. The vector length isn't the bottleneck; the C loads/stores in the loop are. llvm-mca's resource-pressure table (run without the grep) shows the load/store pipes saturated. Hoist first, widen second.

How to use this on a production MLIR backend

For a kernel, sweep 5–10 tile configs through the real pipeline.
Rank with llvm-mca on the target core (-mcpu= of the Snapdragon's cores; llc -mtriple=aarch64 -mcpu=help lists the models LLVM has).
Take the top 2–3 to the device and measure. The model narrows the search; the device decides.

Caveats: llvm-mca assumes every load hits L1 and knows nothing about the front-end, branch prediction or other cores. It ranks microkernels, not whole programs. A tile that wins in the model can still lose to cache misses in the full problem (that's the job of the outer cache-level tiling, KC= in gen.sh).

Try this yourself

Re-rank STEP 1 for -mcpu=cortex-a78 and -mcpu=cortex-x4. Does the winner change?
Try 8×12 (TM=8 TN=12): 24 accumulators. Is it better than 8×8 in the model?
Run llvm-mca -mtriple=aarch64 -mcpu=neoverse-v1 -timeline on the 8×16 loop and find where the spill reloads stall the fmla chain.