what this lab teaches

The Scalable Matrix Extension in the smallest program that exercises its three new ideas: 1. Streaming SVE mode: a CPU mode (smstart/smstop) with its OWN vector length (SVL), possibly different from the normal SVE VL, and a different set of legal instructions. 2. ZA: a matrix accumulator of SVL x SVL bytes, stored as tiles. 3. The outer product: `fmopa` multiplies a column vector by a row vector and accumulates the whole tile in ONE instruction: the core of every SME matmul kernel. Under qemu you can vary the streaming VL and watch the tile change size; on a CPU without SME the program says so instead of dying.

first, get the code

run this

$ git clone https://github.com/adit4443ya/debug_labs.git && cd debug_labs

then build it

run this

$ (cd aarch64 && ./build.sh)   -> bin/05_sme, bin/05_sme.s
$ __arm_locally_streaming and __arm_new("za") are the ACLE keywords; clang
$ inserts smstart/smstop and the ZA lazy-save calls (__arm_tpidr2_save)
$ that compiler-rt's builtins provide: without that archive the link
$ fails on `__arm_tpidr2_save`, which is how you learn that SME code has
$ a runtime ABI, not just instructions.

walk it, one step at a time

run this

$ for vl in 16 32 64; do $A64_QEMU -cpu max,sme-default-vector-length=$vl -L $A64_SYSROOT ./bin/05_sme; done

what it printed on the machine that wrote this lab

SME present. streaming VL = 128 bits -> ZA tile is 4 x 4 f32
trace of outer(x, ones) over the tile = 10 (expected sum 1..4 = 10)
SME present. streaming VL = 256 bits -> ZA tile is 8 x 8 f32
trace of outer(x, ones) over the tile = 36 (expected sum 1..8 = 36)
SME present. streaming VL = 512 bits -> ZA tile is 16 x 16 f32
trace of outer(x, ones) over the tile = 136 (expected sum 1..16 = 136)

run this

$ a64run -cpu cortex-a57 ./bin/05_sme

what it printed on the machine that wrote this lab

no SME on this CPU (HWCAP2_SME clear); nothing to run
Read it: x = (1, 2, 3, ...), y = (1, 1, 1, ...); outer(x, y)[r][c] = x[r]
so the trace (diagonal sum) is 1 + 2 + ... + N with N = SVL / 32 bits.
10, 36, 136 at N = 4, 8, 16: the single fmopa did the whole N x N
product each time, and the code never mentioned N.

run this

$ grep -E 'smstart|smstop|fmopa|zero|rdsvl' bin/05_sme.s | sort | uniq -c

what it printed on the machine that wrote this lab

      2 smstart sm        // enter streaming mode (per __arm_locally_streaming function)
      2 smstop  sm
      1 smstart za        // enable the ZA storage
      1 smstop  za
      2 zero {za}         // svzero_za()
      1 fmopa za0.s, p0/m, p0/m, z0.s, z1.s     // the outer product, predicated on both axes
      4 rdsvl x, #1       // read the STREAMING vector length (svcntsb/svcntsw)
  `rdsvl` is not `rdvl`: streaming and non-streaming lengths are separate
  registers of the machine, and code that mixes them up computes wrong
  loop bounds on hardware where they differ.

WHAT TO KNOW BEFORE TOUCHING A REAL SME KERNEL:
  - Streaming mode changes what is legal: many NEON and some SVE
    instructions are not available in streaming mode (unless sme_fa64);
    calling a normal function from streaming code costs smstop/smstart
    around the call unless it is marked streaming-compatible.
  - ZA is caller-saved via a lazy scheme (TPIDR2_EL0 + __arm_tpidr2_save):
    the ABI, not you, saves the tile when a non-ZA function is called.
    Keep hot loops free of calls.
  - Tiles: ZA holds SVL/8 x SVL/8 bytes; as 32-bit floats that is 4 tiles
    (za0.s..za3.s) of (SVL/32)^2 elements. A matmul kernel keeps several
    tiles live and streams panels of A and B through them: fmopa per K
    step, then reads rows out (svread_hor_za32) to store C.
  - Validation: outer products on small integer-valued inputs (exact in
    float), at multiple SVLs under qemu, against a scalar reference; then
    on real SME hardware for performance (Apple M4, and Armv9.2 cores with
    SME2). qemu is again correctness-only.
  - The dispatch checklist of lab 04 applies with more force: an SME
    kernel wins only above some problem size (mode switches and tile
    load/store are not free), so "has SME" must never mean "use SME".

the rules

Z1. Streaming VL != SVE VL. Use svcntsw()/rdsvl inside streaming code. Z2. No calls inside the ZA-live hot loop; mark helpers __arm_streaming_compatible where possible. Z3. Correctness at several SVLs under qemu first; then measure the crossover size on real hardware before dispatching to SME.

take it further

1. Change x to (1, 2, 3, ...) and y to (2, 2, 2, ...): trace doubles. Then use svmopa with the two predicates restricted (svwhilelt_b32(0,3)) and see only a 3x3 corner accumulate. 2. Accumulate two fmopa calls (K = 2) before reading the tile: the trace doubles again. That is the K loop of a matmul. 3. Read the ZA tile row by row with svread_hor_za32 into a C array and print it as a matrix at SVL 128: the outer product laid bare. ============================================================

read the source · aarch64/05_sme_first_look.c

#include <stdio.h>
#include <sys/auxv.h>
#include <asm/hwcap.h>
#include <arm_sme.h>
/* __arm_locally_streaming: the compiler enters streaming SVE mode on entry and leaves on return (smstart/smstop). */
__arm_locally_streaming static unsigned long streaming_vl_bits(void) { return svcntsb() * 8; }
/* ZA is the matrix accumulator: N x N bytes where N = SVL in bytes. One outer product accumulates a tile. */
__arm_locally_streaming __arm_new("za") static float outer_trace(const float *x, const float *y) {
    svbool_t pg = svptrue_b32();
    svfloat32_t vx = svld1(pg, x), vy = svld1(pg, y);
    svzero_za();
    svmopa_za32_f32_m(0, pg, pg, vx, vy);                  /* ZA0 += x (col) * y (row): a full tile in ONE instruction */
    float trace = 0; unsigned n = svcntsw();
    for (unsigned r = 0; r < n; r++) {                     /* read row r of tile 0, pick element r */
        svfloat32_t row = svread_hor_za32_f32_m(svdup_f32(0), pg, 0, r);
        float tmp[64]; svst1(pg, tmp, row); trace += tmp[r];
    }
    return trace;
}
int main(void) {
    unsigned long h2 = getauxval(AT_HWCAP2);
    if (!(h2 & HWCAP2_SME)) { printf("no SME on this CPU (HWCAP2_SME clear); nothing to run\n"); return 0; }
    float x[64], y[64]; for (int i = 0; i < 64; i++) { x[i] = (float)(i + 1); y[i] = 1.0f; }
    unsigned long svl = streaming_vl_bits();
    printf("SME present. streaming VL = %lu bits -> ZA tile is %lu x %lu f32\n", svl, svl / 32, svl / 32);
    printf("trace of outer(x, ones) over the tile = %.0f (expected sum 1..%lu = %lu)\n", (double)outer_trace(x, y), svl / 32, (svl / 32) * (svl / 32 + 1) / 2);
    return 0;
}

◇

Finish the lab: write the evidence note

A lab is done when you can say what it showed on your machine — not when you have read it.

SME first look — streaming mode, the streaming vector length, an outer product

Which role are you preparing for?