what this lab teaches

The same C11 atomics built three ways, run on a CPU with LSE and on one without: excl -march=armv8-a -mno-outline-atomics load/store-exclusive loops; runs anywhere lse -march=armv8.1-a single-instruction atomics; SIGILL on old cores outline -march=armv8-a -moutline-atomics calls a helper that picks at runtime plus the link error you hit when the helpers' library is missing, which is the most common "why doesn't my aarch64 build link" question.

first, get the code

run this

$ git clone https://github.com/adit4443ya/debug_labs.git && cd debug_labs

then build it

run this

$ cd aarch64 && ./build.sh

produces bin/03_excl, bin/03_lse, bin/03_outline (+ .s for each)

walk it, one step at a time

excl: .LBB0_1: ldxr x10, [x21] ; add x10, x10, #1 ; stxr w11, x10, [x21] ; cbnz w11, .LBB0_1 lse: ldadd x9, x10, [x21] casal x19, x8, [x21] outline: bl __aarch64_ldadd8_relax bl __aarch64_cas8_acq_rel Read it: the v8.0 loop retries until the exclusive store succeeds (w11 == 0). LSE does it in one instruction that the cache system performs atomically (like x86's lock prefix). Outline atomics call a small function (in compiler-rt or libgcc) that checks HWCAP once at startup and then runs the LSE or the exclusive version: portable binaries that still get LSE where it exists, at the cost of a call. This is the default for aarch64 Linux in GCC and clang, which is why the helpers must be linkable.

run this

$ a64cc -O2 -march=armv8-a -moutline-atomics 03_atomics_run.c -o /tmp/outl

what it printed on the machine that wrote this lab

ld.lld: error: undefined symbol: __aarch64_ldadd8_relax
ld.lld: error: undefined symbol: __aarch64_cas8_acq_rel
The helpers live in libgcc.a (GCC) or libclang_rt.builtins-aarch64.a
(clang). A cross setup without one of them cannot link default aarch64
builds; the fixes are to provide the library (tools/a64-setup.sh fetches
compiler-rt's, root-free), or to build with -mno-outline-atomics, or
to target -march=armv8.1-a+ where LSE is inline anyway.

run this

$ for b in 03_excl 03_lse 03_outline; do a64run -cpu max ./bin/$b; done

what it printed on the machine that wrote this lab

HWCAP atomics(LSE)=1 | counter=1000000 | CAS(expect 0) -> failed, observed 1000000    (x3, identical)

run this

$ for b in 03_excl 03_lse 03_outline; do a64run -cpu cortex-a57 ./bin/$b; done

what it printed on the machine that wrote this lab

03_excl     HWCAP atomics(LSE)=0 | counter=1000000 | CAS(expect 0) -> failed, observed 1000000
03_lse      qemu: uncaught target signal 4 (Illegal instruction) - core dumped
03_outline  HWCAP atomics(LSE)=0 | counter=1000000 | CAS(expect 0) -> failed, observed 1000000

Read it: the LSE binary dies on a Cortex-A57 (v8.0): `ldadd` does not exist there. The outline build runs on both and picks LSE on `max` (HWCAP_ATOMICS set). The exclusive build runs on both, always slow-path.

which to ship

- You control the hardware (a trading box, a phone SoC you target): -march for that core (armv8.2-a+... or -mcpu=neoverse-v2), inline LSE, no helper calls. - You ship one binary to unknown Arm servers: outline atomics (the default) or a runtime-dispatched build (lab 04). The call costs a few cycles per atomic; in a hot spin loop that matters, so hot paths often use -march=armv8.1-a with a HWCAP check at startup that refuses to run on older cores. - Never -mno-outline-atomics with armv8.0 on a server: you throw away LSE and get retry loops under contention.

the rules

X1. Know which of the three shapes your build emits: `grep -E 'ldxr|ldadd|__aarch64_' out.s`. X2. On a link error for __aarch64_* symbols, the runtime library is missing, not your code. X3. Test with qemu `-cpu cortex-a57` (v8.0) AND `-cpu max` (everything): SIGILL on the first is a feature-dispatch bug that would ship.

take it further

1. `objdump -d bin/03_outline | grep -A12 '<__aarch64_ldadd8_relax>'`: read the helper: it tests a global set from HWCAP and branches to the ldadd or the ldxr/stxr path. 2. Add memory_order_seq_cst variants and compare `ldaddal` vs `ldadd`, `ldaxr/stlxr` vs `ldxr/stxr` (concurrency lab 03 table). 3. Run with `-cpu neoverse-n1` and `-cpu cortex-a76` in qemu: both have LSE; check HWCAP in the output. ============================================================

read the source · aarch64/03_atomics_run.c

#include <stdatomic.h>
#include <stdio.h>
#include <stdint.h>
#include <sys/auxv.h>
#include <asm/hwcap.h>
static atomic_long counter;
int main(void) {
    for (long i = 0; i < 1000000; i++) atomic_fetch_add_explicit(&counter, 1, memory_order_relaxed);
    long expect = 0; int ok = atomic_compare_exchange_strong(&counter, &expect, 7);   /* fails: counter != 0 */
    printf("HWCAP atomics(LSE)=%d | counter=%ld | CAS(expect 0) -> %s, observed %ld\n",
           !!(getauxval(AT_HWCAP) & HWCAP_ATOMICS), atomic_load(&counter), ok ? "swapped" : "failed", expect);
    return 0;
}

◇

Finish the lab: write the evidence note

A lab is done when you can say what it showed on your machine — not when you have read it.

Atomics on AArch64 — exclusives vs LSE vs outline atomics, run on two emulated CPUs

Which role are you preparing for?