[!IMPORTANT] TL;DR — what you must remember: A modern x86 core is a 6-wide, deeply out-of-order machine whose speed is set by whichever of four things binds first: the dependency chain through your loop (latency), the execution ports (throughput), the branch predictor (a mispredict flushes ~20 cycles of work), or memory (a line from DRAM is ~100+ ns and only overlappable if the loads are independent). Every number in this guide was either measured in the lab on an i7-1260P (Golden Cove P-cores) or is stated as a range with the source you would check. The lab labs that measure each point are named so you can reproduce rather than believe.
The Arm counterpart is Guide 13; the OS half is Guide 17.
FRONT END BACK END (out of order) MEMORY
fetch → decode (6-wide) → µop cache ─► rename/allocate (ROB ~512 entries) ─► scheduler ─► 12 ports ─► retire
│ ▲
branch predictor steers fetch; └─ load/store buffers, store buffer, L1D ◄─ L2 ◄─ L3 ◄─ DRAM
a mispredict discards everything after the branch
The two numbers you use most: the latency of an instruction (cycles until its result exists) and its reciprocal throughput (cycles between independent issues). llvm-mca prints both per instruction and simulates the dispatch (codegen/03): imul latency 3, throughput 1; add 1/0.25; addsd 2 on Golden Cove (4 on older cores); FMA 4; a load ~5 cycles from L1.
The most important microarchitectural question about any loop: is each iteration waiting for the previous one?
| Loop | Bound by | Lab measurement |
|---|---|---|
x = x * K + a[i] (one recurrence) | latency: imul 3 + add 1 = 4 cycles per element | codegen/03: 2.29 ns/elem ≈ 4.8 cycles; llvm-mca predicts 6 (over-charges the fused load) |
| four interleaved recurrences | ports: 4 multiplies per 4 cycles | 0.51 ns/elem ≈ 1.1 cycles: 4.5x |
s += a[i] in double, no reassociation | latency: addsd chain, 2 cycles | cache/03: 1.05 ns/elem; with -ffast-math 0.48 (bandwidth-bound) |
s += a[i]*b[i] float with FMA | latency: FMA 4 cycles per element | codegen/02: -march=native made it 2x slower than mul+add (2-cycle add off the critical path) |
| dependent random loads | memory latency: ~140 ns each | cache/05: 145 ns/load; 16 independent chains: 12 ns/load |
Rules: count the longest dependent path through one iteration in cycles; that is your floor. Break it with independent accumulators (the compiler does this for integer sums and, with permission, for FP), with batching (cache/05), or by removing the dependency (a different algorithm). More instructions with shorter chains beat fewer instructions with longer chains: the FMA case is the canonical example.
branch-misses × 24 ≈ the cycle delta between sorted and random input), 15–25 on most current cores.cmov/setcc/blends compute both sides and select. Wins when the branch is unpredictable and both sides are cheap (codegen/04: 7.3 → 1.0 ns/elem); loses on predictable branches by adding a data dependency, and the compiler often makes the decision for you based on the shape of the source (the "load both then select" form got vectorized into a blend; the naive ternary became a branch). Read the asm.switch jump tables, function pointers) are predicted per site; a monomorphic site is cheap, a polymorphic one that alternates pays the mispredict. The larger cost of a virtual call is the inlining it prevents (codegen/01, codegen/05).perf stat -e branches,branch-misses; branch-misses / branches above ~1% in a hot loop is worth a look; perf record -e branch-misses finds the branch.Measured on the lab machine (cache/02, dependent loads; sizes from lscpu):
| Level | Size (P-core) | Latency measured | Notes |
|---|---|---|---|
| L1D | 48 KiB, 12-way | 2.4 ns (~5 cycles) | 2 loads + 2 stores per cycle; 64-byte lines |
| L2 | 1.25 MiB | 4–12 ns | private per core; the rise inside L2 is the L1 dTLB running out |
| L3 | 18 MiB shared | 24 ns at 2 MiB, rising to 80+ | shared with E-cores; latency depends on slice distance |
| beyond STLB reach (~8 MB, 4 KiB pages) | +15–100 ns per access | page walks, not a cache level: cache/02 and cache/06 | |
| DRAM | 128–184 ns | plus the walk; ~15 GB/s single-core streaming bandwidth (cache/01) |
Mechanisms to know by name:
volatile accumulator costs ~5 cycles per iteration (perf/04: 8x slower). Forwarding fails on size/alignment mismatches (a 4-byte load of half an 8-byte pending store): a stall of ~15 cycles.xmm, baseline x86-64), AVX/AVX2 (256-bit, ymm, 2013+ desktop), AVX-512 (512-bit, zmm, servers; disabled on Alder Lake client parts because the E-cores lack it). -march=x86-64-v3 = AVX2+FMA+BMI; -march=x86-64-v4 adds AVX-512. clang -march=native -### shows what the host enables.-Rpass=loop-vectorize -Rpass-missed=loop-vectorize -Rpass-analysis=loop-vectorize.axpy at 8-wide was not faster than 4-wide: bandwidth).vzeroupper costs a state transition on some cores; compilers insert it. Intrinsics users: keep everything VEX.mov stores, and 0% with xchg.mov for relaxed/acquire/release loads and stores; xchg (implicit lock) for a seq_cst store, ~20–40 cycles including a store-buffer drain; lock xadd/lock cmpxchg for RMWs regardless of ordering (~18 cycles uncontended, cache/04 baseline 8.9 ns); mfence or lock or [rsp] for a full fence. A locked instruction on a line held by another core costs the line transfer.std::mutex is two atomics, ~25 ns, no syscall; under contention the cost is the line moving (50–100 ns) before any futex sleep.pause: in spin loops it reduces the power and the memory-order-machine-clear penalty when the spun-on line changes; it is 10–140 cycles depending on the core generation (that variability is why spin loops with a fixed count of pauses behave differently across CPUs).rdtsc reads a counter that ticks at a constant rate (2.496 GHz on the lab laptop, perf/04) regardless of the core's actual frequency (constant_tsc, nonstop_tsc); convert ticks to ns with a calibrated rate, not the nominal one. Serialize for short regions: lfence; rdtsc before, rdtscp; lfence after (~24 ns for the pair, vs ~12 ns for a bare rdtsc, vs ~38 ns for the vDSO clock_gettime).perf stat cycles counts actual core cycles; time depends on the frequency, which the governor, turbo and thermal limits move around (perf/01: the first two reps of every run at half speed under powersave; 60.1M cycles in 28.8 ms means the core ran at ~2.09 GHz). Compare cycles for microarchitectural questions and time for what users feel.perf reports the two PMUs in separate sections (bigcode/04: read the one with the samples). Pin (tools/pin.sh) before measuring anything.| Question | Events (perf stat -e …:u) | Reading |
|---|---|---|
| Is the core busy or waiting? | cycles, instructions → IPC | < 1 waiting (memory/branches); ~2–4 healthy; near 6 front-end limit |
| Is it the branches? | branches, branch-misses | misses / branches; × ~24 cycles ≈ cycles lost |
| Is it memory, and which level? | L1-dcache-load-misses, LLC-load-misses, dTLB-load-misses | demand misses only: prefetched lines do not count (perf/02) |
| Is it the front end? | topdown-fe-bound via perf stat --topdown / -M TopdownL1 (may need root for the slots event) | the Top-down method: front-end, bad speculation, back-end (memory vs core), retiring |
| Where? | perf record on the same event (-e branch-misses, -e cache-misses) then perf annotate | attribution has skid: the reported instruction is a few after the culprit |
| Is it the kernel or waiting? | context-switches, cpu-migrations, page-faults (software events, allowed unprivileged) | perf/05's counters; strace -c for syscalls |
Names differ per CPU: perf list, and for exact events the vendor's list (cpu_core/event=0x..,umask=0x../). On this hybrid machine, use cpu_core/…/ prefixes or pin to a P-core.
The latency of the chain: imul 3 + add 1 = 4 cycles per element regardless of how wide the core is (codegen/03 measured 4.8). Four independent chains run at the port limit (~1 cycle per element). The fix is structural (accumulators, batching), not "faster instructions".
The compiler fuses s += a*b into one FMA with 4-cycle latency on the accumulation chain; before, only the 2-cycle add was on the chain and the multiply ran in parallel. Fewer instructions, longer chain, 2x slower (codegen/02). -ffp-contract=off or restructuring with several accumulators fixes it.
The prefetcher fetched every line before the load needed it; counters count demand misses. The tell is that time scales with bytes moved and matches the machine's bandwidth ceiling (~15 GB/s per core here: perf/02, cache/01).
~24 cycles here: perf/02 ran the same instructions on sorted and random data; the cycle difference divided by the extra branch misses gave the per-miss cost. The usual range is 15–25 on current x86.
A store followed by a load of a different address can appear reordered because the load reads cache before the store leaves the store buffer. xchg (seq_cst store) or mfence prevents it; concurrency/05 shows 24.6% of trials reordered with plain stores and 0% with xchg.
False sharing: the counters share a 64-byte line and every increment forces a coherence transfer (cache/04). Pad to separate lines; the same instructions then scale perfectly.
rdtsc not a cycle counter, and when does that matter?It ticks at a constant nominal rate independent of the core's actual frequency. For elapsed time it is ideal (after calibration); for "how many cycles did this take" use perf stat -e cycles, because under powersave or turbo the core's frequency differs from the TSC's (perf/01: 2.09 GHz measured vs 2.496 GHz TSC).
No E-cores and no hybrid PMU split; AVX-512 exists (and may lower frequency under heavy use); addsd latency 4 not 2 (reductions get slower); larger L2 on newer parts, different L3 topology (mesh, per-slice latency); mitigations and microcode differ. Re-measure the noise floor (perf/01) and the working-set curve (cache/02) first; do not carry numbers across.
llvm-mca's scheduling models approximate).perf stat --topdown.It decides which problems come first, which drills open, which labs you see and how the Learn surface is organised. Change it any time from the switch in the top bar, or press Escape to look around first.