what this lab teaches

The optimizer works one translation unit at a time. A tiny hot function defined in another .cpp is a CALL: no inlining, no vectorization of the loop around it, no constant propagation into it. Link-time optimization (LTO) merges the units before optimizing, and the call disappears. This lab is two files: 05_lib.cpp defines `scale`, this file calls it 200M times.

first, get the code

run this

$ git clone https://github.com/adit4443ya/debug_labs.git && cd debug_labs

then build it

run this

$ cd codegen && ./build.sh

produces bin/05_separate, bin/05_lto (-flto), bin/05_lto_thin (-flto=thin)

walk it, one step at a time

run this

$ for b in 05_separate 05_lto 05_lto_thin; do printf "%-14s " $b; ../tools/pin.sh -q ./bin/$b; done

what it printed on the machine that wrote this lab

05_separate    1.317 ns/call
05_lto         0.968 ns/call
05_lto_thin    0.947 ns/call

separate: 4004e3: call 400590 <_Z5scalel> # one call per iteration 4004e8: add %rax,%r13 4004f1: jne 4004e0 <main+0x50> lto: 400524: paddq %xmm2,%xmm5 # scale() inlined, then the loop VECTORIZED 400553: paddq %xmm5,%xmm8 ... 400577: add $0xfffffffffffffffc,%rcx 40057b: jne 400520 <main+0x90> With the definition visible, `scale` was inlined into main and the whole accumulation loop became packed 64-bit adds, four per step. The 1.3 ns was never "the cost of a call" (a predicted call is a few cycles); it was the cost of everything the call PREVENTED.

run this

$ clang++ -O2 -Rpass=inline 05_main.cpp 05_lib.cpp -o /dev/null            # separate: only bench_ns inlines

what it printed on the machine that wrote this lab

05_main.cpp:8:19: remark: '_ZL8bench_nsv' inlined into 'main' with (cost=35, threshold=487) ...

run this

$ clang++ -O2 -flto -fuse-ld=lld -Wl,-mllvm,-pass-remarks=inline 05_main.cpp 05_lib.cpp -o /dev/null

what it printed on the machine that wrote this lab

<unknown>:0:0: '_Z5scalel' inlined into 'main' with (cost=-15025, threshold=337)

Read it: under LTO the inlining decision happens inside the linker's optimization pipeline, so the remark must be requested through the linker (-Wl,-mllvm,-pass-remarks=inline with lld). The cost is hugely negative: inlining `scale` makes the caller SMALLER after simplification, so it is a certain win. Cost/threshold is the inliner's currency; -Rpass-missed=inline tells you when it refused and why.

what lto costs and when it matters

- Full LTO merges everything into one module: best optimization, slow serial link, lots of memory. ThinLTO summarizes per module, imports only what it needs, links in parallel: nearly the same result here (0.947 vs 0.968) at a fraction of the build cost. Default to ThinLTO. - It matters when hot paths cross TU boundaries through small functions: accessors, math helpers, comparators, allocators. It does nothing for a hot loop that is already in one TU, or for calls into a shared library (the boundary is the .so, not the TU: header-only or static-link the hot helpers instead). - The alternatives without LTO: put the small function in a header (`inline`), or make it a template. That is why performance libraries are header-heavy. PGO IN ONE PARAGRAPH (try it):

run this

$ clang++ -O2 -fprofile-instr-generate 05_main.cpp 05_lib.cpp -o bin/05_pgo_gen
$ LLVM_PROFILE_FILE=bin/05.profraw ./bin/05_pgo_gen && llvm-profdata merge -o bin/05.profdata bin/05.profraw
$ clang++ -O2 -fprofile-instr-use=bin/05.profdata 05_main.cpp 05_lib.cpp -o bin/05_pgo_use

what it printed on the machine that wrote this lab

  With a profile the inliner raises its threshold at HOT call sites and
  lowers it at cold ones, and lays out hot blocks together. On branchy
  code PGO is often the single largest win available; on this tiny loop it
  cannot beat LTO's inlining across the TU wall.

the rules

I1. When a profile shows a small function called millions of times, the cost is the missed inlining. Check whether the call crosses a TU or a shared-library boundary before touching the function's body. I2. Use -Rpass=inline / -Rpass-missed=inline (via the linker under LTO) to see decisions instead of guessing from the source. I3. ThinLTO by default for release builds; header/template for the handful of helpers that must inline everywhere; PGO when branches and layout dominate.

take it further

1. Mark `scale` `__attribute__((noinline))` in 05_lib.cpp and rebuild the LTO binary: it returns to ~1.3 ns. Inlining, not linking, was it. 2. Put `scale` into a shared library (-shared -fPIC) and call it through the PLT: measure; then compare with -fno-plt / -Bsymbolic. 3. Run the PGO steps above and diff `-Rpass=inline` output with and without the profile: look for "hot callsite" in the remark text. ============================================================

read the source · codegen/05_main.cpp

#include "../tools/bench.h"
#include <cstdint>
int64_t scale(int64_t x);                       /* defined in 05_lib.cpp: invisible to this TU without LTO */
int main(int argc, char **argv) {
    uint64_t n = argc > 1 ? strtoull(argv[1], 0, 10) : 200000000ull;
    int64_t s = 0;
    uint64_t t0 = bench_ns();
    for (uint64_t i = 0; i < n; i++) s += scale((int64_t)i);
    BENCH_KEEP(s);
    printf("%.3f ns/call\n", (double)(bench_ns() - t0) / (double)n);
    return 0;
}

◇

Finish the lab: write the evidence note

A lab is done when you can say what it showed on your machine — not when you have read it.

Inlining across translation units — separate compilation vs LTO

Which role are you preparing for?