← Labscache-06Memory hierarchy●●●○

TLB reach and transparent huge pages

cache/06_tlb_hugepages.cneeds: tools/bench.h, tools/pin.sh, perf (dTLB events)2 steps

0/2 steps

what this lab teaches

Every load translates a virtual address. The TLB caches translations: ~96 first-level entries and 2048 second-level entries here, so with 4 KiB pages random access beyond ~8 MB misses the TLB on nearly every load and pays a page walk (several dependent memory reads of its own). A 2 MiB huge page covers 512x the memory per entry. Linux can give you huge pages transparently (THP): this box is in `madvise` mode, so you ask with madvise(MADV_HUGEPAGE) and, since 6.1, can force the matter with MADV_COLLAPSE. This lab does dependent random loads over a region with 4 KiB pages vs huge pages, prints how many huge pages you actually got, and counts dTLB misses.

evidence to leave with

147 -> 130 ns with full THP on 256 MB; 226 -> 189 ns with a third of 1 GiB huge (ENOMEM on collapse); miss rate 93% -> 65%

first, get the code

run this

$ git clone https://github.com/adit4443ya/debug_labs.git && cd debug_labs

then build it

run this

$ cd cache && ./build.sh

produces bin/06_tlb_hugepages [4k|huge] [loads] [MB]

walk it, one step at a time

run this

$ ../tools/pin.sh -q perf stat -e dTLB-load-misses:u,dTLB-loads:u ./bin/06_tlb_hugepages 4k

what it printed on the machine that wrote this lab

mode=4k   region=1024 MB  huge after fault=0 MB, after MADV_COLLAPSE=0 MB (rc=0 ok) | dependent random loads: 226.3 ns/load
         4,967,751      cpu_core/dTLB-load-misses/       #   92.89% of all dTLB cache accesses
         5,347,937      cpu_core/dTLB-loads/

run this

$ ../tools/pin.sh -q perf stat -e dTLB-load-misses:u,dTLB-loads:u ./bin/06_tlb_hugepages huge

what it printed on the machine that wrote this lab

mode=huge region=1024 MB  huge after fault=324 MB, after MADV_COLLAPSE=324 MB (rc=12 Cannot allocate memory) | dependent random loads: 188.9 ns/load
         3,417,438      cpu_core/dTLB-load-misses/       #   64.90% of all dTLB cache accesses
         5,265,403      cpu_core/dTLB-loads/

Read it: with 4 KiB pages 93% of the 5M dependent loads miss the TLB (each is a fresh random page). Asking for huge pages got only 324 MB of the 1 GiB as huge pages, and MADV_COLLAPSE could not finish the job (ENOMEM: the kernel has no free 2 MiB-contiguous physical chunks on this busy laptop). Result: a third of the loads hit huge pages, misses drop to 65%, time drops 226 -> 189 ns. Partial success, honestly reported by the program itself: NEVER assume you got huge pages; measure AnonHugePages (/proc/self/smaps) and the miss rate.

run this

$ ../tools/pin.sh -q ./bin/06_tlb_hugepages 4k   5000000 256

what it printed on the machine that wrote this lab

  mode=4k   region=256 MB  huge after fault=0 MB,   after MADV_COLLAPSE=0 MB   (rc=0 ok) | dependent random loads: 146.7 ns/load

run this

$ ../tools/pin.sh -q ./bin/06_tlb_hugepages huge 5000000 256

what it printed on the machine that wrote this lab

  mode=huge region=256 MB  huge after fault=256 MB, after MADV_COLLAPSE=256 MB (rc=0 ok) | dependent random loads: 129.9 ns/load

run this

$ ../tools/pin.sh -q ./bin/06_tlb_hugepages 4k   5000000 64

what it printed on the machine that wrote this lab

  mode=4k   region=64 MB   ...                                                          | dependent random loads: 142.1 ns/load

run this

$ ../tools/pin.sh -q ./bin/06_tlb_hugepages huge 5000000 64

what it printed on the machine that wrote this lab

  mode=huge region=64 MB   huge after fault=64 MB, ... (rc=0 ok)                       | dependent random loads: 127.8 ns/load
  Read it: with the whole region on huge pages (128 entries cover 256 MB,
  no misses), a dependent random load costs ~128-130 ns = DRAM latency.
  With 4 KiB pages it costs ~142-147 ns: the page walk adds ~15-17 ns
  here, because for a 256 MB region the page-table entries (512 KB of
  them) stay cache-resident. For the 1 GiB region they don't, and the
  4 KiB cost climbs to 226 ns: a walk that misses cache is ~100 ns on
  top of the data miss. TLB cost is not a constant; it depends on whether
  the page tables themselves are hot.

WHEN HUGE PAGES MATTER (and when they don't):
  - Random access over a working set beyond ~8 MB (STLB reach): yes:
    10-40% on latency-bound code, more when page tables miss cache.
  - Sequential streaming: little; the hardware prefetcher and the STLB
    keep up, and the walk cost is amortized over 64 loads per page.
  - Fits in L2/L3 and under 8 MB: nothing to gain.

Costs: 2 MiB granularity wastes memory for small objects; THP `always` can cause allocation stalls and khugepaged CPU use, which is why latency-sensitive systems prefer `madvise` (this box) plus explicit madvise/MAP_HUGETLB on the big tables, or hugetlbfs reserved at boot (root) so the pages are guaranteed instead of best-effort.

the rules

T1. Beyond ~8 MB of random reach with 4 KiB pages, count a page walk per access. Huge pages move the cliff to gigabytes. T2. THP is best-effort. Check AnonHugePages and dTLB-load-misses; on a fragmented machine you may get a fraction, or nothing. T3. For guaranteed huge pages in production: hugetlbfs / MAP_HUGETLB reserved at boot (root), pre-faulted, mlocked.

take it further

1. Run the 1 GiB `huge` case right after a reboot (little fragmentation): you will likely get all 1024 MB and ~130 ns. 2. Replace the dependent chain with independent random loads (drop the `p[idx] +`): the TLB cost mostly vanishes behind memory-level parallelism (lab 05). TLB misses hurt latency-bound code most. 3. `cat /sys/kernel/mm/transparent_hugepage/enabled` and `.../defrag`; read what `madvise` and `defer+madvise` mean for a latency-critical process. ============================================================

read the source · cache/06_tlb_hugepages.c

#include "../tools/bench.h"
#include <sys/mman.h>
#include <errno.h>
#ifndef MADV_COLLAPSE
#define MADV_COLLAPSE 25                        /* Linux 6.1+: synchronously collapse into huge pages */
#endif
static long anon_huge_kb(void) {                 /* sum of AnonHugePages over /proc/self/smaps */
    FILE *f = fopen("/proc/self/smaps", "r"); if (!f) return -1;
    char line[256]; long kb = 0;
    while (fgets(line, sizeof line, f)) if (!strncmp(line, "AnonHugePages:", 14)) kb += atol(line + 14);
    fclose(f); return kb;
}
int main(int argc, char **argv) {
    const char *mode = argc > 1 ? argv[1] : "4k";
    uint64_t loads = argc > 2 ? strtoull(argv[2], 0, 10) : 5000000ull;
    uint64_t mb = argc > 3 ? strtoull(argv[3], 0, 10) : 1024;          /* region size in MB (power of two) */
    uint64_t bytes = mb << 20, slots = bytes / 8;
    uint64_t *p = mmap(NULL, bytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (p == MAP_FAILED) { perror("mmap"); return 1; }
    int huge = !strcmp(mode, "huge");
    madvise(p, bytes, huge ? MADV_HUGEPAGE : MADV_NOHUGEPAGE);
    for (size_t i = 0; i < slots; i += 512) p[i] = 0;            /* touch every 4 KiB page */
    long kb_fault = anon_huge_kb();
    int rc = 0;
    if (huge) { rc = madvise(p, bytes, MADV_COLLAPSE); if (rc) rc = errno; }   /* ask the kernel to finish the job */
    long kb_now = anon_huge_kb();
    /* Dependent random chain WITHOUT a permutation: the next index depends on the loaded value (all zero),
     * so the load must complete before the next address exists. */
    uint64_t seed = 99, idx = 0;
    uint64_t t0 = bench_ns();
    for (uint64_t i = 0; i < loads; i++) idx = (p[idx] + bench_rand(&seed)) & (slots - 1);
    uint64_t dt = bench_ns() - t0; BENCH_KEEP(idx);
    printf("mode=%-4s region=%llu MB  huge after fault=%ld MB, after MADV_COLLAPSE=%ld MB (rc=%d %s) | dependent random loads: %.1f ns/load\n",
           mode, (unsigned long long)mb, kb_fault / 1024, kb_now / 1024, rc, rc ? strerror(rc) : "ok", (double)dt / (double)loads);
    return 0;
}

◇

Finish the lab: write the evidence note

A lab is done when you can say what it showed on your machine — not when you have read it.