BIGCODE LAB #01: Build topology — how a 7,500-file C++ project is put together

TRACK: bigcode | DIFFICULTY: 2 | NEEDS: a CMake + Ninja build directory (here: the local LLVM 23 build), jq

What this teaches

Before you can profile, fix or even read a large codebase you need three answers: what gets built from what, with which flags, and how do I rebuild just one piece. CMake writes those answers into the build directory; Ninja and compile_commands.json let you query them without reading a single CMakeLists.txt. Every command below ran on $LLVM_BUILD (Release, Ninja, 7,568 translation units).

STEP 1 — How big is it, and what are the entry points?

$ cd $LLVM_BUILD
$ ninja -t targets all | wc -l
61629
$ ninja -t targets | grep -E "^(clang|opt|llc|check-clang|check-llvm|FileCheck)[: ]"
FileCheck: phony
check-clang: phony
check-llvm: phony
clang: phony
llc: phony
opt: phony

61,629 build edges; the names you type (ninja clang, ninja check-llvm) are phony targets that fan out to the real ones. ninja -t targets all lists everything; ninja -t targets rule CXX_COMPILER__… lists by rule; ninja -t browse (if built with it) serves a graph in a browser.

STEP 2 — The compile database: every TU's exact command

$ ls -la compile_commands.json | awk '{print $5" bytes"}'
12376961 bytes
$ jq 'length' compile_commands.json
7568
$ jq '.[] | select(.file | endswith("/lib/Support/APInt.cpp")) | .command' compile_commands.json | cut -c1-600
"/usr/lib64/ccache/c++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
 -I/home/.../llvm-project/build/lib/Support -I/home/.../llvm-project/llvm/lib/Support -I/home/.../llvm-project/build/include
 -I/home/.../llvm-project/llvm/include -I/home/.../llvm-project/llvm/../third-party/siphash/include -I/home/.../llvm-project/llvm/../libc
 -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-pa…

Read one command completely once: the compiler actually used (/usr/lib64/ccache/c++: GCC through ccache, not clang: this LLVM was built with g++), the defines (_GNU_SOURCE, the __STDC_*_MACROS), the include paths in order (build dir first: generated headers shadow source headers), and the codegen flags (-fPIC -fno-semantic-interposition -fvisibility-inlines-hidden are the LLVM shared-library hygiene set).

Aggregate questions are one jq away:

$ jq -r '.[].command' compile_commands.json | grep -oE ' -O[0-3s]' | sort | uniq -c
   7568  -O3
$ jq -r '.[].command' compile_commands.json | grep -c ' -g'
0

Every TU is -O3 and none has -g: a pure Release build. If a profile of this clang showed only addresses, this is why (lab 02, lab 04). The same two lines answer "is anything built without -fPIC?", "which TUs get -fno-exceptions?", "who uses -march?".

STEP 3 — What does target X consist of, and how is one object built?

$ ninja -t query lib/libLLVMSupport.a | head -6
lib/libLLVMSupport.a:
  input: CXX_STATIC_LIBRARY_LINKER__LLVMSupport_Release
    lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3.c.o
    lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_dispatch.c.o
    lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_portable.c.o
    lib/Support/BLAKE3/CMakeFiles/LLVMSupportBlake3.dir/blake3_neon.c.o
$ ar t lib/libLLVMSupport.a | wc -l
180
$ ninja -t commands lib/Support/CMakeFiles/LLVMSupport.dir/APInt.cpp.o | tail -1 | cut -c1-300
/usr/lib64/ccache/c++ -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE ... -o lib/Support/CMakeFiles/LLVMSupport.dir/APInt.cpp.o -c /home/.../llvm/lib/Support/APInt.cpp
$ ninja -t deps lib/Support/CMakeFiles/LLVMSupport.dir/APInt.cpp.o | grep -c "\.h"
258

-t query = inputs and outputs of any edge (the archive is 180 objects, including a blake3_neon.c.o that exists on x86 too: dispatch code). -t commands = the exact commands to rebuild a target, last line being the compile itself: copy it, add -ftime-trace or -S or -E, and you have reproduced one TU outside the build (lab 05 does exactly this). -t deps = the header dependency list Ninja recorded: 258 headers for one file, which is the number to reduce if compile time hurts.

STEP 4 — Rebuild one thing, see what would rebuild

ninja lib/Support/CMakeFiles/LLVMSupport.dir/APInt.cpp.o     # just that object
ninja -n clang | head                                        # dry run: what is out of date and why
ninja -d explain clang 2>&1 | head                           # WHY each edge is dirty (timestamp, missing, flags changed)
ninja -v lib/libLLVMSupport.a                                # print commands while building

-d explain is the tool for "why does everything rebuild after I touched one header": it names the dirty input.

The procedure for an unfamiliar CMake/Ninja tree

ninja -t targets | grep -i <thing> to find the target name.
jq the compile database for the flags of the file you care about; note the real compiler and the include order.
ninja -t query <target> down to objects; ninja -t commands <obj> | tail -1 to reproduce one compile by hand.
ninja -t deps <obj> for its header fan-in; -d explain for rebuild surprises.
Only then open CMakeLists.txt, searching for the target name (rg "add_llvm_component_library\(LLVMSupport"), because now you know what you are looking for.

Other build systems map onto the same questions: Android/Soong (m <module>, out/soong/build.ninja is Ninja underneath: ninja -f out/combined-*.ninja -t query), Bazel (bazel query, bazel aquery for commands), plain Make (make -n, make -p).

Try this yourself

Which TUs include the most headers? for o in $(ninja -t targets rule CXX_COMPILER__LLVMSupport_Release); do echo "$(ninja -t deps $o | grep -c .h) $o"; done | sort -rn | head.
Find every TU compiled with -fno-exceptions vs with exceptions in compile_commands.json and explain the split (hint: LLVM_ENABLE_EH).
Touch llvm/include/llvm/ADT/StringRef.h, then ninja -n clang | wc -l: how much of the world depends on one header?