← Labssystems-02Systems and syscalls●○○○

What a system call costs, and what only looks like one

systems/02_syscall_cost.cneeds: tools/bench.h, strace2 steps

0/2 steps

what this lab teaches

"Avoid syscalls on the hot path" needs a number. Seven calls, one million times each: a vDSO call that never enters the kernel, the cheapest possible real syscall, reads and writes of one byte through the VFS, a scheduler entry, and a path lookup. Then the same program under strace, to see that the tool that counts syscalls makes each one ~100x slower.

evidence to leave with

vDSO 38 ns, getpid 142 ns, open+close 1.7 us; getpid 23.6 us under strace

first, get the code

run this

$ git clone https://github.com/adit4443ya/debug_labs.git && cd debug_labs

then build it

run this

$ cd systems && ./build.sh

produces bin/02_syscall_cost

walk it, one step at a time

run this

$ ../tools/pin.sh -q ./bin/02_syscall_cost

what it printed on the machine that wrote this lab

clock_gettime(MONOTONIC)         37.9 ns   vDSO: no kernel entry
getpid()                        142.5 ns   the cheapest real syscall: pure user<->kernel transition
syscall(SYS_getpid)             142.3 ns   same, via the generic wrapper
read(/dev/zero, 1 byte)         217.7 ns   transition + VFS + copy_to_user
write(/dev/null, 1 byte)        184.5 ns   transition + VFS
sched_yield()                   333.5 ns   transition + scheduler (nothing else runnable)
open+close(/dev/null)          1667.0 ns   path lookup + fd table: two syscalls

run this

$ strace -c ./bin/02_syscall_cost

what it printed on the machine that wrote this lab

clock_gettime(MONOTONIC)         29.6 ns   vDSO: no kernel entry              <- unchanged: not a syscall
getpid()                      23629.7 ns   ...                                <- 166x slower
sched_yield()                 29032.6 ns
open+close(/dev/null)         62196.2 ns
% time     seconds  usecs/call     calls    errors syscall
 33.12    6.532653           3   2000000           getpid
 23.76    4.686517           4   1000000           sched_yield
 21.80    4.299017           4   1000001           write

how to read it

- clock_gettime is ~38 ns and strace does not see it: the vDSO maps a page of kernel code and data into every process, and the time is read from that page (TSC + a scale factor) with no mode switch. Use it freely for microsecond timing; use rdtsc for nanosecond timing (perf lab 04: 12 ns). - getpid is the floor: ~140 ns for the transition into the kernel and back on this CPU with its mitigations (Spectre/Meltdown-era entry code added a lot to this number across the industry; it used to be ~50 ns). Anything a syscall does is on top of this. - read/write of one byte: +40-75 ns for the VFS layer and the copy. Real reads of real data are dominated by the data, not by this. - sched_yield: 330 ns to enter the scheduler and find nothing to switch to. A REAL context switch (another thread runs, caches cool, TLB partially flushed) is 1-5 us of direct cost and more of indirect. - open+close: 1.7 us. Path lookup (dcache), permission checks, fd table. Anything with a path is expensive; keep files open. - Under strace every syscall becomes a ptrace stop: two context switches to the tracer and back. strace -c is for COUNTING what you call, never for timing it. (Same for ltrace, lab 01.) For low-overhead syscall accounting use `perf trace` or bpftrace (root), or just count with strace and measure without it.

why this matters on a latency-critical path

140 ns is not the problem; a syscall's VARIANCE is. Entering the kernel means the scheduler may run, an interrupt may be serviced, a lock may be taken. That is why the hot path does no syscalls at all: no logging (write), no clock_gettime when rdtsc will do, no epoll_wait (busy-poll instead, lab 03), no malloc (mmap, lab 01), no locking that can sleep (futex, concurrency lab 04).

the rules

Y1. Know the floor on your machine (getpid). Everything else is floor + work. Y2. vDSO calls (clock_gettime, gettimeofday, getcpu) are not syscalls; real syscalls in a hot loop show up in `strace -c` counts and in `perf stat -e context-switches` when they sleep. Y3. Count syscalls with strace; time them without it.

take it further

1. `perf stat -e syscalls:sys_enter_getpid ./bin/02_syscall_cost` (needs tracepoint access; may require root here): the low-overhead way. 2. Boot parameter `mitigations=off` (root, a VM is a good place) and rerun: getpid drops; that difference is the security tax. 3. Replace getpid with `gettid` via syscall(): same cost. Replace with `getcpu()`: vDSO on x86-64: ~vDSO speed. ============================================================

read the source · systems/02_syscall_cost.c

#include "../tools/bench.h"
#include <fcntl.h>
#include <sys/syscall.h>
#include <unistd.h>
#define REPS 1000000
static void row(const char *name, uint64_t ns, const char *note) { printf("%-28s %8.1f ns   %s\n", name, (double)ns / REPS, note); }
int main(void) {
    uint64_t t0, acc = 0; char c = 0; struct timespec ts;
    int zero = open("/dev/zero", O_RDONLY), null = open("/dev/null", O_WRONLY);
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) { clock_gettime(CLOCK_MONOTONIC, &ts); acc += ts.tv_nsec; } BENCH_KEEP(acc);
    row("clock_gettime(MONOTONIC)", bench_ns() - t0, "vDSO: no kernel entry");
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) acc += (uint64_t)getpid(); BENCH_KEEP(acc);
    row("getpid()", bench_ns() - t0, "the cheapest real syscall: pure user<->kernel transition");
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) acc += (uint64_t)syscall(SYS_getpid); BENCH_KEEP(acc);
    row("syscall(SYS_getpid)", bench_ns() - t0, "same, via the generic wrapper");
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) acc += (uint64_t)read(zero, &c, 1); BENCH_KEEP(acc);
    row("read(/dev/zero, 1 byte)", bench_ns() - t0, "transition + VFS + copy_to_user");
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) acc += (uint64_t)write(null, &c, 1); BENCH_KEEP(acc);
    row("write(/dev/null, 1 byte)", bench_ns() - t0, "transition + VFS");
    t0 = bench_ns(); for (int i = 0; i < REPS; i++) acc += (uint64_t)sched_yield(); BENCH_KEEP(acc);
    row("sched_yield()", bench_ns() - t0, "transition + scheduler (nothing else runnable)");
    t0 = bench_ns(); for (int i = 0; i < REPS / 10; i++) { int fd = open("/dev/null", O_WRONLY); close(fd); }
    printf("%-28s %8.1f ns   %s\n", "open+close(/dev/null)", (double)(bench_ns() - t0) / (double)(REPS / 10), "path lookup + fd table: two syscalls");
    return 0;
}

◇

Finish the lab: write the evidence note

A lab is done when you can say what it showed on your machine — not when you have read it.