Part I — From Source Text to a Running Process#

1. The problem before linkers#

Begin with a small Rust program. We will follow it through every representation in this part.

fn twice(x: i32) -> i32 {
	x + x
}

fn main() {
	println!("{}", twice(21));
}

Source text gives names and meaning to a human: twice takes an i32, and main prints 42. A processor does not execute Rust text. It fetches numbered bytes from memory and interprets some of them as instructions for one particular instruction set.

One compiler could translate the entire program and every library it needs in one operation. That simple design fails at scale. Any library edit would require translating every program that uses it; separately developed languages could not easily cooperate; and the operating system would have no standard package of bytes to load.

Separate compilation divides the work. A source file can become an object file before the final addresses of its functions are known. This creates the pre-linker problem: one object may emit a call to twice, while another object supplies twice, but neither knows where the final instruction bytes will live. A linker combines their partial knowledge.

The key invariant is: every machine-level reference must have a meaning that can eventually be resolved under the target's rules. Compilation preserves source-level intent selectively, but forgets comments, most spelling choices, and often source structure. Linking preserves executable behavior and selected metadata, but usually forgets object-file boundaries as a semantic concept.

2. One end-to-end mental model#

Use this model before learning exceptions:

Rust source
  | compiler: meaning -> target operations
  v
assembly-like operations
  | assembler: operations -> bytes + unresolved facts
  v
object files
  | linker: combine, assign addresses, resolve references
  v
executable file
  | kernel: validate and map file regions
  v
initial process image
  | runtime loader: map shared libraries and finish dynamic binding
  v
running process -> startup code -> Rust main

Concrete trace: the compiler proves that 21 fits in i32 and translates twice(21) into target operations. An assembler (possibly integrated inside the compiler) encodes operations into bytes. The object records bytes for code, names such as a printing routine, and relocations for addresses not yet known. The linker chooses locations and produces an executable. The kernel creates an address space and maps that executable. On dynamically linked systems, a runtime loader maps needed shared libraries and repairs remaining references. Startup code initializes the language environment and eventually calls Rust's generated path to main.

These boxes are responsibilities, not necessarily separate programs or files. Rust compilers often contain an integrated assembler. Some systems let the kernel and runtime loader divide loading differently. Preserve the model while allowing ownership to vary.

3. Bits, bytes, and hexadecimal#

A bit is one of two states, written 0 or 1. A byte is usually eight bits on targets Rust supports. Eight bits have 256 possible patterns. Binary is exact but verbose, so binary tools display hexadecimal. One hexadecimal digit represents four bits; two digits represent one byte.

binary:       0010 1010
hexadecimal:     2    a   -> 0x2a
decimal:                  -> 42

0x says that following digits are hexadecimal. Hex 0a, decimal 10, and binary 00001010 are the same value, not different encodings. A hex dump shows bytes; meaning comes from context. The byte 0x2a could be an integer, part of an instruction, a character *, or compressed data.

Prediction. Does changing a hex viewer's display from lowercase 2a to uppercase 2A alter the file? No. The display spelling changes; the byte does not.

Invariant: a consumer must agree on both the byte boundaries and the interpretation. Hex preserves every bit but forgets semantic labels unless shown alongside metadata.

4. Endianness: byte order for wider values#

A 32-bit integer occupies four bytes. Endianness defines how significance maps to increasing byte addresses. For 0x12345678, little-endian order is 78 56 34 12; big-endian order is 12 34 56 78.

Endianness does not reverse bits within each byte, reverse text, or necessarily describe instruction encoding as a whole. File formats may prescribe an order independent of the host processor.

This complete stable Rust program reads explicitly little-endian data rather than assuming the host's order:

fn read_u32_le(bytes: [u8; 4]) -> u32 {
	u32::from_le_bytes(bytes)
}

fn main() {
	assert_eq!(read_u32_le([0x78, 0x56, 0x34, 0x12]), 0x1234_5678);
}

Invariant: producers and consumers agree on byte order for each multi-byte field. A violation often produces plausible but absurd sizes or addresses. Test the earliest decoded header field instead of blaming later layout code.

5. Integers are encodings, not abstract numbers#

An unsigned n-bit field represents values from zero through 2^n - 1. Signed machine integers commonly use two's complement: the top bit contributes a negative weight. Thus byte ff means 255 as u8 and -1 as i8. Width matters: ff ff may be 65535 or -1.

Addresses are commonly represented by unsigned integers, but an address is not merely a number: it belongs to an address space and has an interpretation. Relocation addends may be signed because a target can lie before a reference site.

Prediction. If a 32-bit relocation computes a value larger than 32 bits, may a linker silently keep the low bits? Usually that would change the requested reference. A correct tool must follow the relocation specification, commonly diagnosing overflow unless the relocation explicitly defines truncation.

Invariant: each field has a width, signedness, byte order, and overflow rule. Parsing preserves the represented value only if all four are honored.

6. Files and memory are different arrangements#

A file is a durable sequence of bytes indexed by file offsets. Memory is a process-visible collection of byte locations indexed by virtual addresses. Loading is not necessarily “copy the whole file.” Some file bytes are metadata used only by tools. Some memory, such as zero-initialized storage, may occupy almost no file space. Different file regions can be mapped with different permissions.

executable file offsets                  process virtual addresses
0x000 headers (tool input)               not necessarily mapped
0x200 code bytes       --map read/exec--> 0x400200 code
0x800 data bytes       --map read/write-> 0x600800 data
no stored bytes        --create zeros---> 0x601000 zeroed storage

Concrete trace: if the byte encoding part of twice starts at file offset 0x220, the loader might map it at virtual address 0x400220. Equality of low digits here is a convenient layout choice, not a universal rule.

Invariant: every requested file range is within the file, and every mapped memory range has a declared source or initialization rule. Never use an untrusted offset and length before checking their sum.

fn checked_range(data: &[u8], offset: usize, length: usize) -> Option<&[u8]> {
	let end = offset.checked_add(length)?;
	data.get(offset..end)
}

fn main() {
	let data = [10, 20, 30, 40];
	assert_eq!(checked_range(&data, 1, 2), Some(&data[1..3]));
	assert_eq!(checked_range(&data, usize::MAX, 2), None);
}

7. Offset, address, index, and name#

These identifiers are easy to confuse because tools print all of them as numbers or strings.

The number 7 might be a byte offset, an index, or a value; arithmetic does not reveal which. Likewise, a symbol called twice is not its address. Resolution may associate the name with a definition; layout may then give that definition an address.

Invariant: every coordinate carries its space. Robust code uses distinct types or names such as file_offset and virtual_address. Conversion requires explicit layout information and may be impossible for metadata that is never mapped.

8. CPU instructions and data#

A processor repeatedly fetches an instruction from its current instruction address, decodes it, and changes registers or memory. Registers are small processor-held values. Instructions can perform arithmetic, move data, compare values, branch, or call.

The same bytes can be instructions or data depending on how they are reached and mapped. Object formats group likely code and data so tools can preserve intent, but bytes do not carry intrinsic labels.

For our program, a target might keep 21 directly inside an instruction, compute 42, and pass it toward formatting code. Another compiler may precompute 42; an optimizing compiler may remove the body of twice entirely. The Rust language guarantees observable behavior, not a particular instruction sequence.

Separate three levels:

• Language guarantee: the Rust abstract behavior, subject to documented rules.
• ABI guarantee: binary-level calling and representation rules for an agreed interface.
• Implementation choice or tool convention: instruction selection, section names, and optimization details unless a platform specification fixes them.

9. Pages, mappings, and protections#

Operating systems generally manage virtual memory in fixed-size units called pages. A mapping associates a page-aligned virtual range with file bytes, anonymous zero-filled memory, or another backing source. Permissions commonly include read, write, and execute.

virtual pages in one process
[ code ]  read + execute  -> executable file region
[ data ]  read + write    -> executable file region
[ heap ]  read + write    -> anonymous memory
[ guard]  no access       -> catches some invalid growth

Concrete trace: startup needs to execute the page containing twice, so that mapping needs execute permission. The program need not write its code. Keeping writable data non-executable reduces the consequences of memory-corruption bugs.

Page size, permission combinations, and mapping APIs are platform properties. “All code starts at 0x400000” is a myth; address-space randomization, executable type, architecture, and operating system change placement.

Invariant: an access must target a mapped page with suitable permission. A protection fault is the visible symptom; the earliest broken invariant may be an incorrect relocation that produced the bad address.

10. Processes and address spaces#

A process is a running program instance with an address space and operating-system-managed resources. Two processes can use the same virtual address for unrelated bytes. Virtual addresses are therefore meaningful only with a process, or with a not-yet-running image layout.

An executable file is not a process. It lacks live registers, threads, runtime allocations, and open resources. One executable can start many processes. Shared library pages may share physical storage while appearing at different virtual addresses.

The loader establishes an initial invariant: mapped bytes, permissions, stack state, and entry address satisfy the platform process-start contract. It preserves executable contents and startup metadata but may choose runtime addresses that were not fixed in the file.

11. Functions, calls, and entry points#

A source function is a language concept. At machine level, a call generally transfers control while recording enough information to return. The caller and callee must agree where arguments, return values, and saved state reside.

An entry point is the address where execution begins under a particular contract. The executable entry is usually not Rust's main. It points to startup code prepared to receive the kernel's initial state. A shared library may have initialization entries. A thread may begin at yet another function under another contract.

Prediction. Could the kernel call main directly? Only if main obeyed the kernel's process-entry contract and all required runtime initialization happened elsewhere. Typical Rust environments use startup layers, so direct equivalence is false.

Invariant: every control transfer reaches code and both sides obey the same call contract. A valid address with the wrong contract can corrupt state before any crash.

12. Why source language is insufficient: ABIs#

An application binary interface (ABI) is a binary contract. It can specify register use, stack alignment, data layout, symbol spelling, object format, relocation meanings, and system-call boundaries. There are several related ABIs, not always one monolithic document.

Rust source types alone cannot connect independently built components. Should the first integer argument use a register or the stack? How is a structure returned? Which registers survive a call? Source syntax does not answer these target-specific questions.

Rust's ordinary ABI is not generally promised stable between compiler versions. extern "C" requests a supported C ABI for a function boundary, but it does not make every Rust type C-compatible. Use explicitly compatible representations and types.

Invariant: all separately compiled participants agree on the relevant ABI. Link success proves name/reference resolution, not type or call compatibility.

13. Targets and target triples#

A target name often resembles x86_64-unknown-linux-gnu. Its dimensions describe at least an architecture, a vendor field, an operating system, and an environment or ABI family. Exact spelling and fields are toolchain conventions; consult rustc --print target-list and the target specification for your compiler.

Architecture affects instructions, register rules, relocation forms, and endianness. Operating system affects executable format and startup contracts. Environment can select a C library or ABI variant. Pointer width is a target property, not inferred safely from the machine running the compiler during cross-compilation.

Counterexample: two targets with the same CPU architecture can use different object formats and calling rules. Conversely, a file format can support several architectures. “It is 64-bit” is not enough compatibility information.

Invariant: every input and output agrees with the selected target dimensions, or an explicit conversion exists. Linkers diagnose many mismatches, but not every semantic ABI mismatch is encoded for them to see.

14. Symbols: definitions, requests, and visibility#

A symbol is a named record used to connect facts. A defined symbol associates a name with a location or value. An undefined symbol requests a matching definition. Local symbols normally serve one object; global symbols can participate in broader resolution. Actual formats add weak, hidden, versioned, and other forms, postponed here.

Imagine splitting the example:

caller object: bytes for main; undefined symbol twice; relocation at call site
math object:   bytes for twice; defined global symbol twice at code offset 0

The linker resolves the request, then layout turns the definition's section-relative location into an address. Names preserve identity across separate compilation. Stripping local names can reduce metadata but makes diagnostics and debugging less descriptive.

Invariant: each required reference resolves to an allowed definition with compatible meaning. Multiple strong definitions or a missing required definition are ordinarily errors. A same-spelled function with the wrong ABI is a counterexample: name resolution succeeds while execution remains invalid.

15. Sections group bytes by purpose#

An object section groups bytes or reserved space that should receive similar treatment. Typical purposes include executable code, read-only constants, writable initialized data, zero-initialized storage, relocations, names, and debugging data. Exact names and rules depend on the format.

For the example, code for twice may occupy one code section, its symbol points to an offset within that section, and a call from another code section has a relocation. Formatting strings may occupy read-only data. Optimization can merge, split, or remove these groups.

Sections preserve purpose, alignment, and connection metadata so the linker can reorganize bytes. They forget much high-level source structure: a section is not necessarily one function, one module, or one memory page.

Invariant: section sizes, alignments, links, and contained ranges are internally valid. Do not infer “executable” solely from a familiar section name when the format provides authoritative flags.

16. Relocations are deferred equations#

A relocation says: when enough layout facts are known, compute a value and encode it at a particular place. It identifies a location to patch, a relocation kind, often a symbol, and sometimes an addend.

A common teaching equation for a PC-relative reference is:

value = S + A - P

S = final address of the referenced symbol
A = addend, a constant adjustment
P = address associated with the relocation place

The exact meaning of P, where A is stored, scaling, range checks, and bit placement come from the target relocation specification. The equation is not universal.

This complete program performs the abstract arithmetic with checks before narrowing:

fn relative_value(symbol: u64, addend: i64, place: u64) -> Option<i32> {
	let value = i128::from(symbol) + i128::from(addend) - i128::from(place);
	i32::try_from(value).ok()
}

fn main() {
	assert_eq!(relative_value(0x1010, -4, 0x1004), Some(8));
	assert_eq!(relative_value(u64::MAX, i64::MAX, 0), None);
}

Invariant: the chosen definition, arithmetic, range, and encoding all satisfy the relocation kind. Relocations preserve unresolved relationships rather than guessing addresses early.

17. A two-object hand link#

Use a fictional fixed-width machine to isolate the idea. Each instruction is four bytes. CALL rel32 is represented here as four little-endian bytes containing only a signed displacement. This is an educational encoding, not x86, Arm, or any real ABI.

The linker lays out two code sections:

There is padding from 0x1004 through 0x100f. math.o defines twice at its section offset zero, so S = 0x1010. The relocation field is at P = 0x1000. Our fictional instruction defines its displacement relative to the next instruction, represented with A = -4.

S + A - P = 0x1010 + (-4) - 0x1000 = 0x000c
encoded little-endian field: 0c 00 00 00

After linking:

Prediction. If math.o moves to 0x1020, what changes? S changes, so the field becomes 1c 00 00 00. The body bytes need not change. If the signed field cannot hold the result, the linker must report overflow or use a target-authorized alternative; wrapping is not faithful.

18. Static and dynamic binding#

Static binding completes a reference while producing the executable or library. Dynamic binding leaves work for process startup or, on some systems, first use. Static linking can copy selected library code into an output. Dynamic linking usually records dependencies and enough metadata for a runtime loader to locate definitions.

This is a timing distinction, not “static means no loader.” Even a fully statically linked executable still needs an operating-system loader to create mappings and begin execution. Dynamic binding can reduce duplicate on-disk code and permit library servicing, but introduces deployment, search-path, compatibility, and startup obligations.

Mechanism is symbol lookup and relocation. Policy decides search order, permitted providers, and whether replacement is allowed. Optimization may defer or cache lookup. Keep these separate when debugging.

Invariant: the definition selected at runtime obeys the promised identity and ABI. The executable may preserve names and relocation records that a fully statically resolved image could discard.

19. Sections are not segments#

Sections organize content for linking and analysis. Segments—or the analogous load commands in another format—describe ranges for loading into memory. One loadable region can contain several sections with compatible permissions. Some sections, such as debug information, need no runtime mapping.

link view:  [code section][constant section][data section][debug section]
                 |             |             |             X not loaded
load view:  [read+execute region]       [read+write region]

Concrete trace: bytes for twice and nearby read-only constants might be separate sections during linking yet covered by one or more load regions according to platform policy. Debug records can describe twice while remaining outside the process image.

Invariant: load descriptions cover the intended file and memory ranges without violating alignment or protection rules. Saying “the loader loads sections” is a useful beginner shorthand but is inaccurate for many executable formats.

20. Startup happens before main#

At successful execution, the kernel validates the executable, creates process state, maps required regions, establishes initial stack or equivalent startup data, and transfers control to the executable entry. A runtime loader may run first for dynamically linked images. Startup code initializes libraries and language support, processes constructors where the platform defines them, and calls toward Rust's main wrapper.

kernel entry transfer
  -> optional runtime loader
  -> executable startup entry
  -> C/platform runtime initialization
  -> Rust runtime wrapper
  -> user main
  -> termination path

Concrete trace: our main does not parse the kernel's raw startup stack, establish dynamic symbol bindings, or translate its return into process termination. Layers before and after it do that work. Names and exact ordering vary by target and toolchain.

Invariant: each layer receives the state promised by the previous contract. A crash before main can originate in malformed load metadata, a failed dynamic dependency, an initializer, or ABI-incompatible startup code.

21. Archives: libraries as searchable bags of objects#

An archive is, at intuition level, a container of object members plus an index that helps find members defining requested symbols. A static linker commonly extracts members needed to satisfy unresolved references rather than copying the entire archive.

Order can matter with traditional command-line linkers: when scanning an archive, the current unresolved set influences extraction. Linker groups or different linker algorithms can change this behavior; do not universalize one command's rule.

For our split example, libmath.a may contain math.o. If caller.o already requests twice, the archive index directs the linker to that member. Another unused member need not enter the executable.

Invariant: the archive index corresponds to its members, and extraction reaches all definitions required under the linker's documented policy. An archive preserves member boundaries and names, but final linking may discard both.

22. Debug and unwind metadata#

Debug metadata maps machine locations back toward source files, lines, variables, and types. It cannot perfectly reconstruct information optimized away. Unwind metadata describes how to recover caller state while walking the call stack; it can support exceptions, diagnostics, or profiling depending on platform and language runtime.

These records are metadata, not comments. Removing debug information generally preserves normal execution, while removing required unwind information can change behavior on systems that use it for language or platform unwinding. Requirements are target-specific.

Optimization may inline twice, leaving no standalone call while debug metadata reports an inlined source frame. This is not dishonesty: source identity and machine address are many-to-many after optimization.

Invariant: metadata ranges and rules match the code version and layout they describe. A stale separate debug file can produce believable but wrong stack traces.

23. Find the earliest broken invariant#

The first visible failure is often late. Use a causal chain rather than treating the crash site as the cause.

wrong target input
  -> relocation interpreted under wrong rule
  -> incorrect call address emitted
  -> executable loads successfully
  -> CPU branches into data
  -> protection fault appears

Concrete debugging workshop: suppose the example builds, starts, and faults before printing.

1. Establish whether main was reached with a debugger or temporary observable marker.
2. Inspect the faulting address and mapping permissions. Was execution attempted in non-code memory?
3. Decode the control-transfer instruction under the executable's actual architecture.
4. Identify the relocation or dynamic binding that supplied its destination.
5. Verify S, A, P, width, signedness, and selected symbol.
6. Then move earlier: verify each input object's target and ABI.

Do not patch the last symptom first. Restore the earliest invariant that should have made every later state valid.

24. First command-line laboratory#

This lab assumes a Unix-like shell and an ELF-based Linux target with GNU-style or compatible cc, readelf, objdump, nm, and ar. Command flags and output differ on macOS, Windows, BSDs, embedded targets, and LLVM-only installations. The concepts survive; consult native tools such as otool, dumpbin, or llvm-readobj there.

Create a disposable directory, then save the opening Rust program as main.rs.

rustc --version
rustc -vV
cc --version
rustc --emit=obj -C opt-level=0 main.rs -o main.o
file main.o
readelf -h main.o
readelf -S main.o
readelf -s main.o
readelf -r main.o
nm -a main.o | head
objdump -dr main.o | less
rustc -C opt-level=0 main.rs -o main
readelf -l main
./main

Before running nm, predict whether the source spelling twice must appear. Answer: not necessarily. Rust symbol mangling, internalization, code generation units, and optimization affect names. At optimization level zero it is often discoverable as part of a mangled symbol, but that is observation, not a stable ABI promise.

To see an archive without requiring Rust symbol compatibility, create a tiny C member:

printf '%s\n' 'int answer(void) { return 42; }' > answer.c
cc -c answer.c -o answer.o
ar rcs libanswer.a answer.o
ar t libanswer.a
nm libanswer.a

Questions to record: Which object sections have bytes? Which have only a memory size? Which undefined symbols remain? Which relocations correspond to calls or data references? Which executable regions are writable or executable? Do not infer semantics solely from names; compare flags and target documentation.

The laboratory preserves evidence: keep raw command output when diagnosing. Tools render metadata and may demangle names, so their display can forget exact stored spelling unless asked for raw output.

25. Foundational myths and counterexamples#

Judgment comes from asking which contract establishes a claim. If no language, ABI, format, or platform specification establishes it, treat observed behavior as a toolchain implementation choice.

26. Exercises, glossary, and synthesis#

Prediction and implementation exercises#

1. Bytes 34 12 are read as a little-endian u16 and as a big-endian u16. Predict both values, then verify with u16::from_le_bytes and u16::from_be_bytes. Answer: 0x1234 and 0x3412.
2. In the fictional machine, set S = 0x0ff0, A = -4, and P = 0x1000. Predict the signed result before calculating. Answer: -20, encoded according to the fictional signed 32-bit rule.
3. Extend checked_range to return a custom error distinguishing arithmetic overflow from out-of-file range. Test empty ranges at the end and one byte beyond it.
4. Define newtypes FileOffset(u64) and VirtualAddress(u64). Permit addition of a checked size, but deliberately provide no direct conversion between them. Explain what mapping record a conversion needs.
5. Split a C caller and provider into separate objects, inspect undefined and defined symbols, archive the provider, then link. Predict output before each command. Repeat with archive position changed and record whether your linker is order-sensitive.
6. Build the Rust example at optimization levels zero and three. Compare symbols and disassembly. Explain changes as implementation choices while identifying the preserved language behavior.
7. Debugging challenge: a loader reports a gigantic section size from a small file. List three earliest invariants to test before adding allocation limits. Answer should include format identity, byte order, and checked field/range decoding.
8. Counterexample design: invent two functions with the same external symbol spelling but incompatible argument conventions. Explain why the linker's available metadata may be insufficient to reject them.

Compact glossary#

Synthesis#

Our program began as Rust names and behavior. Compilation selected target operations and discarded source details unnecessary for later stages. Assembly encoded bytes while preserving unresolved relationships as symbols and relocations. Linking selected and arranged inputs, assigned addresses, checked equations such as S + A - P, and emitted a loadable image. The kernel created a process and mappings; a runtime loader could finish dynamic work; startup code established contracts before user main printed 42.

No stage makes uncertainty disappear for free. A representation makes some questions cheap and forgets answers to others. Symbols preserve identity but do not prove ABI compatibility. Relocations preserve arithmetic relationships but require exact target rules. Sections preserve link purpose; segments preserve load intent. Debug metadata preserves selected provenance but cannot reverse optimization perfectly.

The reusable debugging rule is to walk backward from the symptom and locate the earliest broken invariant. The reusable design rule is to label every coordinate, encoding, ownership boundary, and contract.

Source and reading map#

Prefer the specification matching the actual target and toolchain revision.

• Rust Reference: linkage — Rust compilation and linkage model; normative where explicitly stated by the Reference.
• Rust Reference: external blocks and ABI — Rust's source-level rules for foreign interfaces.
• rustc platform support — current target tiers and target-specific notes.
• System V ABI repository — x86-64 processor supplement and related ABI material; relevant only to matching System V targets.
• Arm ABI specifications — official Arm procedure-call and object-format specifications.
• ELF generic ABI — executable/object format concepts for ELF systems; Part III will study details.
• Linux execve(2) — Linux process-image replacement behavior, not a universal OS contract.
• GNU Binutils manuals and LLVM command guides — tool behavior and flags, which are conventions rather than language guarantees.
• DWARF standard — authoritative debug-information format material; later parts can study its machinery.

Readiness checklist for Part II#

You are ready to continue when you can:

• trace twice from source through object bytes, link decisions, mappings, startup, and execution;
• distinguish file offsets, section offsets, virtual addresses, indexes, and symbol names;
• decode an integer only after naming width, signedness, endianness, and overflow policy;
• explain why separate compilation requires symbols and relocations;
• calculate the fictional S + A - P example and state why real relocation rules are target-specific;
• distinguish sections from load regions and executable files from processes;
• separate language guarantees, ABI guarantees, tool conventions, and implementation choices;
• use basic inspection tools while stating their platform assumptions;
• investigate a late crash by searching for the earliest broken invariant.

Part II — Static Linking: Resolution, Layout, and Relocation#

1. The Static Linker's Contract#

Part I established that an object file is a typed collection of bytes plus names and fixups. Static linking turns several such collections into one output image. Its smallest useful model is:

ordered arguments
  │ carries files, options, and archive boundaries
  ▼
parse → resolve names → choose live sections → build output sections → assign addresses
                                                                       │
                                                                       ▼
                                                scan relocations → write relocations → emit

The order is conceptual. Real linkers overlap passes, but may do so only while preserving the same observable decisions.

Three categories keep the design honest:

The main invariant is stronger than “all names were found”: every retained reference denotes the selected definition, every allocated byte has one non-overlapping output location, and every encoded relocation represents its mathematical value within the target field's rules.

2. Inputs and Command Semantics#

A linker command is an ordered program, not merely a bag of paths. Inputs commonly include relocatable objects, static archives, shared-library import descriptions, linker scripts, and LTO carrier objects. Options may change the interpretation of later arguments: library search paths, whole-archive mode, symbol wrapping, and static/dynamic preference are examples.

-lfoo asks policy to search configured directories for a platform spelling such as libfoo.a; it is not an object itself. -u name creates an artificial undefined-symbol demand. --whole-archive changes archive extraction policy. A response file expands into arguments at its position, so expansion must retain order.

ELF, COFF, and Mach-O package these ideas differently. This part compares them only when that difference changes an algorithm. Do not infer command-line compatibility from object-format compatibility: GNU ld, gold, lld, mold, MSVC link, and Apple ld have distinct interfaces.

Prediction. If main.o needs parse, why can cc -lparse main.o fail while cc main.o -lparse succeeds with a traditional one-pass archive policy?

Answer. At the first archive encounter no unresolved parse demand exists, so no member is extracted. Ordinary object files are included unconditionally and create the demand too late. This is compatibility behavior of the driver/linker, not an ELF file-format law.

Information preserved after argument decoding: exact order, option scope, and provenance. Information discarded: response-file spelling and often search candidates not selected. Preserve these in diagnostics when practical.

3. The Relocatable Object Model#

Represent each input with stable identities rather than pointers into a growable buffer:

InputFile { id, path, kind, sections[], symbols[], relocations[] }
InputSection { file_id, index, name, type, flags, alignment, bytes }
Symbol { file_id, index, name, binding, visibility, section-or-special, value, size }
Relocation { target_section, offset, kind, symbol_index, addend-source }

An ELF ET_REL symbol value is normally relative to its defining section. COFF uses section numbers plus values. Mach-O uses section ordinals and nlist metadata. Normalize only what later passes need; retain raw type, flags, indices, and relocation kind for diagnostics and target-specific handling.

A section is not yet an output region. A symbol is not yet an address. A relocation is not yet arithmetic until resolution and layout provide its operands. Confusing those stages causes stale-address bugs.

Invariant after normalization: every internal reference identifies an existing input entity or a recognized special value, and no untrusted file offset remains unchecked.

4. Section Contributions and Provenance#

Each retained input section contributes an interval to an output section. Give every contribution a record:

Contribution {
  input_section_id,
  output_section_id,
  output_offset,
  size,
  alignment,
  live,
}

This record translates section-relative symbol values and relocation places. It also answers “which input caused output address 0x4012ac?” Keep it through map-file generation.

Input sections may be split, synthesized, merged, folded, or discarded. Therefore input identity and output location are different concepts. Layout discards freedom about ordering; it must preserve byte ownership, required alignment, and provenance. If garbage collection discards a section, references from retained sections to it must already have been redirected legally or diagnosed.

5. Strings, Symbols, and Relocations#

ELF string tables store NUL-terminated bytes; symbol records refer to offsets. A symbol table's linked section identifies its string table, and relocation sections identify both their target section and symbol table. SHT_REL stores an implicit addend in relocated bytes; SHT_RELA stores an explicit addend. The target ABI chooses which forms and relocation calculations apply.

Never treat a string-table offset as trusted text. Check that the offset is in range and that a terminating NUL exists. Never assume symbol zero or section zero means ordinary content; formats reserve sentinel entries.

COFF relocations identify symbol-table entries, while long names may use a string table. Mach-O scattered and paired relocation history makes normalization target-specific. The common invariant remains: a relocation's place lies wholly inside its target section, its referenced symbol exists, and its addend can be recovered without an out-of-bounds read.

6. Parsing Hostile Objects#

Object input can be truncated, malicious, or merely produced by a newer tool. Parsing is a security boundary. Check arithmetic before slicing:

fn table_range(offset: u64, count: u64, entry_size: u64, file_len: u64)
    -> Result<std::ops::Range<usize>, &'static str>
{
    let bytes = count.checked_mul(entry_size).ok_or("table size overflow")?;
    let end = offset.checked_add(bytes).ok_or("table end overflow")?;
    if end > file_len { return Err("table outside file"); }
    let start = usize::try_from(offset).map_err(|_| "offset exceeds host usize")?;
    let end = usize::try_from(end).map_err(|_| "end exceeds host usize")?;
    Ok(start..end)
}

fn main() {
    assert_eq!(table_range(8, 3, 4, 20), Ok(8..20));
    assert!(table_range(u64::MAX - 1, 2, 8, u64::MAX).is_err());
}

The complete program is stable and dependency-free. Apply the same pattern to alignment, relocation width, string termination, decompression limits, and index conversion. Check entry_size against the format's minimum before reading known fields; accepting larger entries may be forward-compatible if ignored tails are permitted.

Invariant after parsing: all stored slices refer to validated ranges, allocations obey configured limits, and every error names file, table, and offending index. The parser may discard padding bytes, but should not discard provenance or unknown bits needed to reject unsupported semantics.

7. Definition Kinds#

Resolution begins by classifying candidates, not by assigning addresses.

ELF binding and SHN_COMMON define the raw categories; language front ends and options such as GCC's historical -fcommon affect what is emitted. COFF “common” behavior and weak externals use different records. Mach-O tentative definitions have their own rules. Normalize semantic candidates without pretending their encodings are identical.

Visibility, symbol versioning, import status, and output kind can constrain eligibility. Those are policy filters around the core state machine.

8. A Formal Resolution State Machine#

For each global name, keep state Unseen, Undefined, Common, Weak, Strong, or Error, plus the prevailing candidate and all provenance. One bounded policy table is:

* is policy-sensitive. ELF linkers commonly prefer a defined weak symbol over common, while warnings and size/alignment handling vary. “Choose weak” is usually first-in-command-order for compatibility, but that is not a gABI guarantee. A real implementation should encode target/flavor policy explicitly rather than hiding it in enum ordering.

The following complete program implements a deterministic, simplified ELF-like resolver:

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum Kind { Undefined, Common { size: u64, align: u64 }, Weak, Strong }

#[derive(Clone, Debug, Eq, PartialEq)]
struct Choice { file_order: usize, kind: Kind }

fn resolve(mut old: Option<Choice>, new: Choice) -> Result<Choice, &'static str> {
    use Kind::*;
    let Some(ref current) = old else { return Ok(new) };
    match (current.kind, new.kind) {
        (Strong, Strong) => Err("duplicate strong definition"),
        (Strong, _) | (_, Undefined) => Ok(old.take().unwrap()),
        (_, Strong) | (Undefined, _) => Ok(new),
        (Weak, _) => Ok(old.take().unwrap()),
        (_, Weak) => Ok(new),
        (Common { size: a, align: aa }, Common { size: b, align: ba }) => Ok(Choice {
            file_order: current.file_order.min(new.file_order),
            kind: Common { size: a.max(b), align: aa.max(ba) },
        }),
    }
}

fn main() {
    let weak = Choice { file_order: 0, kind: Kind::Weak };
    let strong = Choice { file_order: 1, kind: Kind::Strong };
    assert_eq!(resolve(Some(weak), strong.clone()), Ok(strong));

    let undefined = Choice { file_order: 2, kind: Kind::Undefined };
    let common = Choice {
        file_order: 3,
        kind: Kind::Common { size: 8, align: 8 },
    };
    assert_eq!(resolve(Some(undefined), common.clone()), Ok(common));
}

Determinism comes from command order and explicit tie rules, not hash-map iteration. Production code also records visibility, versions, archive provenance, and why a candidate lost.

9. Resolution Policies and Counterexamples#

The mechanism gathers candidates and computes transitions. Policy decides duplicate handling, weak tie-breaking, common coalescing, interposition, and whether unresolved symbols are legal in the selected output kind.

Counterexample: “largest definition wins” is appropriate for common size, but disastrous for two strong functions: silently choosing one hides an ODR or build error. Counterexample: “first candidate always wins” makes an early undefined symbol suppress a later definition. Counterexample: using lexical path order rather than command order can change weak selection and archive behavior.

Preserve every candidate long enough to issue a diagnostic containing both locations. Resolution discards non-prevailing semantic choices, but should retain their provenance and reason codes.

Prediction. A weak definition appears before a strong definition in an archive member that is never extracted. Which wins?

Answer. The weak definition. An unextracted member is not an input candidate. Archive extraction and symbol resolution interact; an archive index is a promise about possible members, not a set of definitions already present.

10. Archive Indexes and Demand Extraction#

An archive is a member container plus, usually, an index from global names to member offsets. The linker extracts a member when a currently unresolved demand can be satisfied by that member. Extracting it may add new undefined demands, so use a worklist.

use std::collections::{BTreeMap, BTreeSet, VecDeque};

fn extract<'a>(
    index: &BTreeMap<&'a str, usize>,
    members: &[Vec<&'a str>],
    initial: &[&'a str],
) -> Vec<usize> {
    let mut queue: VecDeque<&str> = initial.iter().copied().collect();
    let mut needed: BTreeSet<&str> = initial.iter().copied().collect();
    let mut loaded = BTreeSet::new();
    let mut order = Vec::new();
    while let Some(name) = queue.pop_front() {
        let Some(&member) = index.get(name) else { continue };
        if !loaded.insert(member) { continue; }
        order.push(member);
        needed.remove(name);
        for &new_need in &members[member] {
            if needed.insert(new_need) { queue.push_back(new_need); }
        }
    }
    order
}

fn main() {
    let index = BTreeMap::from([("parse", 0), ("scan", 1), ("alloc", 2)]);
    let members = vec![vec!["scan", "alloc"], vec![], vec![]];
    assert_eq!(extract(&index, &members, &["parse"]), vec![0, 1, 2]);
}

This educational program models each member only by new demands; production extraction must immediately feed all definitions and references through resolution. Exact trace:

Invariant: each member is extracted at most once, and the worklist reaches a fixed point for the archive scope. Thin archives store external member paths; validate those files and preserve archive/member provenance.

11. Order, Rescanning, and Groups#

Traditional Unix archive processing searches an archive at its command position and rescans that archive's members until no new member is needed. It does not necessarily return to earlier archives. Thus -lA -lB can fail if an extracted B member introduces a demand satisfied only by A.

--start-group -lA -lB --end-group requests repeated searches until a group-wide fixed point. This is linker command behavior, not ELF semantics, and may cost much more because indexes and resolution state are revisited. Repeating -lA -lB -lA can solve a bounded cycle but encodes implementation detail into the command.

Whole-archive policy extracts every member, useful for registration objects but likely to increase size and duplicate-definition exposure. A modern linker may implement a faster equivalent algorithm, but observable extraction, weak choices, constructors, and diagnostics must match its promised flavor.

12. COMDAT and Duplicate Section Families#

COMDAT-like facilities permit duplicate contributions to be selected as a unit. They are mechanism for front ends to emit templates, inline functions, and generated metadata in many objects.

ELF SHT_GROUP with GRP_COMDAT groups sections under a signature symbol: the linker keeps one matching group and discards duplicate groups as units. COFF COMDAT puts selection policy on section definitions (ANY, SAME_SIZE, EXACT_MATCH, LARGEST, and others). An associative COMDAT is retained or discarded with its associated leader; it is not merely another independently keyed duplicate. Mach-O commonly relies on coalesced/weak definitions and dead stripping rather than ELF section groups.

Do not reduce all three to “same name means deduplicate.” ELF section names do not establish COMDAT identity; COFF selection kinds can demand validation; associative edges affect liveness. Preserve group membership and signature before resolution and garbage collection. Once a winner is selected, redirect references consistently and record discarded-to-winner provenance for diagnostics.

13. Merging Contributions#

Output construction typically groups compatible live input sections by output name, type, flags, permissions, and script rules. Compatibility is policy: merging writable data into executable text may violate security expectations even if bytes fit.

For each contribution, align the current output offset, assign it, then advance by size. Alignment is usually a nonzero power of two in ELF, but validate target rules rather than relying on the formula accidentally.

fn align_up(value: u64, align: u64) -> Result<u64, &'static str> {
    if align == 0 || !align.is_power_of_two() { return Err("invalid alignment"); }
    let mask = align - 1;
    value.checked_add(mask).map(|v| v & !mask).ok_or("alignment overflow")
}

fn layout(sizes_and_alignments: &[(u64, u64)]) -> Result<Vec<u64>, &'static str> {
    let mut cursor = 0u64;
    let mut offsets = Vec::with_capacity(sizes_and_alignments.len());
    for &(size, align) in sizes_and_alignments {
        cursor = align_up(cursor, align)?;
        offsets.push(cursor);
        cursor = cursor.checked_add(size).ok_or("section size overflow")?;
    }
    Ok(offsets)
}

fn main() {
    assert_eq!(layout(&[(3, 1), (2, 4), (1, 8)]), Ok(vec![0, 4, 8]));
}

Padding contains no contribution but is output state; choose deterministic fill bytes. Layout preserves contribution order, bytes, and alignment while discarding arbitrary placement freedom.

14. Output Sections and Segments#

An output section owns a file interval and, if allocated, a virtual-address interval. Program segments describe loader mappings and may cover several sections. Static linking still constructs segments when producing an executable; section headers and program headers answer different consumers.

input .text.a ─┐
input .text.b ─┼─► output .text ─┐
thunk bytes ───┘                 ├─► executable PT_LOAD mapping
output .rodata ──────────────────┘

NOBITS storage such as .bss consumes memory size but normally no file payload. File offset and virtual address therefore cannot share one cursor universally. Segment congruence and page alignment are target/output constraints.

Invariant after output construction: each contribution has exactly one owner or an explicit discard reason; output permissions and types are coherent; file and memory sizes are separately represented.

15. Address Assignment#

After output order and alignments stabilize, assign virtual addresses. For a defined symbol:

S = output_section.address + contribution.output_offset + input_symbol.value
P = output_section.address + contribution.output_offset + relocation.offset

Check each addition. Verify the symbol value and relocation field fit inside the input contribution before translation. Absolute symbols bypass section translation; common symbols first receive allocated storage; undefined weak behavior is ABI/policy-specific.

Layout can require iteration: inserting thunks changes sizes, which changes addresses, which can create additional out-of-range branches. A safe algorithm iterates monotonically or uses a proven spacing scheme and detects failure to converge.

The address pass discards placement freedom but preserves ordering, alignment, non-overlap, and segment constraints. Emit a map snapshot here; it is the best evidence when relocation failure is only the first visible symptom.

16. Relocation Algebra#

ABI documents use conventional terms:

Letters are notation, not a universal executable language. The target ABI defines each relocation's expression, field, scaling, signedness, overflow rule, and instruction validation. For example, S + A - P is a mathematical integer before encoding; truncating first changes the question.

ELF gABI describes relocation structure, but processor supplements are normative for processor-specific kinds. The System V gABI 4.3 draft is the current generic reference cited here; the AMD64 psABI and Arm AAELF64 define target formulas and constraints.

17. x86-64 Relocations by Hand#

For these examples, compute in unbounded mathematical integers, then range-check.

Absolute 64-bit. R_X86_64_64 writes S + A as 64 bits. If S = 0x401000 and A = 0x18, result 0x401018 is encoded little-endian as 18 10 40 00 00 00 00 00.

PC-relative 32-bit. R_X86_64_PC32 writes S + A - P into a signed 32-bit field. Let S = 0x401080, A = -4, and P = 0x401020:

0x401080 - 4 - 0x401020 = 0x5c = 92
bytes: 5c 00 00 00
range: -2^31 ≤ 92 ≤ 2^31-1, so valid

The -4 commonly accounts for x86's next-instruction convention encoded by the assembler, but use the actual relocation addend rather than guessing instruction length.

PLT-relative. R_X86_64_PLT32 uses L + A - P; a static linker may relax or resolve it directly when binding is non-preemptible, as allowed by psABI/linker policy. Conceptually L remains distinct from S.

32 versus 32S. R_X86_64_32 requires that the 64-bit result equal zero-extension of its low 32 bits. 0xffff_ffff passes; 0x1_0000_0000 fails. R_X86_64_32S requires equality after sign-extension: -1 passes and encodes ff ff ff ff; positive 0xffff_ffff does not represent the same mathematical value after sign-extension and fails.

Prediction. With S=0x9000_0000, A=0, P=0, does PC32 fit? Answer: no; 0x9000_0000 = 2,415,919,104, greater than 2,147,483,647.

18. Checked Relocation Writes#

Separate expression evaluation from field encoding. This complete program writes a signed little-endian 32-bit displacement:

fn write_pc32(buf: &mut [u8], offset: usize, s: i128, a: i128, p: i128)
    -> Result<(), &'static str>
{
    let value = s.checked_add(a).and_then(|v| v.checked_sub(p))
        .ok_or("relocation arithmetic overflow")?;
    let value = i32::try_from(value).map_err(|_| "PC32 out of range")?;
    let end = offset.checked_add(4).ok_or("write offset overflow")?;
    let dst = buf.get_mut(offset..end).ok_or("relocation write outside section")?;
    dst.copy_from_slice(&value.to_le_bytes());
    Ok(())
}

fn main() {
    let mut bytes = [0u8; 4];
    write_pc32(&mut bytes, 0, 0x401080, -4, 0x401020).unwrap();
    assert_eq!(bytes, [0x5c, 0, 0, 0]);
    assert!(write_pc32(&mut bytes, 0, 0x9000_0000, 0, 0).is_err());
}

For unsigned 32-bit, convert to u32; for 32S, convert to i32 and compare the sign-extended value to the original. Avoid casts whose truncation silently “proves” fit. Validate the destination before mutation so an error leaves output deterministic.

19. AArch64 Breaks the Plain-Integer Illusion#

AArch64 relocations often patch instruction fields rather than contiguous integers. CALL26/JUMP26 encode a signed, scaled branch displacement: the low two address bits are implicit, and the immediate occupies selected instruction bits. The linker must verify instruction shape where required, alignment, divisibility by four, and signed range before inserting bits.

Page relocations form addresses in pieces. ADRP computes a page-relative delta, conceptually Page(S + A) - Page(P), while a paired low-12 relocation supplies page offset bits to ADD or a load/store whose field may be scaled by access size. Pairing is semantic even when records are not adjacent. Applying each as an arbitrary integer write would corrupt opcodes.

P: ADRP x0, page_delta(symbol)  ──► x0 = page(S + A)
   ADD  x0, x0, lo12(symbol)    ──► x0 = S + A
        instruction fields preserve opcode/register bits

A veneer is synthesized code placed within branch range; the original branch targets the veneer, which reaches the final destination through a longer sequence. This demonstrates that relocation is not always “write the answer”: it may require layout changes and new symbols/relocations.

Normative details come from AAELF64 and calling constraints from AAPCS64. Release dates matter because Arm ABI documents evolve; record the cited release in production design notes.

20. Scan First, Apply Later#

Relocation scanning discovers requirements before bytes are final: GOT entries, PLT entries, dynamic relocations, TLS models, thunk candidates, text-relocation hazards, and liveness edges. Application happens only after those decisions and layout stabilize.

scan relocation --kind/binding/range--> allocate synthetic structures
       │                                      │ changes sizes
       └──────────── record decision ◄────────┘
                              │ stable layout supplies S/P/L/GOT
                              ▼
                         apply field

The scan preserves relocation identity, target, and decision reason; it may discard impossible alternatives after policy commits. The apply pass must not unexpectedly allocate. Its invariant is stronger: all prerequisites exist and every write is in bounds and range.

The GOT is an address table and the PLT is a call trampoline mechanism. That intuition is enough here. Their dynamic binding, lazy resolution, hardening, and target-specific forms belong to Part III.

21. Thunks and Relaxation#

Thunks or range-extension stubs solve branches whose direct encoding cannot reach. Policy chooses where islands go and which calls share a thunk; mechanism emits target-valid bytes and relocations. Iterative insertion must terminate or report a bounded convergence failure.

Relaxation replaces a general sequence with a smaller or faster equivalent after binding and distance are known—for example, converting an indirect form to a direct form where the ABI permits. It is an optimization, not resolution. It must preserve the observation model: values, control flow, required instruction boundaries, unwind information, and externally observable symbol addresses.

Relaxation may shrink sections and invalidate addresses. Designs either reserve sizes, apply a monotone shrink algorithm with recomputation, or use target proofs that later changes cannot break earlier choices. Preserve a transformation log; otherwise a disassembler mismatch is difficult to explain.

22. Section Garbage Collection#

With section GC enabled, construct a directed graph. Nodes are input sections or indivisible groups. A relocation from live section A to definition in B adds A → B. Associative COMDAT and metadata add special edges.

Roots include the entry section, script KEEP selections, exported symbols under relevant policy, retained/init arrays, and linker-synthesized necessities. Traverse from roots; discard unreachable nodes.

[entry .text] ─call─► [parse .text] ─data─► [table .rodata]
      root
[unused .text] ─────► [unused .rodata]     both unreachable: discard

Undefined references in discarded sections normally cease to matter; unresolved diagnostics should therefore respect liveness timing. But options and output modes differ. The graph mechanism is stable; root and diagnostic timing are policy.

Counterexample: treating every symbol-table definition as a root defeats GC. Treating only call relocations as edges can delete referenced data or unwind helpers. The invariant is that every semantically required section is reachable under the selected root policy.

23. ICF and Mergeable Constants#

Identical code folding partitions functions by bytes, relocation structure, and target semantics, then merges equivalent partitions. It is an optimization beyond COMDAT: candidates can have different names. Safe and aggressive modes differ over whether distinct function addresses are observable.

fn a() {}
fn b() {}

Source-level emptiness does not prove that folding a and b is legal if a program compares their addresses or tooling expects distinct identities. Language, ABI, debug, sanitizer, and linker options define the observation model. Therefore ICF needs explicit policy and a map from folded identity to representative.

ELF mergeable sections (SHF_MERGE, optionally SHF_STRINGS) allow fixed-size constants or strings to share storage. String suffix merging can make "world" point inside "hello world". Validate entity size and terminators; relocations into mergeable entities complicate equality. Merging discards storage identity while preserving content and permitted references.

24. Scripts, Orphans, and Initialization#

Linker scripts can define output sections, ordering, addresses, memory regions, symbols, assertions, and retention. This is layout policy expressed as a language. Parse expressions with checked arithmetic and distinguish absolute values from section-relative values.

An orphan is an allocatable input section unmatched by explicit script rules. Placement differs among linkers and versions; it is compatibility behavior, not a generic ELF guarantee. Security-sensitive builds should use explicit sections or orphan warnings/errors rather than assume a location.

Initialization arrays such as ELF .preinit_array, .init_array, and .fini_array contain function pointers interpreted by runtime rules. The linker must preserve required ordering, apply priority conventions supported by its flavor, retain entries under GC, and relocate them. COFF uses ordered .CRT$... contributions by toolchain convention; Mach-O uses initialization sections with its own runtime contract. Similar purpose does not imply interchangeable sorting.

Debugging lab. Add a uniquely named input section, link with a map file, and inspect its output location with readelf -SW. Then introduce an explicit script rule. Predict the old orphan placement before relinking; compare map provenance, flags, and segment permissions.

25. Unwind and Debug Information#

Unwind tables are runtime correctness data, not optional decoration. On ELF, .eh_frame records contain encoded references and may be deduplicated or accompanied by a synthesized header. GC and ICF must update or discard records with their code. A malformed range can make exceptions or stack walking fail far from link time.

Debug sections may be non-allocated but still contain relocations, string-offset tables, address/range lists, and cross-unit references. A linker can apply relocations, compress sections, or emit separate debug artifacts. Preserve enough provenance for source-level debugging; aggressive merging must update every dependent offset.

Mechanism parses and rewrites records. Policy chooses stripping and compression. Optimization deduplicates or indexes them. Invariant: every retained metadata reference denotes the final code/data range it describes, or is explicitly represented as unavailable according to the format.

26. LTO as a Protocol#

An LTO input carries intermediate representation plus a symbol summary rather than final machine sections. The linker first performs enough global resolution to tell the compiler plugin which symbols prevail, which may be internalized, and which are visible or referenced externally. The backend returns native objects, after which ordinary resolution/layout continues.

IR summaries + native symbols
          │
          ▼
provisional resolution ──prevailing/visibility/liveness facts──► LTO backend
          ▲                                                        │
          └──────────── generated native objects ──────────────────┘

This boundary is a protocol: the linker must not claim a weak IR definition prevails when a strong native definition wins. Archive extraction may be summary-driven. ThinLTO can compile modules in parallel, but cache keys must include semantic options and prevailing-symbol information.

Information discarded by internalization is external name visibility; it is legal only after the protocol proves no required observer needs it. Preserve module/symbol provenance through generated objects for diagnostics.

27. Determinism, Build IDs, and Diagnostics#

Reproducible output requires deterministic iteration, stable tie-breaking, normalized timestamps, controlled paths, deterministic archive metadata, and either deterministic parallel reduction or ordered commit. A deterministic archive index is insufficient if member headers contain varying times or IDs.

A build ID is an identifier derived or assigned according to linker policy. Hash-based IDs must define which bytes are covered and avoid a circular dependency by reserving/normalizing the ID field during hashing. They identify content only under that algorithm; they are not a universal security signature.

Map files should report output ranges, input contributions, selected symbols, archive extraction reasons, common allocation, discarded sections, and synthetic structures. Good errors state the earliest known broken invariant:

app.o:(.text.start+0x17): R_X86_64_PC32 against `target` = 0x90000000
place 0x0, addend 0: result 2415919104 exceeds signed 32-bit range
selected definition: far.o:(.text.target); consider code model or layout

The exact style is illustrative, not a claim about one linker.

28. Performance Cost Model#

Let F be input files, S symbols, R relocations, C contributions, A archive members considered, and B emitted bytes. A conventional design aims near O(S + R + C + B) expected time, plus archive index lookups and format parsing. Hash tables offer expected symbol lookup; deterministic trees cost O(log S) but stable iteration can also be achieved by sorting output separately.

Memory is often dominated by mapped input, symbol/relocation records, strings, and output buffers: roughly O(S + R + C + B) unless streaming or sparse output reduces residency. Group rescanning can revisit archive state; naive ICF pairwise comparison is O(n²) and should use hashes plus structural refinement.

Measure representative workloads: many tiny objects, a few huge debug objects, archive cycles, high relocation density, LTO, and cold versus warm filesystem cache. Record wall time, CPU time, peak RSS, bytes read/written, and per-pass counts. Do not optimize parse speed by dropping bounds checks; improve batching, allocation, interning, or parallelism while retaining invariants.

Parallel parsing is usually safe because files are independent. Resolution commit, archive extraction, layout, and deterministic diagnostics need ordered coordination. An optimization is valid only if it preserves command semantics and reproducible output.

29. A Complete Static-Link Trace#

Consider ld start.o -lmath util.o with archive libmath.a:

start.o: .text.start defines _start; undefined square; PC32 relocation to square
libmath.a(square.o): .text.square defines weak square; undefined table
libmath.a(table.o): .rodata.table defines table
util.o: .text.square defines strong square; .init_array points to init

Assume traditional positional extraction and no duplicate error between weak and strong.

Notice that square.o remains extracted even though its weak definition later loses; its table reference may become dead if GC discards that section. Extraction is not retroactively undone unless a linker implements a semantics-preserving advanced optimization.

Prediction. If the square PC32 relocation overflows, is relocation necessarily the first broken pass? Answer: no. The wrong square may have prevailed during resolution, an orphan policy may have placed it far away, excess alignment may have expanded layout, or a missing thunk decision may have occurred during scan. Relocation application is merely where representability becomes unavoidable.

30. Failure Workshops and Symptom Map#

Object-level lab 1. Compile two tiny files with cc -c -ffunction-sections, inspect readelf -Ws -r -SW, then link with a map and GC report. Before linking, predict each relocation's target section and which unused function dies.

Object-level lab 2. Patch a test object's relocation symbol index to an out-of-range value in a disposable copy. A robust parser must reject it before resolution, naming the relocation record. Do not run malformed binaries.

Object-level lab 3. Place caller and callee far apart with a linker script on a target/test setup where that distance exceeds the branch field. Record S, A, and P; distinguish script layout from missing range extension. Keep this bounded to throwaway artifacts.

Debug from the earliest invariant: bytes and indices, candidate eligibility, extraction, liveness, placement, scan decision, then encoding. The first visible relocation error may be delayed evidence of a resolution or layout mistake.

31. Implementation Path, Review Checklist, and Sources#

Build in milestones, with differential tests against a chosen linker flavor rather than an imaginary universal linker:

1. Beginner: parse one little-endian relocatable format subset; dump sections, symbols, and relocations with checked ranges.
2. Builder: implement local/global/weak/common resolution and exact transition-table tests, including permutation tests for intentional order sensitivity.
3. Builder: add indexed archive extraction and fixed-point traces; test cycles, stale indexes, thin members, and groups.
4. Intermediate: merge sections, allocate common storage, assign aligned addresses, and emit a map before writing an executable.
5. Intermediate: implement a tiny target subset such as x86-64 64, PC32, 32, and 32S; differential-test bytes and overflow diagnostics.
6. Advanced: add COMDAT units, graph GC, init arrays, and unwind retention. Create regressions where each metadata edge is the only live edge.
7. Advanced: implement AArch64 branch relocation and bounded veneer insertion; prove termination/range coverage for the chosen layout.
8. Contributor: find one upstream linker issue involving archive order, relaxation, or diagnostics; reduce it to objects, locate the earliest broken invariant, add a target test, and explain compatibility impact.

Production review checklist:

• [ ] Every file offset, count, multiplication, alignment, address addition, and write range is checked.
• [ ] Resolution policy is explicit, target/flavor-scoped, deterministic, and retains losing provenance.
• [ ] Archive extraction reaches the promised fixed point without extracting members twice.
• [ ] COMDAT/group/associative relationships survive resolution and GC.
• [ ] Every contribution is live, discarded with reason, or synthesized with provenance; intervals do not overlap.
• [ ] Relocation scan allocates all required GOT/PLT/thunk structures before application.
• [ ] Each relocation checks expression arithmetic, destination bounds, signedness, scaling, instruction shape, and field range.
• [ ] Relaxation, ICF, mergeable data, GC, unwind, and debug transformations state their observation model.
• [ ] Scripts, orphans, constructors, common storage, and undefined handling have tested policies.
• [ ] LTO receives correct prevailing/visibility facts and generated objects re-enter ordinary checks.
• [ ] Outputs, archives, maps, diagnostics, and build IDs are reproducible under varied parallel schedules.
• [ ] Resource limits and cancellation cover huge tables, strings, debug data, and adversarial archives.

Authoritative source map:

• Generic ELF, normative format semantics: System V ABI gABI 4.3 draft, especially Chapters 4–7 for object files, sections, symbols, relocation, and program loading. Processor formulas are outside its generic scope.
• x86-64 ELF, normative ABI plus maintained project text: AMD64 psABI repository, relocation types and linker optimization chapters. Pin a revision when behavior matters.
• AArch64 ELF and calls, normative Arm ABI releases: AAELF64 and AAPCS64 release bundle. Read AAELF64 relocation operations beside AAPCS64 calling rules; cite a release tag.
• PE/COFF, normative Microsoft format documentation: PE Format, including COFF symbols, relocations, COMDAT selection, and PE images. Tool command behavior remains implementation-specific.
• Current implementation reading: LLVM lld's ELF/, COFF/, and MachO/ directories; mold's ELF input, symbol, relocation, and output-chunk code; GNU binutils BFD/ld archive and script machinery. Source code describes that revision, not an ABI guarantee.

For source reading, follow one symbol end to end: object parser → candidate insertion → archive trigger → prevailing selection → input-section liveness → output contribution → address → relocation scan → target relocation writer → map diagnostic. Then follow one discarded COMDAT and one range-extension thunk. At each boundary, write down what information is preserved, what is discarded, and which invariant would make the next visible failure possible.

Part III — ELF and Dynamic Linking from Bytes to Bindings#

This part follows one question: how does a name in one file become an address in a running process? We begin with untrusted bytes, build the load image, and finish at symbol lookup, relocation, thread-local storage, and hardening. “Must” below belongs only to the authority named in the sentence.

1. Authority: which document gets to decide?#

ELF is a family of contracts, not one universal file format. Read claims in this order:

The 2025 ELF draft is at Xinuos ELF 4.3 public review. The AMD64 supplement is x86-64 psABI 1.0, a continuously maintained document whose implementation-sensitive citations should pin a commit, and Arm's versioned releases are at abi-aa 2025Q4.

Rule of evidence. A GNU loader accepting DT_GNU_HASH does not make that tag a gABI guarantee. An AMD64 relocation equation says nothing about AArch64. Current glibc behavior is implementation evidence, not a promise to every ELF consumer.

Prediction. A file obeys the generic ABI but uses an unknown machine number. Can a loader run it? Answer: no. Generic structure is readable, but instruction semantics and relocation rules come from the matching architecture ABI.

2. Class, byte order, and the identification bytes#

Every ELF file starts with 16 identification bytes, e_ident:

file offset  00 01 02 03 04       05        06       07       08........0f
meaning      7f  E  L  F  CLASS    DATA      VERSION  OSABI    padding
CLASS        1 = ELF32, 2 = ELF64
DATA         1 = little-endian, 2 = big-endian

ELF32 and ELF64 are different layouts, not merely different pointer interpretations. For example, an ELF64 header is 64 bytes and puts eight-byte fields at offsets 24, 32, and 40. An ELF32 header is 52 bytes and uses four-byte versions there. EI_DATA controls every multi-byte integer in the file. Never cast a byte pointer to a Rust or C structure: alignment, host endianness, padding, and untrusted lengths all make that wrong.

The complete, dependency-free Rust module below reads the fixed ELF64 header. It is complete library code (there is deliberately no main) and rejects unknown class, byte order, short input, wrong record size, and impossible table geometry.

#[derive(Debug, Clone, Copy)]
pub enum Endian { Little, Big }

#[derive(Debug)]
pub struct Elf64Header {
    pub endian: Endian,
    pub kind: u16,
    pub machine: u16,
    pub entry: u64,
    pub phoff: u64,
    pub shoff: u64,
    pub phentsize: u16,
    pub phnum: u16,
    pub shentsize: u16,
    pub shnum: u16,
    pub shstrndx: u16,
}

fn u16_at(b: &[u8], p: usize, e: Endian) -> Result<u16, &'static str> {
    let a: [u8; 2] = b.get(p..p + 2).ok_or("truncated u16")?.try_into().unwrap();
    Ok(match e { Endian::Little => u16::from_le_bytes(a), Endian::Big => u16::from_be_bytes(a) })
}
fn u64_at(b: &[u8], p: usize, e: Endian) -> Result<u64, &'static str> {
    let a: [u8; 8] = b.get(p..p + 8).ok_or("truncated u64")?.try_into().unwrap();
    Ok(match e { Endian::Little => u64::from_le_bytes(a), Endian::Big => u64::from_be_bytes(a) })
}

pub fn parse_elf64(b: &[u8]) -> Result<Elf64Header, &'static str> {
    if b.get(0..4) != Some(b"\x7fELF") { return Err("bad magic"); }
    if b.get(4) != Some(&2) { return Err("not ELF64"); }
    let endian = match b.get(5) {
        Some(1) => Endian::Little,
        Some(2) => Endian::Big,
        _ => return Err("unknown byte order"),
    };
    if b.get(6) != Some(&1) { return Err("unsupported ELF identification version"); }
    if b.len() < 64 || u16_at(b, 52, endian)? != 64 { return Err("bad ELF64 header size"); }
    let h = Elf64Header {
        endian, kind: u16_at(b, 16, endian)?, machine: u16_at(b, 18, endian)?,
        entry: u64_at(b, 24, endian)?, phoff: u64_at(b, 32, endian)?,
        shoff: u64_at(b, 40, endian)?, phentsize: u16_at(b, 54, endian)?,
        phnum: u16_at(b, 56, endian)?, shentsize: u16_at(b, 58, endian)?,
        shnum: u16_at(b, 60, endian)?, shstrndx: u16_at(b, 62, endian)?,
    };
    if h.phnum != 0 && h.phentsize != 56 { return Err("bad ELF64 program-header size"); }
    if h.shnum != 0 && h.shentsize != 64 { return Err("bad ELF64 section-header size"); }
    Ok(h)
}

e_version at offset 20 must also be checked by a full parser. So must e_machine, e_type, ABI policy, and every later referenced range. Parsing a header establishes structure, not trust.

3. Overflow-safe ranges and bounded table slices#

For file length F, table offset O, count N, and entry size E, safety requires:

E >= required_record_size
N * E does not overflow
O + N * E does not overflow
O + N * E <= F

Converting a file's u64 to usize can itself fail on a 32-bit host. This complete library function performs the conversions and checked arithmetic before returning a bounded slice:

pub fn table_slice(
    file: &[u8], offset: u64, count: u64, stride: u64, minimum: u64
) -> Result<&[u8], &'static str> {
    if stride < minimum || (count != 0 && stride == 0) { return Err("invalid stride"); }
    let bytes = count.checked_mul(stride).ok_or("table size overflow")?;
    let end = offset.checked_add(bytes).ok_or("table end overflow")?;
    let start = usize::try_from(offset).map_err(|_| "offset too large")?;
    let end = usize::try_from(end).map_err(|_| "end too large")?;
    file.get(start..end).ok_or("table outside file")
}

Do not “fix” malformed values by clamping them. Clamping changes one corrupt object into another object whose meaning the producer never supplied.

4. Extended numbering and section zero#

Sixteen-bit header fields cannot count all possible sections or program headers. The gABI uses reserved values and section-header entry zero as an escape record:

This creates an ordering rule: validate and read section header zero before calculating a table from an escaped count. A file with no section table has e_shoff == 0; do not mistake that for an extended zero count. After decoding, re-run multiplication and bounds checks with the wide count.

Counterexample. e_phnum = 0xffff, e_shoff = 0, and a valid-looking program table is not self-describing extended numbering. The required carrier record is absent; reject it.

5. Section headers and string tables#

A section header describes a linker-oriented region. ELF64 fields are:

offset  size  field
0x00      4  sh_name       byte offset in section-name string table
0x04      4  sh_type
0x08      8  sh_flags
0x10      8  sh_addr       runtime address when allocated
0x18      8  sh_offset     file offset
0x20      8  sh_size
0x28      4  sh_link       type-specific section index
0x2c      4  sh_info       type-specific value/index
0x30      8  sh_addralign  0, 1, or power of two
0x38      8  sh_entsize    fixed entry size, or zero

e_shstrndx chooses a SHT_STRTAB; sh_name indexes its bytes. A valid string starts inside that table and reaches a NUL before the table ends. Names such as .text and .dynamic are convention, not loading instructions. Other string tables hold symbol names or version dependency names; sh_link identifies the relevant one.

SHT_NOBITS occupies memory but no file bytes. Therefore sh_offset + sh_size <= file_size is not the right check for SHT_NOBITS; still validate its alignment and address arithmetic.

6. Program headers: the execution view#

For ELF64, each 56-byte program header contains p_type, p_flags, then six 64-bit values: p_offset, p_vaddr, p_paddr, p_filesz, p_memsz, p_align. Unlike ELF32, flags come second.

Important types are PT_LOAD, PT_DYNAMIC, PT_INTERP, PT_NOTE, PT_PHDR, and PT_TLS, plus GNU extensions PT_GNU_RELRO, PT_GNU_STACK, and PT_GNU_PROPERTY. The kernel follows program headers. It does not look up .text, .data, or any other section name to load a process.

For every segment, first demand p_filesz <= p_memsz, a bounded file range, bounded virtual addition, and suitable alignment. PT_INTERP must be a bounded NUL-terminated path under the platform ABI. A loader commonly maps PT_LOAD records and transfers control through the named interpreter for dynamic executables.

7. Sections versus segments: exact containment#

Sections support linking and analysis; segments support mapping and execution. One load segment may contain many sections, and a section can be absent from all runtime segments.

For an allocated, file-backed section S and segment P, containment requires both:

P.p_offset <= S.sh_offset
S.sh_offset + S.sh_size <= P.p_offset + P.p_filesz
P.p_vaddr <= S.sh_addr
S.sh_addr + S.sh_size <= P.p_vaddr + P.p_memsz
S.sh_addr - P.p_vaddr = S.sh_offset - P.p_offset

For SHT_NOBITS, use the memory interval, not file extent. TLS sections require the TLS segment's special mapping rules. Non-SHF_ALLOC sections need not occur in a PT_LOAD at all.

file:    [ELF][PHDR][ RX bytes: .text .rodata ][ RW bytes: .data ][debug]
                         | one PT_LOAD |          | PT_LOAD |      no load
memory:              [RX mapping................][RW mapping.....zero tail]

Section-to-segment listings from readelf -l are explanatory reconstructions. The program headers remain authoritative at execution time.

8. PT_LOAD, load bias, zero fill, and congruence#

For a load segment, the gABI requires page congruence when p_align > 1:

p_vaddr ≡ p_offset (mod p_align)
load_bias = chosen_runtime_address - link_time_virtual_address
runtime(x) = load_bias + x

The loader maps file bytes [p_offset, p_offset + p_filesz) to virtual bytes starting at load_bias + p_vaddr. It initializes [p_vaddr + p_filesz, p_vaddr + p_memsz) to zero. The tail of the last file page needs care: zero only bytes belonging to this object, while preserving data sharing and permissions correctly. Subsequent full pages may be anonymous zero pages.

ET_EXEC is traditionally linked for fixed virtual addresses. ET_DYN uses relative virtual addresses and a selected bias. ASLR chooses that bias subject to alignment, non-overlap, address limits, and reserved mappings. The invariant is not “first segment maps at zero”; it is that every translated segment and relocation uses one coherent bias.

9. Symbol records, binding, type, and visibility#

An ELF64 symbol is 24 bytes: st_name:u32, st_info:u8, st_other:u8, st_shndx:u16, st_value:u64, st_size:u64.

binding = st_info >> 4
type    = st_info & 0x0f
visibility = st_other & 0x03

Common bindings are STB_LOCAL, STB_GLOBAL, STB_WEAK, and GNU STB_GNU_UNIQUE. Common types include NOTYPE, OBJECT, FUNC, SECTION, FILE, COMMON, TLS, and GNU IFUNC. Undefined symbols use SHN_UNDEF; absolute symbols use SHN_ABS; large section indexes can use SHN_XINDEX plus SHT_SYMTAB_SHNDX.

Visibility controls export and preemption. DEFAULT participates normally in lookup. HIDDEN is not visible outside the defining component and permits local binding. PROTECTED remains exported but references from its defining component bind locally. INTERNAL has processor-specific meaning. Binding, type, visibility, definition state, and version are separate dimensions.

10. .symtab, .dynsym, and what stripping removes#

SHT_SYMTAB usually contains the full static-link/debug symbol table, including local names. It can be removed from a final executable. SHT_DYNSYM is the smaller dynamic-link symbol set reached through dynamic metadata. The leading local-symbol partition ends at the index recorded in sh_info; dynamic lookup structures refer to dynamic symbol indexes.

The dotted names are conventional. At runtime, the dynamic linker can find symbol records using DT_SYMTAB, strings using DT_STRTAB, and the entry width using DT_SYMENT, without finding a section named .dynsym. Stripping section headers and .symtab therefore need not break dynamic linking. Removing bytes required by the dynamic tags would.

Prediction. Does strip make exported function names disappear? Answer: not if dynamic relocations and lookup still need them. They remain in the dynamic symbol/string data.

11. REL, RELA, and architecture equations#

A relocation says where to write (r_offset), what operation to perform (type), and often which symbol (sym). ELF64 packs the latter two as:

ELF64_R_SYM(r_info)  = r_info >> 32
ELF64_R_TYPE(r_info) = r_info & 0xffff_ffff

Elf64_Rel stores r_offset, r_info; its addend A is read from the relocation field. Elf64_Rela adds signed r_addend; the field's old contents do not supply A. Architectures choose which form and equations apply. AMD64 predominantly uses RELA; AArch64's dynamic ABI also uses RELA. Generic ELF does not license substituting one for the other.

Typical AMD64 equations use symbol value S, addend A, place P, GOT base GOT, and load bias B:

This complete dependency-free calculation helper checks signed PC-relative fit:

pub fn abs64(s: u64, a: i64) -> Option<u64> { s.checked_add_signed(a) }
pub fn relative64(b: u64, a: i64) -> Option<u64> { b.checked_add_signed(a) }
pub fn pc32(s: u64, a: i64, p: u64) -> Result<u32, &'static str> {
    let value = i128::from(s) + i128::from(a) - i128::from(p);
    let signed = i32::try_from(value).map_err(|_| "R_X86_64_PC32 overflow")?;
    Ok(signed as u32)
}

Writing also requires checking that B + r_offset falls in a mapped writable target of the expected width. An equation alone is not a safe relocator.

12. RELR: exact address and bitmap decoding#

SHT_RELR/DT_RELR compress relative relocation offsets. Entries have the ELF address width. An even entry is an address. An odd entry is a bitmap describing the next word_bits - 1 words. For ELF64:

even E: relocate(E); cursor = E + 8
odd E:  for bit i in 1..64, if E & (1 << i) != 0, relocate(cursor + (i-1)*8)
        cursor = cursor + 63*8

The low bit is the discriminator, not a relocation bit. The following complete library function decodes virtual offsets and rejects bitmap-first, overflow, and an output limit breach:

pub fn decode_relr(entries: &[u64], limit: usize) -> Result<Vec<u64>, &'static str> {
    let mut out = Vec::new();
    let mut cursor = None;
    for &entry in entries {
        if entry & 1 == 0 {
            if entry & 7 != 0 { return Err("unaligned RELR address"); }
            if out.len() == limit { return Err("too many relocations"); }
            out.push(entry);
            cursor = Some(entry.checked_add(8).ok_or("RELR overflow")?);
        } else {
            let base = cursor.ok_or("RELR bitmap before address")?;
            for bit in 1..64 {
                if entry & (1u64 << bit) != 0 {
                    let delta = (bit - 1) * 8;
                    let address = base.checked_add(delta).ok_or("RELR overflow")?;
                    if out.len() == limit { return Err("too many relocations"); }
                    out.push(address);
                }
            }
            cursor = Some(base.checked_add(63 * 8).ok_or("RELR overflow")?);
        }
    }
    Ok(out)
}

The decoder finds places; the architecture defines the relative relocation applied there. GNU and LLVM adopted RELR before its generic standardization history settled, so check producer, consumer, and gABI revision compatibility rather than inferring support from file class.

13. Find the dynamic section without section names#

PT_DYNAMIC points to an array of address-sized tag/value pairs in the runtime image. DT_NULL terminates it. Values may be integers, sizes, string offsets, or virtual addresses depending on the tag. A dynamic loader follows the segment, not a .dynamic section name.

This complete ELF64 little-endian reader is intentionally policy-neutral. It requires an aligned pair array, bounded bytes, termination, and a caller-selected entry limit. Adapt byte order from EI_DATA in a general implementation.

pub fn dynamic_tags_le(bytes: &[u8], max: usize) -> Result<Vec<(i64, u64)>, &'static str> {
    if bytes.len() % 16 != 0 { return Err("partial dynamic entry"); }
    let mut result = Vec::new();
    for chunk in bytes.chunks_exact(16).take(max) {
        let tag = i64::from_le_bytes(chunk[0..8].try_into().unwrap());
        let value = u64::from_le_bytes(chunk[8..16].try_into().unwrap());
        result.push((tag, value));
        if tag == 0 { return Ok(result); }
    }
    Err("unterminated or over-limit dynamic table")
}

To turn a dynamic virtual address V into file bytes for offline analysis, find a PT_LOAD with:

p_vaddr <= V < p_vaddr + p_filesz
file_offset(V) = p_offset + (V - p_vaddr)

At runtime use B + V, after checking it lies in a mapped segment. Do not add bias to integer tags such as DT_STRSZ or string-table offsets.

14. DT_NEEDED and the dynamic tag graph#

Key generic tags include:

Common GNU tags include DT_GNU_HASH, DT_VERSYM, DT_VERDEF*, DT_VERNEED*, and historical search-path tags. RELR uses DT_RELR, DT_RELRSZ, and DT_RELRENT in current generic practice.

DT_NEEDED records names, not resolved paths. Search order, secure-execution restrictions, DT_RPATH versus DT_RUNPATH, cache use, $ORIGIN, namespaces, and environment variables are platform/loader policy. Never concatenate an untrusted dependency name to a privileged search path.

15. System V hash: simple lookup with a chain#

The generic DT_HASH table is:

nbucket, nchain, bucket[nbucket], chain[nchain]
candidate = bucket[elf_hash(name) % nbucket]
while candidate != STN_UNDEF: compare name; candidate = chain[candidate]

nchain also bounds dynamic symbol indexes. Every bucket and chain read must be below nchain, and cycle/work limits protect malformed tables. This complete hash function operates on bytes, matching symbol strings before their terminating NUL:

pub fn sysv_elf_hash(name: &[u8]) -> u32 {
    let mut h = 0u32;
    for &byte in name {
        h = h.wrapping_shl(4).wrapping_add(u32::from(byte));
        let high = h & 0xf000_0000;
        if high != 0 { h ^= high >> 24; }
        h &= !high;
    }
    h
}

Hash equality is only a candidate test. Exact bounded string comparison decides identity.

16. GNU hash and Bloom-filter false positives#

GNU hash is an extension documented by GNU toolchain sources and practice. Its header is nbuckets, symoffset, bloom_size, bloom_shift, followed by native-class Bloom words, buckets, and chains. GNU's hash is djb2 with 32-bit wrapping:

pub fn gnu_hash(name: &[u8]) -> u32 {
    name.iter().fold(5381u32, |h, &b| h.wrapping_mul(33).wrapping_add(u32::from(b)))
}

pub fn gnu_bloom_maybe(hash: u32, words: &[u64], shift: u32) -> bool {
    if words.is_empty() { return false; }
    let word = words[(hash as usize / 64) % words.len()];
    let mask = (1u64 << (hash % 64)) | (1u64 << ((hash >> shift) % 64));
    word & mask == mask
}

pub fn gnu_chain_hash_matches(wanted: u32, stored: u32) -> bool {
    (wanted | 1) == (stored | 1)
}

These are complete dependency-free components, not a complete table parser. A negative Bloom test proves absence from this object. A positive test is only “maybe”: unrelated hashes can set both bits. The bucket gives the first symbol index; chain words correspond from symoffset, and bit 0 marks the last chain entry. Compare (chain | 1) with (hash | 1), then compare the full name.

Prediction. The Bloom filter says yes but no chain name matches. Is the file corrupt? Answer: no. False positives are the intended space/time tradeoff.

17. GNU symbol versioning: three linked tables#

GNU versioning is an extension, not core generic symbol semantics:

Versym index 0 means local and 1 means global/unversioned; larger indexes select definition or requirement records. The high VERSYM_HIDDEN bit affects default-version behavior and is masked before indexing. Definition and need records are variable-length linked records with relative next offsets. Validate every auxiliary count, next offset, string offset, alignment, and cycle bound.

Lookup is conceptually on (name, required version), not name alone. foo@V1 names a particular version; GNU source/linker syntax may designate foo@@V2 as the default exported version. A symbol with the right spelling but wrong version is not a successful binding.

18. PIC, PIE, DSO, and dynamic ET_EXEC#

Position-independent code avoids embedding addresses that must be rewritten in executable pages. It uses PC-relative references, the GOT, PLT, and load-bias-relative data. This improves sharing and ASLR, but it does not mean “no relocations.” Data, imported symbols, TLS, and function pointers still need dynamic work.

ET_DYN alone cannot distinguish PIE from DSO. Interpreter presence, entry point, flags, platform conventions, and launch context contribute. A dynamic ET_EXEC still has PT_INTERP, dependencies, and dynamic relocations; “executable” does not mean statically linked.

19. Why the GOT exists#

Machine instructions cannot always encode an arbitrary final address, and imported definitions may move or be preempted. The Global Offset Table gives code a nearby, addressable cell whose contents the dynamic linker can initialize.

PIC instruction --PC-relative displacement--> GOT slot --runtime pointer--> object/function
       fixed at static link                 writable during relocation

The GOT separates instruction encoding from final identity. It costs an indirection and creates a write target. RELRO later removes writes where possible. On AMD64, RIP-relative addressing makes nearby GOT slots convenient. AArch64 commonly forms a page-relative GOT address with ADRP and loads through it; that sequence and its relocation pair are architecture-specific.

-fno-plt-style code can call through a GOT slot instead of a classic PLT entry. That changes the call sequence and relocation timing; it does not remove dynamic symbol resolution.

20. AMD64 PLT: first call and second call#

The classic GNU/System V AMD64 lazy PLT is an ABI/toolchain pattern. Exact instruction sequences can vary with linker options and security features.

first call:
caller -> foo@PLT -> jmp *foo@GOTPLT
                       slot initially points back into foo@PLT
                    -> push relocation index
                    -> PLT0 -> push link-map; jump resolver
                    -> lookup foo; apply R_X86_64_JUMP_SLOT
                    -> rewrite slot with resolved S; tail-call foo

second call:
caller -> foo@PLT -> jmp *foo@GOTPLT -> resolved foo

On entry to an ordinary AMD64 function, stack alignment must still satisfy the psABI calling rule; the resolver trampoline compensates for its own pushes. Do not infer the relocation index format from disassembly alone—interpret it according to the ABI and loader's PLT contract.

Eager binding (DF_BIND_NOW, DT_BIND_NOW, or loader policy such as LD_BIND_NOW) resolves PLT slots before normal execution. Lazy binding defers cost and failure until first call. Eager binding allows full GOT protection earlier and makes missing symbols fail deterministically at startup.

21. Static versus dynamic relocations, copies, and IFUNC#

The static linker consumes input relocations while laying out the output. It resolves locals, relaxes instructions, emits fixed values, and preserves only work needing runtime facts. The dynamic linker then applies emitted dynamic relocations after choosing biases and symbol definitions.

Copy relocation hazard. A non-PIC executable referencing a DSO data symbol may receive storage in the executable; R_*_COPY asks the loader to copy the DSO's initial bytes there. Lookup may then redirect references to the copy. This can split identity, constrain size/ABI evolution, dirty pages, and surprise protected/interposed symbols. Prefer PIC access through the GOT and avoid exported mutable data interfaces.

GNU STT_GNU_IFUNC makes a symbol's value come from calling a resolver. R_*_IRELATIVE similarly calls a resolver at B + A and writes its result. This is extension/architecture territory. Resolvers run during relocation under severe constraints: ordering may be incomplete, recursion and loader-lock interactions are dangerous, and arbitrary application services may not be ready. Validate the resolver address before transfer; treat the returned address according to platform policy. IFUNC can select optimized code, but complicates reproducibility, auditing, and static analysis.

22. TLS: PT_TLS, four classic models, and TLSDESC#

PT_TLS describes the initialization image: copy p_filesz bytes, zero the remainder to p_memsz, and honor p_align. It is a template, not one process-global variable block. Each thread gets an instance. STT_TLS values are offsets in a TLS model, not ordinary virtual addresses.

The static linker may relax GD→IE→LE or LD→LE only when stronger assumptions are proven. Relaxation rewrites instructions and relocations while preserving observable address identity. Loading a DSO later with dlopen makes static-TLS capacity and model choice operational concerns.

AMD64 uses its own thread-pointer conventions and R_X86_64_*TLS* relocations. AArch64 has distinct instruction sequences and R_AARCH64_TLS*/TLSDESC relocations. Never transplant the AMD64 formula or register assumptions to AArch64. Arm's AAELF64 and TLS documents in the 2025Q4 release are the authority for AArch64.

23. AArch64 PLT/GOT, BTI, and pointer authentication#

AArch64 has no AMD64 RIP-relative instruction encoding. Typical code uses ADRP to form a 4 KiB page-relative base, then LDR/ADD with a low-12-bit relocation. Its PLT stubs commonly load the destination from GOT/PLT into x17 and branch with br x17; register and stub details follow AAELF64 and linker choices. Dynamic relocations are R_AARCH64_*, not renamed x86 equations.

Arm BTI marks valid indirect-branch landing pads. GNU property notes can advertise BTI requirements; linker/loader/kernel cooperation determines enforcement. Pointer Authentication (PAuth) signs and authenticates pointers or return addresses under architecture and platform rules. A signed pointer is not merely an integer plus load bias: signing context, key, discriminator, and relocation type matter. The 2025Q4 Arm ABI documents PAuth-related ELF and relocation contracts.

AArch64 call boundary (simplified)
caller -> PLT landing pad [BTI when required]
       -> page-relative GOT address -> loaded target
       -> authenticate target [when ABI sequence requires]
       -> indirect branch

BTI constrains where an indirect branch may land. PAuth constrains which pointer/context is accepted. Neither replaces symbol resolution, relocation bounds checks, or W^X.

24. Constructors, destructors, notes, and identity#

DT_PREINIT_ARRAY is for the main executable where supported. Dependency-aware initialization then invokes DT_INIT and DT_INIT_ARRAY; finalization uses DT_FINI_ARRAY (normally reverse array order) and DT_FINI, with loader-defined dependency ordering and process-exit behavior. Array size must be a multiple of pointer width, each target must be validated, and code must not run before required relocations are complete. Constructor order across dependency graphs is not source-file order.

ELF notes contain aligned name and descriptor byte strings:

namesz:u32, descsz:u32, type:u32,
name bytes, padding to note alignment,
descriptor bytes, padding to note alignment

GNU build ID (NT_GNU_BUILD_ID) is a GNU identity convention used to associate binaries with debug files. Its digest algorithm and uniqueness are producer policy; it is not a cryptographic signature. GNU property notes carry machine properties such as x86 CET or AArch64 BTI/PAuth feature bits. Unknown notes should normally be skipped with checked aligned arithmetic, not interpreted by name alone. Both owner and type participate in meaning.

25. RELRO, stack policy, and W^X#

PT_GNU_RELRO is a GNU extension marking memory that the loader should make read-only after relocations. Page granularity matters: the linker arranges sensitive data into coverable pages.

These names describe toolchain policy, not new generic ELF types beyond the GNU segment. Verify the actual program headers, dynamic flags, and runtime mappings.

PT_GNU_STACK records requested stack permissions; an executable-stack flag is hazardous. Absence is interpreted by platform/toolchain policy and is not universally identical to non-executable. W^X means a page should not be writable and executable at the same time. Segment overlap and page rounding can accidentally create combined permissions, so validate permissions after page-level mapping, not only per raw header. Text relocations fight sharing and W^X and should generally be rejected or tightly controlled.

26. Unwinding and debugging are related, not identical#

.eh_frame contains Common Information Entries and Frame Description Entries used for stack unwinding. Encoded pointers can be PC-relative, data-relative, indirect, and width/signedness qualified by DWARF EH pointer encodings. .eh_frame_hdr is a compact search index used by common runtimes. These names and encodings involve platform ABI/toolchain conventions layered around DWARF call-frame instructions.

DWARF 5 debug sections such as .debug_info, .debug_abbrev, .debug_line, .debug_str, and .debug_rnglists describe source-level entities and locations. They are usually non-allocated and may live in separate debug files associated by build ID or debug link. Their absence does not stop loading; their corruption should not grant memory access in a debugger.

The official standard is DWARF Version 5. Distinguish its normative debugging format from architecture unwind ABI, GNU .eh_frame_hdr practice, and a particular unwinder implementation.

27. Interposition, preemption, and local binding#

Default-visible global definitions can be preempted: the lookup scope may choose a definition in a different object. This powers interposition but limits optimization and changes address identity. Weak definitions lose to suitable strong definitions; an unresolved weak reference often becomes zero under ABI rules. Version and visibility still apply.

-Bsymbolic is a static-linker policy commonly binding references inside a DSO to that DSO's own definitions. It can improve locality but break interposition assumptions and create two observed addresses for what source code expected to be one object. -Bsymbolic-functions narrows the scope. Hidden visibility is often the clearer interface decision because it says the name is not exported.

Counterexample. “The nearest definition in dependency order always wins” is incomplete. Scope construction, executable precedence, namespaces, symbolic binding, protected/hidden visibility, versions, weak rules, GNU unique symbols, and deep-binding extensions can all matter.

28. A stripped object: what program headers can still reveal#

Suppose e_shoff = 0, e_shnum = 0, and no section names exist. From the ELF header and program headers a loader can still discover:

1. architecture, entry point, interpreter, and loadable file/memory ranges;
2. permissions, load bias constraints, zero-fill, TLS template, notes, RELRO, and stack request;
3. PT_DYNAMIC, then dependencies, strings, symbols, hashes, versions, relocations, PLT work, constructors, destructors, and flags through dynamic tags;
4. runtime addresses by translating dynamic virtual addresses through mapped segments.

It cannot recover arbitrary discarded local symbols, section names, exact source boundaries, or debug information that was removed. It also cannot infer the byte extent of .dynsym from DT_SYMTAB alone in every possible object; hash tables, relocation indexes, neighboring mapped objects, and policy supply bounds, and a hardened consumer must refuse ambiguity rather than scan unbounded memory.

ELF header
  -> program-header table
      -> PT_LOAD: map image and establish B
      -> PT_INTERP: select dynamic linker
      -> PT_DYNAMIC: tag graph
           -> strtab -> DT_NEEDED names
           -> symtab + hash + versions -> lookup candidates
           -> relocation tables -> writes
           -> init arrays -> controlled calls

29. One complete dynamic-binding trace#

Trace an AMD64 PIE calling imported puts, assuming classic lazy PLT and no version mismatch:

1. The kernel validates the executable enough for platform policy, maps its PT_LOAD segments, obtains its interpreter from PT_INTERP, and enters the dynamic linker with auxiliary-vector facts. Exact division of work is operating-system implementation.
2. The linker derives the PIE bias B_main, walks PT_DYNAMIC, and reads DT_NEEDED through the bounded dynamic string table.
3. It locates and maps dependencies, recording each object's load bias and dynamic tag graph.
4. It builds the platform-defined lookup scope. Dependency graph order and scope are not simply “alphabetical libraries.”
5. It applies relative relocations (B + A) and symbol relocations. For each symbol it checks name, GNU version, binding, visibility, definition state, and machine relocation constraints.
6. With lazy binding, puts's R_X86_64_JUMP_SLOT remains directed to its PLT resolver path.
7. It applies eligible RELRO protections, leaving lazy GOT/PLT cells writable, runs dependency-safe constructors, and transfers to the program entry sequence.
8. The first puts@PLT call reaches PLT0. The resolver identifies the relocation safely, searches the established scope through GNU hash (Bloom → bucket → chain → exact name), finds the required version, computes S, and atomically/safely updates the GOT slot according to implementation.
9. The resolver tail-calls puts. The second call jumps directly through the now-resolved slot.

With eager binding, step 8 moves into step 5 and full RELRO can protect the slot before constructors. With -fno-plt, the instruction path differs. On AArch64, use AArch64 stubs and relocations; this AMD64 trace must not be projected onto it.

Earliest-invariant debugging. A crash in puts may begin with a wrong load bias, malformed GNU chain, wrong version match, PC-relative overflow, premature RELRO, or stack misalignment. Start at the earliest trace step whose output is wrong, not the last visible fault.

30. Adversarial parsing and loader proof obligations#

Treat every offset, count, alignment, index, string, chain, and address as hostile. A hardened consumer should enforce:

Do not allocate count * size before checking it. Do not follow DT_NEEDED recursively without depth and object limits. Do not trust DT_STRSZ unless its entire addressed interval is in a file- backed/mapped segment. Do not write relocations to RX or unmapped pages merely because a relocation requests it. Temporarily changing permissions is mechanism with security cost, not validation.

Debugging workshop: a plausible but crashing PIE#

You see R_X86_64_RELATIVE write 0x7f40_0012_3450, while /proc/.../maps begins the object at 0x7f40_1000_0000.

1. Predict before inspecting: symbol lookup or bias translation?
2. Record r_offset, r_addend, chosen bias, target mapping, and computed value.
3. Check target = B + r_offset and value = B + A separately.
4. Compare the lowest PT_LOAD link address with its mapped address; do not assume it is zero.

Answer: no symbol lookup participates in a RELATIVE relocation. A bias derived as “mapping start” rather than mapped_address - p_vaddr_page is the leading hypothesis.

Debugging workshop: startup fails only with -z now#

Predict first. Likely classes are an otherwise-unused missing/versioned symbol, an IFUNC ordering failure, or a relocation/RELRO timing bug. Use readelf -dWr, loader diagnostics, and a debugger break at the dynamic-linker error path. Lazy mode hid the problem by never executing that binding.

31. Diagnostic laboratories and tools#

Work on disposable binaries; tool output is evidence, not authority.

cc -fPIE -pie -Wl,-z,relro,-z,now -Wl,--build-id -o hello hello.c
readelf -hW hello
readelf -lW hello
readelf -dWr --dyn-syms --version-info hello
objdump -drwC -Mintel hello
nm -a hello
ldd hello                 # do not use on untrusted files
gdb --args ./hello

1. Byte lab: use xxd -g1; manually decode e_phoff, one ELF64 program header, and verify congruence. Prediction: changing only section names does not change mappings.
2. Strip lab: copy and strip a PIE. Compare section tables, PT_DYNAMIC, dependencies, and a successful run. Then use objcopy --strip-sections only if your installed tool supports the intended mode; check rather than assume option semantics.
3. Binding lab: build lazy and -z now variants. Use LD_DEBUG=bindings,reloc only in a safe non-privileged environment; compare first-call behavior and RELRO mappings.
4. Hash lab: calculate both hashes for exported names, then deliberately find a GNU Bloom false positive. Confirm exact comparison rejects it.
5. TLS lab: compile one thread_local access as PIE and DSO at different optimization levels. Identify model relocations and linker relaxations on AMD64, then independently repeat on AArch64.
6. Corruption lab: mutate counts, DT_NULL, string terminators, RELR bitmap order, and chain indexes. Your parser must return structured errors without panic, huge allocation, or loop.

Safer inspection tools include readelf and custom bounded parsers. ldd has historical and platform-dependent execution risks; never point it at an untrusted executable. A sandbox does not turn malformed native code into inert data.

32. Myths, exercises, and mastery route#

Implementation milestones#

1. Extend the Rust header reader to ELF32 and test both endiannesses without casting.
2. Decode extended numbering from section zero, with table-limit tests.
3. Build a program-header mapper that reports file bytes, zero fill, bias equations, and final page permissions. Keep mapping simulation separate from OS mapping calls.
4. Starting only at PT_DYNAMIC, print bounded DT_NEEDED, SysV/GNU hash geometry, versions, RELA, and RELR places for a stripped object.
5. Implement lookup as (name, optional version) with explicit scope input. Differential-test results against readelf on your own generated corpus, not hostile files.
6. Implement AMD64 relocation calculation into a fake byte vector. Test overflow, overlap, wrong symbol indexes, read-only targets, and duplicate writes before considering native memory.

Prediction and counterexample exercises#

1. Two PT_LOAD records overlap after page rounding but not as byte intervals. Predict final page permissions and explain why header-order dependence is unsafe.
2. A GNU bucket index is below symoffset. Decide whether subtracting with wrapping is acceptable. Construct the smallest rejecting test.
3. A versym entry is 0x8003. Show the hidden flag and table index separately; test a name match against version 4.
4. Decode RELR [0x4000, 0b1011] on ELF64. Answer: places 0x4000, 0x4008, and 0x4018; bit zero marks a bitmap, while bits 1 and 3 select offsets 0 and 16 from cursor 0x4008.
5. Explain why applying RELRO before IFUNC and PLT relocations can fail, and how eager binding changes the legal order.
6. Design a DSO API that avoids copy relocations and exported mutable data. State the ownership and versioning tradeoff.

Contribution and teaching capstones#

• Add malformed-ELF regression cases to an open-source parser. Find its earliest bounds-checking module, fuzz target, diagnostic type, and architecture dispatch before proposing a patch.
• Read one dynamic linker's map/load, lookup, relocation, TLS, and RELRO modules. Draw the actual control flow and mark implementation policy versus ABI requirement.
• Give a 30-minute talk built around the complete trace in Chapter 29. Include one AMD64 disassembly, one independent AArch64 sequence, one RELR bitmap, and one failed invariant diagnosis.
• Build a non-executing inspector capped at 64 dependencies, one million relocations, 64 MiB of metadata, and 32 graph depth. Publish the omissions: no mapping, no resolver calls, no trust claim.

33. Versioned source ledger and reading map#

Use specifications for contracts and source trees for current behavior. Dates and status matter.

Recommended order: gABI identification → sections → program loading → symbols → relocation → dynamic linking; then the matching psABI; then DWARF; finally compare GNU/Linux implementations. When source disagrees with a draft, report both: the draft states intended generic semantics, while the source states what that revision actually does.

The reusable model is now precise: program headers create mapped address space; dynamic tags make a bounded metadata graph; lookup chooses a versioned definition under scope and visibility rules; architecture relocations turn that choice into checked writes; protection transitions close the write window before application code receives control. Every optimization—hashing, PLT laziness, TLS relaxation, RELR, IFUNC—must preserve that model's observable meaning under explicitly stated ABI and implementation assumptions.

Part IV — Loaders: From execve to main, dlopen, and Unload#

1. The loader contract and who has authority#

A loader turns a file representation into a running program. That short sentence hides several jobs:

1. recognize and validate an executable format;
2. create memory mappings with suitable permissions;
3. establish the machine's initial register and stack state;
4. load shared objects, find symbols, and apply relocations;
5. initialize language runtimes and transfer control to the program;
6. support later loading, lookup, and sometimes unloading.

On Linux these jobs are split. The kernel recognizes ELF and maps the main executable and, for a dynamically linked program, the named ELF interpreter. The interpreter—normally glibc's ld-linux or musl's dynamic linker—loads DT_NEEDED libraries and relocates them. The kernel does not load libc merely because libc appears in DT_NEEDED.

Keep five kinds of rule separate:

ELF is a parser at a trust boundary. Lengths, offsets, alignments, integer additions, overlapping mappings, and strings all come from a file that may be hostile. The earliest invariant is: no file-derived range is used until it is proved to fit both the file and the address space. A later segmentation fault may merely be the first visible result of an earlier unchecked overflow.

2. execve: replacement, inheritance, and failure#

execve(path, argv, envp) replaces the calling process's memory image. The process ID normally remains. Open file descriptors remain unless marked close-on-exec. Threads other than the caller disappear. Caught signal dispositions reset; many other attributes change as specified by execve(2). A shell normally performs fork/clone, then the child calls execve; creation and replacement are distinct.

There is no successful return from execve. On failure the old image still exists and the call returns -1. During a successful exec Linux reaches a point of no return after dismantling enough of the old image; a rare later failure cannot restore it. This distinction matters when diagnosing a process that dies without the caller observing an errno.

Information crosses exec selectively. Argument and environment bytes are copied; close-on-exec descriptors are discarded; other descriptors retain their open-file descriptions; and the calling thread becomes the only thread. The old virtual addresses are discarded. A launcher should choose inherited descriptors, signals, credentials, directory, and limits deliberately rather than treating exec as a clean slate.

3. The Linux kernel exec path#

The high-level Linux route, current around Linux 6.x, is:

userspace execve
  | pathname, argv, envp
  v
do_execveat_common()                 fs/exec.c
  | opens file; prepares linux_binprm; copies strings
  v
bprm_execve() -> exec_binprm()
  | tries registered linux_binfmt handlers
  +--> script handler               fs/binfmt_script.c
  +--> ELF handler load_elf_binary  fs/binfmt_elf.c
  +--> optional binfmt_misc         fs/binfmt_misc.c

The generic exec layer handles limits and format dispatch. A handler seeing an unsupported magic returns an error that permits another handler to try. binfmt_misc is optional policy, not part of ELF. Script handling can select an interpreter and begin another exec interpretation; details and recursion limits are Linux implementation behavior.

The generic layer preserves the request in linux_binprm while format handlers decide whether they own it. Moving commitment earlier would simplify cleanup but could destroy a valid process for malformed input; some work cannot move entirely before commitment because mappings belong to the new address space. This tension creates the point-of-no-return boundary.

4. ELF recognition and validation#

In current fs/binfmt_elf.c, load_elf_binary() checks ELF magic, executable type (ET_EXEC or ET_DYN), architecture via elf_check_arch, and a usable file mapping operation. It reads the program-header table with load_elf_phdrs(), rejects impossible entry sizes/counts and malformed ranges, examines headers such as PT_INTERP, PT_GNU_STACK, and PT_LOAD, and later maps segments. Exact checks evolve: cite a kernel revision when depending on them, rather than converting source details into an ABI promise.

Validation must precede allocation and mapping. Check multiplication and addition before using their results: program-header table size, p_offset + p_filesz, and p_vaddr + p_memsz must fit. Preserve the failing header index in diagnostics; normalized ranges alone discard useful provenance. ELF magic selects a parser—it does not prove that the remaining bytes are safe.

Prediction: if section headers are stripped but program headers remain, can the program run? Usually yes. Execution uses program headers. Sections primarily serve linking and analysis.

5. PT_LOAD: pages, permissions, and zero fill#

Each PT_LOAD describes bytes to map: file offset p_offset, virtual address p_vaddr, file bytes p_filesz, memory bytes p_memsz, alignment, and PF_R/PF_W/PF_X flags. The invariant is p_filesz <= p_memsz. The tail from p_filesz to p_memsz is zero-filled; this is how .bss exists without occupying equivalent file space. Page boundaries require careful treatment because mappings operate in pages while ELF ranges need not.

Concrete trace: with p_vaddr=0x401000, p_filesz=0x1800, and p_memsz=0x2400, file bytes end at 0x402800 and zero fill ends at 0x403400. The final partial file page must be cleared after its file bytes; later pages are anonymous zeroed memory. Mapping whole file pages without clearing the tail can expose unrelated file data as .bss.

6. Load bias, ASLR, and the initial brk#

For an ET_EXEC, addresses are usually linked at fixed virtual addresses, though a toolchain may produce unusual layouts. A position-independent executable is normally ET_DYN; Linux chooses a randomized load bias. Shared objects are also ET_DYN, but the kernel maps only the executable and interpreter during exec. ASLR is constrained by alignment, architecture, address-space layout, kernel policy, and entropy. It is defense in depth, not permission checking.

elf_map()/elf_load() and surrounding code in fs/binfmt_elf.c map load segments and arrange zeroing; set_brk() establishes the initial brk region. arch_randomize_brk() and architecture hooks participate in randomization. The effective page permissions come from segment flags plus kernel/architecture policy. Modern linkers avoid writable-and-executable load segments; the loader cannot recover security that a bad layout discarded.

7. PT_INTERP and the kernel–rtld division#

A dynamically linked executable usually contains a PT_INTERP string such as /lib64/ld-linux-x86-64.so.2 or /lib/ld-musl-x86_64.so.1. Linux opens that exact path in the process's filesystem context, validates it as a suitable ELF object, maps its loadable segments, builds the initial stack, and enters the interpreter's entry point.

Linux does not interpret DT_NEEDED, search for libc.so, resolve symbols, run constructors, or call main. Those are runtime-loader and libc jobs. For a static executable without PT_INTERP, Linux enters the executable's ELF entry directly.

8. The initial stack and machine entry#

The psABI defines the machine-level entry convention; Linux supplies the strings and auxiliary vector. A simplified x86-64 stack, with addresses increasing downward in this picture, is:

low address
  argc = 2
  argv[0] ----+                         pointers are machine words
  argv[1] --+ |                         argv[argc] is null
  NULL      | |
  envp[0] -----------+                  envp ends with null
  envp[1] --------+  |
  NULL           |  |
  AT_PHDR,  0x400040                    auxv is (type, value) pairs
  AT_PHNUM, 13
  AT_ENTRY, 0x401080
  AT_BASE,  0x7f...                     interpreter load base
  AT_RANDOM, --------+
  AT_SECURE, 0       |
  AT_SYSINFO_EHDR, 0x7f...
  AT_NULL, 0         |
  padding/alignment  |
  random[16] <-------+                  pointed-to bytes
  "LANG=C\0" <---------+
  "MODE=test\0" <------+
  "input.txt\0" <---+ |
  "./viewer\0" <-----|-+
high address          |
                      +-- argv[1]

The drawing is conceptual: exact string order, padding, extra auxv entries, and register state are platform-specific. On x86-64, %rsp points at argc at process entry and other required register conventions come from the psABI.

9. The auxiliary vector as a typed handoff#

Important Linux auxiliary entries include:

• AT_PHDR: in-memory address of the main executable's program headers;
• AT_ENTRY: main executable entry address, not necessarily where execution begins when an interpreter exists;
• AT_BASE: interpreter load base (zero when inapplicable);
• AT_SECURE: nonzero tells libc/runtime loader to use secure-execution behavior;
• AT_RANDOM: pointer to 16 kernel-supplied random bytes, used for defenses such as stack canaries;
• AT_SYSINFO_EHDR: base of the kernel-provided vDSO ELF image.

getauxval(3) is the normal libc interface. Reading beyond AT_NULL, trusting pointers without process context, or assuming every architecture supplies every tag breaks the representation invariant.

This complete stable Rust program is an educational safe model. It parses indices into one owned word slice; it does not dereference a real process stack or replace libc startup.

#[derive(Debug, PartialEq)]
struct Initial<'a> {
    argv: &'a [usize],
    envp: &'a [usize],
    auxv: Vec<(usize, usize)>,
}

fn parse_initial(words: &[usize]) -> Result<Initial<'_>, &'static str> {
    let argc = *words.first().ok_or("missing argc")?;
    let argv_end = 1usize.checked_add(argc).ok_or("argc overflow")?;
    let argv = words.get(1..argv_end).ok_or("short argv")?;
    if words.get(argv_end) != Some(&0) {
        return Err("argv lacks terminator");
    }

    let mut i = argv_end + 1;
    while words.get(i).copied().ok_or("short envp")? != 0 {
        i = i.checked_add(1).ok_or("index overflow")?;
    }
    let envp = &words[argv_end + 1..i];
    i += 1;

    let mut auxv = Vec::new();
    loop {
        let kind = *words.get(i).ok_or("short auxv type")?;
        let value = *words.get(i + 1).ok_or("short auxv value")?;
        i += 2;
        if kind == 0 { break; } // AT_NULL
        auxv.push((kind, value));
    }
    Ok(Initial { argv, envp, auxv })
}

fn main() {
    let words = [2, 100, 200, 0, 300, 0, 9, 0x401080, 23, 0, 0, 0];
    let got = parse_initial(&words).unwrap();
    assert_eq!(got.argv, &[100, 200]);
    assert_eq!(got.envp, &[300]);
    assert_eq!(got.auxv, vec![(9, 0x401080), (23, 0)]);
}

10. Credentials and AT_SECURE#

Exec is a security transition. Set-user-ID/set-group-ID bits and file capabilities may change effective credentials, subject to mount flags, tracing, and no_new_privs; see execve(2), capabilities(7), and current security/commoncap.c. The kernel computes secure-exec state (bprm->secureexec through security hooks) and communicates it as AT_SECURE. The runtime loader must not infer trust merely from environment strings.

In glibc secure-execution mode, environment-controlled loader behavior is removed or restricted: for example LD_LIBRARY_PATH, LD_PRELOAD, LD_AUDIT, LD_DEBUG, and related variables cannot be allowed to redirect privileged execution in their ordinary form. Details and exceptional rules belong to the specific glibc release; ld.so(8) is the user-facing contract. A privileged program should also sanitize its own environment, file descriptors, working directory assumptions, locale inputs, and inherited resource limits.

The invariant is not merely “real ID differs from effective ID.” Kernel hooks know about capabilities, mount policy, tracing, and no_new_privs; recomputing secure state in userspace would discard that context. AT_SECURE is the authoritative handoff for libc policy, but applications must still validate their own inputs.

11. The vDSO: kernel code in an ELF-shaped mapping#

The vDSO is a small ELF image mapped by the kernel and advertised with AT_SYSINFO_EHDR. It lets libc call selected kernel-provided routines—commonly time-related operations—without an ordinary syscall when safe. It is not a general shared library from disk. Names and symbol versions are architecture ABI. glibc locates suitable symbols; falling back to a syscall preserves behavior when necessary. See vdso(7), Documentation/ABI/stable/vdso, and arch/*/entry/vdso.

12. Runtime-linker self-relocation#

At first entry, the dynamic linker itself cannot assume that its normal global variables and function calls are relocated. It finds its own load bias, processes a restricted bootstrap set of relocations, establishes enough TLS and machine state, and only then uses ordinary C machinery. This is a carefully staged fixed point: the relocator needs data structures that relocation itself makes usable.

In current glibc, architecture _start code reaches _dl_start and _dl_start_final in elf/rtld.c; _dl_sysdep_start in sysdeps/unix/sysv/linux/dl-sysdep.c parses the initial stack and auxv. elf/dl-reloc.c, architecture dl-machine.h files, and elf/dynamic-link.h participate in relocation. Names are implementation details, not Linux ABI.

The bootstrap invariant is: no instruction consumes an address before the relocation that defines it. General allocation, TLS, auditing, and lazy binding cannot all be prerequisites for the code that makes them usable, so self-relocation deliberately supports a small staged mechanism first.

13. Discovering the dynamic section#

The loader finds the executable's PT_DYNAMIC by scanning program headers. The dynamic array then points to string tables, symbol tables, hashes, relocation tables, and DT_NEEDED string offsets. Section names such as .dynsym are not required at runtime. Every pointer is normally interpreted relative to the correct load bias, with ABI-specific exceptions.

kernel maps main + interpreter
        |
        v enters interpreter ELF entry
locate own headers/load bias
        |
self-relocate permitted bootstrap forms
        |
parse main PT_DYNAMIC -> load dependency closure
        |
build scopes; relocate; establish TLS; enforce RELRO
        |
run initialization -> transfer to executable _start

14. Building the dependency graph#

Each object's DT_NEEDED entries name direct dependencies. Loading recursively forms a directed graph, not necessarily a tree: two objects can share a dependency, and cycles are legal. Identity also involves canonicalized file/device information and namespaces; treating each spelling as a separate object can duplicate state.

The following complete stable Rust program is an educational planner. It condenses strongly connected components (SCCs), then topologically orders those components. It does not claim to reproduce glibc constructor policy.

fn scc_plan(edges: &[Vec<usize>]) -> Vec<Vec<usize>> {
    fn visit(v: usize, e: &[Vec<usize>], seen: &mut [bool], out: &mut Vec<usize>) {
        if seen[v] { return; }
        seen[v] = true;
        for &w in &e[v] { visit(w, e, seen, out); }
        out.push(v);
    }
    let n = edges.len();
    let mut seen = vec![false; n];
    let mut finish = Vec::new();
    for v in 0..n { visit(v, edges, &mut seen, &mut finish); }

    let mut rev = vec![Vec::new(); n];
    for (v, ws) in edges.iter().enumerate() {
        for &w in ws { rev[w].push(v); }
    }
    seen.fill(false);
    let mut groups = Vec::new();
    for &v in finish.iter().rev() {
        if seen[v] { continue; }
        let mut group = Vec::new();
        visit(v, &rev, &mut seen, &mut group);
        group.sort_unstable();
        groups.push(group);
    }
    groups.reverse(); // dependencies before users for edges user -> dependency
    groups
}

fn main() {
    // app -> a -> b -> a, and app -> c
    let graph = vec![vec![1, 3], vec![2], vec![1], vec![]];
    let plan = scc_plan(&graph);
    assert!(plan.iter().any(|g| g == &[1, 2]));
    assert_eq!(plan.last(), Some(&vec![0]));
}

15. glibc search policy: RPATH, RUNPATH, and identity#

For a DT_NEEDED name without a slash, the simplified glibc order from ld.so(8) is:

1. the requester's DT_RPATH if it has no DT_RUNPATH, including applicable ancestor RPATH behavior;
2. LD_LIBRARY_PATH, unless secure mode suppresses it;
3. the requester's DT_RUNPATH, but only for its direct dependencies;
4. /etc/ld.so.cache (with hardware-capability selection and options affecting defaults);
5. default directories such as /lib and /usr/lib, architecture-adjusted.

This is a teaching summary: slash-containing names, command-line loader options, DF_1_NODEFLIB, glibc-hwcaps, cache configuration, and already-loaded objects add cases. The crucial distinction is transitivity: old RPATH can affect descendants; RUNPATH applies only to direct children, so each child needing a private directory should carry its own RUNPATH.

$ORIGIN expands to the directory containing the referring executable or shared object in applicable dynamic strings. Quote it in a shell ('$ORIGIN/../lib') so the shell does not expand $ORIGIN first. $LIB and $PLATFORM are other glibc tokens. Relative and writable search paths are dangerous across privilege or packaging boundaries.

Concrete trace: app needs plugins/render.so; render needs libcodec.so. If app has RUNPATH=$ORIGIN/plugins, glibc can find render, but that RUNPATH alone does not find codec for render. Give render RUNPATH=$ORIGIN/../lib, install codec in a configured directory, or use another explicit deployment design.

musl policy differs. Current behavior is concentrated in ldso/dynlink.c; paths derive from explicit names, LD_LIBRARY_PATH when permitted, the main program's runpath/rpath handling, /etc/ld-musl-$ARCH.path, and built-in defaults. Do not assume glibc's cache, hwcaps, token, or RPATH-transitivity details apply. Verify against the target musl release.

16. Secure environment and preloads#

LD_PRELOAD inserts objects before ordinary dependencies and can interpose definitions. /etc/ld.so.preload is a system-wide administrative mechanism with a large blast radius. In secure mode glibc strips, ignores, or restricts dangerous controls according to ld.so(8); outside secure mode they remain a product-security concern. Use a minimal launch environment and trusted, read-only search trees.

17. Symbol scopes, versions, and interposition#

A lookup occurs in an ordered scope of loaded objects. Within each object, ELF hash tables accelerate finding a dynamic symbol; binding, visibility, definition status, and symbol version then decide usability. Default-visible global symbols can be interposed: an earlier definition may satisfy later references. STV_PROTECTED, -Bsymbolic, DF_SYMBOLIC, direct bindings on other platforms, and compiler optimizations alter opportunities.

GNU symbol versioning associates definitions and requirements with version nodes. foo@V1 and default foo@@V2 are not merely names containing punctuation. glibc implements broad GNU version semantics; musl accepts common version metadata for compatibility but does not provide every glibc version-selection behavior. Versioning is processor-independent ELF extension policy, not a kernel service.

This complete stable Rust example is an educational scope model, omitting weak symbols, visibility, hash tables, namespaces, and version-definition graphs:

#[derive(Debug)]
struct Def<'a> { name: &'a str, version: Option<&'a str>, address: usize }
#[derive(Debug)]
struct Object<'a> { name: &'a str, defs: Vec<Def<'a>> }

fn lookup<'a>(scope: &'a [Object<'a>], name: &str, version: Option<&str>)
    -> Option<(&'a str, usize)>
{
    scope.iter().find_map(|object| {
        object.defs.iter()
            .find(|d| d.name == name && version.map_or(true, |v| d.version == Some(v)))
            .map(|d| (object.name, d.address))
    })
}

fn main() {
    let scope = [
        Object { name: "preload", defs: vec![Def { name: "draw", version: Some("V1"), address: 10 }] },
        Object { name: "library", defs: vec![Def { name: "draw", version: Some("V1"), address: 20 }] },
    ];
    assert_eq!(lookup(&scope, "draw", Some("V1")), Some(("preload", 10)));
}

18. Relocation ordering, IFUNC, lazy binding, and RELRO#

Relocation writes addresses or values derived from symbols and load biases into mapped locations. Exact types and formulas are psABI rules. REL uses an addend already stored at the target; RELA carries an explicit addend. Relative relocations need only an object's load bias and are cheap, so loaders commonly process or optimize them early. Packed relative formats such as RELR are ABI extensions negotiated by toolchains and loaders.

A defensible order is constrained rather than universal:

1. bootstrap the loader's own usable state;
2. map the dependency closure and establish lookup scopes;
3. allocate TLS layout needed by initial objects;
4. apply relative and ordinary non-PLT relocations in a safe architecture-specific order;
5. resolve required IFUNC/IRELATIVE work only when its resolver prerequisites are valid;
6. perform eager PLT relocations, or prepare lazy resolver slots;
7. make RELRO ranges read-only;
8. run constructors.

An STT_GNU_IFUNC definition is selected by executing a resolver at load/lookup time. A resolver runs in an unusually fragile environment: it must obey architecture calling rules and avoid services not yet initialized, recursive lazy binding, unsafe TLS assumptions, and loader-lock surprises. glibc's exact ordering is architecture and release sensitive; see elf/dl-reloc.c, elf/ifuncmain*.c, and sysdeps/*/dl-irel.h.

With lazy binding, a PLT call initially reaches the runtime resolver; it looks up the symbol and patches a GOT slot. LD_BIND_NOW=1, DT_BIND_NOW, or DF_1_NOW requests eager binding. Eager binding moves failure to startup and allows more data to become read-only; lazy binding can reduce unused startup work but adds first-call latency and a complex reentrant path.

PT_GNU_RELRO marks a range writable during relocation and read-only afterward. Partial RELRO usually leaves lazy-binding GOT slots writable. Full RELRO combines a suitable layout with eager binding so those slots can be protected. RELRO narrows writable control data; it does not validate arbitrary pointers or stop all code reuse.

19. TLS, the DTV, and late loading#

Every TLS-using object supplies an initialization image and zero-fill size. The loader assigns module IDs and records TLS metadata. Each thread has a thread pointer and a dynamic thread vector (DTV) or ABI-equivalent route to module storage. TLS access models trade flexibility for speed:

• local-exec can use a fixed offset but belongs in the main startup set;
• initial-exec assumes space in static TLS and is unsafe for arbitrary late plugins;
• local-dynamic/global-dynamic permit more dynamic lookup;
• TLSDESC is an architecture ABI mechanism with its own relocation rules.

glibc reserves some static TLS surplus so selected late-loaded modules can use optimized/static layouts, but the reserve is finite and policy changes. Late dlopen may allocate dynamic TLS lazily for existing threads. New threads need templates for all current modules. A stale TLS address after unload is still a stale pointer.

The invariant is: a TLS address is valid only for its owning thread, while its module and allocation generation remain alive. Module IDs, DTV entries, initialization, and unload synchronization must agree. See the relevant psABI, glibc elf/dl-tls.c, and musl ldso/dynlink.c plus thread-pointer architecture code.

20. Constructors, destructors, and cycles#

After relocation, constructors run in dependency-aware order. ELF provides DT_INIT and DT_INIT_ARRAY; toolchains and libc add conventions. Dependencies generally initialize before users, but cycles have no perfect topological order. A loader must detect SCCs, choose a deterministic order within each cycle, and prevent duplicate calls. Code should not rely on an unspecified order between peers.

Constructors run under a partially born process. Starting threads, taking application locks, calling dlopen, throwing across foreign frames, or depending on another cycle member can expose reentrancy and order bugs. Destructors (DT_FINI_ARRAY, DT_FINI, __cxa_atexit machinery) normally reverse relevant lifetime relationships, but process exit and dlclose are different events.

Cycles preserve a real uncertainty: no order can put every dependency before its user. A deterministic SCC tie-break aids reproduction but does not create an ordering contract for peer constructors. The loader must mark initialization progress before calling code that can reenter it, and each constructor should publish state only after its own invariant holds.

21. From _start through libc to main#

For a typical glibc dynamic executable, the chain is:

interpreter bootstrap
  -> load/relocate objects
  -> arrange initialization and startup state
  -> jump/call executable ELF entry (_start)
  -> crt startup gathers argc/argv and runtime hooks
  -> __libc_start_main
       -> libc/program initialization
       -> main(argc, argv, envp)
       -> exit(result) and registered termination work

Details vary by architecture, PIE/static-PIE mode, and glibc release. Inspect elf/rtld.c, csu/libc-start.c, sysdeps/*/start.S, and compiler-provided CRT files. Musl's path is implemented through ldso/dynlink.c and src/env/__libc_start_main.c. _start is not main, and language runtimes may insert substantial work between them.

When a program crashes before main, debug the earliest invariant, not the final instruction alone:

22. dlopen: flags, handles, and scopes#

dlopen(path, flags) adds an object and dependencies to a runtime namespace. Common glibc/POSIX flags are:

• RTLD_NOW or RTLD_LAZY: eager or permitted-lazy function relocation;
• RTLD_LOCAL or RTLD_GLOBAL: whether definitions enter a broader lookup scope;
• RTLD_NOLOAD: query/promote an already loaded object (GNU extension);
• RTLD_NODELETE: retain mappings/state after close (GNU extension);
• RTLD_DEEPBIND: prefer a new object's own scope in glibc (GNU extension, often surprising).

Flags do not make an untrusted DSO safe. Loading executes its machine code through relocations, IFUNC resolvers, and constructors before the caller can inspect a friendly plugin API. Never dlopen arbitrary uploads in the host security domain; use a separate, restricted process and a validated protocol.

RTLD_LOCAL limits export into later scopes; it is not confidentiality or memory isolation. A handle records loader bookkeeping, not every pointer obtained from an object. Keep resolved file identity, namespace, mode, handle, and product generation together so aliases and reloads cannot silently exchange lifetimes.

23. dlsym, dlvsym, RTLD_NEXT, and dlerror#

dlsym(handle, "name") searches according to the handle's scope. RTLD_DEFAULT requests normal global lookup; glibc's RTLD_NEXT starts after the calling object, useful for wrappers but dependent on caller identity and scope. dlvsym requests a GNU symbol version. Clear old errors with dlerror(), perform the operation, then call dlerror() again: a successful lookup can legally produce a null address on ELF, so null alone is ambiguous. See dlopen(3), dlsym(3), and dlerror(3) in man-pages 6.18.

The diagnostic string is thread-associated state and may be overwritten by another loader call, so copy it promptly. RTLD_NEXT also depends on the true calling object; an extra wrapper layer can change the start point. A cached result is valid only for the namespace, scope, version, and loaded generation in which lookup succeeded.

24. dlclose and unloading hazards#

dlclose decrements a reference count. It does not promise immediate unloading: dependencies, other handles, global use, TLS, RTLD_NODELETE, implementation policy, and language/runtime registrations can retain an object. If unmapped, every code pointer, data pointer, vtable, callback, unwind record, TLS reference, and borrowed string into it becomes invalid. Reference counts cannot discover raw pointers hidden in application memory.

Stale-pointer workshop: a plugin registers callback P in a host queue, then its handle closes. Hours later the queue calls P; the address may be unmapped or, worse, remapped to unrelated code. The earliest broken invariant occurred at registration: the queue lifetime was not bounded by the plugin lease. Test by recording mapping generation and owner token at registration, not merely by reproducing the final crash.

Safe unloading needs quiescence across threads, callbacks, TLS destructors, unwinders, and foreign runtime registrations. A timeout proves only elapsed time. If quiescence cannot be proved, retaining the generation, using RTLD_NODELETE, or replacing a worker process is a correctness policy.

25. dlmopen, auditing, r_debug, and loader locks#

dlmopen(LM_ID_NEWLM, ...) creates another glibc link-map namespace. It can reduce accidental symbol collision, but it is not a sandbox: namespaces share a process, kernel credentials, address space, file descriptors, signals, and vulnerabilities. Some libraries assume process-global uniqueness and fail when duplicated. Namespace count and sharing details are glibc implementation constraints.

The auditing interface (LD_AUDIT, rtld-audit(7), glibc elf/rtld-audit.c) loads auditor DSOs and reports object/search/binding events, with architecture hooks for PLT traffic. It changes timing and runs sensitive code inside the process. Secure mode restricts it. Treat auditors as trusted instrumentation, not passive logs.

Debuggers rendezvous through structures conventionally exposed by DT_DEBUG: glibc's _r_debug has a state and link_map chain, and calls _dl_debug_state around map changes. GDB sets a breakpoint there to learn loaded-object addresses. This debugger protocol is implementation ABI, not a license for application code to mutate link_map. See gABI dynamic tags, glibc elf/dl-debug.c, and GDB's solib-svr4.c.

Loader locks and reentrancy#

Mapping and scope changes require internal locks. Constructors, destructors, IFUNC resolvers, audit callbacks, lazy binders, and allocation can call surprising code. Calling dlopen from a signal handler is not async-signal-safe. Holding an application mutex while calling into loader operations, while another thread holds a loader lock and calls application code that wants that mutex, creates lock inversion.

A robust policy keeps plugin lifecycle changes on a coordinator thread, publishes immutable snapshots to workers, does not unload while callbacks are possible, and performs no user callback while holding product-loader locks. glibc has recursive and staged lock machinery, but that cannot repair arbitrary application lock ordering.

26. Musl is a different loader policy#

Current musl centralizes much of dynamic linking in ldso/dynlink.c. Important practical differences from glibc include eager relocation/binding behavior, no /etc/ld.so.cache, different path policy, narrower GNU symbol-version semantics, no glibc dlmopen/audit ecosystem, and dlclose behavior that does not effectively unload mapped DSOs. These are musl implementation policies, not ELF guarantees. “Works on musl” must include tests for lookup and ABI assumptions rather than just recompilation.

27. A Rust plugin boundary with explicit lifetime#

Rust's native ABI and data layouts are not a stable cross-build plugin contract. Do not exchange String, Vec, trait objects, references with unproved lifetime, panics, allocator ownership, or compiler-generated enums across a DSO boundary. Use an explicit C ABI, fixed-width fields, size/version negotiation, function pointers, opaque handles, and paired allocation/destruction.

This stable Rust excerpt is a versioned ABI data model, not a complete loader. It does not load a DSO or call unknown code. unsafe extern "C" marks proof obligations at the eventual call site.

use std::ffi::{c_char, c_void};

pub const PLUGIN_ABI_V1: u32 = 1;

#[repr(C)]
pub struct HostV1 {
    pub abi_version: u32,
    pub struct_size: u32,
    pub context: *mut c_void,
    pub log: Option<unsafe extern "C" fn(*mut c_void, u32, *const c_char)>,
}

#[repr(C)]
pub struct PluginV1 {
    pub abi_version: u32,
    pub struct_size: u32,
    pub context: *mut c_void,
    pub run: Option<unsafe extern "C" fn(*mut c_void, *const u8, usize) -> i32>,
    pub destroy: Option<unsafe extern "C" fn(*mut c_void)>,
}

pub type PluginEntry = unsafe extern "C" fn(
    host: *const HostV1,
    out: *mut PluginV1,
) -> i32;

fn accepts(version: u32, size: u32) -> bool {
    version == PLUGIN_ABI_V1 && (size as usize) >= std::mem::size_of::<PluginV1>()
}

fn main() {
    assert!(accepts(PLUGIN_ABI_V1, std::mem::size_of::<PluginV1>() as u32));
    assert!(!accepts(2, std::mem::size_of::<PluginV1>() as u32));
}

The eventual unsafe wrapper must prove: pointers are aligned and valid for the call; the returned table was fully initialized before reading; every function pointer has the declared ABI; buffers live for the call; no panic unwinds across C; thread-safety claims are enforced; and destroy runs before the library lease ends. Include explicit ownership rules in the ABI specification, not only in Rust types.

Represent a loaded plugin as an owning lease and make callable wrappers borrow it. Safe Rust can then prevent ordinary code from outliving the library owner, although FFI code can still lie or register an escaping callback. Version and size preserve ABI evolution while deliberately discarding unstable Rust layout; encoding, allocation, nullability, threading, panic containment, and cancellation still need a language-neutral contract.

28. Loader security and product policy#

A product loader should separate mechanism from policy:

1. Discovery: read signed/owned metadata without executing candidates.
2. Compatibility: check product ABI, architecture, required features, and policy.
3. Isolation: prefer a worker process for third-party or updateable code; apply least privilege, resource limits, and a narrow IPC protocol.
4. Activation: stage a generation, initialize it, then atomically publish a lease.
5. Use: each request owns a generation lease; callbacks cannot outlive it.
6. Retirement: stop new leases, cancel/drain work, unregister callbacks, destroy plugin state, and only then close—or deliberately never unload.
7. Observability: log stable plugin identity, build ID, generation, dependency decisions, ABI rejection, and lifecycle transitions without leaking secrets.

Hot replacement is often safer as process replacement. If in-process unloading is required, model states (Staged, Active, Draining, Retired) and prove that only Active gets new work. Timeout does not prove quiescence. Leaking one old generation may be safer than executing a stale pointer.

Adversarial hardening#

• Treat ELF offsets, counts, hashes, version chains, and strings as hostile in inspection tools; cap work and detect arithmetic overflow and cycles.
• Use PIE plus ASLR, non-executable writable memory, full RELRO/eager binding where startup cost permits, stack protection, control-flow defenses supported by the architecture, and current toolchains.
• Avoid ambient LD_LIBRARY_PATH; use controlled deployment paths and read-only ownership. $ORIGIN is safe only if the origin tree is trusted.
• Do not grant setuid to complex dynamically extensible applications. Environment filtering is one layer, not a complete privilege design.
• Pin or authenticate plugin artifacts and dependencies; a matching filename is not identity.
• Keep writable directories out of privileged search paths. Audit symlink, mount-namespace, and race assumptions.
• Fuzz parsers with malformed, huge, overlapping, cyclic, and architecture-mismatched inputs under memory/time limits.

Do not run ldd on an untrusted executable. Historically and in unusual cases it may invoke an interpreter or otherwise cross into target-controlled behavior; even safer tracing modes entrust parsers with hostile bytes. Prefer non-executing inspection (readelf -d, objdump -p) inside a sandbox, and remember that static inspection cannot perfectly reproduce runtime namespaces, tokens, or policy.

29. Observation and debugging workshops#

Start observation with a copy of a small program you control:

readelf -h -l -d -r -sW ./app
readelf --version-info ./app
objdump -p ./app
LD_DEBUG=libs,reloc,bindings ./app
strace -f -e trace=execve,openat,mmap,mprotect,munmap ./app
gdb --args ./app
cat /proc/$PID/maps

readelf shows file declarations, not necessarily runtime choices. LD_DEBUG is glibc policy, noisy, timing-changing, and filtered in secure mode. strace sees syscalls but not pure userspace hash lookup. /proc/PID/maps shows current mappings but not why a scope selected one symbol. GDB's info sharedlibrary, maintenance info solib, breakpoints on _dl_debug_state, and an early entry breakpoint connect maps to execution. Permissions may limit /proc and ptrace.

Workshop A: trace exec to main#

Build a PIE that calls one libc function. Predict PT_INTERP, load segments, and needed objects before inspection. Record AT_ENTRY, AT_BASE, and AT_SYSINFO_EHDR with getauxval. Break at the interpreter entry, executable _start, and main. Explain each transition and identify which agent—the kernel, rtld, CRT, or libc—owns it. Repeat with a static executable and explain the missing interpreter stage.

Workshop B: RPATH versus RUNPATH#

Create app -> liba -> libb in separate controlled directories. Link app once with old dtags/RPATH and once with new dtags/RUNPATH. Predict whether app's path reaches libb; verify with readelf -d and LD_DEBUG=libs. Then add a RUNPATH to liba. State exactly which requester's metadata controlled each lookup.

Workshop C: crash before main#

Use a test DSO whose constructor intentionally aborts. First run with eager binding, then set breakpoints on constructor symbols and _start. Capture mappings and a backtrace. The visible failure is abort; the earliest violated application invariant is “constructors complete before application entry.” Extend the test with a missing versioned symbol and distinguish lookup failure from constructor failure.

Workshop D: stale callback#

In a disposable test process, load a DSO you wrote, obtain a callback, close incorrectly, and observe only under a debugger. Then repair the host with generation leases and RTLD_NODELETE as defense in depth. Do not generalize one observed dlclose outcome: compare glibc and musl and explain why an apparently working stale call is evidence of luck, not validity.

Myths to retire#

• “The kernel loads all shared libraries.” It maps the executable and named interpreter; rtld follows DT_NEEDED.
• “ELF sections drive execution.” Program headers and dynamic tags are the runtime core.
• “LD_LIBRARY_PATH always wins.” RPATH cases, secure mode, explicit slash paths, and already-loaded objects complicate that claim.
• “RUNPATH is inherited.” In glibc it governs direct dependencies, unlike old transitive RPATH behavior.
• “dlsym null means failure.” Check dlerror because a valid symbol value can be null.
• “dlclose invalidates the handle and therefore all use stops.” Raw pointers and callbacks are outside that bookkeeping.
• “dlmopen is isolation.” It separates lookup namespaces, not process authority or memory safety.
• “ASLR makes writable code pointers safe.” It randomizes placement; it does not enforce pointer integrity.
• “glibc behavior is ELF behavior.” Search, auditing, versions, and unload policy are runtime implementations.

30. Mastery exercises and a versioned source map#

1. Draw the ownership boundary from execve through main. Label every arrow with data, not just control.
2. Given three PT_LOAD records, compute page-aligned file/memory ranges and zero-fill. Reject one with overflowing p_vaddr + p_memsz.
3. Extend the safe stack parser to return named auxv constants and reject duplicate singleton tags. Explain why real pointers remain unsafe.
4. Explain why AT_ENTRY and the first instruction differ for a dynamic executable.
5. Build the RPATH/RUNPATH experiment and write a prediction before each run.
6. Extend the graph planner to output the condensed DAG and deterministic cycle diagnostics. Property-test that every inter-component dependency precedes its user.
7. Extend the scope model with weak definitions, hidden visibility, and exact/default versions. List where it still differs from glibc.
8. Measure startup and first-call costs under lazy and eager binding. Preserve program meaning and report filesystem-cache state.
9. Design a TLS stress test with threads created before and after repeated dlopen. Compare glibc and musl without assuming unload.
10. Write a plugin ABI specification covering ownership, cancellation, concurrency, panic containment, errors, and evolution. Add compile-time layout checks from C.
11. Implement a process-isolated plugin prototype with authenticated framing, bounded messages, timeouts, and crash restart. Threat-model malicious plugins.
12. In a current kernel checkout, trace do_execveat_common to load_elf_binary; identify the exact point of no return and submit a documentation correction if comments and behavior diverge.
13. In current glibc, follow one x86-64 R_X86_64_JUMP_SLOT from relocation table to _dl_fixup; add a focused test for a discovered edge case before proposing code.
14. In musl, trace load_library, dependency loading, relocation, and dlclose in ldso/dynlink.c; document which observations are policy and which are ABI.

Versioned authoritative source map#

Use these as reading anchors, and record the exact commit or release in serious analysis:

• Linux kernel: current source fs/exec.c (do_execveat_common, bprm_execve, exec_binprm), fs/binfmt_elf.c (load_elf_binary, load_elf_phdrs, elf_map/elf_load, create_elf_tables), fs/binfmt_script.c, fs/binfmt_misc.c, security/commoncap.c, architecture arch/*/entry/vdso, and Documentation/ABI/stable/vdso.
• Linux userspace ABI documentation: Linux man-pages 6.18: execve(2), getauxval(3), proc_pid_maps(5), vdso(7), capabilities(7), ld.so(8), dlopen(3), dlsym(3), dlerror(3), and rtld-audit(7).
• ELF standards: System V gABI chapters “Program Loading and Dynamic Linking,” “Program Header,” “Dynamic Section,” “Symbol Table,” and “Relocation”; then the target processor's current psABI (for example x86-64 psABI) for entry state, relocations, TLS, PLT/GOT, and GNU extensions it adopts.
• glibc: current release/commit elf/rtld.c, elf/dl-load.c, elf/dl-deps.c, elf/dl-lookup.c, elf/dl-reloc.c, elf/dl-runtime.c, elf/dl-open.c, elf/dl-close.c, elf/dl-tls.c, elf/dl-debug.c, elf/rtld-audit.c, elf/dynamic-link.h, csu/libc-start.c, sysdeps/unix/sysv/linux/dl-sysdep.c, and target sysdeps/*/dl-machine.h/start.S. Read tests beside elf/ before inferring intent from one function.
• musl: current release/commit ldso/dynlink.c, src/env/__libc_start_main.c, src/ldso/dlopen.c, src/ldso/dlsym.c, architecture relocation/TLS headers, and musl's dynamic-linker documentation. Much policy is intentionally centralized.
• debugger implementation: current GDB solib-svr4.c for r_debug/link_map rendezvous consumption; compare it with glibc elf/dl-debug.c rather than treating either side alone as a universal ELF guarantee.

The durable debugging rule is simple: walk backward from the first visible failure to the earliest broken invariant. A bad call in main may begin with wrong symbol scope; a constructor crash may begin with a cycle assumption; a stale callback crash begins when lifetime escaped its lease; and an rtld fault may begin when hostile file arithmetic was trusted. Loaders are understandable when representation, authority, ordering, and lifetime are kept explicit.

Part V — Build a Linker from Scratch in Stable Rust#

1. The contract: a bounded linker, not a miniature claim#

This part builds LinkLab, an educational static linker, and then derives a bounded ELF64 linker from it. LinkLab reads textual .lobj files and writes deterministic .lhex files. It does not read ELF, create native machine code, or execute its output. Calling it “ELF-like” would hide important differences, so we will not.

Our contract is deliberately narrow:

The linker never runs .lhex. Treating untrusted bytes as executable would cross a separate security boundary.

We will repeatedly separate four concerns:

• Mechanism records facts and performs transformations.
• Policy chooses among legal outcomes, such as which weak definition wins.
• Optimization makes the same observable result cheaper.
• Presentation turns retained provenance into diagnostics and map text.

The central invariant is: every reference either has a proved definition and checked address, or linking stops with a diagnostic. Every pass must say what it forgets. Parsing forgets textual spacing but retains spans. Resolution forgets losing candidates but records why the winner won. GC discards unreachable sections. Layout converts symbolic sizes into addresses. Relocation application finally overwrites placeholder bytes, so it must happen last.

2. A semantic oracle before an implementation#

A semantic oracle is a tiny, slow description of correct behavior. For LinkLab it is:

1. Begin with every eager member.
2. Resolve symbols. A strong definition beats weak definitions; two active strong definitions are an error. An unresolved weak reference may become zero.
3. If a required unresolved symbol is defined by a lazy member, activate the first such member in deterministic input order. Repeat to a fixed point.
4. Select one active member for each nonempty COMDAT-like unit key.
5. Starting from roots, follow relocation edges and retain reachable sections.
6. Assign aligned addresses in stable section-ID order.
7. Evaluate each relocation with unbounded mathematical integers, then reject values that do not fit its field.
8. Serialize retained bytes and the map in a prescribed order.

This is both specification and differential-test oracle. A faster implementation may index lazy members or parallelize parsing, but it must preserve these observations.

Prediction: may archive extraction run only once before resolution? No. A newly extracted member can introduce a new undefined symbol that requires another member.

text --tokens with spans--> normalized members --activation fixed point--> winners
  winners + reloc edges + roots --reachability--> live sections
  live sections --alignment/layout--> addresses --checked patches--> .lhex + map

Each arrow carries IDs, not borrowed pointers. That makes reordering storage less dangerous.

3. The .lobj grammar and one fixture#

Whitespace separates tokens; # starts a comment. Names are ASCII identifiers containing letters, digits, _, ., or -. Integers are decimal, except section bytes are hexadecimal.

file     := { member }
member   := "member" name ("eager" | "lazy") ["unit=" name] newline
            { section | definition | undefined | relocation } "end" newline
section  := "section" name "align=" uint ("root" | "noroot") hexbytes newline
definition := "def" name section uint ("local" | "strong" | "weak") newline
undefined  := "undef" name ("strong" | "weak") newline
relocation := "reloc" section uint ("ABS64" | "PC32") name sint newline
hexbytes := "-" | an even number of hexadecimal digits

Offsets are byte offsets. A definition's offset may equal its section length; a relocation field must fit entirely inside the section. Local symbols are visible only within their member. Global names share one namespace. unit=- means no COMDAT-like unit.

member start eager unit=-
section .text align=16 root 0000000000000000
def _start .text 0 strong
undef answer strong
reloc .text 0 ABS64 answer 0
end
member answer lazy unit=answer_v1
section .rodata align=8 noroot 2a00000000000000
def answer .rodata 0 strong
end

The expected image starts .text at address 0 and .rodata at address 8. The first eight bytes therefore become little-endian 08 00 00 00 00 00 00 00, followed by the encoded number 42.

4. Stable IDs and the cost of identity#

Never use a vector index without naming what it indexes:

#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
struct MemberId(u32);
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
struct SectionId(u32);
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
struct SymbolId(u32);

These newtypes prevent accidental cross-table lookup. IDs remain stable while vectors grow. The invariant is that an ID is allocated once and never reused during a link. Conversion from usize uses u32::try_from; exhaustion is an ordinary error, not truncation.

An ID preserves identity but forgets ownership. Tables must retain member: MemberId on sections and symbols. A pointer makes access cheap but identity dependent on storage; an ID moves one bounds check into lookup and makes serialization, diagnostics, and deterministic sorting easier.

5. Checked text and byte parsing#

The parser is a trust boundary. Before semantic work it checks line length, token count, ASCII, integer syntax, duplicate member-local section names, alignment, and hexadecimal length. A span is {file, line, column, length}. Parsing 18446744073709551616 must report an overflow at that token, not silently wrap.

Hex decoding illustrates the rule “prove the range, then write”:

// Complete dependency-free helper.
fn hex_bytes(text: &str, limit: usize) -> Result<Vec<u8>, String> {
	if text == "-" { return Ok(Vec::new()); }
	if text.len() % 2 != 0 { return Err("hex byte string has odd length".into()); }
	let count = text.len() / 2;
	if count > limit { return Err("section exceeds byte budget".into()); }
	let mut out = Vec::with_capacity(count);
	for i in 0..count {
		let pair = &text[i * 2..i * 2 + 2];
		out.push(u8::from_str_radix(pair, 16)
			.map_err(|_| format!("invalid hex byte at column {}", i * 2 + 1))?);
	}
	Ok(out)
}

Slicing is safe here because the complete parser first rejects non-ASCII input. Without that precondition, byte offsets could split UTF-8. Parsing forgets comments and whitespace; spans retain where meaningful tokens came from.

6. Diagnostics are preserved provenance#

An error should identify the earliest broken invariant and both sides of a conflict:

main.lobj:8:5: duplicate strong definition of `answer`
  first definition: library.lobj:14:5 (member old_answer)
  second definition: main.lobj:8:5 (member replacement)

Store spans on definitions, undefined references, relocations, sections, and member declarations. Do not rebuild locations by searching source text later. Diagnostic ordering is presentation policy: sort by (file input number, byte offset, error kind), never by hash-table iteration.

A useful diagnostic object has a primary span, message, zero or more labeled secondary spans, and notes. Rendering belongs outside resolution. This permits JSON or editor output without changing link semantics.

7. Normalize once, validate relationships once#

The parser produces a normalized model:

struct Section { id: SectionId, member: MemberId, name: String, align: u64, bytes: Vec<u8>, root: bool }
enum Binding { Local, Strong, Weak }
struct Definition { name: String, member: MemberId, section: SectionId, value: u64, binding: Binding }
struct Undefined { name: String, member: MemberId, weak: bool }
enum RelocKind { Abs64, Pc32 }
struct Reloc { section: SectionId, offset: u64, kind: RelocKind, target: String, addend: i64 }
struct Member { id: MemberId, name: String, eager: bool, unit: Option<String> }

Normalization turns section names in definitions and relocations into SectionIds. It proves that definition offsets are in range, relocation fields fit, alignments are nonzero powers of two, and local targets are unambiguous. Later passes never parse names to discover structure.

The pass forgets the original order of declarations inside a member because that order has no semantic meaning. It preserves member and section input order as IDs because those are tie-breakers.

8. Resolution is an explicit state machine#

For each global name, resolution state is one of:

Unseen
Referenced { at least_one_required }
DefinedWeak { winner, other_weak_candidates }
DefinedStrong { winner }
Conflict { first_strong, second_strong }

Transitions are easier to audit than a pile of replacement conditions. Strong + Strong becomes Conflict; Weak + Strong becomes DefinedStrong; Strong + Weak keeps the strong. Multiple weak definitions choose the lowest active MemberId. That tie-break is policy, not an ELF rule.

Local definitions never enter this map. A relocation first searches its member's locals, then the global map. Only a target explicitly declared with undef name weak may evaluate to zero; an unresolved strong target or an undeclared relocation name is an error. Resolution forgets losing weak candidates after recording enough provenance for a map explanation.

Counterexample: putting locals in the global map can make two unrelated files' .Ltmp0 symbols conflict. The visible duplicate error occurs during resolution, but the first broken invariant was namespace normalization.

9. Lazy archive extraction reaches a fixed point#

Build a deterministic index from global name to lazy defining members. A BTreeMap<String, Vec<MemberId>> is sufficient. Sort each vector by input order. Then:

activate all eager members
repeat
  resolve active members
  find the lexicographically first required unresolved name
  activate its lowest-ID lazy provider, if one exists
until no member was activated
resolve once more; report remaining required undefined names

Activation is monotonic, so at most member_count iterations can change state. This proves termination. A work queue is an optimization; the fixed-point meaning is the mechanism. Real Unix archive command-line ordering and group rescans have richer policy than LinkLab's global index.

Experiment: let lazy member A define a and require b; B defines b and requires c; C defines c. Root code requires a. Record activation after every iteration. If B or C remains inactive, the implementation performed a one-shot scan.

10. COMDAT-like units are atomic selection groups#

Several members may carry the same nonempty unit key. LinkLab keeps the lowest active MemberId and discards all sections and definitions of later active members with that key. Selection occurs before final resolution so discarded strong definitions cannot conflict.

This is only a teaching model. ELF SHT_GROUP with GRP_COMDAT groups sections within ET_REL objects; signatures and prevailing definitions matter. COFF COMDAT has additional selection kinds.

The invariant is atomicity: either every section belonging to the selected unit participates, or none does. Selection forgets discarded implementations, while a map may retain their IDs and the reason duplicate-unit. Selecting per section would permit code from one implementation and data from another—a near miss that can link and then fail at runtime.

11. The live-section graph#

Make each retained section a node. Every relocation from section A to a definition in section B adds edge A → B. Root-marked sections seed a graph traversal. A relocation to unresolved weak zero adds no edge. A member being active does not itself make all its sections live.

.text.start [root] --answer relocation--> .rodata.answer
       |
       +--helper relocation--> .text.helper --table--> .rodata.table

.debug.note                     no incoming path: collected

Use a sorted adjacency vector and a BTreeSet<SectionId> work set for reproducible traces. DFS or BFS is mechanism-equivalent because only the final set is observable. GC forgets bytes and symbols in dead sections; the map retains a discard reason. A definition in a dead section cannot satisfy a live relocation after GC, so resolution and graph construction must agree on selected definitions.

12. Alignment and layout without overflow#

For positive power-of-two alignment a, alignment is:

fn align_up(value: u64, a: u64) -> Result<u64, String> {
	if a == 0 || !a.is_power_of_two() { return Err("bad alignment".into()); }
	let mask = a - 1;
	value.checked_add(mask).map(|v| v & !mask)
		.ok_or_else(|| "address overflow during alignment".into())
}

LinkLab lays out live sections by SectionId, inserts zero padding, and starts at address zero. For each section it proves start + len fits u64 and the configured image-size budget. The invariant after layout is that live ranges do not overlap and every start satisfies its alignment.

Layout policy could instead group code and data. That would change addresses and therefore output, so it is not merely optimization. Layout forgets abstract placement freedom by choosing concrete addresses. Preserve section-to-range mappings for diagnostics and the map.

13. Scan before writing: synthetic needs#

Product linkers scan relocations before final layout because a relocation may require a GOT entry, PLT entry, thunk, dynamic relocation, or architecture-specific stub. These are synthetic sections: output data not copied directly from one input section.

LinkLab's two relocation kinds need no synthetic sections, but it still has an explicit scan pass. The pass validates kinds, identifies edges, and computes exact output/map size budgets. Keeping the pass teaches an architecture that can grow without circular layout:

resolve -> scan relocations -> create synthetic requirements -> layout -> apply
                 ^                    |
                 +--- repeat only if sizes can change

The invariant before layout is that every size-affecting need is known. A relaxation that changes instruction size violates that invariant and requires a bounded convergence strategy.

14. Relocation arithmetic is integer algebra#

Let S be the resolved symbol address, A the signed addend, and P the address of the relocation field. LinkLab defines:

Compute in i128, check the mathematical range, convert, and only then write little-endian bytes. Do not calculate in u64: a negative addend would wrap before validation.

fn abs64(s: u64, a: i64) -> Result<[u8; 8], String> {
	let v = i128::from(s) + i128::from(a);
	let n = u64::try_from(v).map_err(|_| "ABS64 overflow")?;
	Ok(n.to_le_bytes())
}
fn pc32(s: u64, a: i64, p: u64) -> Result<[u8; 4], String> {
	let v = i128::from(s) + i128::from(a) - i128::from(p);
	let n = i32::try_from(v).map_err(|_| "PC32 overflow")?;
	Ok(n.to_le_bytes())
}

The relocation write invariant is that [offset, offset + width) lies within both the input section and its output range. Applying writes to a cloned image makes failure atomic: no partial output is published.

15. The deterministic .lhex format#

.lhex is text plus hex, not an executable format:

LHEX1
size 16
section 0 .text 0 8 16
section 1 .rodata 8 8 8
symbol _start 0 strong
symbol answer 8 strong
data 08000000000000002a00000000000000

Lines end in LF. Numbers are canonical decimal; hex is lowercase; symbols are sorted by name; sections retain ID order. There is no timestamp, host path, random hash seed, or locale-sensitive formatting. Serialization checks an output-byte budget before allocating.

This format preserves final bytes, ranges, symbols, and binding. It forgets relocations, dead sections, source spans, and lazy-member history; those belong in a separate map. Determinism means identical normalized inputs and options produce identical bytes. It does not mean different input orders are equivalent, because input order is an explicit weak/COMDAT policy.

16. LinkLab: a coherent complete implementation#

The following dependency-free stable-Rust program is complete and runnable. To keep it reviewable, its surface grammar is a canonical subset of Chapter 3: one section per member, declarations after that section, ASCII input, and unit=- for no unit. It visibly includes spans in parse errors, alignment, local/strong/weak/undefined symbols, relocations, lazy members, unit selection, GC, checked layout and writes, limits, map output, and deterministic serialization. It never executes its output.

Save it as linklab.rs; run rustc linklab.rs && ./linklab input.lobj output.lhex output.map.

use std::{collections::{BTreeMap, BTreeSet}, env, fs};

const MAX_INPUT: usize = 1 << 20;
const MAX_MEMBERS: usize = 4096;
const MAX_IMAGE: usize = 16 << 20;

#[derive(Clone)] struct Sec { name:String, align:u64, root:bool, bytes:Vec<u8> }
#[derive(Clone, Copy, PartialEq)] enum Bind { Local, Strong, Weak }
#[derive(Clone)] struct Def { name:String, off:u64, bind:Bind, line:usize }
#[derive(Clone)] struct Und { name:String, weak:bool, line:usize }
#[derive(Clone, Copy)] enum Kind { Abs64, Pc32 }
#[derive(Clone)] struct Rel { off:u64, kind:Kind, name:String, add:i64, line:usize }
#[derive(Clone)] struct Mem { name:String, eager:bool, unit:Option<String>, sec:Sec,
	defs:Vec<Def>, unds:Vec<Und>, rels:Vec<Rel> }
fn err(file:&str,line:usize,msg:impl AsRef<str>)->String {
	format!("{}:{}:1: {}",file,line,msg.as_ref())
}
fn uint(file:&str,line:usize,s:&str)->Result<u64,String>{
	s.parse().map_err(|_|err(file,line,format!("invalid unsigned integer `{s}`")))
}
fn hex(file:&str,line:usize,s:&str)->Result<Vec<u8>,String>{
	if s=="-" { return Ok(vec![]) } if !s.is_ascii()||s.len()%2!=0{return Err(err(file,line,"bad hex bytes"))}
	if s.len()/2>MAX_IMAGE{return Err(err(file,line,"section exceeds limit"))}
	(0..s.len()/2).map(|i|u8::from_str_radix(&s[i*2..i*2+2],16)
		.map_err(|_|err(file,line,"bad hex bytes"))).collect()
}
fn parse(file:&str,text:&str)->Result<Vec<Mem>,String>{
	if text.len()>MAX_INPUT{return Err(err(file,1,"input exceeds limit"))}
	if !text.is_ascii(){return Err(err(file,1,"input must be ASCII"))}
	let mut out=Vec::new(); let mut cur:Option<Mem>=None;
	for (n,raw) in text.lines().enumerate(){let line=n+1;let code=raw.split('#').next().unwrap().trim();
		if code.is_empty(){continue} let t:Vec<_>=code.split_whitespace().collect();
		match t[0] {
			"member" if t.len()==4=>{if cur.is_some(){return Err(err(file,line,"nested member"))}
				if out.len()>=MAX_MEMBERS{return Err(err(file,line,"member limit"))}
				let eager=match t[2]{"eager"=>true,"lazy"=>false,_=>return Err(err(file,line,"expected eager or lazy"))};
				let u=t[3].strip_prefix("unit=").ok_or_else(||err(file,line,"expected unit="))?;
				cur=Some(Mem{name:t[1].into(),eager,unit:if u=="-"{None}else{Some(u.into())},
					sec:Sec{name:String::new(),align:1,root:false,bytes:vec![]},defs:vec![],unds:vec![],rels:vec![]});}
			"section" if t.len()==5=>{let m=cur.as_mut().ok_or_else(||err(file,line,"section outside member"))?;
				if !m.sec.name.is_empty(){return Err(err(file,line,"only one section per member in canonical subset"))}
				let a=uint(file,line,t[2].strip_prefix("align=").ok_or_else(||err(file,line,"expected align="))?)?;
				if a==0||!a.is_power_of_two(){return Err(err(file,line,"alignment must be a power of two"))}
				m.sec=Sec{name:t[1].into(),align:a,root:match t[3]{"root"=>true,"noroot"=>false,_=>return Err(err(file,line,"expected root or noroot"))},bytes:hex(file,line,t[4])?};}
			"def" if t.len()==5=>{let m=cur.as_mut().ok_or_else(||err(file,line,"def outside member"))?;
				if t[2]!=m.sec.name{return Err(err(file,line,"definition names unknown section"))}
				let off=uint(file,line,t[3])?;if off>m.sec.bytes.len() as u64{return Err(err(file,line,"definition offset out of range"))}
				let bind=match t[4]{"local"=>Bind::Local,"strong"=>Bind::Strong,"weak"=>Bind::Weak,_=>return Err(err(file,line,"bad binding"))};
				m.defs.push(Def{name:t[1].into(),off,bind,line});}
			"undef" if t.len()==3=>{let m=cur.as_mut().ok_or_else(||err(file,line,"undef outside member"))?;
				m.unds.push(Und{name:t[1].into(),weak:match t[2]{"weak"=>true,"strong"=>false,_=>return Err(err(file,line,"bad undefined binding"))},line});}
			"reloc" if t.len()==6=>{let m=cur.as_mut().ok_or_else(||err(file,line,"reloc outside member"))?;
				if t[1]!=m.sec.name{return Err(err(file,line,"relocation names unknown section"))}
				let off=uint(file,line,t[2])?;let kind=match t[3]{"ABS64"=>Kind::Abs64,"PC32"=>Kind::Pc32,_=>return Err(err(file,line,"bad relocation kind"))};
				let width=match kind{Kind::Abs64=>8,Kind::Pc32=>4};if off.checked_add(width).filter(|&x|x<=m.sec.bytes.len() as u64).is_none(){return Err(err(file,line,"relocation field out of range"))}
				let add=t[5].parse().map_err(|_|err(file,line,"bad addend"))?;m.rels.push(Rel{off,kind,name:t[4].into(),add,line});}
			"end" if t.len()==1=>{let m=cur.take().ok_or_else(||err(file,line,"end outside member"))?;
				if m.sec.name.is_empty(){return Err(err(file,line,"member has no section"))} out.push(m);}
			_=>return Err(err(file,line,"unknown or malformed directive")),
		}}
	if cur.is_some(){return Err(err(file,text.lines().count(),"unterminated member"))} Ok(out)
}
fn globals(ms:&[Mem],active:&BTreeSet<usize>)->Result<BTreeMap<String,(usize,usize)>,String>{
	let mut g:BTreeMap<String,(usize,usize)>=BTreeMap::new();
	for &mi in active{for (di,d) in ms[mi].defs.iter().enumerate(){if d.bind==Bind::Local{continue}
		match g.get(&d.name).copied(){None=>{g.insert(d.name.clone(),(mi,di));},Some((om,od))=>{
			let old=&ms[om].defs[od];if old.bind==Bind::Strong&&d.bind==Bind::Strong{return Err(format!("duplicate strong `{}` at lines {} and {}",d.name,old.line,d.line))}
			if old.bind==Bind::Weak&&d.bind==Bind::Strong{g.insert(d.name.clone(),(mi,di));}}}}
	}
	Ok(g)
}
fn align(v:u64,a:u64)->Result<u64,String>{let m=a-1;v.checked_add(m).map(|x|x&!m).ok_or("layout overflow".into())}
fn link(ms:&[Mem])->Result<(String,String),String>{
	let mut active:BTreeSet<usize>=ms.iter().enumerate().filter(|(_,m)|m.eager).map(|(i,_)|i).collect();
	loop{let g=globals(ms,&active)?;let mut need=BTreeSet::new();for &i in &active{for u in &ms[i].unds{if !u.weak&&!g.contains_key(&u.name)&&!ms[i].defs.iter().any(|d|d.bind==Bind::Local&&d.name==u.name){need.insert(u.name.clone());}}}
		let pick=need.iter().find_map(|n|ms.iter().enumerate().find(|(i,m)|!active.contains(i)&&!m.eager&&m.defs.iter().any(|d|d.bind!=Bind::Local&&d.name==*n)).map(|(i,_)|i));
		if let Some(i)=pick{active.insert(i);}else{break}}
	let mut units=BTreeMap::new();let snapshot:Vec<_>=active.iter().copied().collect();for i in snapshot{if let Some(u)=&ms[i].unit{if units.insert(u.clone(),i).is_some(){active.remove(&i);}}}
	let g=globals(ms,&active)?;for &i in &active{for u in &ms[i].unds{if !u.weak&&!g.contains_key(&u.name)&&!ms[i].defs.iter().any(|d|d.bind==Bind::Local&&d.name==u.name){return Err(format!("undefined `{}` at line {}",u.name,u.line))}}}
	for &i in &active{for r in &ms[i].rels{let local=ms[i].defs.iter().any(|d|d.bind==Bind::Local&&d.name==r.name);let declared=ms[i].unds.iter().any(|u|u.name==r.name);if !local&&!g.contains_key(&r.name)&&!declared{return Err(format!("relocation target `{}` has no definition or declaration at line {}",r.name,r.line))}}}
	let target=|mi:usize,n:&str|->Option<(usize,u64,Bind)>{if let Some(d)=ms[mi].defs.iter().find(|d|d.bind==Bind::Local&&d.name==n){return Some((mi,d.off,d.bind))}g.get(n).map(|&(m,d)|(m,ms[m].defs[d].off,ms[m].defs[d].bind))};
	let mut live:BTreeSet<usize>=active.iter().filter(|&&i|ms[i].sec.root).copied().collect();loop{let before=live.len();for i in live.clone(){for r in &ms[i].rels{if let Some((m,_,_))=target(i,&r.name){live.insert(m);}}}if live.len()==before{break}}
	let mut starts=BTreeMap::new();let mut size=0u64;for &i in &live{size=align(size,ms[i].sec.align)?;starts.insert(i,size);size=size.checked_add(ms[i].sec.bytes.len() as u64).ok_or("image overflow")?;if size>MAX_IMAGE as u64{return Err("image limit".into())}}
	let mut image=vec![0u8;size as usize];for &i in &live{let s=starts[&i] as usize;image[s..s+ms[i].sec.bytes.len()].copy_from_slice(&ms[i].sec.bytes)}
	for &i in &live{for r in &ms[i].rels{let p=starts[&i].checked_add(r.off).ok_or("place overflow")?;let value=match target(i,&r.name){Some((m,o,_))if live.contains(&m)=>Some(starts[&m].checked_add(o).ok_or("symbol overflow")?),Some(_)=>return Err(format!("live relocation reaches dead section at line {}",r.line)),None=>None};
		let at=p as usize;match (r.kind,value){(Kind::Abs64,Some(s))=>{let v=u64::try_from(i128::from(s)+i128::from(r.add)).map_err(|_|"ABS64 overflow")?;image[at..at+8].copy_from_slice(&v.to_le_bytes())},(Kind::Pc32,Some(s))=>{let v=i32::try_from(i128::from(s)+i128::from(r.add)-i128::from(p)).map_err(|_|"PC32 overflow")?;image[at..at+4].copy_from_slice(&v.to_le_bytes())},(Kind::Abs64,None)=>image[at..at+8].fill(0),(Kind::Pc32,None)=>image[at..at+4].fill(0)}}}
	let mut out=format!("LHEX1\nsize {size}\n");let mut map=String::new();for &i in &live{out+=&format!("section {i} {} {} {} {}\n",ms[i].sec.name,starts[&i],ms[i].sec.bytes.len(),ms[i].sec.align);map+=&format!("live member={} section={} address={}\n",ms[i].name,ms[i].sec.name,starts[&i]);}
	for (n,(m,d)) in &g{if live.contains(m){let x=&ms[*m].defs[*d];out+=&format!("symbol {n} {} {}\n",starts[m]+x.off,if x.bind==Bind::Strong{"strong"}else{"weak"});}}
	out+="data ";for b in image{out+=&format!("{b:02x}")}out+="\n";for (i,m) in ms.iter().enumerate(){if !live.contains(&i){map+=&format!("discard member={} reason={}\n",m.name,if active.contains(&i){"unreachable"}else{"inactive-or-duplicate-unit"});}}Ok((out,map))
}
fn run()->Result<(),String>{let a:Vec<_>=env::args().collect();if a.len()!=4{return Err("usage: linklab INPUT.lobj OUTPUT.lhex OUTPUT.map".into())}let text=fs::read_to_string(&a[1]).map_err(|e|e.to_string())?;let ms=parse(&a[1],&text)?;let (image,map)=link(&ms)?;fs::write(&a[2],image).map_err(|e|e.to_string())?;fs::write(&a[3],map).map_err(|e|e.to_string())}
fn main(){if let Err(e)=run(){eprintln!("linklab: {e}");std::process::exit(1)}}

The complete program deliberately uses member index as its stable ID to keep the listing compact. The earlier newtypes are the first hardening exercise. Its unit policy chooses the first active unit member; its archive policy chooses the first provider in input order. Neither is implied by ELF.

17. Trace every state change#

For the Chapter 3 fixture, the exact abstract trace is:

Expected .lhex is the Chapter 15 text. The map is:

live member=start section=.text address=0
live member=answer section=.rodata address=8

Prediction: change .rodata alignment to 16. The patched value becomes 16 and the image gains eight bytes of padding. Nothing in resolution changes; if it does, pass boundaries are leaking.

18. Tests, properties, and adversarial experiments#

Start with golden tests for the fixture, duplicate strong definitions, weak replacement, missing strong symbols, undeclared relocation targets, explicitly declared unresolved weak zero, lazy chains, duplicate units, dead sections, alignment padding, and both relocation overflows. Compare complete output bytes and stable diagnostics.

Useful properties require no external crate:

1. Parse, print canonically, and parse again; normalized models must match.
2. Linking the same input twice produces byte-identical image and map.
3. Every live section start is aligned and ranges do not overlap.
4. Every applied PC32, decoded as i32, equals S + A - P.
5. Adding an unreachable lazy member changes neither image nor existing map lines.
6. Activation and liveness only grow during their respective fixed points.

For fuzzing, a small deterministic generator can enumerate token streams and mutations. A coverage-guided fuzzer is useful later, but core parsing should remain crate-free. Differentially compare the optimized linker against the Chapter 2 oracle, not against ELF tools: the formats have different semantics.

Debugging workshop: a PC32 value is four too small. Log S, A, P, width, symbol owner, and section starts. If P means the end of the field rather than its address, the arithmetic pass has violated its formula. If P is already wrong before application, inspect layout instead.

19. Resource budgets and cancellation#

Limits are part of correctness under hostile input. Configure maximum input bytes, lines, token length, members, sections, symbols, relocations, archive activations, graph edges, output bytes, and diagnostic count. Every accumulation uses checked addition. Reject before allocating when possible.

Cancellation can be a dependency-free trait:

trait Cancel { fn cancelled(&self) -> bool; }
fn checkpoint(c: &dyn Cancel) -> Result<(), String> {
	if c.cancelled() { Err("link cancelled".into()) } else { Ok(()) }
}

Check at bounded intervals in parsing, archive extraction, graph traversal, relocation scanning, and writing. Cancellation is policy; passing a never-cancel token preserves mechanism. Publish via a temporary file followed by rename so cancellation never leaves an apparently valid partial image. Account temporary vectors as well as final bytes. “Input is only 1 MiB” does not prevent a quadratic symbol table or a huge diagnostic storm.

20. Where the toy model stops#

LinkLab taught pass ordering, identity, provenance, fixed points, graph reachability, layout, and checked patching. It did not teach ELF's binary encoding, section groups, symbol visibility, architecture relocation details, segments, permission policy, or startup ABI.

Moving to ELF changes representation, not the core reasoning:

Do not mutate LinkLab's parser until it “accepts ELF.” Create an InputReader boundary that emits a richer normalized model. Otherwise file-format concerns contaminate resolution and layout.

21. Bounded ELF64 little-endian x86-64 contract#

The real-format extension accepts only:

• ELF64, little-endian, version 1, EM_X86_64, ET_REL;
• ordinary SHT_PROGBITS and SHT_NOBITS allocatable sections;
• one SHT_SYMTAB and its linked SHT_STRTAB per object;
• SHT_RELA relocations with R_X86_64_64, PC32, PLT32, 32, and 32S;
• local, global, and weak symbols; undefined and absolute symbols;
• a caller-selected _start; no dynamic linker, shared libraries, TLS, notes, unwind synthesis, GNU properties, symbol versions, linker scripts, archives, COMDAT, relaxation, or PIE.

It emits one Linux x86-64 ET_EXEC with an ELF header, program-header table, two or three PT_LOAD segments, an entry address, and no section headers in the smallest milestone. A later milestone may emit .symtab, .strtab, and .shstrtab for tools. This is enough for tiny syscall-only programs, not a general hosted C or Rust environment.

The invariant is that unsupported input is rejected by name and location. Silently ignoring an unknown allocatable section, relocation, or symbol state is worse than refusing it.

22. Reading ELF without unsafe#

Never cast file bytes to a Rust struct. Besides alignment and endianness, Rust layout is not the ELF wire format. This complete cursor uses checked ranges:

#[derive(Clone, Copy)] pub struct Reader<'a> { pub b: &'a [u8] }
impl<'a> Reader<'a> {
	pub fn bytes(&self, off:u64, len:u64)->Result<&'a [u8],String>{
		let s=usize::try_from(off).map_err(|_|"offset too large")?;
		let n=usize::try_from(len).map_err(|_|"length too large")?;
		let e=s.checked_add(n).ok_or("range overflow")?;
		self.b.get(s..e).ok_or_else(||"range outside file".into())
	}
	pub fn u16(&self,o:u64)->Result<u16,String>{let x=self.bytes(o,2)?;Ok(u16::from_le_bytes([x[0],x[1]]))}
	pub fn u32(&self,o:u64)->Result<u32,String>{let x=self.bytes(o,4)?;Ok(u32::from_le_bytes(x.try_into().unwrap()))}
	pub fn u64(&self,o:u64)->Result<u64,String>{let x=self.bytes(o,8)?;Ok(u64::from_le_bytes(x.try_into().unwrap()))}
	pub fn i64(&self,o:u64)->Result<i64,String>{Ok(self.u64(o)? as i64)}
}

The unwrap calls cannot fail because bytes proved exact fixed lengths; replacing them with a small array-copy helper can satisfy a no-unwrap product policy. No unsafe is needed anywhere in the linker.

First validate the 16-byte identification: magic 7f 45 4c 46, class 2, data 1, version 1. Then read the ELF header fields at specified offsets. Require e_ehsize=64, e_shentsize=64, and prove e_shoff + e_shnum*64 fits the file. Extended section numbering is explicitly unsupported here.

23. Sections, symbols, strings, and RELA relationships#

An ELF64 section header has name index, type, flags, address, file offset, size, link, info, alignment, and entry size. For ET_REL, allocatable sections normally have no final address. Validate sh_addralign as zero or a power of two; normalize zero to one. SHT_NOBITS occupies memory but no file range. Other sections require a checked file range.

SHT_SYMTAB.sh_link indexes its string table; sh_entsize must be 24. sh_info separates local symbols from nonlocals. An Elf64_Sym contains st_name, info, other, section index, value, and size. Prove each string index reaches a NUL within the linked string table. SHN_UNDEF is a reference; SHN_ABS is an absolute value; an ordinary index identifies the defining section.

SHT_RELA.sh_info identifies the section being patched and sh_link identifies the symbol table. Each 24-byte entry contains r_offset, r_info, and signed r_addend:

fn rela_symbol(info:u64)->Result<u32,String>{u32::try_from(info>>32).map_err(|_|"bad symbol index".into())}
fn rela_type(info:u64)->u32{info as u32}

The symbol number is local to that symbol table, so normalized references use (ObjectId, SymbolIndex), not only a name. Parsing forgets file offsets after preserving spans and source indices for diagnostics.

24. Mapping ET_REL into output sections#

Map each supported SHF_ALLOC input section to an output class:

Within a class, preserve (object input order, section index) and each section's alignment. Merging sections forgets input boundaries unless the map stores each contribution's output offset. Keep that table because symbols and relocations are expressed in input-section coordinates.

Local symbols resolve through their owning object and section contribution. Global/weak symbols enter the same state machine as LinkLab, with ELF visibility either rejected or implemented explicitly. STT_SECTION locals are important relocation targets. SHN_COMMON is omitted; reject it rather than pretending it is .bss.

25. The x86-64 relocation backend#

For AMD64 psABI notation, S is symbol value, A explicit RELA addend, and P place. The bounded backend implements:

enum X64 { R64, Pc32, Plt32, U32, S32 }
fn relocate(k:X64,s:u64,a:i64,p:u64)->Result<Vec<u8>,String>{
	let absolute=i128::from(s)+i128::from(a);
	match k {
		X64::R64=>Ok(u64::try_from(absolute).map_err(|_|"R_X86_64_64 overflow")?.to_le_bytes().to_vec()),
		X64::Pc32|X64::Plt32=>Ok(i32::try_from(absolute-i128::from(p)).map_err(|_|"PC-relative overflow")?.to_le_bytes().to_vec()),
		X64::U32=>Ok(u32::try_from(absolute).map_err(|_|"R_X86_64_32 overflow")?.to_le_bytes().to_vec()),
		X64::S32=>Ok(i32::try_from(absolute).map_err(|_|"R_X86_64_32S overflow")?.to_le_bytes().to_vec()),
	}
}

Treating PLT32 as direct is valid only under this linker's no-preemption static contract. A general linker may need a PLT. The backend must not know ELF file offsets or symbol-name policy; it receives proved values and returns bytes or a diagnostic.

26. Writing ET_EXEC, segments, and .bss#

Choose a page size, for example 0x1000, and a base virtual address such as 0x400000. Emit the 64-byte ELF header followed by 56-byte program headers. Use ET_EXEC=2, EM_X86_64=62, current version, e_phoff=64, and e_ehsize=64. Set e_entry to resolved _start.

Create R-X and RW- PT_LOAD segments, optionally a separate R-- segment. For every load segment:

p_offset mod p_align == p_vaddr mod p_align
p_filesz <= p_memsz
all referenced file and virtual ranges fit without overlap

Page congruence lets the kernel map file pages at the requested virtual pages. Layout headers in the first readable segment; page-align the next segment's file offset and virtual address. .bss extends p_memsz but not p_filesz; initialize no file bytes for it. Symbols in .bss still get addresses inside the memory tail.

A writer should provide checked put_u16/u32/u64(offset, value) over a pre-sized Vec<u8>. Build a layout plan first, allocate once after checking the file budget, write headers and content, apply relocations to the clone, validate the finished plan, then atomically publish. Writing while still deciding sizes creates self-referential bugs.

27. A no-libc _start acceptance program#

This x86-64 GNU assembler source makes only Linux syscalls. It needs no CRT or libc:

.global _start
.section .text
_start:
    mov $1, %rax
    mov $1, %rdi
    lea message(%rip), %rsi
    mov $3, %rdx
    syscall
    mov $60, %rax
    xor %rdi, %rdi
    syscall
.section .rodata
message:
    .ascii "ok\n"

Assemble with as --64 -o hello.o hello.s. The bounded linker should consume hello.o, resolve _start, apply the likely PC-relative relocation to message, and produce an ET_EXEC. Execution is a separate test step outside the linker and only after inspection. LinkLab itself never executes anything.

A hosted program beginning at main is not equivalent. CRT startup supplies _start, interprets the initial stack, initializes libc and language runtimes, runs constructors, calls main, and performs termination. libc brings archives, symbol versions, TLS, IFUNC, unwind metadata, linker scripts, dynamic linking or large static dependency graphs, and ABI policy. Supporting one syscall object is evidence for the bounded contract, not a general linker.

28. Validate with independent tools#

Use tools as observers, not as specifications:

readelf -h -l -s tiny
readelf -r hello.o
objdump -dr hello.o
objdump -d tiny
cmp tiny first-build

Check Type: EXEC, machine x86-64, entry inside an executable load segment, nonoverlapping load ranges, congruent offsets/addresses, and filesz <= memsz. In disassembly, independently calculate each patched displacement. If section headers are intentionally absent, explain the resulting tool limitations rather than mistaking warnings for loader rejection.

Differential testing against ld.lld, mold, or GNU ld must compare an observation model—loaded segment bytes, exported addresses, entry behavior—not whole files. Product linkers legitimately differ in build IDs, section ordering, padding, metadata, and relaxation.

29. Incremental milestones and architecture boundaries#

Build in this order, keeping every milestone executable as a test suite:

1. Reader primitives and ELF-header rejection tests.
2. Section/string/symbol/RELA parsing into stable IDs.
3. One object, one .text, no relocations; emit valid headers and one segment.
4. Multiple sections with alignment and contribution maps.
5. Local symbols and PC32; then all five relocation kinds.
6. Multiple objects, global/weak resolution, and undefined diagnostics.
7. R-X/R--/RW- segments, .bss, and _start validation.
8. GC and a deterministic map, if included in the bounded contract.

Keep boundaries narrow:

InputReader -> NormalizedGraph -> Resolver -> RelocScanner -> LayoutPlan
                                                        |          |
ArchitectureBackend <--- proved relocation requests ----+          v
OutputWriter <---------------- immutable plan + patched bytes ------+
Diagnostics <---------------- provenance from every boundary

The reader owns binary syntax. The resolver owns names and binding policy. The backend owns formulas and range checks. The writer owns ELF encoding, not semantic decisions. The layout planner owns addresses and segment congruence. This separation permits a second architecture or file format without copying resolution.

30. Hardening, source reading, and the product-grade gap#

Normative references are the System V ABI generic ABI (gABI) chapters “Object Files” and “Program Loading and Dynamic Linking,” plus the System V AMD64 ABI supplement sections on ELF, relocation types, and process initialization. Read the version matching the platform/toolchain; processor supplements evolve. Linux elf(5) and kernel fs/binfmt_elf.c explain Linux-facing details but do not replace those ABI contracts.

For current implementations, read by responsibility rather than line number:

• LLVM lld ELF: ELF/InputFiles.*, Symbols.*, Relocations.*, SyntheticSections.*, and Writer.*;
• mold: ELF input parsing, symbol resolution, archive extraction, relocation scanning, output chunks/sections, and architecture backends under its ELF source tree;
• GNU BFD ld: ld/ldlang.* for language/layout, ld/ldfile.*, archive handling through BFD, and bfd/elf64-x86-64.c for the x86-64 backend.

These are current implementation areas, not language or ABI guarantees; directories and names can move. Trace one symbol and one relocation end to end before proposing a change.

The explicit product-grade gap remains large:

• real ar archives, command-line order/groups, thin archives, and archive indexes;
• ELF groups/COMDAT, common symbols, visibility, protected semantics, versions, and wrapping;
• shared objects, PIE, GOT, PLT, dynamic relocations, hash tables, and interpreter metadata;
• TLS models, IFUNC, copy relocations, GNU properties, notes, build IDs, and RELRO;
• linker scripts, orphan policy, memory regions, PHDR control, and section sorting;
• relaxation, thunks, branch islands, identical-code folding, mergeable strings, and LTO plugins;
• debug/unwind preservation, split DWARF, symbol tables, reproducible metadata, and stripping;
• more architectures, endianness/classes, OS ABIs, huge inputs, parallelism, incremental linking;
• hardened path handling, atomic output, sandboxing, telemetry, compatibility suites, and stable CLI.

31. Review map, debugging practice, and capstone rubric#

When output fails late, find the earliest broken invariant:

Capstone acceptance rubric:

• Parsing: rejects every out-of-range table, unterminated string, unsupported class/type, and excessive resource use with stable source provenance.
• Semantics: local/global/weak resolution, undefined handling, archive fixed point, unit selection, and GC match a written oracle.
• Layout: all additions and conversions are checked; contribution ranges do not overlap; alignments and page congruence are mechanically tested; .bss consumes memory but not file data.
• Relocations: all five x86-64 formulas use i128, enforce signedness/width, and report symbol, object, section, offset, formula inputs, and range on failure.
• Output: ET_EXEC headers and load segments pass independent structural checks; _start lies in R-X memory; two runs are byte-identical; output publication is atomic.
• Evidence: golden, property, malformed-input, differential-observation, cancellation, and budget tests pass; the syscall-only program is inspected with readelf and objdump before optional execution in an isolated test environment.
• Honesty: every omission above is rejected or documented, no LinkLab rule is presented as ELF, no internal behavior is claimed as an ABI guarantee, and no unsafe is needed in the linker.

The capstone is complete only when another reader can locate a failure by pass and invariant, not merely when one fixture prints ok.

Part VI — Build a Loader from Scratch in Stable Rust#

1. The contract: turn bytes into a constrained computation#

A loader is a boundary between an untrusted description and a running computation. Its job is not “copy bytes and jump.” For this part, loading succeeds only if all of these postconditions hold:

1. every input range was checked before indexing;
2. every virtual range fits one bounded address space;
3. mappings agree with policy, including write-xor-execute (W^X);
4. file bytes and zero-filled tails have their specified values;
5. each relocation wrote an allowed place using an allowed source;
6. imports came from an explicit trusted scope;
7. RELRO is no longer writable;
8. the startup stack is valid; and
9. control enters only our bytecode interpreter.

Failure is atomic: no executable object is published. Parsing, planning, mapping, relocating, resolving, initialization planning, policy, and execution are separate stages. The threat model is hostile bytes, huge counts, integer overflow, overlaps, forged names, and programs that never halt. It does not include a memory-safety bug in Rust's standard library or a malicious host resolver.

bytes -> Parser -> Parsed -> Planner + Policy -> Plan
                                              |
                                              v
trusted modules -> Resolver -> Mapper -> Relocator -> seal RELRO -> Loaded
                                                               |
                                       InitializerPlanner ------+-> Executor(VM)
Any error -------------------------------------------------------> discard transaction

Invariant before side effects: validation and planning are pure. The safe mapper builds a private Vec; an error drops it. A native mapper must instead keep a rollback journal.

2. An image is not a process#

An image describes bytes, addresses, imports, and an entry. A process also has kernel-created VMAs, threads, credentials, signal state, file descriptors, an ABI stack, TLS registers, and an address space identity. Part V's .lhex was a deterministic static-link oracle, not an executable format. This part deliberately introduces a separate binary .limg format that models only the first group plus a small startup stack. Its final capstone adds a linker backend that emits this format.

Counterexample: mapping an ELF into the current process does not remove old mappings, stop sibling threads, reset signal dispositions, install the vDSO, or recreate credentials. It is not execve.

Prediction: can simulated X make attacker bytes execute natively? Answer: no. It is only a bit checked by the VM before instruction fetch.

3. The LinkLab .limg wire format#

All integers are little-endian. Addresses are offsets in a 64 KiB maximum image. Names are UTF-8. This is not a binary encoding of Part V's textual .lhex; it adds permissions, imports, unresolved load-time relocations, TLS, constructors, RELRO, and an executable bytecode entry. Keeping the two formats distinct prevents the static-link oracle from silently acquiring loader semantics.

Flags are R=1, W=2, X=4; unknown bits fail. Relocation kind 0 is RELATIVE: *(place)=base+addend; kind 1 is SYMBOL: *(place)=symbol(source)+addend. Writes are little-endian u32. Constructors are import indices: we order them but never call them.

This compact format is educational, not compatible with ELF and not a stable interchange standard. It intentionally stores segment payloads inline so truncation checks are easy to see.

4. Budgets are part of parsing#

The parser must reject work before allocating proportional memory. Our limits are 16 segments, 64 relocations, 32 imports, 32 constructors, 64-byte names, 64 KiB memory, and 10,000 VM steps. Use checked_add, convert widths explicitly, and take slices with get.

#[derive(Clone, Copy)]
struct Budget {
	max_mem: u32,
	max_steps: u64,
}

fn range(start: u32, len: u32, limit: u32) -> Result<std::ops::Range<usize>, String> {
	let end = start.checked_add(len).ok_or("address overflow")?;
	if end > limit { return Err("range exceeds address space".into()); }
	Ok(start as usize..end as usize)
}

The conversion is safe because the format is bounded to u32 and supported Rust hosts have a usize capable of indexing the allocated 64 KiB vector. A production parser should state its host width policy rather than inherit it accidentally.

5. Address types prevent category mistakes#

File offsets, image virtual addresses, host indices, symbol values, and bytecode program counters are not interchangeable even when represented by integers. The implementation uses u32 on the wire, but each stage gives values a meaning. A larger implementation should use newtypes such as FileOff(u64), Vaddr(u64), and HostIndex(usize).

The central translation is index = vaddr - min_vaddr; LinkLab fixes min_vaddr at zero, making the load bias zero. ELF will not. Keeping translation in one function prevents “works for ET_EXEC, fails for PIE” bugs.

Exercise: change LinkLab to choose a deterministic nonzero base. List every equation that must change. The VM's virtual operands should change; vector indices should not.

6. Planning segments before allocation#

The planner establishes these invariants:

• file_len <= mem_len;
• every segment is nonempty, in bounds, and has R;
• flags contain no unknown bit and policy rejects W|X;
• segments do not overlap in memory;
• the entry is inside one executable segment;
• every relocation write lies wholly in a writable segment;
• RELRO lies wholly in mapped writable bytes; and
• TLS, if present, names a readable non-executable segment.

Sort temporary (start,end,index) tuples by start; adjacent pairs expose overlap in O(S log S) time. Planning costs O(S log S + R*S) here; an interval map would reduce repeated containment queries for large formats. We keep original segment order for payload identity.

7. Overlap, alignment, and congruence#

LinkLab has byte granularity, so it requires only non-overlap. Real mappings have page granularity. For ELF, the required congruence is p_vaddr % p_align == p_offset % p_align when alignment is greater than one. Two byte-disjoint segments may still share a page and request incompatible final permissions.

page 0                     page 1
[ RX segment tail | RW segment head ]
^ byte ranges do not overlap, but one page cannot be both policies without compromise

Do not silently round each segment independently. First derive the union of page intervals, detect collisions, and assign a final protection per page. Reject a page that would become W+X under a strict policy.

Counterexample: checking only start < previous_end before sorting misses an overlap hidden by input order.

8. Map, copy, and zero#

Mapping starts with vec![0; max_end]; therefore all anonymous tails are zero. For each segment, copy exactly file_len bytes into [vaddr, vaddr+file_len). The remainder [vaddr+file_len, vaddr+mem_len) retains zeroes. Never round the file copy up to a page: bytes after the file extent might belong to another file object or be absent.

file payload:  [11 22 30 00]
memory:        [11 22 30 00 00 00 00 00]
               <--- file ---><-- zero -->

The mapper also builds one permission byte per address. This costs memory but makes the semantic oracle exact and simple. Production uses page tables, where permission granularity changes policy.

9. Permissions and W^X are transitions#

Relocation needs writable destinations; execution needs non-writable code. LinkLab requires code to arrive R|X and relocations to target R|W, so it never makes executable bytes writable. RELRO starts writable, receives relocations, then loses W.

Real loaders sometimes need text relocations or JIT transitions. A strict product should reject text relocations. “Temporarily W+X” expands the time in which corruption becomes executable.

10. Relocation phases and equations#

Relocation is split into scan and apply. Scan validates kinds, indices, write ranges, arithmetic, and duplicate places. Only then does apply mutate the private vector.

LinkLab uses B=0, signed A, and requires the final value to fit u32. Two relocations to the same word are rejected because order-dependent writes make audit and differential testing harder. The addend lives in the relocation record, so applying twice gives the same bytes; that still does not make double application acceptable.

Prediction: should S + A use wrapping arithmetic? Answer: no. A wrapped pointer can pass a later range check by becoming small. Compute in i64, then range-check.

11. Import scope is capability scope#

The resolver receives explicit trusted modules; it never reads environment variables or searches a filesystem. Modules are searched in supplied order, and duplicate exported names across modules are an error rather than an accidental interposition rule.

#[derive(Clone)]
struct Module { name: &'static str, exports: &'static [(&'static str, u32)] }

fn resolve(name: &str, modules: &[Module]) -> Result<u32, String> {
	let mut found = None;
	for m in modules {
		for &(n, value) in m.exports {
			if n == name {
				if found.is_some() { return Err(format!("ambiguous import {name}")); }
				found = Some(value);
			}
		}
	}
	found.ok_or_else(|| format!("unresolved import {name}"))
}

The module name is useful provenance for diagnostics even though this compact resolver returns only the value. Production should return (module identity, symbol identity, value, version).

12. A safe TLS model#

If tls_index != 0xff, the named segment is a TLS template. The loader copies its file bytes and zero tail into a separate vector for each modeled thread. VM instructions TlsLoad8 and TlsStore8 address that vector; no architecture thread-pointer register is touched.

Invariant: TLS offsets are relative to the template, not image addresses. Creating a second VM clones the initial template, proving that writes are thread-local.

This model forgets alignment, static versus dynamic TLS allocation, module IDs, dynamic thread vectors, TLS descriptors, and destructors. Those omissions are explicit because pretending a Vec is ELF TLS would teach the wrong ABI.

13. Constructors are a dependency-ordering problem#

Constructors can run arbitrary code, reenter the loader, start threads, or fail without a language level exception boundary. LinkLab therefore returns an initialization plan: resolved constructor names in dependency order. It never invokes them.

For one image, listed order is retained after validating import indices. For multiple images, create edges dependency -> dependent, topologically sort, and reject cycles or apply a documented SCC policy. Destructors reverse only successfully completed initialization, not merely the full plan.

Exercise: add module dependencies and Kahn's algorithm. Record why each node became ready so a cycle diagnostic can name an actual edge.

14. RELRO closes the mutation window#

RELRO is a range writable during relocation and read-only afterward. Sealing clears the simulated W bit for every byte in the range. The loader then verifies every byte lacks W; verification is not replaced by trusting the transition loop.

RELRO must happen after all eager writes and before publication or execution. Lazy binding conflicts with full RELRO because it wants to patch GOT slots later; choices include eager binding, a separate writable indirection area, or a carefully synchronized temporary transition. LinkLab chooses eager binding.

15. The modeled startup stack#

The VM receives typed startup data rather than an architecture byte stack:

#[derive(Debug, PartialEq)]
struct Startup {
	argc: u32,
	argv: Vec<Vec<u8>>,
	env: Vec<(Vec<u8>, Vec<u8>)>,
	aux: Vec<(u32, u32)>,
}

Policy bounds argument count and total bytes, rejects embedded NUL for easy future serialization, and supplies deterministic auxiliary values: (1, page_size), (2, entry), (0, 0) where key zero terminates the list. This is not the AMD64 initial stack ABI; it is an inspectable model.

Exact model for argv=["demo","7"]: argc=2, two byte vectors, no implicit C terminators, and auxiliary entries [(1,4096),(2,entry),(0,0)].

16. A deliberately tiny bytecode machine#

Instructions are fetched only from simulated executable memory. Data access separately checks R/W.

Each instruction consumes one fuel unit. Operand bytes must also be executable; integer operations are checked. There are no branches, host calls, syscalls, pointers, or native handoff. Small is a security property and makes traces finite.

17. Complete safe simulator: model and parser#

The next three blocks are successive parts of one file, not independently complete. Save them in order as linklab.rs. The final block supplies main and tests. It uses stable Rust and std only.

use std::collections::BTreeSet;

const R: u8 = 1; const W: u8 = 2; const X: u8 = 4;
const MAX_MEM: u32 = 65_536;

#[derive(Clone, Debug)] struct Segment { va:u32, mem:u32, data:Vec<u8>, flags:u8 }
#[derive(Clone, Debug)] struct Reloc { place:u32, kind:u8, source:u16, addend:i32 }
#[derive(Clone, Debug)] struct Image {
	entry:u32, relro:(u32,u32), tls:Option<usize>, segs:Vec<Segment>,
	relocs:Vec<Reloc>, imports:Vec<String>, ctors:Vec<u16>,
}
#[derive(Clone)] struct Module { name:&'static str, exports:&'static [(&'static str,u32)] }
#[derive(Debug, PartialEq)] struct Startup {
	argc:u32, argv:Vec<Vec<u8>>, env:Vec<(Vec<u8>,Vec<u8>)>, aux:Vec<(u32,u32)>,
}
struct Cur<'a> { b:&'a [u8], p:usize }
impl<'a> Cur<'a> {
	fn take(&mut self,n:usize)->Result<&'a [u8],String>{
		let e=self.p.checked_add(n).ok_or("file offset overflow")?;
		let s=self.b.get(self.p..e).ok_or("truncated image")?; self.p=e; Ok(s)
	}
	fn u8(&mut self)->Result<u8,String>{Ok(self.take(1)?[0])}
	fn u16(&mut self)->Result<u16,String>{let x=self.take(2)?;Ok(u16::from_le_bytes([x[0],x[1]]))}
	fn u32(&mut self)->Result<u32,String>{let x=self.take(4)?;Ok(u32::from_le_bytes(x.try_into().unwrap()))}
	fn i32(&mut self)->Result<i32,String>{Ok(self.u32()? as i32)}
}
fn parse(b:&[u8])->Result<Image,String>{
	let mut c=Cur{b,p:0}; if c.take(4)?!=b"LIMG" {return Err("bad magic".into())}
	if c.u8()?!=1{return Err("unsupported version".into())}
	let ns=c.u8()? as usize; let nr=c.u8()? as usize; let ni=c.u8()? as usize;
	if ns>16||nr>64||ni>32{return Err("table budget exceeded".into())}
	let entry=c.u32()?; let relro=(c.u32()?,c.u32()?); let ti=c.u8()?;
	let nc=c.u8()? as usize; if nc>32{return Err("constructor budget exceeded".into())}
	if c.u16()!=Ok(0){return Err("reserved field is nonzero".into())}
	let mut segs=Vec::with_capacity(ns);
	for _ in 0..ns { let va=c.u32()?;let mem=c.u32()?;let n=c.u32()?;
		let flags=c.u8()?;if n>mem{return Err("file exceeds memory".into())}
		let data=c.take(n as usize)?.to_vec();segs.push(Segment{va,mem,data,flags}); }
	let mut relocs=Vec::with_capacity(nr);
	for _ in 0..nr {relocs.push(Reloc{place:c.u32()?,kind:c.u8()?,source:c.u16()?,addend:c.i32()?})}
	let mut imports=Vec::with_capacity(ni);
	for _ in 0..ni {let n=c.u8()? as usize;if n>64{return Err("name too long".into())}
		imports.push(std::str::from_utf8(c.take(n)?).map_err(|_|"invalid UTF-8")?.to_owned());}
	let mut ctors=Vec::with_capacity(nc);for _ in 0..nc{ctors.push(c.u16()?)}
	if c.p!=b.len(){return Err("trailing bytes".into())}
	let tls=if ti==0xff{None}else{Some(ti as usize)};
	Ok(Image{entry,relro,tls,segs,relocs,imports,ctors})
}

try_into().unwrap() cannot panic here: take(4) has already returned a slice of exactly four bytes. Avoiding all unwrap is a style option; explaining the established invariant is the proof.

18. Complete safe simulator: policy, plan, map, relocate#

fn rg(s:u32,n:u32,limit:u32)->Result<std::ops::Range<usize>,String>{
	let e=s.checked_add(n).ok_or("address overflow")?;
	if e>limit{return Err("address-space budget exceeded".into())} Ok(s as usize..e as usize)
}
fn covers(s:&Segment,a:u32,n:u32,need:u8)->bool{
	match (a.checked_add(n),s.va.checked_add(s.mem)) {
		(Some(e),Some(se))=>a>=s.va&&e<=se&&(s.flags&need)==need, _=>false
	}
}
fn plan(i:&Image)->Result<u32,String>{
	if i.segs.is_empty(){return Err("no segments".into())}
	let mut iv=Vec::new();let mut end=0;
	for (k,s) in i.segs.iter().enumerate(){
		if s.mem==0||s.flags&!7!=0||s.flags&R==0||s.flags&(W|X)==(W|X){return Err("bad segment policy".into())}
		let r=rg(s.va,s.mem,MAX_MEM)?;end=end.max(r.end as u32);iv.push((r.start,r.end,k));
	}
	iv.sort();for p in iv.windows(2){if p[0].1>p[1].0{return Err("overlapping segments".into())}}
	if !i.segs.iter().any(|s|covers(s,i.entry,1,X)){return Err("entry is not executable".into())}
	let rr=rg(i.relro.0,i.relro.1,end)?;
	if !rr.is_empty()&&!i.segs.iter().any(|s|covers(s,i.relro.0,i.relro.1,R|W)){return Err("bad RELRO".into())}
	if let Some(t)=i.tls {let s=i.segs.get(t).ok_or("bad TLS segment")?;
		if s.flags&X!=0{return Err("executable TLS".into())}}
	let mut places=BTreeSet::new();
	for q in &i.relocs {if q.kind>1{return Err("unknown relocation".into())}
		if !places.insert(q.place){return Err("duplicate relocation place".into())}
		if !i.segs.iter().any(|s|covers(s,q.place,4,R|W)){return Err("relocation target not writable".into())}
		if q.kind==1&&q.source as usize>=i.imports.len(){return Err("bad symbol index".into())}}
	for &c in &i.ctors {if c as usize>=i.imports.len(){return Err("bad constructor index".into())}}
	Ok(end)
}
fn resolve(n:&str,ms:&[Module])->Result<u32,String>{
	let mut hit=None;for m in ms {for &(name,v) in m.exports {if name==n {
		if hit.is_some(){return Err(format!("ambiguous import {n}"))}hit=Some((m.name,v));}}}
	hit.map(|(_,v)|v).ok_or_else(||format!("unresolved import {n}"))
}
struct Loaded { mem:Vec<u8>, perm:Vec<u8>, entry:u32, tls:Vec<u8>, init:Vec<String> }
fn load(i:&Image,ms:&[Module])->Result<Loaded,String>{
	let end=plan(i)?;let values=i.imports.iter().map(|n|resolve(n,ms)).collect::<Result<Vec<_>,_>>()?;
	let mut mem=vec![0;end as usize];let mut perm=vec![0;end as usize];
	for s in &i.segs {let all=rg(s.va,s.mem,end)?;perm[all.clone()].fill(s.flags);
		let file=rg(s.va,s.data.len() as u32,end)?;mem[file].copy_from_slice(&s.data);}
	let mut writes=Vec::new();
	for q in &i.relocs {let src=if q.kind==0{0}else{values[q.source as usize]};
		let v=(src as i64).checked_add(q.addend as i64).ok_or("relocation overflow")?;
		let v=u32::try_from(v).map_err(|_|"relocation does not fit u32")?;writes.push((q.place,v));}
	for (p,v) in writes {let r=rg(p,4,end)?;mem[r].copy_from_slice(&v.to_le_bytes());}
	let rr=rg(i.relro.0,i.relro.1,end)?;for p in &mut perm[rr.clone()]{*p&=!W}
	if perm[rr].iter().any(|p|p&W!=0){return Err("RELRO seal failed".into())}
	let tls=match i.tls {None=>Vec::new(),Some(k)=>{let s=&i.segs[k];mem[rg(s.va,s.mem,end)?].to_vec()}};
	let init=i.ctors.iter().map(|&k|i.imports[k as usize].clone()).collect();
	Ok(Loaded{mem,perm,entry:i.entry,tls,init})
}

Although mutation has begun inside load, it is private. Every ? drops mem and perm; no caller can observe a half-relocated image. Resolution happens before allocation and mutation.

19. Complete safe simulator: executor, fixture, and tests#

This final block makes the combined file complete and runnable with rustc --edition=2021.

fn startup(entry:u32,args:&[&[u8]])->Result<Startup,String>{
	if args.len()>32||args.iter().map(|x|x.len()).sum::<usize>()>4096{return Err("argument budget".into())}
	if args.iter().any(|x|x.contains(&0)){return Err("NUL argument".into())}
	Ok(Startup{argc:args.len() as u32,argv:args.iter().map(|x|x.to_vec()).collect(),env:vec![],
		aux:vec![(1,4096),(2,entry),(0,0)]})
}
fn run(l:&mut Loaded,fuel:u64)->Result<u32,String>{
	let mut pc=l.entry;let mut r0=0u32;
	fn bytes(l:&Loaded,p:u32,n:u32,need:u8)->Result<std::ops::Range<usize>,String>{
		let r=rg(p,n,l.mem.len() as u32)?;if l.perm[r.clone()].iter().any(|x|x&need!=need){return Err("permission fault".into())}Ok(r)
	}
	fn imm(l:&Loaded,pc:&mut u32)->Result<u32,String>{let r=bytes(l,*pc,4,X)?;*pc=pc.checked_add(4).ok_or("pc overflow")?;Ok(u32::from_le_bytes(l.mem[r].try_into().unwrap()))}
	for _ in 0..fuel {let op=l.mem[bytes(l,pc,1,X)?][0];pc=pc.checked_add(1).ok_or("pc overflow")?;
		match op {1=>r0=imm(l,&mut pc)?,2=>{let a=imm(l,&mut pc)?;r0=u32::from_le_bytes(l.mem[bytes(l,a,4,R)?].try_into().unwrap())},
		3=>{let a=imm(l,&mut pc)?;let r=bytes(l,a,4,W)?;l.mem[r].copy_from_slice(&r0.to_le_bytes())},
		4=>r0=r0.checked_add(imm(l,&mut pc)?).ok_or("VM add overflow")?,
		5=>{let a=imm(l,&mut pc)? as usize;r0=*l.tls.get(a).ok_or("TLS read fault")? as u32},
		6=>{let a=imm(l,&mut pc)? as usize;*l.tls.get_mut(a).ok_or("TLS write fault")?=r0 as u8},
		0xff=>return Ok(r0),_=>return Err("unknown opcode".into())}}
	Err("fuel exhausted".into())
}
fn fixture()->Vec<u8>{
	fn u16(v:&mut Vec<u8>,x:u16){v.extend(x.to_le_bytes())}fn u32(v:&mut Vec<u8>,x:u32){v.extend(x.to_le_bytes())}
	let code=[2,0x20,0,0,0,4,1,0,0,0,0xff];let mut v=b"LIMG".to_vec();v.extend([1,3,1,1]);
	u32(&mut v,0);u32(&mut v,0x20);u32(&mut v,4);v.push(2);v.push(1);u16(&mut v,0);
	for (va,mem,data,fl) in [(0,16,code.as_slice(),R|X),(0x20,4,&[][..],R|W),(0x30,4,&[7][..],R|W)] {
		u32(&mut v,va);u32(&mut v,mem);u32(&mut v,data.len() as u32);v.push(fl);v.extend(data)}
	u32(&mut v,0x20);v.push(1);u16(&mut v,0);u32(&mut v,0);
	v.push(4);v.extend(b"seed");u16(&mut v,0);v
}
fn main()->Result<(),String>{
	let modules=[Module{name:"host",exports:&[("seed",41)]}];let image=parse(&fixture())?;
	let mut loaded=load(&image,&modules)?;let st=startup(loaded.entry,&[b"demo",b"7"])?;
	let result=run(&mut loaded,100)?;
	println!("init={:?} argc={} result={}",loaded.init,st.argc,result);Ok(())
}
#[cfg(test)] mod tests {use super::*;
	fn modules()->[Module;1]{[Module{name:"host",exports:&[("seed",41)]}]}
	#[test] fn end_to_end(){let i=parse(&fixture()).unwrap();let mut l=load(&i,&modules()).unwrap();
		assert_eq!(&l.mem[0x20..0x24],&41u32.to_le_bytes());assert_eq!(l.perm[0x20]&W,0);
		assert_eq!(l.tls,vec![7,0,0,0]);assert_eq!(l.init,vec!["seed"]);assert_eq!(run(&mut l,20),Ok(42));}
	#[test] fn truncations_fail(){let f=fixture();for n in 0..f.len(){assert!(parse(&f[..n]).is_err())}}
	#[test] fn wx_fails(){let mut i=parse(&fixture()).unwrap();i.segs[0].flags|=W;assert!(load(&i,&modules()).is_err())}
	#[test] fn unresolved_fails_atomically(){let i=parse(&fixture()).unwrap();assert!(load(&i,&[]).is_err())}
}

20. One exact load-and-run trace#

For the fixture, the states are:

The earliest invariant that would prevent a bad RELRO write is relocation-target containment during planning, not the later VM permission fault.

21. Malformed images and diagnostic precedence#

Reject at the earliest layer with enough provenance to act:

Avoid diagnostics that print all hostile bytes or secrets from paths. Include record kind/index, checked range, and violated limit. Error precedence should be deterministic so fuzz regressions are reproducible.

22. Tests, differential checks, fuzzing, and cancellation#

The simulator is a semantic oracle because it has no OS mapping races or native execution. Add:

• unit tests for checked ranges, signed addends, and every opcode;
• table tests at every budget boundary;
• truncation and one-bit mutation tests;
• properties: zero tails remain zero before relocation, no W+X byte, RELRO never writable on return;
• differential tests against a second simple parser or a format encoder/decoder round trip;
• corpus fuzzing that calls parse, plan, and load, but runs the VM only with fuel;
• deterministic cancellation checks between records and VM instructions.

rustc alone has no built-in coverage-guided fuzzer. A production fuzz harness may use external tooling, while the oracle remains dependency-free. Never weaken budgets for fuzz convenience.

Exercise: generate all images up to a tiny size and assert that load either errors or satisfies the permission invariants. Shrink failures by deleting whole records before deleting bytes.

23. Crossing from LinkLab to ELF64#

ELF changes scale and authority, not the validation discipline. Parse the ELF identification and header, then program headers. A runtime loader does not need section headers. The System V gABI defines ELF structures and dynamic tags; the processor supplement defines machine relocations and entry conventions.

Bound the program-header count and require e_phentsize to match the supported ELF64 record size. Check e_phoff + e_phnum * e_phentsize without overflow. Validate class, byte order, machine, type, and version before interpreting machine-specific values.

Do not copy Part IV's startup narrative into code. Ask implementation questions: which bytes are authoritative, which ranges can alias, and what invariant must hold before the first mapping?

24. Deriving the PT_LOAD plan and load bias#

Collect only PT_LOAD records, require p_filesz <= p_memsz, validate file ranges, permissions, alignment power-of-two rules, and congruence. Sort by p_vaddr. Let:

lo = page_down(min p_vaddr), hi = page_up(max(p_vaddr+p_memsz)).

For ET_DYN, reserve hi-lo bytes at an OS-chosen base M; load bias is B=M-lo. Runtime address is B+p_vaddr. For a fixed ET_EXEC, the requested virtual geometry constrains placement; an in-process loader risks collisions and should normally reject it.

Plan page ownership and final protections before mmap. File bytes cover [B+p_vaddr, B+p_vaddr+p_filesz). Zero the partial final file page after p_filesz, then provide anonymous zero pages through p_memsz. Never write beyond the reserved span.

25. Native mapping, protection transitions, and rollback#

A bounded Linux-only implementation can reserve one PROT_NONE span, map/copy into controlled pages, relocate while writable but non-executable, apply RELRO, then install final protections. Every successful operation enters a journal; any later failure unmaps the entire reservation.

An optional FFI boundary might look like this; it is an excerpt, not a complete loader, and must never receive unvalidated lengths or execute input:

#[cfg(target_os = "linux")]
mod os {
	use std::ffi::c_void;
	unsafe extern "C" {
		fn mmap(a:*mut c_void,n:usize,p:i32,f:i32,fd:i32,o:isize)->*mut c_void;
		fn munmap(a:*mut c_void,n:usize)->i32;
	}
	pub struct Reservation { pub base:*mut u8, pub len:usize }
	impl Drop for Reservation {
		fn drop(&mut self) {
			// SAFETY: constructor records exactly the successful mmap base and nonzero
			// length; ownership is unique; no references may outlive Reservation. Failure
			// leaks an address-space reservation but must not trigger a second unmap.
			unsafe { let _ = munmap(self.base.cast(),self.len); }
		}
	}
	// A real constructor must check page-rounded nonzero length, flag constants,
	// MAP_FAILED, pointer arithmetic, and reservation ownership before returning.
}

Caller guarantees: validated page geometry and resource limits. Implementation assumptions: Linux ABI constants and isize offsets are correct for the target. Failure consequence: rollback unmaps the reservation; no pointer is published. Additional proof is required for copying, mprotect, file descriptor lifetime, shared pages, cache coherency on relevant architectures, and concurrent access.

26. PT_DYNAMIC, RELA, and RELR#

Locate PT_DYNAMIC through program headers and bound it by its segment, stopping at DT_NULL. Dynamic pointers are image virtual addresses translated with B; they are not file offsets and must land in mapped ranges. Record tags first, reject contradictory duplicates, then validate referenced tables as complete units.

For AMD64 R_X86_64_RELATIVE in RELA, write B + r_addend to B + r_offset. Validate the target before writing. Other relocation types require symbol semantics and should be rejected by the bounded loader.

RELR entries alternate between direct addresses and bitmaps. A direct even entry sets the cursor, relocates that word, then advances one word. An odd entry uses bits above bit zero to select the next word_bits-1 words, then advances by that whole window. Bound entry count, every cursor advance, every selected target, and require writable relocation-phase memory. Scan all writes before apply.

27. Symbols, TLS, IFUNC, and explicit omissions#

Dynamic symbol lookup requires a proven symbol-table boundary. DT_SYMTAB alone gives no count; derive safe bounds from validated SysV/GNU hash metadata and mapped ranges, and bound every string by DT_STRSZ. Include binding, visibility, version, and defining object in the result. Never linear-scan until memory “looks wrong.”

PT_TLS provides template bytes, zero-fill size, and alignment. A real loader must allocate module IDs, update each thread's dynamic thread vector or static TLS layout, and honor the architecture TLS ABI. Existing threads make this a synchronization problem.

This bounded design omits symbol relocations, PLT lazy binding, copy relocations, IFUNC, TLSDESC, auditing, symbol versions, and native constructors. IFUNC executes resolver code during relocation; it is not data parsing and cannot be accepted for hostile images.

28. Initial stack, entry ABI, and why handoff is not a cast#

Linux ELF startup expects an architecture-defined stack containing argc, argv[], a null, envp[], a null, and auxiliary-vector pairs ending in AT_NULL, with required alignment. Useful entries include AT_PHDR, AT_PHENT, AT_PHNUM, AT_ENTRY, AT_PAGESZ, AT_BASE, AT_RANDOM, and AT_EXECFN; their values must refer to live storage.

The AMD64 psABI defines register and stack obligations. A Rust function call does not synthesize the process-entry ABI. Arbitrary entry code may issue syscalls, assume a dynamic linker rendezvous, inspect TLS, unwind, overwrite the host, or call _exit. Sibling threads and old mappings remain.

Therefore this textbook never casts an address to extern "C" fn() and calls it. A native handoff would require architecture assembly, complete ABI state, a security boundary such as a child process, and a proof that the image is trusted. Prefer execve for programs and a documented RPC or Wasm-like VM boundary for plugins.

29. Debuggers, locks, paths, and product architecture#

Dynamic debuggers commonly observe the runtime linker's r_debug/link_map rendezvous and a state transition around map changes. Publishing a half-built list gives debuggers dangling pointers. Prepare private metadata, hold the loader lock, announce a consistent transition, publish atomically, then announce consistency. Exact integration is implementation-specific.

The loader lock protects module graphs, symbol scopes, TLS IDs, and lifetime. Constructors must not run under a non-reentrant global lock; they can recursively load or start threads. Use states such as Planning, Relocating, Ready, Initializing, Live, and Failed, with waiters and cycle checks.

Path resolution is policy, not parsing. In privileged contexts ignore unsafe environment search paths, avoid current-directory defaults, constrain roots, use race-resistant descriptor-relative opens where available, verify identity after open, and cache by stable identity rather than spelling.

untrusted bytes -> parser worker -> immutable plan -> policy decision
trusted fd + plan -> isolated mapper process -> relocation report
                                       |
                              executor sandbox / RPC
No native pointer crosses the product API.

30. Incident playbooks, capstone, readiness, and sources#

Debug from the earliest broken invariant:

Capstone: first add a LinkLab linker backend that lowers normalized .lobj inputs into this binary .limg contract, plus randomized encode/parse round-trip tests. Decide explicitly which references the static linker resolves and which imports/relocations it leaves for the loader. Then add deterministic nonzero bias, multi-module dependency sorting, cancellation, and a relocation audit log. Separately, write an ELF64 _inspector_ that emits a mapping plan and RELA/RELR writes without mapping or running them. Differentially compare geometry with readelf -l on trusted fixtures. Finally design, but do not execute, a Linux child-process mapper with a rollback journal and seccomp/resource-limit policy.

Explicit omissions are native execution, syscalls, real TLS registers, IFUNC, lazy PLT, symbol versioning, constructors, unload, signals, credentials, vDSO, auditing, and cross-architecture cache maintenance. The safe simulator is complete for .limg; the ELF design is intentionally bounded.

Production readiness rubric:

• Correctness: checked arithmetic, hostile corpus, property/differential tests, architecture ABI review, deterministic errors.
• Security: strict budgets, W^X, full RELRO where promised, trusted scope, no ambient path search, process isolation, no hostile native calls.
• Operations: cancellation, rollback, metrics by stage, sanitized provenance, crash containment, reproducible fixtures.
• Concurrency: explicit states, lock ordering, callback boundaries, TLS/thread tests, safe unload policy (or no unload).
• Portability: target matrix, page-size tests, endianness/class rejection, kernel/libc version notes, instruction-cache policy.
• Maintenance: format versioning, compatibility tests, source revision ledger, security review for every new relocation or dynamic tag.

Authoritative source-reading path:

1. System V gABI, Chapter 4 and Chapter 5 for file structures, program loading, dynamic tags, symbols, and relocations.
2. AMD64 psABI for x86-64 relocations, TLS, calling convention, and process initialization. Record the commit read because it evolves.
3. Linux kernel fs/binfmt_elf.c for current execve ELF mapping and stack setup; implementation, not a portable ABI guarantee.
4. glibc elf/ (rtld.c, dl-load.c, dl-reloc.c, dl-tls.c, link.h) for a production dynamic loader and debugger protocol.
5. musl ldso/dynlink.c for a smaller, contrasting implementation. Compare mechanisms and policy rather than assuming identical scope.

Final mastery exercise: explain, without code, why each of these boundaries exists—parse/plan, plan/map, scan/apply, relocate/seal, publish/initialize, and image/process. Then identify the earliest invariant whose failure could explain each incident-row symptom. If the answer starts with “jump to the entry and see,” the loader is not ready.

Part VII — PE/COFF and the Windows Image Loader#

1. One journey: source, object, image, process#

Suppose app.c calls MessageBoxW. A compiler does not know where that function will live in the final process. It emits a COFF object containing machine code, symbols, sections, and relocations. lld-link or link.exe combines that object with other objects and libraries. It chooses one definition for each symbol, lays bytes out, applies object relocations, and writes a PE image. Windows maps that image, resolves imports, applies any needed base relocations, establishes process-visible runtime state, and starts execution.

source.c
  | compiler: instructions plus unresolved facts
  v
COFF .obj ---- static/archive .lib
  | lld-link or link.exe: select, resolve, lay out, relocate
  v
PE .exe ------ import .lib names DLL contracts, not DLL code
  | documented Windows image-loading behavior
  v
mapped image: pages + imports + protections + runtime metadata
  | entry protocol
  v
program code

This is the part's first mental model:

1. An object records questions that remain open.
2. A linker answers enough questions to make an image.
3. An image records the smaller set of questions that can only be answered in a process.
4. The loader answers those questions and transfers control.

The boundaries matter. A COFF relocation is a link-time request. A PE base-relocation entry is a load-time repair. An import address table (IAT) slot is a load-time binding cell. They can all change bytes, but they exist at different times and preserve different information.

We will use four evidence labels:

• Format contract means the current Microsoft PE Format documentation describes the bytes.
• Platform contract means Microsoft Learn documents application-visible Windows behavior.
• LLVM implementation means current llvm-project/lld/COFF source demonstrates what lld-link does now; it is not a Windows guarantee.
• Undocumented observation means debuggers and researchers have observed an ntdll detail. Never build a compatibility promise on it.

The PE documentation itself says it is not guaranteed complete in every respect. Treat reserved fields conservatively and APIs—not private structures—as the operating-system contract.

Prediction. If a DLL is rebuilt at a different preferred base, must every caller be relinked?

Answer. Usually no. Callers import a name or ordinal through an IAT slot. The loader writes the loaded target address into that slot. Rebuilding can still break callers if exports or ABI behavior change.

2. The COFF object envelope#

A normal COFF object begins with a 20-byte file header, followed by 40-byte section headers. Raw section data, relocation arrays, a symbol table, and a string table live at file offsets named by those records. Object files use section-relative values; they are not memory images.

COFF file header, exactly 20 bytes
+0x00  u16 Machine
+0x02  u16 NumberOfSections
+0x04  u32 TimeDateStamp
+0x08  u32 PointerToSymbolTable       -- file offset
+0x0c  u32 NumberOfSymbols            -- includes auxiliary records
+0x10  u16 SizeOfOptionalHeader       -- normally 0 in .obj
+0x12  u16 Characteristics

section header, exactly 40 bytes
+0x00  u8 Name[8]
+0x08  u32 VirtualSize/PhysicalAddress
+0x0c  u32 VirtualAddress
+0x10  u32 SizeOfRawData
+0x14  u32 PointerToRawData            -- file offset
+0x18  u32 PointerToRelocations        -- file offset
+0x1c  u32 PointerToLinenumbers
+0x20  u16 NumberOfRelocations
+0x22  u16 NumberOfLinenumbers
+0x24  u32 Characteristics

Machine identifies the instruction set. Never infer it from the host parsing the file. NumberOfSections sizes the section-header array, but multiplication and addition must be checked before slicing. A parser must not assume records occur in the order listed in prose.

Section names fit in eight bytes or use a slash followed by a decimal offset into the COFF string table. The string table begins immediately after all symbol records; its first four bytes are its total byte size, including those four bytes. A name need not be valid UTF-8. Preserve raw bytes and decode only for presentation.

Important section flags say whether content is code, initialized data, or uninitialized data; whether it is readable, writable, or executable; its alignment; and whether it contains COMDAT content. An object section named .text$mn is commonly sorted with other .text$... contributions by suffix. That convention is linker behavior, not a C-language rule.

Invariant: every range (offset, size) must fit both the integer type and the input slice before any read.

3. Symbols and auxiliary records#

The ordinary COFF symbol record is exactly 18 bytes.

COFF symbol record, exactly 18 bytes
+0x00  union Name {
         u8 ShortName[8]
         { u32 Zeroes; u32 OffsetIntoStringTable; }
       }
+0x08  u32 Value
+0x0c  i16 SectionNumber
+0x0e  u16 Type
+0x10  u8  StorageClass
+0x11  u8  NumberOfAuxSymbols

SectionNumber > 0 identifies a section. Zero is undefined, and negative special values include absolute and debug symbols. Value is interpreted according to symbol kind: for an ordinary definition it is an offset within the section. Storage class distinguishes external, static, file, weak external, and other records.

The symbol count includes auxiliary records. If a symbol says it owns two auxiliaries, iteration advances by three records. Treating every 18-byte record as a symbol creates convincing but false names.

Auxiliary records reuse the same fixed record width but have context-dependent layouts. They can hold a source filename, weak-external fallback, function data, or section-definition data. The section-definition auxiliary record carries section length, relocation and line-number counts, checksum, associated section number, and COMDAT selection.

For ordinary COFF, the section-definition auxiliary bytes relevant to COMDAT are:

section-definition auxiliary record, exactly 18 bytes
+0x00  u32 Length
+0x04  u16 NumberOfRelocations
+0x06  u16 NumberOfLinenumbers
+0x08  u32 CheckSum
+0x0c  i16 Number                    -- associated section, low 16
+0x0e  u8  Selection
+0x0f  u8  Reserved
+0x10  i16 HighNumber                -- associated section, high 16

Combine Number and HighNumber for extended section indices where applicable. The owning symbol and storage class tell you that these bytes have this meaning.

Counterexample: “There are 300 symbols because NumberOfSymbols == 300.” There are 300 records. Some records may be auxiliaries and not independently named symbols.

4. Object relocations are delayed calculations#

A COFF relocation says: at an offset in this section, compute a machine-specific expression involving this symbol. Each relocation is exactly 10 bytes.

COFF relocation, exactly 10 bytes
+0x00  u32 VirtualAddress       -- offset within section in an object
+0x04  u32 SymbolTableIndex
+0x08  u16 Type                 -- interpretation depends on Machine

For AMD64, IMAGE_REL_AMD64_ADDR64 writes an absolute 64-bit address. IMAGE_REL_AMD64_REL32 and its numbered variants write a signed PC-relative displacement with specified bias. ARM64 has different relocation types for branches, page bases, and low 12-bit offsets. Never apply a numeric relocation code without first dispatching on Machine.

Let S be the chosen symbol address, A the addend already encoded at the relocation site, and P the address of the relocation field. A typical relative relocation resembles S + A - P, but the exact width, bias, overflow rule, and instruction-bit insertion come from the architecture's relocation definition.

The linker must check that:

• the relocation site fits within its input section;
• the symbol index names a primary symbol record, not an arbitrary auxiliary;
• the selected definition belongs to a live contribution;
• the result fits the field or instruction encoding;
• relocation pairs and architecture-specific sequences remain valid.

Prediction. Can a linker apply an AMD64 REL32 by blindly adding a value to four bytes?

Answer. No. It must include the place and bias, preserve the encoded addend, check signed range, and report an overflow rather than truncate.

5. BigObj: when ordinary COFF is too small#

The ordinary header's 16-bit section count is insufficient for generated programs with many COMDAT sections. Microsoft BigObj extends object representation. It uses a distinctive anonymous-object header with signatures, a class ID, a 32-bit section count, and a 32-bit symbol count. Its symbol records use a 32-bit section number and are 20 bytes rather than 18.

Do not detect BigObj from a filename or from “too many bytes.” Validate the documented signature and class ID, then switch the complete parsing schema. Mixing an ordinary symbol stride with a BigObj table shifts every later read.

The design lesson is broader: an extension that widens one index often changes dependent record layouts. Normalize both formats into one internal model only after validation:

InputSymbol {
    raw_name: bytes,
    section: Undefined | Absolute | Debug | Section(u32),
    value: u32,
    storage_class: u8,
    aux_range: checked record range,
}

Keep original record indices too. Relocations refer to physical symbol-table indices, including gaps occupied by auxiliary records. A dense “logical symbol number” is not interchangeable.

6. A robust parser begins with ranges#

Parsing is not casting a byte pointer to a C structure. C structures can have padding, host endianness, alignment requirements, and accidental out-of-bounds reads. PE/COFF is little-endian. Read fields explicitly.

Use three layers:

1. Bytes: checked little-endian reads and checked ranges.
2. Records: headers whose internal offsets are not yet trusted.
3. Meaning: cross-reference validation, architecture rules, and policy limits.

This stable Rust excerpt is a reusable checked RVA-to-file conversion. It deliberately returns None for a zero-filled virtual tail. It is a library excerpt, not a complete program.

#[derive(Clone, Copy, Debug)]
struct Section {
	virtual_address: u32,
	virtual_size: u32,
	raw_offset: u32,
	raw_size: u32,
}

fn rva_to_file(rva: u32, len: usize, sections: &[Section]) -> Option<usize> {
	for s in sections {
		let memory_size = s.virtual_size.max(s.raw_size);
		let delta = rva.checked_sub(s.virtual_address)?;
		if delta >= memory_size {
			continue;
		}
		// A mapped address in the zero-filled tail has no source file byte.
		if delta >= s.raw_size {
			return None;
		}
		let off = s.raw_offset.checked_add(delta)? as usize;
		return (off < len).then_some(off);
	}
	None
}

That first checked_sub needs care: ? would return too early for an RVA below the first examined section. The robust version must continue instead:

fn checked_rva_to_file(rva: u32, len: usize, sections: &[Section]) -> Option<usize> {
	for s in sections {
		let Some(delta) = rva.checked_sub(s.virtual_address) else {
			continue;
		};
		let memory_size = s.virtual_size.max(s.raw_size);
		if delta >= memory_size {
			continue;
		}
		if delta >= s.raw_size {
			return None;
		}
		let off = s.raw_offset.checked_add(delta)? as usize;
		return (off < len).then_some(off);
	}
	None
}

The corrected function is the one to use. Keeping the failed version teaches an important review habit: arithmetic can be memory-safe yet logically wrong.

Set limits before allocation: section count, symbol count, archive members, string length, relocation count, import descriptors, and recursion depth. Reject overlapping ranges when your consumer requires uniqueness; otherwise represent overlaps explicitly instead of silently choosing one.

7. Archives and the two meanings of .lib#

A Windows .lib can be a static archive of COFF objects or an import library that contains linker contracts for a DLL. Both use archive machinery. The extension alone does not tell you which meaning applies.

The archive starts with !<arch>\n. Each member has a fixed-size header and an even-byte alignment rule. Special linker members index symbols to member offsets. Longnames support member names that do not fit the short header field.

Static archive extraction is demand-driven:

1. Begin with unresolved symbols from explicitly included objects.
2. Consult the archive index.
3. Extract a member that can satisfy an unresolved symbol.
4. Add that member's new undefined symbols.
5. Repeat to a fixed point.

This policy prevents every helper in a library from entering the image. It also explains archive-order surprises in some linker families. Windows linkers generally build an index-based view, but /WHOLEARCHIVE deliberately changes extraction policy.

Import libraries usually contain small synthetic object members. They communicate the DLL name, imported symbol, name/ordinal mode, machine, and code/data kind. The final executable does not copy the DLL's implementation. The linker synthesizes import tables and perhaps a callable thunk.

Workshop: run llvm-ar t x.lib, llvm-readobj --coff-imports app.exe, and dumpbin /linkermember x.lib. Predict whether each member contributes code, metadata, or an import contract before reading the output.

8. .drectve: command-line fragments inside objects#

The .drectve section contains linker options encoded as text. Compilers use it for such directives as default libraries, exports, manifest dependencies, alternatename aliases, and mismatch checks. It is not mapped as ordinary program data.

This means an object is partly an input program for the linker. A secure build service must not treat uploaded .obj files as passive byte arrays. Options can select libraries, alter exports, or affect output policy.

LLVM implementation: lld/COFF/Driver.cpp, DriverUtils.cpp, and object-file parsing code tokenize and apply directives; exact ownership moves over time. Read the current revision before citing a function name.

Policy guidance:

• report directives in inspection tools;
• allowlist accepted options for untrusted builds;
• preserve command-line precedence rules;
• bound directive size and token count;
• diagnose the object member that introduced an option.

/DEFAULTLIB:foo is a request to search a library; /NODEFAULTLIB changes that policy. /FAILIFMISMATCH:key=value lets separately compiled units insist on compatible settings. Ignoring .drectve can produce silent ABI mismatches; obeying every directive from untrusted input can violate build policy.

9. COMDAT selection and associative families#

COMDAT lets multiple object files provide candidates for a contribution. The linker groups candidates by key and applies a selection mode from the section-definition auxiliary record.

Do not flatten these into “weak symbols.” Selection compares section contributions and can include content, relocation structure, size, and family relationships.

Associative COMDAT is crucial. An unwind record, metadata contribution, or static-initialization fragment can declare that it follows a parent code COMDAT. If the parent loses selection, its child must not survive as an orphan. Build a graph from child to parent, validate indices, detect cycles, then propagate liveness.

This stable Rust excerpt decodes the ordinary 18-byte section-definition auxiliary record. It is intentionally a partial parser: its caller must first establish that the owning symbol is a section definition.

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum ComdatSelection {
	None,
	NoDuplicates,
	Any,
	SameSize,
	ExactMatch,
	Associative,
	Largest,
	Newest,
}

fn parse_section_aux(bytes: &[u8]) -> Option<(u32, u32, ComdatSelection)> {
	let b: &[u8; 18] = bytes.try_into().ok()?;
	let length = u32::from_le_bytes(b[0..4].try_into().ok()?);
	let low = u16::from_le_bytes(b[12..14].try_into().ok()?) as u32;
	let high = u16::from_le_bytes(b[16..18].try_into().ok()?) as u32;
	let associated_section = low | (high << 16);
	let selection = match b[14] {
		0 => ComdatSelection::None,
		1 => ComdatSelection::NoDuplicates,
		2 => ComdatSelection::Any,
		3 => ComdatSelection::SameSize,
		4 => ComdatSelection::ExactMatch,
		5 => ComdatSelection::Associative,
		6 => ComdatSelection::Largest,
		7 => ComdatSelection::Newest,
		_ => return None,
	};
	Some((length, associated_section, selection))
}

10. Dead stripping and identical COMDAT folding#

COMDAT selection removes duplicate candidates with one key. Garbage collection removes contributions not reachable from roots. Identical COMDAT folding (ICF) can merge distinct live contributions whose observable machine-level behavior is considered equivalent.

Typical roots include the entry point, exports, explicitly included symbols, runtime tables, and contributions retained by policy. Relocations form graph edges. Associative COMDAT adds liveness edges. The linker marks reachable nodes and discards the rest.

ICF then compares eligible live sections, often using iterative hashes of bytes and relocation targets. Two functions with equal bytes but references to different final targets are not necessarily equal. Address-taking is the hard boundary: folding changes pointer identity.

LLVM implementation: current lld COFF behavior is chiefly visible in lld/COFF/ICF.cpp, MarkLive.cpp, Chunks.cpp, and Writer.cpp. The implementation chooses eligibility and equivalence rules. Those rules are not the PE format.

Use /OPT:REF and /OPT:ICF deliberately. Debug builds often value stable addresses and simple stepping over size. Release builds may accept folding, but programs must not use distinct function addresses as semantic IDs unless the toolchain promises not to fold them.

Counterexample: two wrappers have identical current instructions. One is used as a registration token by address. Folding preserves call results but destroys the program's identity assumption.

11. Import libraries, tables, and callable thunks#

An import library turns a source-level external reference into PE import metadata. The final image normally contains:

• an import directory table: one descriptor per DLL plus a zero terminator;
• an import lookup table, also called the import name table (INT/ILT);
• hint/name records or ordinal flags;
• an IAT, writable while binding and used by executing code;
• optional architecture-specific thunks.

An import descriptor is exactly 20 bytes:

+0x00 u32 ImportLookupTableRVA / OriginalFirstThunk
+0x04 u32 TimeDateStamp
+0x08 u32 ForwarderChain
+0x0c u32 NameRVA
+0x10 u32 ImportAddressTableRVA / FirstThunk

Each lookup entry is 4 bytes in PE32 and 8 bytes in PE32+. Its high bit selects ordinal import; otherwise the remaining value is an RVA to a two-byte hint followed by a NUL-terminated ASCII name. Arrays end with a zero entry.

The linker may expose __imp_MessageBoxW as the address of the IAT cell. A callable MessageBoxW symbol may be a thunk that jumps indirectly through that cell. Optimized code can call through __imp_... directly and skip one jump.

Exact logical call trace on x64; addresses are illustrative

COFF object before link
  call displacement at .text+0x21
  IMAGE_REL_AMD64_REL32 -> symbol MessageBoxW
        |
        | linker resolves symbol to local import thunk
        v
PE .text at RVA 0x1100
  E8 DB 00 00 00       call RVA 0x11e0
        |
        v
thunk at RVA 0x11e0, bytes FF 25 1A 2E 00 00
  jmp qword ptr [RIP + 0x2e1a] -> IAT RVA 0x4000
        |
        | loader previously wrote 0x00007ffa`12345670
        v
IAT cell at RVA 0x4000: 70 56 34 12 FA 7F 00 00
        |
        v
USER32!MessageBoxW at VA 0x00007ffa`12345670

The first relocation is gone after linking: it created the call displacement. The IAT remains because process-specific binding happens later.

12. Iterating imports without trusting terminators#

Terminator-based tables still need directory-size and file-size bounds. A malicious image can omit the zero descriptor, point a name into a zero-filled tail, or create an unterminated string.

This stable Rust excerpt yields validated import descriptors. It is a partial library excerpt and uses the Section and checked_rva_to_file definitions from Chapter 6.

#[derive(Debug)]
struct ImportDescriptor {
	lookup_rva: u32,
	name_rva: u32,
	iat_rva: u32,
}

fn imports(
	file: &[u8],
	directory_rva: u32,
	directory_size: u32,
	sections: &[Section],
) -> Option<Vec<ImportDescriptor>> {
	let start = checked_rva_to_file(directory_rva, file.len(), sections)?;
	let size = usize::try_from(directory_size).ok()?;
	let end = start.checked_add(size)?.min(file.len());
	let mut pos = start;
	let mut out = Vec::new();
	while pos.checked_add(20)? <= end {
		let r = &file[pos..pos + 20];
		let u32_at = |n| u32::from_le_bytes(r[n..n + 4].try_into().unwrap());
		let lookup_rva = u32_at(0);
		let name_rva = u32_at(12);
		let iat_rva = u32_at(16);
		if lookup_rva == 0 && name_rva == 0 && iat_rva == 0 {
			return Some(out);
		}
		out.push(ImportDescriptor { lookup_rva, name_rva, iat_rva });
		pos += 20;
	}
	None
}

The closure's unwrap is safe only because the outer checked 20-byte slice and constant subranges establish its precondition. In production, a generic reader can remove even this locally proven panic.

Validate each nested table separately. Directory size bounds descriptors, not necessarily all referenced hint/name strings. Bound each string by its containing file-backed region and a policy maximum.

13. Exports: names, ordinals, and forwarders#

The export directory describes one export address table (EAT), a sorted export-name-pointer table, and an ordinal table. The public ordinal is OrdinalBase + EAT index. The name ordinal table contains unbiased EAT indices, not public ordinals.

To resolve a name:

1. Search the lexically sorted name-pointer table.
2. Read its corresponding 16-bit ordinal index.
3. Bounds-check that index against the EAT count.
4. Read the function RVA from the EAT.

Not every EAT slot has a name. Ordinal-only exports are valid. A zero EAT entry is unused. Names are case-sensitive for GetProcAddress.

If an EAT value points inside the export directory's RVA range, it is a forwarder string rather than code. A forwarder resembles OTHERDLL.Function or OTHERDLL.#27. It asks the loader to continue resolution in another module.

This stable Rust excerpt classifies an already-read EAT value, including the forwarder case. Its caller must use checked arithmetic to validate the export-directory range and bound the C string.

#[derive(Debug, PartialEq, Eq)]
enum ExportTarget<'a> {
	AddressRva(u32),
	Forwarder(&'a [u8]),
}

fn bounded_c_string(bytes: &[u8], start: usize, end: usize) -> Option<&[u8]> {
	let tail = bytes.get(start..end.min(bytes.len()))?;
	let nul = tail.iter().position(|&b| b == 0)?;
	Some(&tail[..nul])
}

fn classify_export<'a>(
	file: &'a [u8],
	target_rva: u32,
	export_rva: u32,
	export_size: u32,
	sections: &[Section],
) -> Option<ExportTarget<'a>> {
	let export_end = export_rva.checked_add(export_size)?;
	if (export_rva..export_end).contains(&target_rva) {
		let start = checked_rva_to_file(target_rva, file.len(), sections)?;
		let dir_end = checked_rva_to_file(export_end - 1, file.len(), sections)?
			.checked_add(1)?;
		return Some(ExportTarget::Forwarder(bounded_c_string(file, start, dir_end)?));
	}
	Some(ExportTarget::AddressRva(target_rva))
}

Do not call a forwarded EAT value as an address. Do not assume a DLL suffix is present in the forwarder. Resolution policy belongs to the platform, not a hand-written PE parser.

14. Delay loading is a helper protocol#

Normal imports are resolved as part of loading. Delay imports defer work until a call first needs a symbol. The image contains delay-import descriptors, name/ordinal data, IAT-related tables, and compiler-generated stubs. A runtime helper such as __delayLoadHelper2 performs the first resolution and patches the delay IAT.

Conceptual protocol:

caller -> delay thunk
       -> check delay-IAT state
       -> helper receives descriptor and slot identity
       -> load module if needed
       -> resolve name or ordinal
       -> run documented hook/failure policy supplied by runtime
       -> write target into delay IAT
       -> tail-call target
later calls -> patched slot -> target

The PE format documents the delay-import directory fields. The helper ABI and hooks are toolchain runtime contracts. They are not the same as a kernel loader promise. Different toolchains may synthesize different stubs while producing compatible directory data.

Delay loading moves failure later. Startup can succeed and the first feature use can fail. Test that path, including missing DLL, missing symbol, architecture mismatch, and recursive loading. A plugin system should normally perform explicit LoadLibraryExW and GetProcAddress at a controlled boundary instead of hiding errors behind arbitrary first calls.

15. The PE image envelope#

A PE image retains a DOS header and stub, then points to the NT headers through e_lfanew at DOS-header offset 0x3c.

file offset 0x00: IMAGE_DOS_HEADER, e_magic = "MZ"
                 ...
file offset 0x3c: i32 e_lfanew ----------------------+
                                                        |
file offset e_lfanew: 4 bytes "PE\0\0" <---------------+
                     20-byte COFF file header
                     Optional header (size from COFF header)
                     NumberOfSections * 40-byte section headers
                     section raw data and other file data

“Optional header” is historical terminology. It is required for an executable image. Never assume its size from machine type alone: first bounds-check SizeOfOptionalHeader, then require enough bytes for fields you consume.

Important COFF image-header fields include machine, section count, optional-header size, and characteristics such as executable image, large-address-aware, and DLL. The timestamp field is not a trustworthy security timestamp or unique build identity.

The optional header contains linker versions, code/data sizes, entry-point RVA, image base, section and file alignment, OS/image/subsystem versions, image/header sizes, subsystem, DLL characteristics, stack/heap reservations, and data directories.

Parser order: validate MZ; read checked e_lfanew; validate PE\0\0; parse the 20-byte header; bound the optional header; inspect its magic; parse only available directories; then bound the section-header array.

16. PE32, PE32+, and architecture are separate questions#

Optional-header magic 0x10b means PE32. 0x20b means PE32+. PE32+ removes BaseOfData, widens image base and stack/heap size fields, and uses 64-bit import thunk entries. RVAs and section-header fields remain 32-bit.

Do not equate “PE32+” with “AMD64.” ARM64 images also use PE32+. Machine selects architecture; optional-header magic selects layout class.

The NumberOfRvaAndSizes field bounds available directory entries, but the optional-header byte size is the stronger physical bound. Use the minimum of declared directory count, format maximum relevant to your tool, and bytes actually present.

Cross-check combinations. Reject or clearly flag an unsupported machine/magic pair. A generic parser may preserve unusual combinations; an execution tool should enforce platform requirements.

17. RVA, VA, file offset, and zero fill#

A relative virtual address is measured from image base after mapping. A virtual address is actual_image_base + RVA. A file offset identifies a byte in the disk file. These are three different coordinate systems.

For a normal file-backed section byte:

delta       = RVA - section.VirtualAddress
file_offset = section.PointerToRawData + delta
valid only when delta < section.SizeOfRawData

Memory extent often uses VirtualSize, while raw extent uses SizeOfRawData. If virtual size exceeds raw size, the mapped tail is zero-filled. An RVA in that tail is valid memory after mapping but has no RVA-to-file mapping. Returning a nearby disk offset invents bytes.

Headers can also be mapped. A parser may map an RVA below SizeOfHeaders directly only after validating the header range and applying a clearly stated policy. Overlapping sections, out-of-order sections, and mismatched extents require deterministic handling.

file:   raw section bytes [AAAA BBBB]         no bytes here
                         |          |
memory: RVA 0x2000       [AAAA BBBB 0000 0000]
                         ^ file-backed ^ zero-filled virtual tail

An RVA is not a pointer until combined with a validated mapped-image base. A VA stored in an image may need base relocation. Most directory fields are RVAs—but one famous exception appears next.

18. Sections, alignment, and data directories#

SectionAlignment controls sections in memory. FileAlignment controls raw section positions in the file. SizeOfImage covers the mapped image rounded according to section alignment. SizeOfHeaders covers the headers as represented for mapping.

Section characteristics guide final page protections: code, initialized/uninitialized data, discardability, sharing, read, write, and execute. These are image intent, not a license for a parser to access memory. Linkers merge input contributions into output sections and derive output characteristics.

The optional header's data-directory array gives (VirtualAddress, Size) pairs for exports, imports, resources, exceptions, certificates, base relocations, debug, TLS, load configuration, delay imports, and more. A zero pair means absent. Validate each directory with its own record grammar; do not assume its size is a multiple of one guessed structure.

Certificate-table exception: the security/certificate directory's “VirtualAddress” field is a file offset, not an RVA. Certificates are not mapped as an image section. Its entries are aligned as documented and live outside ordinary image hashing regions. Passing that field through rva_to_file is wrong.

This exception reveals why typed coordinates are valuable:

struct Rva(u32);
struct FileOffset(u32);
struct Va(u64);

Making conversion explicit prevents a whole class of parser and security defects.

19. Base relocations are load-time repairs#

The linker chooses a preferred ImageBase. If the image maps elsewhere, the loader computes delta = actual_base - preferred_base. The base relocation directory identifies fields that need that delta.

Each block begins with an 8-byte header followed by 16-bit entries:

base relocation block
+0x00 u32 PageRVA
+0x04 u32 SizeOfBlock
+0x08 u16 entries[]

entry bits 15..12: type
entry bits 11..0 : offset within 4 KiB page
patch RVA = PageRVA + offset

ABSOLUTE entries are padding and require no patch. HIGHLOW commonly patches a 32-bit value. DIR64 patches a 64-bit value. Other machine-specific types exist. Dispatch on architecture and type, check patch width, and reject arithmetic overflow.

This stable Rust excerpt iterates block entries without applying them. It is a partial library excerpt.

fn base_relocations(bytes: &[u8]) -> Option<Vec<(u32, u8)>> {
	let mut pos = 0usize;
	let mut out = Vec::new();
	while pos < bytes.len() {
		let header = bytes.get(pos..pos.checked_add(8)?)?;
		let page = u32::from_le_bytes(header[0..4].try_into().ok()?);
		let size = u32::from_le_bytes(header[4..8].try_into().ok()?) as usize;
		if size < 8 || size % 2 != 0 {
			return None;
		}
		let end = pos.checked_add(size)?;
		let block = bytes.get(pos + 8..end)?;
		for pair in block.chunks_exact(2) {
			let raw = u16::from_le_bytes(pair.try_into().ok()?);
			let kind = (raw >> 12) as u8;
			let offset = u32::from(raw & 0x0fff);
			let patch_rva = page.checked_add(offset)?;
			out.push((patch_rva, kind));
		}
		pos = end;
	}
	Some(out)
}

Object relocations disappear as the linker computes section layout. Base relocations survive in the PE because actual load address is not yet known.

20. ASLR flags, entropy, and relocatability#

Address-space layout randomization makes image addresses less predictable. IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE announces that the image can be relocated at load time. A useful relocation table must exist for addresses that require repair. Stripping required relocations can force preferred-base loading or make the image unusable under collision.

HIGH_ENTROPY_VA requests support for high-entropy 64-bit address placement when platform and image conditions permit. It is not proof that every run has a particular entropy count. LARGE_ADDRESS_AWARE and PE32/PE32+ choices also affect address-space assumptions.

Security is a conjunction:

• toolchain emits position-tolerant code where appropriate;
• linker records every required absolute fixup;
• image flags request supported mitigations;
• operating system policy enables them;
• no information disclosure defeats unpredictability.

Workshop: compare llvm-readobj --file-headers --coff-basereloc a.exe for builds with and without /DYNAMICBASE. Search disassembly for absolute addresses. Explain whether each absolute field has a matching base-relocation entry.

Do not “improve security” by toggling a header bit after linking. Flags often assert that supporting metadata and code-generation invariants are true.

21. The documented loader sequence—and its limit#

At a documented, application-relevant level, loading an executable or DLL involves these dependencies:

1. Validate enough image metadata to map it as an image.
2. Reserve/map image regions and establish zero-filled storage.
3. Account for a nonpreferred base and process relocation metadata.
4. Load dependencies under the applicable search policy.
5. Resolve imported names/ordinals and populate address cells.
6. Establish runtime metadata and effective page protections.
7. Perform TLS and DLL initialization according to platform rules.
8. Transfer control to the executable entry point or return a module handle.

This list is a dependency model, not a guaranteed instruction-by-instruction order. Microsoft documents format fields, API effects, TLS callbacks, and DllMain restrictions. It does not promise every internal phase boundary.

Undocumented observation: ntdll contains loader routines and maintains structures often discussed as the PEB loader lists and LDR_DATA_TABLE_ENTRY records. Their layouts, lock implementation, hash tables, dependency graph, and private routine names can change. Reading them in a debugger can explain one Windows build; shipping code must use supported APIs such as LoadLibraryExW, GetModuleHandleW, and GetProcAddress.

Find the earliest broken invariant. “Entry point crashed” may begin with a malformed relocation, wrong import ABI, missing unwind metadata, or unsafe initialization much earlier.

22. TLS directory and callbacks#

The TLS directory supports static thread-local storage. It identifies the template's start/end VAs, an address of the TLS index variable, an address of a null-terminated callback-pointer array, zero-fill size, and characteristics. These address fields are VAs, not ordinary RVAs, and therefore participate in base relocation where needed.

The loader creates per-thread storage from the template plus zero fill. TLS callbacks receive notifications related to process/thread attach and detach under documented rules. They execute in a sensitive initialization context similar to DllMain; keep them minimal.

TLS template bytes in image + declared zero fill
                  |
                  v
per-thread TLS block -- index mechanism --> compiled TLS access

callback array: VA(cb1), VA(cb2), ..., 0

Do not parse the callback array until exhaustion. Convert its VA to an RVA only after validating the preferred/actual image-base model, bound every pointer-sized read to mapped image data, and set a callback-count limit.

TLS creates lifetime traps for plugins. A thread can retain plugin-owned TLS state while the host attempts FreeLibrary. The product contract should require all plugin-created threads to stop and all calls to return before unload. Often the safest policy is never to unload plugins during normal process life.

23. x64 and ARM64 exception and unwind data#

On Windows x64, table-based unwind metadata describes nonleaf functions. The exception directory points to sorted runtime-function records containing begin RVA, end RVA, and unwind-information RVA. UNWIND_INFO encodes prologue operations, frame-register use, and optional handler or chained information. The calling convention and unwind encoding constrain legal prologues.

ARM64 also uses table-driven exception metadata, but record encodings and packed forms differ. Do not apply x64 structures to ARM64 because both are PE32+. Consult the current Microsoft x64 and ARM64 exception-handling ABI documentation.

Unwind data supports more than language exceptions:

• stack unwinding and cleanup;
• debugger stack walking;
• exception dispatch;
• profiling and diagnostics.

Hand-written assembly must either stay within documented leaf-function conditions or provide correct unwind directives/metadata. A function can return correctly in normal tests yet make exception dispatch or crash stacks fail.

Associative COMDAT commonly keeps unwind contributions with their code. ICF and dead stripping must preserve runtime-function consistency. The linker sorts and emits final exception tables only after code layout is known.

Debugging workshop: break inside a crashing x64 function and compare llvm-readobj --unwind image.exe with disassembly. Find the first prologue operation not represented by metadata. Do not start by blaming the debugger's final bad frame.

24. Resources, manifests, and side-by-side policy#

The resource directory is a tree of type, name, and language entries leading to data entries. Keys may be integer IDs or UTF-16 names. Offsets inside the resource structure use documented high-bit tags and directory-relative coordinates. Cycles and extreme depth are malicious possibilities even though valid trees are acyclic.

Common resources include icons, version information, dialogs, strings, and manifests. A manifest can declare requested execution level, compatibility, dependencies, and side-by-side assembly identity. The resource compiler and linker package these records into .rsrc.

Side-by-side (SxS) activation and assembly resolution are platform policy, not merely resource parsing. Microsoft Learn's manifests and isolated applications documentation is the authority. Do not reproduce an observed WinSxS path algorithm in application code.

Parser checklist:

• bound recursion depth and total nodes;
• validate every relative offset against the resource directory region;
• decode UTF-16 with explicit invalid-data policy;
• detect cycles by offset, not only depth;
• bound data-entry RVA and size independently;
• treat XML as untrusted input.

Embedding a manifest is not equivalent to signing the image. A requested-execution-level declaration is not authorization, and resource bytes can be inspected without executing a DLL by using suitable data-file loading flags.

25. Load configuration and mitigation metadata#

The load-configuration directory has grown over Windows and toolchain generations. It can identify security cookies, SEH tables on supported 32-bit targets, Control Flow Guard (CFG) tables and flags, code-integrity data, dynamic-relocation metadata, address-taken IAT entries, and newer mitigation-related fields.

Parse it by Size, not by one compile-time C structure. Read a field only if the declared size and enclosing directory/file ranges cover that field. Accept older prefixes. Treat unknown trailing bytes as unknown, not corrupt by default.

Boundaries:

• CFG: compiler instruments indirect calls; linker builds target metadata; image flags describe support; Windows enforces according to platform policy. A flag alone is insufficient.
• CET compatibility: hardware, OS policy, compiler code generation, and image metadata interact. PE fields do not themselves prove shadow-stack or indirect-branch safety.
• SafeSEH: primarily an x86 mechanism using a table of valid exception handlers. It is not the x64/ARM64 unwind model.
• Security cookie: compiler/runtime protocol, not an automatic parser guarantee.

LLVM implementation: inspect lld/COFF/LoadConfig.cpp, Writer.cpp, and architecture-specific thunk/relocation code for current lld synthesis and validation. Verify exact paths in the revision you study.

Security review question: for every advertised mitigation, which producer generated its metadata, which linker preserved/combined it, and which platform component consumes it?

26. Authenticode: hashing is not trust#

The attribute certificate table contains WIN_CERTIFICATE entries. As Chapter 18 emphasized, its directory location is a file offset. Each entry has length, revision, certificate type, and certificate bytes, with documented alignment.

Authenticode hashing excludes or specially handles fields whose values change during signing, including the checksum and certificate-directory entry, and excludes the certificate table itself according to the PE/COFF hashing procedure. Do not invent a “hash every byte except the last blob” algorithm. Overlay data and malformed ranges require exact documented treatment.

A valid cryptographic signature answers only part of the trust question. Windows trust evaluation also considers certificate chain, policy, purpose, revocation behavior, timestamps, and current verification settings. Use supported trust APIs such as WinVerifyTrust; do not equate successful PKCS#7 parsing with trusted software.

PE bytes -- documented Authenticode digest algorithm --> digest
digest + signed attributes -- signature verification --> signer evidence
signer evidence + chain + timestamp + policy + revocation --> trust decision

Security checklist:

• validate certificate file offsets and lengths with checked arithmetic;
• reject overlapping or nonprogressing entries under your policy;
• use Microsoft's documented digest algorithm;
• make revocation/network policy explicit;
• report “unsigned,” “invalid signature,” and “untrusted signer” separately;
• bind plugin approval to a stable policy and expected publisher/product identity;
• revalidate the same opened file that will be loaded to reduce path races.

27. Explicit dynamic loading APIs and safe lifetime#

LoadLibraryExW maps a module or supports special data/image-resource modes according to flags. On success it returns a module handle. GetProcAddress resolves a case-sensitive ASCII export name or an ordinal encoded as documented. FreeLibrary decrements the module reference count for ordinary executable loads and can unmap when the count reaches zero.

Prefer the wide API and an absolute, canonicalized path. For dependencies, use an explicit safe policy such as appropriate LOAD_LIBRARY_SEARCH_* flags, SetDefaultDllDirectories, and scoped AddDllDirectory use. Exact choices depend on supported Windows versions and product layout.

Safe plugin lifetime rules:

1. Verify path and trust policy before loading; avoid a check/use race by controlling directories and file replacement.
2. Load, then resolve every required entry point.
3. Check an ABI version and structure sizes before exchanging complex values.
4. Keep the HMODULE owned by the plugin object.
5. Never let a function pointer, callback, vtable, panic payload, allocator-owned object, TLS object, or worker thread outlive that owner.
6. Stop new calls, unregister callbacks, stop/join threads, destroy plugin objects through plugin-provided destructors, then unload.
7. Serialize unload against all calls. A reference count in Windows cannot know about your raw pointers.
8. Consider process isolation or no-unload policy for untrusted or complex plugins.

Host state: Loaded -> Quiescing -> Unloaded
               |           |
new call allowed|           +-- no callbacks/threads/objects/pointers remain
               v
          active-call guard
               |
               +-- holds module ownership until call returns

GetProcAddress does not verify a C signature. A successful lookup followed by a wrong calling convention or structure layout is still undefined behavior.

28. Loader lock, DllMain, and deadlock#

Windows serializes important DLL initialization activity. Microsoft documents severe restrictions for DllMain: perform only simple initialization, avoid LoadLibrary/LoadLibraryEx, avoid synchronization patterns that can wait on code needing the loader lock, and defer substantial work.

Thread A                              Thread B
--------                              --------
holds loader lock
enters Plugin!DllMain
tries to lock host_mutex  ----------> host_mutex already held
                                      calls LoadLibrary/GetModuleHandle path
                                      waits for loader lock

wait-for cycle:
Thread A -> host_mutex -> Thread B -> loader lock -> Thread A

This is the classic lock-order inversion. Another trap is creating a thread in DllMain and waiting for it: thread startup/notification can need serialized loader activity before the new thread can run the code that releases the wait.

Safe pattern:

• DllMain: store minimal state, initialize trivial lock-free data if permitted, return promptly;
• exported plugin_init: perform fallible work after LoadLibraryExW returns;
• exported plugin_shutdown: quiesce while code is still loaded;
• DLL_PROCESS_DETACH: no dependence on other DLLs during process termination.

Undocumented boundary: developers commonly call the serialization mechanism “the loader lock” and debuggers expose internal ownership. Its private implementation is not a stable API. The documented restrictions and API behavior are sufficient to design correctly.

29. Search order, KnownDLLs, API sets, and security#

DLL resolution is policy-rich. It depends on whether a full path was supplied, packaged versus unpackaged application model, redirection, API-set resolution, SxS activation, already-loaded modules, KnownDLLs, safe DLL search mode, LOAD_LIBRARY_SEARCH_* flags, application/system/user directories, and product configuration. Microsoft Learn's Dynamic-Link Library Search Order page is the current authority; do not freeze its long ordered list into folklore.

KnownDLLs are operating-system-managed module identities considered during documented search behavior. API-set names are contract names that Windows maps to host implementations. The host can vary by Windows version. Do not assume an API-set contract is a disk filename beside the application.

Safer product policy:

• load the top-level plugin by absolute path;
• use LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR for its colocated private dependencies when appropriate;
• include LOAD_LIBRARY_SEARCH_SYSTEM32 for system dependencies as needed;
• establish default directories early;
• never use the current working directory as a trust boundary;
• do not use SearchPath to predict a later LoadLibraryEx result;
• protect plugin directories from untrusted writes;
• log the final module path and signature identity.

Counterexample: the host verifies C:\Program Files\Host\plugins\x.dll, then calls LoadLibraryW(L"x.dll"). Search policy can choose a different file. Verification and loading did not name the same object.

30. PDB, CodeView identity, incremental linking, and hotpatching#

The PE debug directory can point to CodeView data. The common RSDS record carries the RSDS signature, a PDB GUID, an age, and a NUL-terminated PDB path. Debuggers use identity—not merely the basename—to match an image and PDB. The path can disclose a build-machine path, so deterministic/privacy-conscious builds should configure it deliberately.

CodeView RSDS record
+0x00 u8  Signature[4] = "RSDS"
+0x04 u8  Guid[16]
+0x14 u32 Age
+0x18 u8  PdbPath[] terminated by NUL

PDB is not the COFF symbol table. It carries richer type, line, module, and linker information. LLVM's PDB writing code lives under lld/COFF with shared support in LLVM's DebugInfo/PDB and CodeView libraries; source ownership changes, so follow includes and tests in the revision used.

Incremental linking trades final-layout freedom for faster edits by preserving state and reserving space. It can change optimization, ICF, padding, and debug behavior. Do not promise byte-for-byte equivalence to a clean optimized link without measuring your toolchain configuration.

Hotpatchability is a coordinated compiler, linker, binary-layout, unwind, CFG, and deployment property. Function padding alone does not make arbitrary runtime rewriting safe. Threads may execute the bytes, relative branches have ranges, unwind tables describe ranges, and control-flow mitigations track targets. Use a documented servicing/hotpatch system or process restart rather than inventing one from a header flag.

31. Reading current lld/COFF as implementation evidence#

At the time of writing, the current LLVM tree organizes the COFF linker under lld/COFF. Start with this source map, then verify names against the revision you check out:

A productive reading trace is:

coff::link
  -> parse options and enqueue explicit files
  -> InputFile parses symbols/chunks/directives
  -> SymbolTable resolves and extracts archive members
  -> COMDAT selection + mark-live + ICF
  -> Writer assigns RVAs/file offsets
  -> architecture code applies object relocations
  -> Writer synthesizes PE directories and headers
  -> PDB writer emits matching debug identity

This is an orientation map, not a stable internal API. Confirm each arrow with current source and a focused lld/test/COFF case. When documenting behavior, label whether evidence is a Microsoft format rule, command-line compatibility behavior, or one lld implementation choice. link.exe may reach a compatible image through different internals.

Contribution method: reduce a behavior to a tiny .s/.obj lit test, inspect with llvm-readobj, make one implementation change, run the focused test, then the COFF suite. Include malformed tests for parser changes and both AMD64/ARM64 coverage for architecture-sensitive work.

32. Rust FFI and a product-grade Windows plugin ABI#

Rust's native ABI is not a stable cross-version plugin ABI. Export extern "system" or a precisely selected C ABI, use #[unsafe(no_mangle)] under current Rust edition rules where required, and exchange C-layout records with #[repr(C)]. Keep Rust strings, slices, trait objects, enums without fixed representation, unwinding, and allocator ownership behind the boundary.

A small contract might export one query function returning a versioned function table:

use core::ffi::c_void;

#[repr(C)]
pub struct HostV1 {
	pub size: u32,
	pub log: Option<unsafe extern "system" fn(level: u32, ptr: *const u8, len: usize)>,
}

#[repr(C)]
pub struct PluginV1 {
	pub size: u32,
	pub abi_version: u32,
	pub create: Option<unsafe extern "system" fn(*const HostV1) -> *mut c_void>,
	pub run: Option<unsafe extern "system" fn(*mut c_void, *const u8, usize) -> i32>,
	pub destroy: Option<unsafe extern "system" fn(*mut c_void)>,
}

pub type QueryPlugin = unsafe extern "system" fn(requested: u32) -> *const PluginV1;

This is a complete set of declarations, not a complete executable. Every unsafe call has proof obligations:

• the module remains loaded for the full call;
• the function pointer has exactly the declared ABI;
• input pointer/length pairs name readable memory for the call;
• returned pointers follow stated ownership and thread rules;
• no panic or foreign exception crosses the ABI;
• callbacks are unregistered before host state or module code disappears.

Catch Rust panics inside exports if the panic strategy permits, translate them to error codes, and document that aborting builds terminate the process. Allocate and free on the same side unless a matched deallocator is in the ABI. Include size so older hosts can accept shorter structures and newer hosts can append fields.

For hostile plugins, an ABI cannot create a security boundary inside one process. Use a child process, restricted token/job/AppContainer as appropriate, authenticated IPC, resource limits, and a restart policy.

33. Tools, malformed-input workshop, myths, and debugging#

Useful views answer different questions:

Malformed-input workshop#

Build a parser corpus from tiny valid files, then mutate one invariant at a time:

1. section-count multiplication overflow;
2. symbol table ending mid-record;
3. auxiliary count crossing table end;
4. relocation naming an auxiliary record;
5. cyclic associative COMDAT;
6. archive member with nonprogressing size;
7. import descriptor without terminator inside its directory bound;
8. export ordinal index beyond the EAT;
9. forwarder string without NUL;
10. RVA in a section's zero-filled tail;
11. base-relocation block smaller than eight or with odd size;
12. certificate offset mistaken for an RVA;
13. resource cycle;
14. load-config Size shorter than a read field.

For every case require: no panic, no unbounded allocation, deterministic diagnosis, and no partial result mistaken for trusted validation. Differential-test structure against llvm-readobj and another independent parser, but investigate disagreements; consensus is not specification.

Debugging workshop: imported call crashes#

1. Confirm architecture and optional-header class.
2. Disassemble the call and identify direct thunk versus direct IAT call.
3. Compute the RIP-relative target by hand.
4. Verify it lands in the expected IAT slot.
5. Inspect the slot after load.
6. Resolve the module/export independently with debugger symbols.
7. Check forwarders and calling convention.
8. Check that unload did not race the call.

The earliest failed invariant wins. A crash at the target may originate in a wrongly declared Rust function pointer.

Myths to retire#

• “PE is just COFF with a DOS stub.” PE adds mapping, optional-header, directory, and loader contracts.
• “Every directory address is an RVA.” The certificate table uses a file offset.
• “Every valid RVA has a disk byte.” Zero-filled virtual tails do not.
• “A .lib contains implementation code.” Import libraries can contain only import contracts.
• “The IAT is the export table.” The IAT belongs to an importer; the EAT belongs to an exporter.
• “GetProcAddress checks my signature.” It returns an address, not type safety.
• “ASLR is one bit.” Metadata, relocatability, OS policy, and entropy conditions all matter.
• “Signed means safe.” Signature validity, trust policy, and code behavior are separate.
• “PEB loader structures are an ABI.” They are undocumented implementation details.
• “FreeLibrary makes stale pointers harmless.” It makes them more likely to fault or call reused memory.

34. Mastery exercises, contributor entry points, and source ledger#

Mastery exercises#

1. Draw the complete MessageBoxW journey from source declaration through import-library member, COFF relocation, thunk, IAT, export lookup, and target. Label every coordinate as section offset, file offset, RVA, or VA.
2. Write a bounded ordinary-COFF/BigObj symbol iterator. Preserve physical record indices and reject an auxiliary run beyond the table.
3. Implement archive fixed-point extraction. Add /WHOLEARCHIVE as policy, not a parser special case.
4. Implement all COMDAT selection modes and associative liveness. Create a cyclic-association negative test.
5. Add AMD64 REL32 relocation evaluation with signed overflow diagnostics. Explain the numbered bias variants.
6. Parse exports by name and ordinal. Test an ordinal-only export, an empty EAT slot, and both forms of forwarder.
7. Extend Chapter 12 to iterate PE32 and PE32+ lookup entries. Bound every string and reject missing terminators.
8. Map a synthetic section with four raw bytes and eight virtual bytes. Prove that its last four mapped bytes have no file offset.
9. Implement base-relocation application to a simulated byte vector for ABSOLUTE, HIGHLOW, and DIR64. Reject machine/type mismatches.
10. Implement the Authenticode digest ranges from Microsoft's procedure and compare with a trusted signing tool. Do not implement trust-chain policy yourself.
11. Hand-write one x64 assembly function with correct unwind directives, throw or unwind through it, and inspect generated runtime-function data.
12. Design a loader-lock deadlock test without shipping deadlocking code. State which waits establish the cycle.
13. Build the versioned Rust plugin table, fuzz structure sizes, and prove module ownership covers every call guard.
14. Trace one current lld COFF test from command line through input parsing to Writer. Record implementation evidence separately from format requirements.
15. Give a 30-minute talk titled “Three Address Spaces in a PE File.” Include the certificate exception and zero-fill counterexample.

Contributor entry points#

For LLVM, begin in lld/test/COFF. A minimized lit test is often the best first contribution: malformed archive diagnostics, COMDAT selection, import/export synthesis, relocation overflow, load-config metadata, or PDB identity. Implementation changes commonly lead to lld/COFF/InputFiles.cpp, SymbolTable.cpp, Chunks.cpp, ICF.cpp, MarkLive.cpp, Writer.cpp, LoadConfig.cpp, PDB.cpp, or Arch/{AMD64,ARM64}.cpp. Run the focused lit test and the full lld COFF suite; use LLVM's contribution and code-review documentation.

For Microsoft-facing product bugs, reduce behavior to documented API calls and a minimal PE fact. Do not report a private Ldr structure assumption as an API regression. For Rust ecosystem work, parser fuzzing, typed RVA/file-offset APIs, bounded iteration, and Windows CI fixtures are valuable entry points.

Source ledger#

The following sources distinguish authority and evidence. URLs are stable documentation entry points; implementation claims should also record a commit when used in a patch or talk.

Format contract

• Microsoft, PE Format: COFF and BigObj records, PE headers, sections, relocations, imports, exports, resources, exceptions, TLS, load configuration, debug data, certificate table, and Authenticode hashing procedure. This page warns that the document is not complete in every respect.
• Microsoft Learn, architecture tables linked from PE Format: AMD64, ARM64, and other machine-specific COFF relocation definitions.

Documented platform behavior and APIs

• Microsoft Learn, LoadLibraryExW, GetProcAddress, and FreeLibrary.
• Microsoft Learn, Dynamic-Link Library Search Order, Dynamic-Link Library Security, SetDefaultDllDirectories, and AddDllDirectory.
• Microsoft Learn, Dynamic-Link Library Best Practices and DllMain.
• Microsoft Learn, Using Thread Local Storage in a Dynamic-Link Library and PE TLS documentation.
• Microsoft Learn, x64 exception handling and ARM64 exception handling; pair these with current x64/ARM64 calling-convention documentation.
• Microsoft Learn, manifest, isolated applications/SxS, Control Flow Guard, hardware-enforced stack protection, /SAFESEH, and Windows trust-provider documentation. Feature availability and defaults are version-sensitive.
• Microsoft Learn, WinVerifyTrust and Authenticode/portable-executable signature documentation.
• Microsoft Learn, Understanding the Helper Function for the Microsoft delay-load helper protocol.

Current LLVM implementation evidence

• LLVM monorepo, lld/COFF: driver, inputs, symbols, chunks, liveness, ICF, output writing, load configuration, imports/exports, architecture code, and PDB production.
• LLVM monorepo, lld/test/COFF: executable regression evidence for command-line behavior and emitted bytes.
• LLVM object and debug support under llvm/include/llvm/Object, llvm/lib/Object, and llvm/lib/DebugInfo/CodeView/PDB: shared readers and PDB/CodeView machinery used by lld.
• LLD documentation and LLVM's current contributor guide. The documentation itself notes that some linker-internals prose is older; source plus tests are stronger evidence for current details.

Explicitly not a contract

Private PEB fields, LDR_DATA_TABLE_ENTRY layouts, internal ntdll!Ldr* routines, loader graph nodes, lock implementation, and API-set backing structures seen in symbols or reverse engineering are undocumented observations. They are useful for a debugger on a stated Windows build, never a stable application ABI. Preserve that label whenever teaching or diagnosing them.

You have mastered this part when you can inspect one byte range, state which producer wrote it, name its coordinate system, explain which consumer uses it and when, identify its bounds and invariant, and refuse to turn current implementation evidence into a platform promise.

Part VIII — Mach-O, dyld, and Apple Platform Linking#

1. The whole trip: source text to a running image#

A compiler turns each source file into an MH_OBJECT Mach-O file. Its addresses are mostly provisional. It contains sections, symbols, and relocation records saying, in effect, “the four bytes here must eventually reach _f.” An archive (.a) is an indexed collection of such object files; it is not loaded at process startup.

The static linker resolves inputs and emits an image with one of three important file types:

source --compiler--> MH_OBJECT --static linker--> MH_EXECUTE
                         ^       \-------------> MH_DYLIB
archive member ----------|        \------------> MH_BUNDLE

exec(path)
  kernel: validate/map executable, arrange process state
  dyld: discover dependencies -> map -> fix -> initialize -> call entry path

On current Apple systems, the exact kernel handoff and dyld optimizations are implementation matters. The public SDK's <mach-o/loader.h>, <mach-o/nlist.h>, and dyld(3) APIs define the durable vocabulary. Claims here about Loader, PrebuiltLoader, and JustInTimeLoader refer to Apple's published dyld-1378 source. Serialization details and private SPI are not contracts.

Earliest invariants: every byte range is in its containing file or mapping; commands do not overlap arithmetic limits; each selected CPU slice matches the process; each bind resolves according to namespace policy; executable memory is authenticated and signed as required.

2. Four kinds of evidence#

Do not assign every observation the same authority.

The open-source ld64 tree does not fully describe Apple's shipping linker. Use it to learn architecture, not to promise current behavior. In contrast, LLVM's lld/MachO is actively testable source, but it is a different linker. Pin a commit when making implementation claims; this chapter describes the LLVM monorepo Mach-O backend as published in August 2026, not an eternal layout.

3. Universal binaries are containers, not images#

A fat/universal file begins with a big-endian fat_header, followed by fat_arch or fat_arch_64 entries. Each entry names CPU type/subtype and a file range containing an independent thin Mach-O. Selecting a slice precedes parsing its Mach header. Never add offset + size without checked arithmetic; never assume ranges are disjoint or aligned merely because a producer should make them so.

This complete stable-Rust example selects an exact CPU pair from 32-bit fat entries. It deliberately rejects FAT64 and swapped magic rather than guessing.

use std::convert::TryInto;

const FAT_MAGIC: u32 = 0xcafebabe;

fn be32(b: &[u8], at: usize) -> Option<u32> {
    Some(u32::from_be_bytes(b.get(at..at.checked_add(4)?)?.try_into().ok()?))
}

fn select_slice(file: &[u8], cpu: u32, subtype: u32) -> Option<&[u8]> {
    if be32(file, 0)? != FAT_MAGIC { return None; }
    let count = usize::try_from(be32(file, 4)?).ok()?;
    if count > 64 { return None; }
    let table_end = 8usize.checked_add(count.checked_mul(20)?)?;
    if table_end > file.len() { return None; }
    for i in 0..count {
        let p = 8 + i * 20;
        if be32(file, p)? == cpu && be32(file, p + 4)? == subtype {
            let off = usize::try_from(be32(file, p + 8)?).ok()?;
            let len = usize::try_from(be32(file, p + 12)?).ok()?;
            return file.get(off..off.checked_add(len)?);
        }
    }
    None
}

fn main() {
    let mut f = vec![0u8; 32];
    f[0..4].copy_from_slice(&FAT_MAGIC.to_be_bytes());
    f[4..8].copy_from_slice(&1u32.to_be_bytes());
    f[8..12].copy_from_slice(&0x0100_000cu32.to_be_bytes());
    f[12..16].copy_from_slice(&0u32.to_be_bytes());
    f[16..20].copy_from_slice(&28u32.to_be_bytes());
    f[20..24].copy_from_slice(&4u32.to_be_bytes());
    assert_eq!(select_slice(&f, 0x0100_000c, 0), Some(&f[28..32]));
}

Prediction: can one fat file's arm64 and x86-64 slices have different dependencies? Answer: yes. They are separate images; inspect each slice.

4. The Mach header and a checked command walk#

mach_header_64 identifies magic, CPU, file type, ncmds, sizeofcmds, and flags. It is followed by a packed region of load commands. Every command begins with cmd and cmdsize; commands are not a null-terminated list. A safe walker bounds the whole region first, requires at least eight bytes per command, rejects zero/small sizes, and checks that exactly ncmds commands fit. Unknown commands must be skippable unless their required bit says otherwise.

use std::convert::TryInto;

fn le32(b: &[u8], p: usize) -> Option<u32> {
    Some(u32::from_le_bytes(b.get(p..p.checked_add(4)?)?.try_into().ok()?))
}

fn walk_commands<F>(file: &[u8], header: usize, mut visit: F) -> Result<(), &'static str>
where F: FnMut(u32, &[u8]) -> Result<(), &'static str> {
    let n = usize::try_from(le32(file, header.checked_add(16).ok_or("overflow")?).ok_or("header")?).map_err(|_| "count")?;
    let bytes = usize::try_from(le32(file, header.checked_add(20).ok_or("overflow")?).ok_or("header")?).map_err(|_| "size")?;
    if n > 65_536 { return Err("command count limit"); }
    let mut p = header.checked_add(32).ok_or("overflow")?;
    let end = p.checked_add(bytes).ok_or("overflow")?;
    if end > file.len() { return Err("command region"); }
    for _ in 0..n {
        let size = usize::try_from(le32(file, p + 4).ok_or("command header")?).map_err(|_| "size")?;
        if size < 8 { return Err("small command"); }
        let next = p.checked_add(size).ok_or("overflow")?;
        if next > end { return Err("command outside region"); }
        visit(le32(file, p).ok_or("command header")?, &file[p..next])?;
        p = next;
    }
    if p != end { return Err("unclaimed command bytes"); }
    Ok(())
}

fn main() {
    let mut image = vec![0u8; 40];
    image[16..20].copy_from_slice(&1u32.to_le_bytes());
    image[20..24].copy_from_slice(&8u32.to_le_bytes());
    image[32..36].copy_from_slice(&2u32.to_le_bytes());
    image[36..40].copy_from_slice(&8u32.to_le_bytes());
    let mut seen = 0;
    walk_commands(&image, 0, |cmd, _| { seen = cmd; Ok(()) }).unwrap();
    assert_eq!(seen, 2);
}

The offsets in that educational walker are specifically for mach_header_64. Real code first determines endianness and 32/64-bit shape.

5. Segments, sections, and address translation#

A segment is a virtual-memory mapping request: file range, VM range, maximum/current protections. A section is a linker's semantic subdivision inside a segment, such as __TEXT,__text or __DATA_CONST,__const. Sections guide linking and tools; pages enforce permissions. __PAGEZERO normally reserves an unmapped low range. __LINKEDIT holds tables and encoded metadata. Zero-fill sections occupy VM space but no corresponding file bytes.

Mach-O has no ELF “RVA” term, but tools often need vmaddr - image_base. To map a VM address to file data, require it to lie in both the segment's VM range and file-backed prefix:

#[derive(Clone, Copy)]
struct Segment { vm: u64, vm_size: u64, file: u64, file_size: u64 }

fn vm_to_file(s: Segment, address: u64, image_len: usize) -> Option<usize> {
    let delta = address.checked_sub(s.vm)?;
    if delta >= s.vm_size || delta >= s.file_size { return None; }
    let off = s.file.checked_add(delta)?;
    let off = usize::try_from(off).ok()?;
    (off < image_len).then_some(off)
}

fn main() {
    let s = Segment { vm: 0x1000, vm_size: 0x2000, file: 0x400, file_size: 0x800 };
    assert_eq!(vm_to_file(s, 0x1100, 0x2000), Some(0x500));
    assert_eq!(vm_to_file(s, 0x1900, 0x2000), None); // zero-fill tail
}

Invariant: a section's VM range lies within its segment; its file range must also fit when it is not zero-fill. Overlapping mappings are hostile input even when each range independently fits.

6. Load-command families form the image's plan#

Important families include:

Strings in commands are offsets relative to that command, not arbitrary file offsets. Validate a terminating NUL inside the command.

7. Symbols: nlist, ordinals, weak, coalesced, common#

An nlist_64 carries a string-table index, type bits, section ordinal, description bits, and value. N_STAB entries are debugging records. N_UNDF is undefined; N_SECT is section-defined; N_ABS is absolute. N_EXT marks external visibility. The string table is untrusted input.

For undefined references, n_desc can encode a library ordinal: a two-level lookup target. Special ordinals support the executable, self, and flat lookup. Weak references may resolve to zero; weak definitions may be replaced according to coalescing rules. Historically, coalesced sections and weak definitions let duplicate eligible definitions select one winner. Tentative “common” symbols are undefined externals with nonzero size; the linker allocates storage unless a real definition wins. Modern language options often emit real zero-fill definitions instead.

Do not confuse a symbol table with the export interface. Stripping can remove local symbol names while the exports trie still supplies runtime exports.

8. Archives are demand extraction machines#

The static linker starts with explicit objects and unresolved symbols. An archive index maps names to members. A member is extracted when it can satisfy a currently unresolved reference, and its references can trigger more extraction. Merely putting an object in libA.a does not guarantee inclusion.

Objective-C categories can be discoverable only through metadata rather than an ordinary unresolved symbol. Apple's documented linker option -ObjC asks for archive members implementing Objective-C classes or categories; -all_load loads every member from every archive (-force_load path scopes that force). These alter extraction policy, increasing size and duplicate-symbol exposure.

Prediction: a C constructor in an otherwise unreferenced archive member—does it run? Answer: normally no, because its member was never linked. Force extraction or create a real reference.

9. Dead stripping and atom boundaries#

-dead_strip computes reachability from roots: entry points, exported/kept items, initializers, and other policy roots. MH_SUBSECTIONS_VIA_SYMBOLS says an object permits the linker to treat symbol-delimited pieces as separate dead-strip units. Without suitable atom boundaries, one live item can retain neighboring bytes.

Relocations form edges; metadata can form hidden edges. Objective-C and Swift reflection, selector, conformance, or registration records need linker-aware treatment. “No source call” does not mean “dead.” Compare a link map and size, then test runtime discovery after enabling dead strip.

10. Relocations are architecture-specific equations#

A relocation identifies a place, width/type, target, and often whether the computation is PC-relative. The linker computes a value; it does not blindly copy an address.

On x86-64, common forms include signed RIP-relative relocations, branches, GOT/GOT-load references, and subtractor-plus-unsigned pairs. On arm64, instructions split values: ARM64_RELOC_PAGE21 commonly pairs with PAGEOFF12; branches use bounded BRANCH26; GOT and TLV variants express different indirections. ARM64_RELOC_SUBTRACTOR pairs with UNSIGNED. Pair order and addend rules are ABI facts, not suggestions.

arm64 address materialization:
  adrp x0, target@PAGE       // page(target) - page(PC)
  add  x0, x0, target@PAGEOFF

The two relocations preserve one logical address computation.

If a branch cannot reach, the linker may synthesize a branch island/thunk in range; that island performs a farther transfer. Layout therefore affects correctness, not only speed. lld's thunk/island passes iterate because adding islands changes addresses.

11. The indirect symbol table connects code and names#

LC_DYSYMTAB points to the indirect symbol table. Certain section types use reserved1 as a starting index. Each stub or pointer slot selects an indirect entry, which usually selects an nlist symbol. Special markers mean local or absolute rather than an external bind.

The chain is:

stub/pointer slot index
  -> section.reserved1 + index
  -> indirect-symbol-table entry
  -> symbol-table index
  -> string-table name and undefined-symbol ordinal

Validate every step. A section byte count must be divisible by stub size (reserved2) where required, and all resulting indices must fit.

12. Stubs, lazy pointers, and non-lazy pointers#

A call to an external function usually targets a local stub. The stub loads/jumps through a pointer slot. A non-lazy pointer is bound before use. In classic lazy binding, a lazy pointer initially reaches a helper that asks dyld to resolve and patch it; subsequent calls go directly. Modern dyld and cache/prebuilt mechanisms may arrange equivalent results differently, so assembly shape is not a public promise.

Data references often use GOT-like non-lazy slots because rewriting text would violate sharing and code-signing assumptions. Thread-local variables use a separate TLV mechanism. Diagnose all three layers: call-site relocation, synthesized stub, and slot fixup.

13. Classic rebase and bind opcode streams#

LC_DYLD_INFO_ONLY can identify compact bytecode streams for rebasing, binding, weak binding, lazy binding, and an export trie. Opcodes maintain state—segment, offset, type, symbol, ordinal, addend—and perform repeated actions. ULEB128/SLEB128 integers require bounded decoding.

SET_SEGMENT_AND_OFFSET -> SET_SYMBOL -> SET_ORDINAL -> DO_BIND
       state mutation       state         state       writes one location

Security invariant: each state transition is bounded, each write falls in an appropriate mapped segment, repeat counts cannot exhaust resources, and stream termination is present. Opcode details are public in SDK headers; which encoding a current linker chooses is implementation behavior.

14. The exports trie: compressed lookup with limits#

The export trie shares name prefixes. A node begins with terminal-payload length, then terminal data, child count, and children consisting of NUL-terminated edge text plus a ULEB child offset. Flags distinguish regular exports, re-exports, and stub/resolver forms. Offsets are relative to the trie start.

This complete validator checks structural traversal under explicit budgets. It intentionally validates shape rather than interpreting all terminal flag variants.

fn uleb(b: &[u8], p: &mut usize) -> Option<u64> {
    let mut out = 0u64;
    for shift in (0..64).step_by(7) {
        let x = *b.get(*p)?; *p += 1;
        out |= u64::from(x & 0x7f).checked_shl(shift)?;
        if x & 0x80 == 0 { return Some(out); }
    }
    None
}

fn validate_exports(trie: &[u8]) -> bool {
    let mut work = vec![0usize];
    let mut seen = std::collections::HashSet::new();
    while let Some(node) = work.pop() {
        if seen.len() >= 100_000 || !seen.insert(node) { return false; }
        let mut p = node;
        let term = match uleb(trie, &mut p).and_then(|x| usize::try_from(x).ok()) { Some(x) => x, None => return false };
        p = match p.checked_add(term) { Some(x) if x <= trie.len() => x, _ => return false };
        let children = match trie.get(p) { Some(x) => usize::from(*x), None => return false }; p += 1;
        for _ in 0..children {
            let rest = match trie.get(p..) { Some(x) => x, None => return false };
            let nul = match rest.iter().position(|x| *x == 0) { Some(x) if x > 0 => x, _ => return false };
            p += nul + 1;
            let child = match uleb(trie, &mut p).and_then(|x| usize::try_from(x).ok()) { Some(x) => x, None => return false };
            if child >= trie.len() { return false; }
            work.push(child);
        }
    }
    true
}

fn main() {
    assert!(validate_exports(&[0, 0])); // non-terminal leaf
    assert!(!validate_exports(&[0, 1, b'x', 0, 99]));
}

A production lookup also caps accumulated name length and distinguishes a DAG from malicious cycles. Re-export terminals redirect lookup to another image and optionally another name.

15. Chained fixups: pages containing linked lists#

LC_DYLD_CHAINED_FIXUPS points to a fixups blob. Its header leads to starts-in-image information and an imports table/symbol pool. For each participating segment, starts data describes a page size, pointer format, segment offset, maximum-valid-pointer policy, page count, and per-page start values. A page can have no chain, one start, or (through overflow entries) multiple starts.

At each chain location, one encoded pointer contains both payload and a next field. The pointer format determines stride. The next location is:

next_location = current_location + next_field * format_stride

next == 0 ends that chain. A bind payload selects an imports-table ordinal and carries format-specific addend information; a rebase payload encodes a target. Both must be decoded according to the exact pointer format. The chain stays inside its page. It is not a file-wide linked list and next is not a byte count in every format.

fn validate_chain(page: &[u8], start: usize, stride: usize) -> bool {
    if stride == 0 || start % stride != 0 { return false; }
    let mut at = start;
    let max_steps = page.len() / stride + 1;
    for _ in 0..max_steps {
        let raw = match page.get(at..at.saturating_add(8))
            .and_then(|x| <[u8; 8]>::try_from(x).ok()) {
            Some(x) => u64::from_le_bytes(x), None => return false
        };
        // Educational format: top 11 bits are `next`; real formats differ.
        let next = ((raw >> 51) & 0x7ff) as usize;
        if next == 0 { return true; }
        let jump = match next.checked_mul(stride) { Some(x) => x, None => return false };
        at = match at.checked_add(jump) { Some(x) if x < page.len() => x, _ => return false };
    }
    false
}

fn main() {
    assert!(validate_chain(&[0; 16], 0, 8));
    assert!(!validate_chain(&[0; 7], 0, 8));
}

The snippet's bit allocation is explicitly educational. Real code switches on public DYLD_CHAINED_PTR_* definitions, validates page-start overflow indexing, import count, symbol offsets, target ranges, and permissions before touching memory.

16. arm64e pointer authentication is a boundary, not decoration#

Some chained pointer formats encode authenticated arm64e rebases or binds: target/ordinal information plus diversity, address-diversity, and key selection. dyld reconstructs and signs a pointer for its storage location and intended use. A generic parser may inspect bits, but only architecture-aware runtime code with the right context can create a usable authenticated pointer.

Treat signing as a trust boundary:

• never “preserve” an authenticated pointer by copying it to another address;
• distinguish unauthenticated data parsing from pointer materialization;
• reject unsupported formats rather than masking unknown bits;
• do not claim private pointer encodings are stable beyond published headers/ABI.

A pointer-authentication failure often appears later at indirect branch or return. Find the earliest broken invariant: wrong format, wrong target, wrong key/diversity, or wrong storage address.

17. Two-level namespaces and library ordinals#

In the normal two-level namespace, an undefined symbol records both a name and the dependency ordinal expected to provide it. Thus two dylibs can export _open without every client becoming ambiguous. Re-exports can make another image's export visible through the named provider.

Two-level namespaces do not remove all interposition. Flat-namespace modes, special ordinals, weak-definition coalescing, documented interpose mechanisms in applicable environments, inserted libraries where permitted, and symbol lookup APIs can alter or broaden lookup. Compiler direct binding, visibility, hardened-runtime policy, and cache optimizations can narrow practical interception. Never promise that arbitrary calls are hookable.

18. Dependency edges: normal, weak, and re-exported#

LC_LOAD_DYLIB is a required edge. LC_LOAD_WEAK_DYLIB permits absence under specified weak-linking behavior; code must availability-check before use. LC_REEXPORT_DYLIB says the downstream library's exports are exposed through this image. LC_LOAD_UPWARD_DYLIB supports special upward relationships and is not a general cycle cure.

An install name is the identity recorded by LC_ID_DYLIB and copied into clients' dependency commands. It is not necessarily the file's build-time pathname. -install_name, framework conventions, and @ tokens make deployment location an ABI decision. Changing it after clients ship breaks discovery unless compatibility machinery covers the change.

19. @rpath is a contextual stack, not one directory#

@loader_path expands relative to the directory of the image containing the load command. @executable_path expands relative to the main executable. For an @rpath/Tail dependency, dyld searches run-path entries available in the dependency-loading context; entries can themselves use @loader_path or @executable_path. Dependency chains contribute context, so think of an ordered run-path stack, not a global RPATH=/one/place variable.

Exact conceptual trace:

/App/My.app/My                  LC_RPATH @executable_path/Frameworks
  loads @rpath/A.framework/A
    candidate: /App/My.app/Frameworks/A.framework/A

/App/My.app/Frameworks/A.framework/A
  LC_RPATH @loader_path/Frameworks
  loads @rpath/B.framework/B
    first contextual candidate:
      @loader_path of A = /App/My.app/Frameworks/A.framework
      -> /App/My.app/Frameworks/A.framework/Frameworks/B.framework/B
    inherited executable run path can provide another candidate:
      /App/My.app/Frameworks/B.framework/B

Exact ordering is governed by dyld's documented path semantics and current implementation; inspect DYLD_PRINT_RPATHS/dyld diagnostics where platform policy permits. Environment overrides can be restricted for protected or hardened processes.

Misconception: “@rpath means beside the executable.” Correction: only a particular LC_RPATH expansion might point there; the requesting loader and dependency chain matter.

20. Startup: from kernel mapping to user initializers#

Conceptually the kernel recognizes Mach-O, validates architecture and security state, maps enough of the main executable/dyld, and transfers control with process bootstrap data. dyld then constructs loaders, discovers dependency closure, maps images (or uses shared-cache mappings), validates identities, applies rebases/binds, sets protections, registers runtime metadata as needed, runs initializers, and reaches the program entry path.

kernel handoff
  -> instantiate main Loader
  -> recursively discover dependency graph
  -> map/validate images
  -> fix rebases and binds
  -> runtime notifications and initializers
  -> LC_MAIN entry path

This ordering is simplified: published dyld-1378 pipelines overlap and optimize work, and cached/prebuilt information can avoid repeating discovery. The correctness rule is dependency-aware readiness before code uses an image, not “every file follows one serial loop.”

21. dyld4 loaders and the closure vocabulary change#

In published dyld-1378, Loader is the abstraction for an image. PrebuiltLoader consumes precomputed loader information when valid; JustInTimeLoader constructs state from Mach-O at launch/load time. A process can use prebuilt information for some images and JIT loaders for others.

Older dyld3 material and tools commonly discuss closures. Current dyld4 source speaks in terms including prebuilt loaders and prebuilt loader sets. The ideas are related—precompute dependency/fixup decisions—but names and serialized representations are not interchangeable contracts. Never parse or emit private cache/prebuilt serialization as a stable third-party format. Private SPI exposing it is unstable, entitlement-sensitive, and may disappear.

Invalidation matters: path identity, code identity, environment, cache generation, and other policy inputs can make precomputed decisions unusable. Caching moves work from launch to construction and creates freshness obligations; it does not delete linking work.

22. Initializers and language runtime registration#

C/C++-family initializers are represented through initializer sections/routines; dependencies generally initialize before dependents, while order among independent branches should not become program logic. Within an image, source and linker order can influence records, but relying on cross-image incidental order is fragile. Initializers must not assume arbitrary unrelated plug-ins are loaded.

Objective-C runtime notifications and metadata discovery interact with image loading. Swift has metadata, protocol-conformance, reflection, and runtime registration concerns. The bounded lesson is that metadata creates roots and ordering edges invisible in a source call graph. Their exact private layouts and dyld/runtime handshakes are versioned implementation details; use language/runtime ABI documentation and supported compiler output, not hand-authored private records.

Workshop: a category works in a dylib but vanishes when moved into .a. First check archive extraction (-ObjC), then dead-strip roots, then image presence, then runtime registration. Do not begin by changing initializer order.

23. dlopen, dlsym, and dlclose are lifetime operations#

dlopen adds or finds an image according to path and mode, loads dependencies, fixes it, and runs required initialization. dlsym(handle, name) searches according to handle semantics; special handles broaden or alter scope. Check dlerror correctly: clear old error, call, then read the new error, because a null symbol value and failure are conceptually distinct. dlclose releases a reference; it does not guarantee immediate unmapping in every implementation circumstance.

Pointers, function addresses, C++ objects, callbacks, TLS, Objective-C classes, and Rust trait objects can outlive the handle that made their code available. Safe policy:

open handle -> obtain API -> create objects/callbacks -> stop callbacks
 -> destroy all foreign objects -> drop every code/data pointer -> close handle

RTLD_NODELETE-like behavior and dyld/runtime policies may retain images. Design correctness without requiring prompt unmapping.

24. Interposition has hard boundaries#

Interposition means redirecting a symbolic reference, not rewriting every possible machine-code edge. It may affect a bind through a stub/pointer while leaving local calls, hidden symbols, direct branches, inlined code, authenticated pointers, or already-resolved references untouched. Two-level ordinals identify intended providers. Hardened runtime and library validation can prevent injected code from entering the process at all.

Prediction: overriding malloc in a plug-in necessarily captures the host's prior calls. Answer: no. Scope, load time, namespace, binding form, platform policy, and optimized/direct calls all matter.

Use supported instrumentation APIs where available. Treat DYLD_INSERT_LIBRARIES as environment-dependent debugging behavior, not a deployment architecture.

25. Unwinding: compact tables, then DWARF where encoded#

Apple platforms use compact unwind information for common function prologues. It indexes address ranges and compact encodings describing how to recover caller state. Encodings can designate DWARF fallback, whose call-frame information commonly resides in __TEXT,__eh_frame. The linker synthesizes/merges tables and must adjust addresses after final layout.

Unwind correctness affects exceptions, backtraces, profiling, and crash reports. A function can execute correctly yet unwind incorrectly. Test asynchronous-looking samples, thrown exceptions across language boundaries where supported, stripped release builds, and hand-written assembly. Assembly must provide appropriate unwind directives or explicitly establish a supported boundary.

26. Code signing, PIE, hardened runtime, and validation#

LC_CODE_SIGNATURE locates an embedded signature superblob in link-edit data. The kernel and security stack enforce code-directory hashes and policy; signatures also bind selected metadata. Ad hoc signing is not equivalent to an identity accepted for every distribution policy. Mutating signed bytes after signing invalidates hashes.

PIE allows an executable to slide; dyld rebases slide-dependent pointers. ASLR is policy plus relocatable representation, not encryption. Hardened runtime enables restrictions selected by signing/runtime flags and entitlements. Library validation can require loaded code to satisfy team/platform-signing rules, subject to supported exceptions. Entitlements are security claims granted through signing policy, not strings that self-authorize.

Debug order for “image not loaded”:

1. record exact path and dyld error;
2. verify architecture/platform/minimum OS;
3. trace install name and @rpath candidates;
4. inspect signature validity and designated requirements;
5. inspect hardened-runtime/library-validation policy;
6. only then inspect constructors or symbols.

The first visible dlopen failure may originate in a nested dependency.

27. The dyld shared cache is a prepared address-space artifact#

Apple's shared cache combines and optimizes system libraries for mapping, sharing, fixups, and launch performance. It can reorganize link-edit information and data relative to original on-disk dylibs. It is not an archive of original files: extracting bytes does not necessarily reconstruct bit-identical inputs, signatures, paths, or standalone loadability.

Public tools and published dyld source reveal useful current behavior, but cache layouts and private extraction SPI are unstable. Never make a product depend on a private cache/prebuilt serialization. Symbolication should use matching supported symbols/build identifiers, not assume a cache from another OS build is equivalent.

28. lld Mach-O internals and what ld64 evidence can say#

At a pinned LLVM revision, begin in lld/MachO. The driver parses options and inputs; input-file classes parse objects, dylibs, and archives; symbols represent defined/undefined/common/dylib states; input sections and synthetic sections form the output; target backends apply relocations and create thunks; writer passes assign addresses, build link-edit structures, and emit/sign as configured. Tests under lld/test/MachO are often the clearest executable specification for lld behavior.

Useful investigation path:

test case -> Driver options -> InputFiles/SymbolTable resolution
 -> InputSection liveness/relocations -> SyntheticSections
 -> target relocation/thunk logic -> Writer layout

Apple ld64 may differ in option details, diagnostics, ordering, dead-strip policy, fixup selection, and platform integration. Historical open-source ld64 is valuable for atom-based design and old behavior, but it is incomplete evidence for today's proprietary Apple linker. Compare emitted bytes and documented behavior; do not infer current ld64 internals from lld names.

Contribution exercise: add a minimal malformed-input test before changing parsing. For architecture work, add exact bytes/disassembly and boundary-distance tests on both sides of branch range. Run the focused Mach-O test suite on a pinned LLVM checkout; upstream review expects cross-platform test reproducibility and no dependence on private Apple SPI.

29. Rust dylibs and plug-ins: choose a C-shaped boundary#

For Apple targets, Rust cdylib output participates in Mach-O install names, exported symbols, signing, and deployment-target rules. Rust's native ABI, trait-object layout, panic representation, and standard-library type layout are not a stable plug-in contract. Export extern "C" functions with #[unsafe(no_mangle)] on toolchains where that attribute form is required, fixed-width C-compatible types, explicit ownership, and an API version/size field.

Keep allocation paired: memory allocated by one side is freed by that side. Do not unwind a Rust panic across an FFI boundary; catch it where supported or abort by policy. Make callbacks unregisterable and quiesce them before dlclose. Avoid passing borrowed references whose lifetimes cannot be represented to the host.

Host opens plug-in
  -> finds plugin_entry_v1
  -> validates { abi_version, struct_size, capabilities }
  -> uses function pointers while retaining handle
  -> unregisters callbacks and destroys plug-in state
  -> releases pointers, then closes handle

For App Store or hardened hosts, arbitrary third-party native plug-ins may conflict with signing/library-validation policy. An out-of-process protocol can be the correct security boundary.

30. Tools, parser tests, workshops, and a mastery path#

Use tools as competing views of invariants:

LLDB workshop: when a breakpoint symbol exists but never hits, confirm the right UUID/image is loaded, inspect its slide, resolve the address, disassemble the call site, and determine whether it is direct or through a stub. Then check architecture and optimization. A source breakpoint is late evidence.

Security workshop: fuzz from the outside inward—fat table, thin header, command region, command-local strings, segment/section containment, link-edit subranges, ULEB streams, trie graph, then chains. Use truncation at every byte, integer boundaries, overlapping ranges, cycles, excessive counts, malformed UTF-8-independent byte names, and differential tests against independent readers. Parsing success must not authorize mapping or code execution.

Lifetime workshop: load a test plug-in that starts a worker and registers a callback. Predict the crash after immediate close. Then add stop/join, callback unregister, state destruction, pointer invalidation, and finally close. Run under Address Sanitizer and Thread Sanitizer in separate supported builds. The invariant is “no reachable execution or data points into the image before releasing its handle.”

Common misconceptions:

• “A section is mapped independently.” Pages and segments establish VM mappings.
• “The symbol table is the runtime export set.” The exports trie and binding metadata can survive stripping.
• “Weak means random winner.” Weak import/definition and coalescing follow specific linker/runtime policies.
• “Chained fixups are one chain.” Starts are per segment/page, with possibly multiple chains per page.
• “@rpath is one folder.” It is ordered, loader/dependency context.
• “The cache contains original dylib files.” It is an optimized mapping artifact.
• “dlclose makes every pointer invalid immediately—or guarantees unmapping.” Neither simplification is a safe contract.

Milestones and exercises:

1. Extend the universal selector for swapped FAT and FAT64, retaining overlap and alignment checks.
2. Parse a thin header and print unknown commands without interpreting them; property-test that no panic occurs.
3. Build two dylibs exporting the same name and demonstrate two-level ordinals with nm -m and disassembly.
4. Construct nested @loader_path/@rpath dependencies; predict every candidate before running.
5. Compare x86-64 and arm64 relocations from equivalent C, then force a branch-range boundary in an lld test.
6. Write a bounded export lookup preserving re-export provenance and cycle diagnostics.
7. Validate every chained-fixup page start and format without materializing pointers; add malformed overflow cases.
8. Build a versioned Rust cdylib API and prove teardown under concurrent callbacks.
9. Compare lld and Apple linker outputs without treating byte differences as bugs; classify policy versus ABI.
10. Triage a real lld Mach-O issue to the earliest broken invariant and submit the smallest test-first patch.

Authoritative reading order:

1. XNU/SDK public headers: <mach-o/fat.h>, loader.h, nlist.h, reloc.h, and architecture relocation headers.
2. Apple ld(1), dyld(3), dlopen(3), dlsym(3), Code Signing Guide, and platform security documentation for guarantees and policy.
3. Apple ABI documents for Mach-O, arm64, pointer authentication, and language interoperability; note each document's platform/revision.
4. Apple's published dyld-1378 source for current published implementation study: follow Loader, PrebuiltLoader, JustInTimeLoader, Mach-O analysis, fixups, and tests. Do not promote private SPI or serialized layouts into contracts.
5. A pinned LLVM monorepo revision: lld/MachO plus lld/test/MachO; use the commit hash in bug reports and experiments.
6. Historical dyld3/closure talks and open-source ld64 only as historical design evidence; verify all present-tense claims elsewhere.

The realistic contribution limit is important: lld can be built and tested openly, so focused parser, relocation, layout, and diagnostic patches have a clear upstream route. Published dyld can be read and experiments can produce excellent reports, but Apple's shipping integration, signing services, cache production, private tests, and private SPI are not fully reproducible. State that boundary rather than filling it with guesses. Mastery means preserving provenance: know which byte established a fact, which layer made it policy, which revision implemented it, and which earliest invariant a failure violated.

Part IX — Production Linkers, Rust Toolchains, and Debugging#

1. What “production quality” means#

A linker is production-ready when it keeps old programs working, rejects bad inputs clearly, finishes within an operational budget, and emits an image that every supported loader understands. Speed matters, but it is only one axis.

Separate four questions:

1. Mechanism: how are symbols, sections, and relocations represented?
2. Policy: which definition wins, which sections survive, and what is exported?
3. Optimization: which equivalent work or bytes may be merged or parallelized?
4. Operations: how can people bound, reproduce, explain, and roll back the work?

The earliest broken invariant is usually earlier than the visible failure. A crash before main may begin with a wrong symbol version at link time, not with startup code.

Prediction. A linker is twice as fast but occasionally changes output order. Is it production-better? Not without proving that order is outside the product's observation model.

2. The common pipeline and its contracts#

Different linkers fuse and reorder stages, but this model is useful:

arguments + scripts
        | policy
        v
discover inputs -> parse -> resolve symbols -> select/live sections
       archive       |             |                 |
       members       v             v                 v
                 relocations   LTO ownership      ICF/GC
                        \          |              /
                         -> layout -> range repair -> write
                              |          thunks        |
                              v                        v
                         addresses                 artifact + map

Pass invariants are handholds for debugging:

• after parsing, every range lies inside its owning file;
• after resolution, each required symbol has one permitted winner;
• after liveness, every retained relocation target is retained or dynamically resolvable;
• after layout, allocated ranges do not overlap and satisfy alignment;
• after thunk insertion, every instruction relocation is in range;
• after writing, headers describe exactly the emitted bytes.

A pass may invalidate a later fact. Adding thunks changes size, so layout must either reserve space or iterate. LTO creates native definitions, so final resolution cannot pretend bitcode was merely an opaque section.

Counterexample: “undefined symbols are checked immediately after opening each object.” That rejects valid backward references and prevents archive policy from operating.

3. GNU BFD ld: layers and source map#

GNU ld is a policy engine over BFD, the Binary File Descriptor library. In the current binutils-gdb tree, begin with ld/ldmain.c and ld/ldlang.c; argument handling is in ld/lexsup.c, scripts in ld/ldlex.l, ld/ldgram.y, and language statements in ld/ldlang.c. Emulation templates live under ld/emultempl/. ELF-specific BFD machinery is principally under bfd/elf.c, bfd/elflink.c, and target files such as bfd/elf64-x86-64.c. Generated emulation files mean a build tree can be easier to trace than source alone.

BFD abstracts object formats, targets, sections, symbols, and relocations. ld supplies command language and selection/layout policy; BFD reads, transforms, and writes formats. The boundary is not perfectly clean: decades of compatibility have put target hooks and policy-adjacent behavior in both layers.

Read in this order:

1. ld/ldmain.c for process lifetime;
2. ld/lexsup.c and ld/ldfile.c for options and files;
3. ld/ldlang.c for statement expansion, mapping, and assignment;
4. ld/ldwrite.c for final writing;
5. bfd/elflink.c and the target backend for symbol/relocation details;
6. ld/testsuite/ beside every behavioral claim.

This is a moving-main source map, checked conceptually against the repository layout in August 2026. Pin a binutils tag before citing line numbers.

4. Gold: important history, deprecated product#

Gold was designed as a faster ELF linker and is still valuable source reading for object-layout and parallel-work ideas. Its implementation is under gold/ in binutils-gdb, with files such as gold.cc, options.cc, symtab.cc, object.cc, layout.cc, reloc.cc, and target-specific files.

Do not recommend it as the forward default. Gold is deprecated. Since GNU binutils 2.44, released in February 2025, its sources are absent from the ordinary release tarball. Even-numbered releases may provide a separate binutils-with-gold tarball under the announced transition; the project said eventual removal would follow unless maintainers stepped forward. That is narrower and more accurate than saying “gold vanished from every repository.”

Historical gold design papers describe the constraints and machine balance of their time. Label their benchmark numbers historical; do not compare them directly with current lld or mold on different hardware and inputs.

5. lld: several native linkers, not one universal core#

LLVM's lld repository has format front ends under lld/ELF, lld/COFF, lld/MachO, and lld/wasm, plus common support under lld/Common. lld/tools/lld/lld.cpp dispatches a flavor. Sharing LLVM utilities does not mean ELF, PE/COFF, and Mach-O share one policy engine.

That separation is deliberate. Native format concepts remain visible: ELF input sections, COFF chunks and COMDATs, Mach-O atoms/subsections and load commands. A universal intermediate representation would simplify diagrams while moving format-specific exceptions into awkward escape hatches.

The documentation lld/docs/NewLLD.rst is useful orientation but contains design-era prose. Current source and tests are authoritative for current implementation. The public entry point and re-entrancy rules also evolve; embedders must read lld/include/lld/Common/Driver.h and current release notes rather than assuming a stable C library ABI.

6. lld ELF: Driver and context#

Start at lld/ELF/Driver.cpp and lld/ELF/Driver.h. The driver parses the GNU-like command line, expands files and libraries, creates linker state, invokes major phases, and reports errors. lld/ELF/Config.h represents chosen policy. Modern source passes an ELFLinkingContext through much of the implementation; older articles may show process-global state.

Driver ordering is semantic. --as-needed, --whole-archive, --start-group, state-push/pop options, and scripts affect inputs that follow. Sorting arguments before processing would be deterministic yet wrong.

Trace one tiny link with -### at the compiler-driver level, then lld's reproduction facilities where supported. Record both command and response-file bytes. The invariant is not “all paths opened”; it is “each path was interpreted under the option state active at its position.”

7. lld ELF: InputFiles and archive laziness#

lld/ELF/InputFiles.h and .cpp model regular objects, shared objects, archives, LLVM bitcode, and binary inputs. Parsing creates symbols and input sections while retaining ownership/provenance. lld/ELF/InputSection.h and .cpp represent ordinary and special section forms.

Archives are indexes plus member payloads. A lazy symbol advertises that a member _could_ define a name. Resolution extracts the member when an unresolved reference requires it, then the new member may create more requirements. Groups permit rescanning to a fixed point.

Invariant: each extracted member has a recorded cause—at minimum archive, member, and triggering symbol. Without this provenance, “why did binary size grow?” becomes guesswork.

Prediction. libA.a defines a and references b; libB.a defines b. With a traditional one-pass archive policy, does -lB -lA necessarily work? No: libB may be scanned before b is needed. A group can change that policy.

8. lld ELF: SymbolTable and compatibility#

Read lld/ELF/SymbolTable.h, .cpp, and lld/ELF/Symbols.h. Resolution considers binding, visibility, definition kind, common symbols, shared definitions, lazy archive candidates, symbol versions, and bitcode. “Put names in a hash map” misses the policy.

The ABI defines what bindings and visibilities mean. The linker decides diagnostics, tie handling where permitted, archive extraction timing, and compatibility extensions. A strong definition generally defeats a weak one; two impermissible strong definitions should diagnose, not silently select by thread race.

Compatibility inventory:

• symbol spelling and version (foo@V1, default versions);
• weak/common/unique semantics;
• visibility and preemption;
• archive order and group behavior;
• COMDAT or section-group signatures;
• plugin-produced LTO symbols;
• dynamic export policy and interposition.

Never “fix” a duplicate by adding --allow-multiple-definition globally. First identify both providers and why both entered the link.

9. lld ELF: Relocations and the architecture boundary#

Generic relocation scanning and application live around lld/ELF/Relocations.cpp and .h. Target semantics live under lld/ELF/Arch/, selected through target interfaces such as lld/ELF/Target.h. The split is mechanism versus machine law: generic code discovers references and allocates GOT/PLT or dynamic-relocation needs; architecture code knows instruction fields, addends, ranges, and encodings.

A useful equation is:

relocated field = encode(S + A - P)       # one common PC-relative family

S is target address, A addend, and P relocation place. This is not a universal formula. Each ABI relocation defines its own expression, width, signedness, overflow rule, and dynamic behavior.

Earliest invariant: the relocation field and referenced symbol are valid before arithmetic. Next: arithmetic is performed wide enough to detect overflow. Last: truncation occurs only after the ABI range check.

10. lld ELF: SyntheticSections and dynamic linking#

lld/ELF/SyntheticSections.h and .cpp construct bytes with no direct input-section twin: symbol/string tables, GOT, PLT, dynamic sections, hash tables, version tables, relocation sections, build-ID notes, and target-specific support.

Synthetic sections are where policy becomes ABI-visible bytes. A dynamic symbol admitted by export policy changes .dynsym; that can require strings, hashes, versions, and relocations. Their sizes depend on resolution, and layout depends on their sizes.

Trace one imported function:

undefined call relocation
  -> dynamic-symbol decision
  -> PLT/GOT entry allocation
  -> dynamic relocation and tags
  -> loader resolves slot
  -> call reaches provider

The exact path varies by target and flags such as -fno-plt, PIE, and binding policy. Do not teach PLT/GOT as universal loader features; PE and Mach-O organize imports differently.

11. lld ELF: Writer and layout#

lld/ELF/Writer.cpp is the convergence point. It creates output sections, orders content, assigns addresses and file offsets, emits program/section headers, and writes bytes. Linker-script support in lld/ELF/LinkerScript.cpp and .h can override ordinary placement.

Layout must satisfy:

aligned(x, a) = ceil(x / a) * a
next_offset >= current_offset + current_size
load_address mapping obeys segment alignment congruence

Alignment padding is real cost in file or virtual address space. Segment permissions are security policy: writable and executable content should not be combined merely to simplify layout. Section headers may be dispensable at runtime; program headers are the loader contract for ELF executables and shared objects.

Map files turn this pass into evidence. Keep one from a known-good build when investigating movement, growth, or overflow.

12. lld ELF: MarkLive, ICF, and identity#

Garbage collection in lld/ELF/MarkLive.cpp starts from roots and follows relocation edges. Roots include entry points, retained sections, exports or script-kept content according to policy, and runtime-required structures. Invariant: every semantically reachable section remains.

lld/ELF/ICF.cpp performs identical code folding by refining equivalence using bytes and relocation relationships. Equal bytes alone are insufficient: two sections containing the same zeroed relocation fields but referring to different targets need not be equivalent.

ICF changes address identity. Code that expects distinct function addresses can observe folding, even when calls behave identically. Therefore:

Counterexample: two interrupt-vector functions have identical machine code but external hardware uses their addresses as identities. Folding is invalid under that observation model.

13. lld ELF: LTO#

lld/ELF/LTO.cpp bridges resolved linker symbols to LLVM's LTO API. Bitcode participates in preliminary symbol resolution; prevailing definitions and visibility are communicated to LTO; generated native objects return to the ordinary link.

bitcode summaries + native symbols
       | prevailing/export decisions
       v
 LLVM LTO / ThinLTO
       | generated object buffers
       v
 final symbols -> sections -> relocations -> layout

The boundary must preserve names, linkage, visibility, COMDAT intent, and “used” roots. A mistake can appear as an optimizer deletion although the earliest broken invariant was a linker-to-LTO ownership decision.

Full LTO offers broad optimization but centralizes memory and work. ThinLTO uses summaries and partitions for reuse and parallelism, at the cost of cache validity and more moving parts. Cache keys must include compiler revision, target features, options, and relevant input identity.

14. lld ELF: thunks and range repair#

Branches have finite reach. lld/ELF/Thunks.cpp and .h, together with target code in lld/ELF/Arch/, synthesize nearby stubs when final placement puts a target out of range. A thunk transfers control using a longer-range sequence.

This is a feedback problem: layout reveals overflow; adding thunks changes layout. Implementations use target-aware spacing, reservations, or iterative convergence. Invariant: termination and in-range relocations after the final pass.

When an overflow appears only in release mode, compare section order, code size, relaxation, and thunk policy before blaming arithmetic. Capture the exact target triple and linker version; range rules are architecture-specific.

15. lld COFF and Mach-O boundaries#

For PE/COFF, enter through lld/COFF/Driver.cpp; then read InputFiles.cpp, SymbolTable.cpp, Chunks.cpp, Writer.cpp, ICF.cpp, and MinGW.cpp. Chunks, COMDAT selection, import libraries, base relocations, PDB production, Windows subsystem metadata, and /alternatename are native concerns.

For Mach-O, enter through lld/MachO/Driver.cpp; then InputFiles.cpp, Symbols.cpp, InputSection.cpp, SyntheticSections.cpp, Writer.cpp, UnwindInfoSection.cpp, and ICF.cpp. Load commands, dylib ordinals, stubs/lazy pointers, compact unwind, dead stripping, universal-binary tooling boundaries, and code-signature space are Mach-O concerns.

Do not transfer flags by spelling. /OPT:REF, --gc-sections, and -dead_strip express related policies through different object models. Likewise, ELF SONAME, Mach-O install name, and PE import-library identity are analogous operational contracts, not interchangeable fields.

16. mold: pass-oriented and data-parallel engineering#

Current mold source is organized under src/, with ELF templates and target files, plus src/macho/ for its Mach-O work. Start at src/main.cc, then src/cmdline.cc, src/input-files.cc, src/symbol.cc, src/passes.cc, src/output-chunks.cc, src/mapfile.cc, and a target file. Names can move on main; pin a release tag and inspect its build files.

Mold is designed to expose large independent sets—input files, symbols, sections, relocation blocks—to parallel work, while keeping phase boundaries where global facts are required. Typical opportunities include parsing many files, scanning relocations, computing output bytes, and hashing. Resolution and final offsets still require deterministic coordination.

Old mold design documents and launch benchmarks explain motivation, not current guarantees. Hardware, filesystem cache, debug-info volume, compiler driver, and competing load can dominate. Mach-O support and feature parity have changed over time; verify the selected release rather than inferring from an old README.

17. Deterministic concurrency#

Parallelism is safe when workers compute local facts and a deterministic reduction combines them. Never let lock acquisition order decide symbol winners or section order.

This complete stable Rust program models workers producing observations, followed by a canonical reduction. It runs sequentially to expose the proof; the observe calls could run in parallel because they do not mutate shared state.

#[derive(Debug, Eq, Ord, PartialEq, PartialOrd)]
struct Observation {
	name: String,
	input_ordinal: usize,
	bytes: u64,
}

fn observe(name: &str, input_ordinal: usize, bytes: u64) -> Observation {
	Observation { name: name.into(), input_ordinal, bytes }
}

fn main() {
	let mut facts = vec![
		observe("z.o", 2, 10),
		observe("a.o", 0, 7),
		observe("m.o", 1, 5),
	];
	facts.sort_by(|a, b| a.input_ordinal.cmp(&b.input_ordinal).then_with(|| a.name.cmp(&b.name)));
	let total: u64 = facts.iter().map(|fact| fact.bytes).sum();
	assert_eq!(facts[0].name, "a.o");
	assert_eq!(total, 22);
	println!("{facts:?}");
}

For floating-point or overflow-sensitive reductions, order affects values. Define arithmetic and reduction order, not merely output sorting. Stress with randomized scheduling, varying thread counts, and repeated cold and warm runs.

18. Allocation, arenas, and interning#

Linkers create millions of short-lived records and then discard them together. Arenas make allocation cheap and pointer-stable; string interning stores one canonical spelling; compact tagged records improve cache locality.

Costs move elsewhere:

• arenas delay reclamation and can increase peak RSS;
• intern tables require hashing and retain every admitted string;
• pointer identity is process-local and unsuitable for reproducible output;
• compact layouts can make diagnostics lose provenance;
• unbounded deduplication tables are denial-of-service targets.

A useful memory model is:

peak RSS ≈ mapped resident input + parsed metadata + symbol/string state
         + LTO state + output dirty pages + parallel worker scratch

Measure peak RSS and page faults, not only allocator-requested bytes. A faster linker that causes CI swapping can increase fleet latency.

19. Mapped input and output writing#

Memory mapping avoids explicit read copies and lets the kernel fault pages on demand. It does not make I/O free. Random metadata walks cause faults; network filesystems alter costs; truncated files can invalidate mappings; address space matters on 32-bit hosts.

Output may be mapped and filled in parallel when every writer owns a disjoint range. The layout invariant is then a concurrency proof: no overlaps, stable offsets, all bytes initialized, and publication only after success. Write a temporary file, flush as required by the durability contract, then atomically rename where the platform permits. Never leave a partially valid output under the final name.

Sparse files, copy-on-write filesystems, antivirus scanners, and code-signing steps can change measured behavior. Record them in benchmark and incident environments.

20. A performance model and honest benchmarks#

Model wall time before optimizing:

T ≈ T_open + bytes_read / effective_read_bandwidth
  + symbols * C_resolve + relocations * C_reloc
  + bytes_hashed * C_hash + bytes_written / effective_write_bandwidth
  + critical_serial_work + scheduling_overhead

The model predicts regimes, not exact time. Debug links stress byte volume; template-heavy C++ or generic-heavy Rust stresses symbols and relocations; LTO adds compiler work; tiny links expose process startup.

Benchmark matrix:

Pin input corpus, toolchain, flags, worker count, filesystem, and background load. Run enough repetitions, report distributions and confidence, and retain raw data. Do not paste upstream headline numbers as predictions for your build. No measurements are invented here.

Prediction. Why can eight link threads be slower in CI? CPU quota throttling, memory-bandwidth saturation, page-fault contention, or competition with concurrent rustc jobs can dominate available parallel work.

21. Incremental linking and reusable state#

Incremental linking stores a prior interpretation so a small change need not repeat all work. The hard question is dependency granularity. Changing one object's size can move following addresses, invalidate branch ranges, alter debug addresses, and change build IDs.

Reusable state needs:

• content-derived input identity, not timestamps alone;
• versioned serialization and target/options keys;
• dependency edges from symbols and relocations to outputs;
• transactional update and corruption detection;
• a clean fallback path;
• equivalence tests against a full link.

Caching does not remove work; it moves work into key construction, invalidation, storage, and debugging. Fine-grained state reduces recomputation but increases metadata and correctness surface. ThinLTO caches and compiler incremental artifacts may already capture more value than a second linker cache.

22. Reproducibility, hermeticity, and artifact identity#

A reproducible link produces the same declared outputs from the same declared inputs. Hermeticity means undeclared environment cannot affect them. Control absolute paths, locale, time, random seeds, directory enumeration, tool versions, library search paths, and environment-derived flags.

A build ID may identify content, a link invocation, or a release record. State which. This complete stable Rust sketch creates a deterministic, versioned identity without claiming cryptographic security:

use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};

#[derive(Hash)]
struct Artifact<'a> {
	schema: u32,
	target: &'a str,
	toolchain: &'a str,
	ordered_inputs: &'a [(&'a str, u64)],
	flags: &'a [&'a str],
}

fn identity(value: &Artifact<'_>) -> u64 {
	let mut hasher = DefaultHasher::new();
	value.hash(&mut hasher);
	hasher.finish()
}

fn main() {
	let inputs = [("a.o", 17), ("libx.a", 23)];
	let flags = ["--gc-sections"];
	let artifact = Artifact {
		schema: 1,
		target: "x86_64-unknown-linux-gnu",
		toolchain: "rustc-1.90.0",
		ordered_inputs: &inputs,
		flags: &flags,
	};
	println!("artifact-v1-{:016x}", identity(&artifact));
}

DefaultHasher is suitable only for this educational in-process sketch; its algorithm is not a stable persistent format. Production identity must specify a stable encoding and reviewed digest, and usually hashes input contents rather than toy numbers.

23. Structured diagnostics and provenance#

An actionable error says what failed, where the bad fact came from, which policy applied, and what evidence would distinguish causes. Human prose can be rendered from structured data.

use std::fmt;

struct Provenance<'a> {
	file: &'a str,
	member: Option<&'a str>,
	section: Option<&'a str>,
	offset: Option<u64>,
}

struct Diagnostic<'a> {
	code: &'a str,
	message: &'a str,
	symbol: Option<&'a str>,
	origin: Provenance<'a>,
}

impl fmt::Display for Diagnostic<'_> {
	fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
		write!(f, "{}: {}", self.code, self.message)?;
		if let Some(symbol) = self.symbol { write!(f, "; symbol={symbol}")?; }
		write!(f, "; file={}", self.origin.file)?;
		if let Some(member) = self.origin.member { write!(f, "; member={member}")?; }
		if let Some(section) = self.origin.section { write!(f, "; section={section}")?; }
		if let Some(offset) = self.origin.offset { write!(f, "; offset=0x{offset:x}")?; }
		Ok(())
	}
}

fn main() {
	let error = Diagnostic {
		code: "E_RELOC_RANGE",
		message: "branch target does not fit",
		symbol: Some("worker"),
		origin: Provenance {
			file: "libjobs.a",
			member: Some("queue.o"),
			section: Some(".text.poll"),
			offset: Some(0x28),
		},
	};
	println!("{error}");
}

Map files, cross-reference tables, archive extraction traces, discarded-section reports, and reproduction archives answer different questions. Treat paths as potentially sensitive in shared logs; support remapping or access control.

24. Hostile files, limits, and cancellation#

Object files are structured, attacker-controlled bytes whenever a build consumes untrusted packages or uploads. Validate offset-plus-size without wrapping, table counts before allocation, alignment, string termination, index bounds, decompression expansion, recursion depth, and relocation arithmetic.

This complete stable Rust sketch enforces both range safety and a cumulative work budget:

#[derive(Debug)]
enum ParseError { OutOfBounds, BudgetExceeded }

struct Budget { remaining: usize }

impl Budget {
	fn charge(&mut self, amount: usize) -> Result<(), ParseError> {
		self.remaining = self.remaining.checked_sub(amount)
			.ok_or(ParseError::BudgetExceeded)?;
		Ok(())
	}
}

fn slice<'a>(data: &'a [u8], offset: usize, size: usize, budget: &mut Budget)
	-> Result<&'a [u8], ParseError>
{
	budget.charge(size)?;
	let end = offset.checked_add(size).ok_or(ParseError::OutOfBounds)?;
	data.get(offset..end).ok_or(ParseError::OutOfBounds)
}

fn main() {
	let data = [0_u8; 16];
	let mut budget = Budget { remaining: 8 };
	assert!(slice(&data, 4, 8, &mut budget).is_ok());
	assert!(matches!(slice(&data, 0, 1, &mut budget), Err(ParseError::BudgetExceeded)));
}

Budgets may cover bytes, symbols, relocations, decompressed bytes, wall time, and memory. Cancellation checks belong between bounded units; checking only between giant files is not responsive. Clean temporary output on cancellation.

25. Security threat model#

Name the trust boundaries:

Compiler-generated inputs are not automatically trusted if source dependencies can trigger compiler or plugin bugs. A linker commonly runs with developer or CI credentials; it should not have network or production secrets merely because builds historically did.

Security hardening may cost speed. Sandbox startup and duplicate validation are explicit costs worth measuring, not reasons to skip a boundary.

26. Testing the linker stack#

Use layers because each localizes a different failure:

1. parser unit tests for malformed records and arithmetic boundaries;
2. relocation tests with known encodings and overflow edges;
3. symbol/archive/script semantic tests;
4. golden structural inspection with readelf, objdump, or platform equivalents;
5. executable/shared-library loader tests;
6. differential tests across linkers where semantics overlap;
7. property tests for layout non-overlap and deterministic reduction;
8. fuzzing of parsers and semantic pipelines;
9. regression tests reduced from incidents;
10. representative performance and memory gates.

Differential disagreement is evidence, not proof that the minority is wrong: implementations may choose different permitted policy. Compare semantic observations—exports, relocations, execution—not raw bytes unless reproducibility is the contract.

Cross-platform CI should vary host and target, endian/word size where supported, shared/static/PIE, debug/LTO, linker choice, and oldest supported loaders. Emulation is useful but does not replace tests on native loaders, filesystems, signing tools, and debuggers.

27. rustc code generation to link#

For a final artifact, rustc translates crate code through analysis and monomorphization into codegen units. The LLVM backend emits object code or bitcode; rustc collects Rust dependencies, native libraries, runtime objects, and flags; a linker invocation creates the image. Cargo orchestrates crate invocations but is not the machine-code linker.

Cargo graph
  -> rustc crate/codegen units
       -> object or bitcode + metadata
            -> compiler linker-driver construction
                 -> C compiler driver or direct linker
                      -> ELF/PE/Mach-O image
                           -> package/sign -> loader

Current rustc reading path: compiler/rustc_codegen_ssa/src/back/link.rs for orchestration; neighboring linker.rs for linker abstractions; compiler/rustc_target/src/spec/ for target options; LLVM backend code under compiler/rustc_codegen_llvm/; native-library collection in codegen/metadata paths. This is a moving-main map: pin the Rust release tag and follow symbols because files are refactored.

Use cargo build -vv to observe rustc commands and appropriate rustc link-printing/logging facilities for the final invocation. Diagnose the command actually run, not the command remembered from another host.

28. Linker path, flavor, and driver are different#

Three settings are often confused:

• path/program: executable rustc launches, such as cc, clang, rust-lld, or link.exe;
• flavor: command-line dialect and invocation behavior rustc emits;
• underlying linker: program that performs layout, perhaps selected by a driver using -fuse-ld=.

On Unix-like GNU targets, a C compiler driver can add startup objects, library search directories, libc, and target defaults before invoking ld. Pointing directly at an ELF linker can omit that driver policy. On MSVC, link.exe-style and lld-link-style invocations differ from GNU mode even though lld can implement both.

Rust 1.90 narrow fact: stable Rust 1.90 changed the default for the exact target x86_64-unknown-linux-gnu to use the toolchain's rust-lld as the underlying linker through the established driver arrangement. The default program may still be cc; users still need the suitable system driver/startup and system libraries. -C linker-features=-lld opts that target behavior out. Do not generalize this claim to all Linux architectures, all GNU targets, musl, macOS, Windows, or future Rust versions. Confirm with the selected toolchain's target specification and verbose invocation.

29. Cargo build scripts, ordering, and links#

A build.rs program emits cargo::rustc-link-search, cargo::rustc-link-lib, and related instructions. Their order can affect final linker argument order. Emit a library after the objects or libraries that require it when the platform's archive semantics demand that ordering. Do not use directory iteration order.

The package manifest links key declares a native-library ownership name. Cargo permits at most one package with a given links value in a dependency graph and passes metadata from the producer's build script to direct dependents. It does not prove ABI compatibility or stop two differently named packages from bundling the same symbols.

Build scripts execute code and inspect the host environment. For hermetic builds, enumerate rerun inputs, pin external tools, distinguish HOST from TARGET, avoid probing undeclared machine state, and record generated native artifacts. A cross-build script runs on the host while producing information for the target.

Checklist:

• inspect cargo build -vv output;
• check instruction order and exact -L directories;
• identify static versus dynamic native library choice;
• verify host tools do not emit host objects for the target;
• audit every build script as code, not metadata.

30. Rust artifact types, symbols, and FFI#

Rust symbol mangling carries identity needed by Rust compilation and is not a stable C ABI. For FFI, define explicit exported names and ABI with current Rust attributes, use extern "C" where appropriate, and use #[repr(C)] for shared layouts. Still specify integer widths, ownership, allocation/free pairing, panic behavior, thread rules, and versioning. repr(C) alone does not make String, trait objects, or Rust enums universally safe interfaces.

Control exports with version scripts/export lists/.def files as appropriate. Accidentally exporting all Rust symbols enlarges compatibility and attack surfaces. Catch it by comparing a checked-in export allowlist against the produced dynamic symbol table.

31. Rust LTO, debug information, and symbol services#

Rust's -C lto, -C linker-plugin-lto, codegen-unit settings, and Cargo profile options change where optimization occurs and what intermediate form reaches the linker. All participants must agree on compatible LLVM/toolchain expectations for plugin LTO. Measure release throughput, peak memory, runtime, and binary size together.

Debug information connects machine addresses to source through DWARF on common ELF systems, PDB on MSVC, and dSYM/DWARF workflows on Apple platforms. Stripping can separate rather than destroy debug data. The deployable image and debug companion must share a reliable identity: ELF build ID or debug link, PE/PDB GUID-age conventions, or Mach-O UUID as applicable.

A symbol server is an indexed artifact service, not merely a bucket. Preserve exact unstripped/debug artifacts, identity, toolchain and source revision, retention policy, and access controls. Test a crash from a packaged release through symbolication before rollout. Split DWARF and compressed debug formats need debugger, linker, packager, and server compatibility as one system.

32. Rust crates for object and loader tooling#

Choose the abstraction matching the operation:

Crate APIs and MSRVs move; consult the selected release documentation and lockfile. Parsing safety does not imply semantic trust. libloading can keep a handle alive for a symbol's Rust lifetime, but it cannot prove the foreign signature is correct or that spawned foreign threads stopped before unload.

33. Packaging and the runtime loader#

Link success proves only that an image was produced. At runtime, the loader searches dependencies, maps segments, resolves imports, applies relocations, establishes TLS, runs initializers, and transfers control.

On ELF, distinguish link-time -L from runtime DT_RPATH/DT_RUNPATH, loader configuration, and $ORIGIN policy. On Mach-O, reason about install names and @rpath, @loader_path, and @executable_path. On Windows, reason about documented DLL search order, application directories, safe search settings, manifests, and packaged DLLs. Never advise adding an untrusted writable directory globally to a search path.

Signing is downstream of byte production: modifying bytes after signing invalidates identity. Reproducible unsigned images and signed release artifacts are different contracts because signatures may include controlled time or service metadata.

Release checklist:

• enumerate transitive dynamic dependencies on a clean machine;
• verify minimum loader/OS and symbol-version requirements;
• test relocation and initializer behavior before main;
• validate rpaths/install names/search policy;
• strip and attach debug identity correctly;
• sign the final bytes and verify signatures;
• exercise upgrade, rollback, and side-by-side installation.

34. Debugging by the earliest broken invariant#

Freeze evidence before changing flags: source revision, lockfile, target, host, rustc and linker versions, complete command, environment allowlist, response files, input hashes, map, output headers, loader trace, and resource telemetry. Then minimize one axis at a time.

Do not apply five folklore flags at once. Each experiment needs a prediction. If disabling section GC fixes a crash, that proves reachability policy is implicated; it does not prove GC is buggy. The missing root may be an FFI callback hidden from relocation edges.

Incident workshop: before-main crash. Preserve the core/minidump and exact image. Verify loader dependency resolution, inspect dynamic relocations and constructors, compare the first faulting address with the map, and reduce initializers. Earliest candidates are wrong provider/version, malformed relocation, missing TLS setup, or constructor ABI mismatch.

Incident workshop: CI-only OOM. Record cgroup limit and concurrent rustc/link jobs. Compare one linker thread versus normal, LTO off versus on, and debug information reduced versus unchanged. The fix may be scheduler policy, not a slower linker.

35. Incident response, product architecture, and rollout#

Treat linker selection as product infrastructure. Centralize target profiles, but keep escape hatches per target. Build architecture should expose artifacts between compilation, linking, debug splitting, packaging, and signing so each identity can be audited.

Incident sequence:

1. stop promotion and preserve failing artifacts;
2. classify build-time, image-validation, load-time, or runtime failure;
3. restore service with a tested rollback, not an improvised flag;
4. reproduce in a clean environment;
5. locate the earliest broken invariant;
6. reduce and add a regression test;
7. fix forward, then remove temporary bypasses;
8. publish impact, detection gap, and prevention.

Roll out a linker/toolchain change through shadow builds, then a small CI cohort, then release candidates, then broad adoption. Compare exports, dependencies, security properties, loader behavior, tests, size, time, and RSS. Byte equality is appropriate only when promised; semantic comparison is often better.

Compatibility has direction. New linker with old objects, old loader with new image, plugins built by another LLVM, and rollback to an old service are separate matrix cells. Define the minimum supported OS/libc/toolchain and retain the means to rebuild and symbolize every supported release.

36. Contributing and a versioned source-reading map#

Before proposing code, reproduce at a pinned revision, reduce the input, identify the governing ABI text, locate neighboring tests, and separate compatibility change from refactoring. Include command lines and generated objects when licensing and size permit; assembly reproducers are often durable.

Project workflows differ:

• lld/LLVM: read llvm/docs/Contributing.rst, lld docs, lld/test/, and target tests; patches follow LLVM review and style. Begin at the relevant format directory, not generic LLVM optimization code.
• mold: read the repository's current CONTRIBUTING.md, tests, and issue templates at the pinned release/main revision. Validate claims on current hardware and include GNU-compatible behavior expectations.
• GNU binutils: read binutils/MAINTAINERS, top-level contribution guidance, ld/testsuite/, and BFD target tests; copyright and mailing-list procedures matter.
• rustc/Cargo: follow the Rust compiler development guide, CONTRIBUTING.md, UI tests, target specs, and team review. Link changes need target-owner and bootstrap/CI awareness.
• Rust crates: read each crate's MSRV, safety policy, fuzz targets, changelog, and semver expectations. Add malformed-input tests for parsers and platform CI for loaders.

Versioned source map#

This map is a starting path, not a guarantee of filenames forever. “Moving main” entries describe upstream layout observed through current public trees as of 2026-08-06; pin a commit before detailed study.

Contributor exercises#

1. lld trace: pin an LLVM commit. Trace one undefined ELF symbol from Driver through InputFiles and SymbolTable to its diagnostic. Write every invariant and add a test that breaks the earliest one.
2. Archive compatibility: build two archives whose order matters. Test BFD ld, lld, and mold with and without a group. Explain differences from traces, not assumptions.
3. Relocation edge: choose one architecture relocation, derive its legal range from the ABI, generate boundary cases, and locate generic versus Arch/ responsibility.
4. Determinism: perturb object discovery and thread count. Compare semantic structure and bytes separately; identify every unstable field before proposing sorting.
5. Budget: extend the Rust budget sketch with independent symbol and decompression limits. Property-test that rejected inputs never allocate proportional to claimed untrusted sizes.
6. Rust driver: on Rust 1.90.0, capture the exact x86_64-unknown-linux-gnu final link path and then opt out of lld. Explain program, flavor, underlying linker, and startup-object source. Repeat on another target and do not generalize.
7. Loader incident: package a tiny cdylib with a deliberately wrong runtime path. Diagnose it on a clean environment, repair the packaging contract, and retain a regression test.
8. Symbol service: build, split, strip, crash, and symbolize one binary. Prove artifact/debug identity and document retention and access controls.
9. Performance study: design the matrix from Chapter 20 for one real repository. Report raw distributions, peak RSS, cache state, and correctness checks; reject any conclusion unsupported by the cost model.
10. Contribution rehearsal: reduce a real issue to the owning upstream's test format, identify reviewers/modules, and write a patch description separating mechanism, policy, optimization, and operational impact.

Mastery is the ability to move from a production symptom to the first violated contract, while preserving compatibility and evidence. The linker's final bytes are only the end of that reasoning chain.

Part X — Mastery, Product Design, Talks, and the Philosophy of Binding#

1. What mastery means#

Mastery is not remembering every ELF tag or linker option. It is being able to recover the mechanism from evidence. A master can:

• trace a name from source declaration to object symbol, relocation, selected definition, output address, dynamic lookup, and live mapping;
• state the invariant at each boundary and find the earliest one that broke;
• distinguish an ABI guarantee from a toolchain convention and a current implementation choice;
• build a bounded linker or loader, state its omissions, and reject inputs outside its contract;
• compare designs by correctness, compatibility, security, diagnostics, time, and memory;
• explain why a result occurred with bytes, maps, traces, and pinned commands rather than authority.

The evidence matters more than the claim. Keep an inspectable portfolio:

2. A 30/60/90-day plan#

The schedule assumes six focused hours each week. Reduce scope, not evidence, if less time is available.

At each gate, ask: Can another person reproduce the claim from a clean checkout? If not, the missing work is packaging, not polish.

3. The deliberate-practice ladder#

Practice should increase uncertainty one dimension at a time.

1. Predict a field before running an inspector.
2. Decode it by hand and compare.
3. Trace one symbol and one relocation end to end.
4. Change exactly one cause: order, visibility, alignment, reachability, or load scope.
5. Diagnose a seeded failure without source code.
6. Implement the mechanism against a tiny written oracle.
7. Add malformed and adversarial cases.
8. Compare two implementations and explain disagreement.
9. Review someone else's design and identify its hidden policy.
10. Teach the mechanism while answering counterexamples.

Prediction. Two strong definitions have equal names. Which one wins? Do not answer until the output kind, object format, visibility, weak/common rules, command order, and linker policy are known. The practiced response is not a guess; it is a list of missing facts.

4. LinkLab from scratch#

Rebuild LinkLab without copying Part V. Use a textual object format and never execute its output.

bytes -> validated records -> stable IDs -> candidate sets -> selected definitions
                                                        |
archive demand <-> unresolved set                        v
roots -> reachability graph -> live sections -> layout -> checked relocation -> image
             provenance accompanies every arrow ------------------------------^

Milestones are independently testable:

1. checked parser with byte-span errors;
2. stable input, section, symbol, and relocation identities;
3. local/strong/weak/undefined resolution with an explicit policy table;
4. lazy archive extraction to a fixed point;
5. group selection and section reachability;
6. aligned deterministic layout;
7. absolute and PC-relative relocation with overflow checks;
8. map, extraction reason, discard reason, and relocation trace;
9. slow semantic oracle plus generated differential tests.

Acceptance tests require permutation tests for deterministic cases, duplicate-definition errors naming both origins, an archive cycle, a dead section that contains a bad relocation, signed displacement boundaries, and byte-identical output across repeated runs.

5. The bounded real ELF linker capstone#

Write the contract before code: for example, little-endian ELF64 x86-64 ET_REL, selected section and symbol forms, RELA, and a short relocation allowlist. Reject everything else by field and offset. A narrow truthful linker is better evidence than a broad accidental parser.

The capstone passes when it:

• reads without casting file bytes to Rust structs;
• proves every table range, string termination, index, alignment, addition, and multiplication before use;
• links at least two compiler-produced objects under the declared compiler flags;
• emits coherent ELF and program headers inspected by two independent tools;
• reports unsupported TLS, visibility, COMDAT, relocation, or section features rather than guessing;
• matches the declared relocation equations at minimum, maximum, and overflow values;
• produces a deterministic map linking output ranges back to input ranges;
• runs only in a disposable test process after structural inspection.

Differential comparison with ld.lld, GNU ld, or mold is evidence, not an oracle: disagreement may expose either your bug or a policy difference. Minimize the input and classify it.

6. The safe loader simulator capstone#

Build a simulator that allocates a Vec<u8> as a pretend address space. It parses a bounded image, chooses a simulated load bias, copies file bytes, zero-fills memory tails, applies allowlisted relocations, changes simulated permissions, and calls no loaded code.

untrusted file
     |
 validate ranges and limits
     v
mapping plan --overlap/W^X checks--> simulated pages
     |                                  |
     +-> relocation plan -> checked writes
     +-> constructor plan -> printed order only
     +-> retirement plan  -> lease-state trace only

Acceptance tests cover truncated headers, filesz > memsz, wraparound, overlapping segments, forbidden writable-and-executable final pages, unresolved symbols, relocation overflow, cyclic dependencies, constructor ordering, stale-generation handles, and deterministic traces. The simulator must never call mmap with executable permission, jump to input bytes, or model successful validation as permission to execute.

7. Product-grade linker architecture capstone#

Design, then implement one vertical slice of a service that links many builds. Separate these concerns:

Capstone features: cancellation at pass boundaries, resource budgets, deterministic parallel reduction, content-addressed cache keys, machine-readable maps, and compatibility modes named in output metadata. Inject a new relocation, object reader, and diagnostic renderer. If each requires editing the central resolver, the boundaries are false.

Acceptance: cancel without publishing a partial output; reject a cache entry built under another target or policy; reproduce bytes under varied worker counts; explain every retained section; and bound memory on a generated large input.

8. Product loader and plugin service capstone#

Prefer process isolation unless latency or ABI constraints justify in-process loading. The product protocol should expose states, not raw library handles:

Discovered -> Validated -> Staged -> Active -> Draining -> Retired
                              |          |          |
                         publish API  deny leases  release only after proof

Use versioned capability tables, host-owned buffers, explicit ownership, health checks, generation-tagged leases, and a coordinator that never calls plugin code while holding lifecycle locks. Define crash containment, quotas, rollout, rollback, and audit events. “Close the handle” is not a retirement proof.

Acceptance tests include ABI mismatch, plugin crash, hung request, reentrant callback, failed constructor, concurrent upgrade, rollback, leaked lease, old-generation request, and host restart. The safe result of uncertain quiescence is quarantine or process retirement, not an in-process unmap.

9. Debugging workshop: the wrong definition#

Seed three definitions with the same spelling: local, weak, and default-visible global, with one inside an archive. The observed call reaches the surprising body.

Before tools, draw candidate creation and scope order. Then capture command order, archive extraction reasons, symbol binding/visibility/version, resolution map, dynamic dependency order, and loader lookup trace. The earliest broken invariant might be “the expected archive member became active,” not “the final address is wrong.”

Socratic questions: Was the expected object eligible? Did demand exist when the archive was visited? Was the reference bound statically or left for runtime? Could interposition alter it? Which record proves each answer?

10. Debugging workshop: relocation and layout#

Seed a PC-relative overflow that appears only after a linker script moves a section. Require a worksheet containing S, A, P, field width, signedness, computed mathematical value, permitted interval, and output placement provenance.

Compare four remedies: change layout, choose a large code model, insert a target-supported range extension, or reject. Silent truncation is never a remedy. If relaxation is present, trace iterations and prove convergence or state the bound.

11. Debugging workshop: startup and lifetime#

Cases: an executable maps but fails before main; a lazy binding fails on the first rare call; and a callback crashes after plugin upgrade. Build timelines rather than stack-trace stories.

The workshop ends only when the first violated invariant is named and a regression test fails at that boundary.

12. Malformed-input and security workshop#

Create inputs by structural mutation, not only random bit flips: counts near integer limits, offsets near end-of-file, unterminated strings, cyclic version chains, conflicting overlaps, impossible alignment, huge sparse sizes, relocation storms, deep dependency graphs, and hash chains that exhaust work budgets.

For every file-derived range, write the proof:

start <= file_len
size <= file_len - start
count <= configured_limit
entry_size is the required size
count * entry_size is checked before addition
decoded index is in the validated target table

Run parsers with memory, time, recursion, and output limits. Fuzzing finds examples; properties explain correctness. Keep minimized cases with the invariant in the filename or test name.

13. Source-reading routes#

Read by data flow, not directory order. Pin a revision because source describes that revision, not every ABI implementation.

For each route, identify input types, stable identities, state transitions, error exits, limits, locks, provenance loss, and tests. Do not infer a public promise from an internal type name.

14. Contribution assignments#

Choose work small enough to review:

1. improve an error with both referring and defining provenance;
2. add a malformed-object regression at the earliest missing check;
3. document one implementation behavior with a pinned source link;
4. reduce a differential failure to a tiny object;
5. add a deterministic-order test under different thread counts;
6. measure and remove one avoidable allocation without changing semantics.

Before coding, find the issue template, contributor guide, test convention, supported targets, and maintainer position. A technically correct patch can still be wrong for project policy or compatibility.

15. Benchmark and reproducibility portfolio#

Benchmark parsing, resolution, archive extraction, GC, relocation, writing, startup, and lookup separately where possible. Record tool revision, target, flags, input digest, machine, OS, CPU policy, storage state, worker count, warm-up, samples, summary statistic, and raw measurements.

Include representative shapes: many tiny objects, few large objects, many symbols, relocation-heavy debug builds, archive cycles, high dead-code fraction, and a real application. Peak memory and output size belong beside elapsed time.

Design exercise: A cache makes the second link 20 times faster. Report cold and warm results separately; include cache construction and validation cost. Otherwise the number answers “How fast is a cache hit?” while pretending to answer “How fast is linking?”

16. Review rubrics#

Score each category 0–3: absent, partial, convincing, or independently reproducible.

A total score is less useful than one zero in a critical boundary. Security and lifecycle cannot be averaged away by fast benchmarks.

17. A 15-minute talk storyboard#

Use one symbol, render, as the entire story.

End with the earliest broken invariant in the demo, not a summary of terminology.

18. A 30-minute talk storyboard#

Add two traces to the 15-minute spine. First, calculate one relocation with S, A, and P. Second, compare section and segment views of the same bytes. At minute 20, ask the audience to predict which archive member is extracted. Reveal the unresolved-set timeline, then connect demand-driven extraction to lazy runtime binding.

Reserve five minutes for one production choice: eager binding, lazy binding, or process-isolated plugins. Show what work moves in time, what evidence must remain, and what rollback exists.

19. A 60-minute talk and live demo#

Structure: 10 minutes mental model, 15 object inspection, 15 linker trace, 10 loader/lifetime model, 5 product decision, 5 questions. The live demo uses checked-in tiny sources and a script that prints commands, versions, and expected landmarks.

Demo safety rules: never download or execute audience-provided objects; use a disposable directory; inspect before execution; have captured output; and make the teaching point survive tool-output differences. Seed one controlled relocation failure, predict it, show the map, fix the layout, and rerun. A successful command is not the climax; the matched prediction is.

20. Adversarial audience questions#

• “Why not bind everything early?” Early binding narrows uncertainty and startup surprises, but limits replacement, interposition, and deployment flexibility; it also requires all information earlier.
• “Does ASLR change symbol identity?” It changes runtime addresses, not the intended semantic identity. Code that used addresses as identity confused two representations.
• “Can a reference count prove unloading is safe?” Not if raw pointers, callbacks, TLS, unwind metadata, or foreign registrations escape its accounting.
• “If ICF passes tests, is it correct?” Only under the tests' observation model. Address comparison, tooling, or hidden ABI behavior may observe folding.
• “Are sections irrelevant at runtime?” Program headers drive normal ELF mapping, but sections remain useful to linkers, debuggers, analyzers, and some platform conventions. Different questions need different views.
• “Is the system linker the oracle?” It is a compatibility reference for a named version and mode, not a proof of the format or your intended policy.
• “Why preserve provenance if stripping saves space?” Strip runtime output when appropriate, but retain build-side causal records if diagnosis, audit, or reproducibility requires them.

21. Teaching plan#

Teach in four sessions: object evidence, static decisions, runtime authority, and product lifetime. Each session follows prediction → small trace → mechanism → boundary case → lab → explanation by the learner.

Assess with three forms: decode an unfamiliar record, diagnose a seeded failure, and defend a design tradeoff. Vocabulary quizzes cannot show whether a learner can separate symbol identity from address. Pair learners so one operates tools while the other states predictions; swap roles before revealing output.

22. The OSS contribution ladder#

Progress by review risk, not apparent cleverness:

1. reproduce and classify an issue;
2. improve a test or documentation claim;
3. minimize a malformed input;
4. add a diagnostic with preserved provenance;
5. fix a local validation or overflow bug;
6. repair a target-specific relocation under its ABI tests;
7. change shared resolution, layout, or lifecycle policy;
8. propose architecture only after maintaining earlier changes.

Contribution evidence includes review responses and abandoned approaches. Knowing why a patch should not merge is part of production judgment.

23. Ethical performance and résumé claims#

Say “reduced median warm-link time from 4.2 s to 3.1 s on corpus C, revision R, 16 workers” rather than “made linking 35% faster.” Say “implemented a bounded ELF64 linker supporting X and rejecting Y,” not “built an ELF linker” if TLS, shared objects, and scripts are absent.

Never hide failed workloads, compare cold and warm paths without labels, call simulator isolation a security sandbox, or present an implementation observation as an ABI law. Disclose whether code was educational, deployed, independently reviewed, fuzzed, or executed on untrusted input.

24. Names are claims; identities are not coordinates#

A name such as render claims that some entity should be found under a language, object-format, version, visibility, and scope contract. It is not the entity itself. Two local symbols may share spelling yet have distinct identities; one versioned global may have several spellings in tools; a stripped entity may retain identity in machine references without retaining a public name.

Likewise, an input section index identifies a table entry only in one file. A file offset locates bytes in one encoding. An output virtual address locates a result in one image. A runtime address locates one mapping generation. Treating any coordinate as permanent identity makes deduplication, ASLR, incremental layout, and unloading unsafe.

Socratic question: If a function moves but all valid references are repaired, did it change identity? If two functions are folded to one address, did they become one semantic entity? The answers depend on the declared observation model, not pointer arithmetic alone.

25. Relocations represent future work#

A relocation preserves a deferred equation: a place to modify, a target claim, an addend, a kind, and therefore width, signedness, and range rules. The compiler could not finish because final placement or selection was unknown. It did not erase the uncertainty; it encoded enough information for a later agent.

This yields a general design rule with a concrete origin: when a boundary lacks information, preserve the pending operation and its provenance. Guessing an address early makes later correction expensive or impossible. Preserving every high-level type, however, also costs space and compatibility. A good representation carries exactly the future questions the next stage is authorized to answer.

26. Linking negotiates; loading grants authority#

Object files are partial descriptions. One contributes bytes, another demands a symbol, an archive offers definitions conditionally, a script imposes placement policy, and an ABI defines admissible equations. Linking negotiates these claims into one image while reporting conflicts and unresolved demands.

Loading changes the stakes. It commits bytes into a process's authority: mapped data can influence control flow; executable mappings can run; relocations write addresses; constructors execute before ordinary application checks; exported callbacks can outlive the load operation. Validation must therefore precede authority, and final permissions must reflect intended use.

partial claims --link policy--> coherent image --validation/mapping--> process authority
     ^                |               |                              |
 provenance        negotiation     executable contract           lifetime proof

27. Binding time moves work; it does not delete it#

Static, dynamic, and late binding place selection at different times. Earlier work has more build context and produces earlier failures. Later work can see deployment state and replacement choices but adds startup or first-use cost, runtime failure paths, synchronization, and observability needs.

An abstraction can hide these operations from callers, but the resolver, cache, loader, or operator still performs them.

28. Sections, segments, symbols, and forgotten knowledge#

Sections organize bytes for linking questions: which relocations target this contribution, which group is live, which symbols are defined here? Segments organize bytes for loading questions: which file range maps where, with what size, alignment, and permissions? They may cover the same bytes without being redundant because each makes a different query cheap.

Symbols preserve names, binding, visibility, section relationships, values, and sometimes versions or types at object-format granularity. They generally do not preserve a language's complete type system, ownership, generic constraints, or lifetime proof. Stripping saves space and reduces exposed metadata but makes later diagnosis and attribution harder. Representation controls cheap questions by choosing what to index and what to forget.

29. Demand, observation, and optimization#

Archives and lazy loading respond to demand. They avoid work only when nobody asks, but require indexes, fixed-point reasoning, ordering policy, and a path that can perform delayed work safely. The saved cost becomes conditional complexity.

GC removes unreachable contributions under a root-and-edge model. ICF merges contributions considered equivalent. Neither preserves “everything about the original.” Correctness is relative to an observation model: calls and data reads, address equality, unwinding, debugging, sanitizers, constructors, exported names, and ABI metadata may all matter.

Design exercise: Define an observation model for a firmware image with no debugger and no function-pointer equality, then for a hot-patchable server. Explain why identical bytes can be foldable in the first and distinct in the second.

30. Open worlds, address stability, and policy#

Interposition keeps selection open so an earlier runtime definition can replace a later one. This supports preload tools, compatibility techniques, and some instrumentation. It also constrains direct calls, constant propagation, symbol locality, startup lookup, and reasoning about which body runs. Closing the world can improve optimization only by restricting replacement.

Address stability is a separate product promise. Incremental links, ASLR, ICF, hot replacement, and compaction can move or merge addresses while semantic identities remain. If external state stores addresses as durable IDs, the product has silently promised immobility.

Linker scripts expose placement and retention policy as a language. That makes unusual memory maps possible without recompiling the linker, but moves complexity into order-sensitive policy whose orphan rules and expressions need diagnostics, compatibility tests, and version control. Local linker simplicity can create global script complexity.

31. Caches, prebuilt loaders, and controlled causality#

A cache reuses a past answer. Correctness now depends on the key representing every cause that could change meaning: input bytes, order, target, ABI, options, environment-dependent search results, tool revision, scripts, and relevant plugins. Validation and invalidation are semantic work, not housekeeping.

Prebuilt loader metadata similarly shifts parsing, lookup, or relocation work earlier. It needs authentication or trust, compatibility checks, fallback when stale, and a way to diagnose which facts produced it. Faster startup has purchased a second representation and consistency obligations.

Deterministic output means controlling causality: stable input order, tie-breaks, hash iteration, parallel reduction, timestamps, paths, random seeds, environment, and tool identity. It does not mean pretending the environment has no effect. Record intended causes; remove accidental ones.

32. Provenance, uncertainty, and the first broken invariant#

Diagnostics depend on preserved provenance. “Duplicate symbol” is weak; useful output names both files, archive extraction causes, bindings, command positions, and the policy that made them conflict. Once a transform folds, discards, renames, or synthesizes entities without a reverse map, later layers cannot reconstruct that story.

Uncertainty should be represented: unresolved symbol, unknown version, pending relocation, unverified capability, draining generation. Guessing turns uncertainty into false certainty and moves failure farther away. A crash in main can begin with an unchecked file range; a stale callback can begin when retirement accepted new work; a wrong function can begin with archive order. The earliest visible failure often trails the first broken invariant because each later layer trusted the previous representation.

33. Unloading, security boundaries, and exit paths#

Unloading is a temporal proof: no current or future execution may reach code or data from the retiring generation. The proof includes active calls, queued work, callbacks, function pointers, TLS, unwind registrations, destructors, borrowed memory, and foreign-runtime state. Time passing and reference count zero do not prove these facts unless the whole capability system is accounted for.

Security boundaries are where bytes or names gain authority: parser allocation, file search, mapping, relocation writes, permission changes, constructor calls, plugin capability grants, and cache acceptance. Validate before crossing each boundary and minimize what crosses.

Product design needs exit paths: reject an unsupported object, disable an optimization, fall back from stale prebuilt data, roll back a plugin, quarantine a generation, restart a worker, or retain an old mapping rather than unsafely unmap it. A fast path without a safe refusal path is not product complete.

34. Counterfactual laboratory I: compilation and resources#

Change one premise and trace where the cost goes.

Prediction: With free memory, does lazy archive extraction become useless? Not necessarily. It also controls semantic inclusion, duplicate exposure, constructors, and output size; only one pressure disappeared.

35. Counterfactual laboratory II: runtime and trust#

Socratic questions: Would no dynamic libraries eliminate late binding inside a language runtime? Would trusted input permit unchecked arithmetic? Would permanent lifetime make stale semantic versions safe? Each “no” identifies a responsibility that was independent of the removed mechanism.

36. Durable reading map#

Read specifications for guarantees, source for current mechanisms, and design documents for rationale. Pin dated or revision-sensitive material.

• ELF gABI — generic object, program-loading, symbol, and dynamic-linking contracts; note the document status shown by the site.
• x86-64 psABI and Arm ABI releases — processor relocation, calling, TLS, and platform details; cite a revision or release.
• PE format documentation — Microsoft's PE/COFF contract; pair with current Windows loader documentation for product behavior.
• Mach-O loader reference archive and current dyld source where published — separate historical documentation from current platform behavior.
• LLVM lld source, mold source, and GNU binutils — implementations, tests, and policy; not universal format law.
• Linux ELF loader, glibc, and musl — kernel and runtime-loader responsibility boundaries; pin revisions for claims.
• DWARF standard — debug provenance and type information that object symbols alone do not preserve.
• Rust Reference and Unsafe Code Guidelines repository — language guarantees versus evolving design discussion; the latter is not a normative Rust specification.

Revisit this map by question: encoding, relocation, lookup, lifetime, implementation, or diagnostics. Do not read all sources as if they had equal authority.

37. Mastery checklist and commitments#

Evidence of mastery:

• [ ] I can trace a symbol and relocation from input bytes to a mapped address.
• [ ] I keep identity separate from names, indices, offsets, and addresses.
• [ ] I state resolution, reachability, layout, mapping, and lifetime invariants.
• [ ] I classify claims as specification, convention, implementation, history, or proposal.
• [ ] My bounded tools reject unsupported inputs and cap hostile work.
• [ ] LinkLab has an oracle, generated cases, causal maps, and deterministic output.
• [ ] The ELF linker and loader simulator meet their written acceptance tests.
• [ ] My product design includes cancellation, rollback, quarantine, and retirement.
• [ ] My benchmarks include raw data, revisions, cold/warm labels, memory, and corpus digests.
• [ ] My debugging dossiers identify the earliest broken invariant.
• [ ] I can defend an observation model for GC, ICF, and address identity.
• [ ] I have read and annotated one linker and one loader path at pinned revisions.
• [ ] I can give the 15-minute talk and answer the adversarial questions precisely.
• [ ] My résumé and performance claims state scope, baseline, conditions, and omissions.

Make these precise commitments: preserve provenance until its consumers are named; represent unresolved facts instead of guessing; validate ranges before allocation or authority; make policy explicit and testable; never confuse coordinates with identity; state the observation model before optimizing; key caches by semantic causes; require a temporal proof before unloading; provide a safe refusal, fallback, or rollback path; and report only claims another person can reproduce from the recorded evidence.