Chapter 39 / 51
Volume 5 · Extensions, Ops & Projects

Custom extensions: three pieces bring in any C library

Beyond sqlite and xtp2, your own libraries can compile in the same way — the registry was designed as "add one entry and done", with every input validated.

Overview

Positioning: for developers who want to bring their C libraries into XServer. The deliverable is the three-piece set plus one registration — follow sqlite/xtp2's living samples; this chapter explains why each step exists.

39.1 Intro

The design goal in one sentence: a new library = a declaration, not code. The host source carries no per-library branches (if sqlite… else if xtp…) — the registry generates the symbol-aggregation header xs_build_extensions.h at build time and injects uniformly into TCC via XS_TccAddExtensions() at runtime. Which means integrating a new library requires zero host source changes, and the extension's boundary is decided entirely by its declaration file.

39.2 The three-piece set

A real extensions.json entry (verbatim from the repository)verified on a real machinesource: xserver/tools/extensions.json
{
  "mylib": {
    "macro": "XS_USE_MYLIB",                  /* ① compile macro: the script-side visibility switch */
    "headers": {"mylib.h": "lib/mylib.h"},     /* ② header → VFS path mapping */
    "sources": [{"path": "lib/mylib.c",
                 "flags": ["-O2"]}],           /*    sources compiled into the host (with flags) */
    "symbols": "src/script/import_mylib.inc",  /* ③ the symbol table */
    "symbol_macro": "XS_MYLIB_SYMBOL"
  }
}

How the three pieces relate: sources compile into the host (the implementation lives in the process), headers enter the VFS (scripts can include the declarations), the symbol table injects into TCC (script calls resolve by address to the implementation). The macro XS_USE_MYLIB is predefined inside nested compilers — scripts can #ifdef-probe the extension's availability. Single-header libraries (stb style) use the defines field to carry the implementation switch (the xtp2 entry is the XTP2_IMPLEMENTATION exemplar).

39.3 Registry validation

The registry (xs_extensions.py) defensively checks every input; a bad registration explodes at build time:

CheckMisfortune prevented
source_path must exist inside the projectpath escapes (pulling in unintended files outside the repository)
feature macros deduplicated globallytwo extensions grabbing the same macro name
VFS header names forbid .. and are deduplicated across extensionsheaders overwriting each other / path traversal
field whitelistmisspelled fields silently ignored
macro/library name format validationgenerating illegal code

These checks have dedicated unit tests (one of test_extensions.py's 10 cases is exactly "adding a demo entry takes effect, zero code branches").

39.4 The two test tiers

  • Plan tier (test_extensions.py): registry parsing, build plans, symbol-table/header coupling, publish-failure protection — pure logic, seconds
  • Runtime tier (test_extensions_runtime.py): embedded C probes running inside a genuinely built XServer — actually calling extension APIs, extensions usable inside nested TCC too, and the reverse check "with the extension unselected, headers and symbols are truly absent" (compile failure + a named error)

When writing an extension for your library, add both tiers: plan-level keeps the registration sound; runtime-level proves "compiled in means really usable, left out means really absent".

39.5 Contract

  • Declaration: a new library is the three-piece set plus one json entry, zero host changes.
  • Consistency: the VFS header, the symbol table and the host sources must share one version — a symbol table listing functions absent from the header is a time bomb.
  • Testing: the two tiers are the extension's acceptance definition.

Common mistakes

The symbol table out of sync with the header
Symptom
Script linking fails (header without symbols) or runtime jumps to wild addresses (symbols with a stale header).
Cause
The library was upgraded and only one of the two was regenerated. Make symbol-table generation scripted, changing together with the header.
Forgetting the symbol macro wrapper
Symptom
The symbol table .inc breaks host compilation on non-extension builds.
Cause
Import tables must be wrapped in symbol_macro (compare import_sqlite.inc's XS_SQLITE_SYMBOL style) — extension symbols compile into the host only in selected variants.

Exercises

BasicRegistering a toy library

Write a single-header add(int,int) library, register it per the three pieces and confirm with --list-extensions.

Acceptance: build.bat mylib succeeds; a script calling add gets the right result.
AdvancedAvailability probing

Write a script using #ifdef XS_USE_MYLIB to provide two implementations (library when present, builtin otherwise); both variants compile.

Acceptance: default and mylib variants behave identically (from different sources).
ChallengeCompleting both test tiers

Add plan-tier + runtime-tier tests for your library, including the reverse "unavailable when unselected" assertion.

Acceptance: both positive and negative assertions pass; deliberately breaking the registry gets caught by the plan tier.

Summary

PointConclusion
Mechanisma new library = three pieces + one json entry, zero host changes
The three piecessources (compiled in) + headers (into VFS) + symbol table (injected into TCC)
Validationpath escapes / macro clashes / header overwrites / field whitelist
Testingplan tier + runtime tier (including the reverse unavailability assertion)
API references in this chapter
xrt further reading
  • xrt reference · core — the base type conventions shared when extension sources compile into the host