The xllm extension: LLM calls from your scripts

Overview

34.1 Position and boundaries

xllm (vendored from xrt extlibs, core layer v3.0.0) absorbs all the dirty work of "talking to LLM services":

• Three dialects: Chat Completions (OpenAI-compatible / GLM), OpenAI Responses, Anthropic Messages — one request API, the eProvider enum switches wire formats
• SSE streaming: framing, reassembly, incremental callbacks; non-streaming is just the no-callback special case
• Retries and rate limits: retryable errors back off automatically, Retry-After is respected; a diagnostics struct tells you how many attempts ran
• Cancellation: wired into the xrt cancellation system (xcancel), with per-request deadlines

The boundaries are equally explicit: it does not execute tools (tool calls come back to you — execution is your business) and keeps no conversation history (each request is stateless; history and compaction belong to layers above — xrt's xllm-session/xllm-memory are not part of this variant). Networking, TLS, JSON and cancellation all run on the host's built-in xrt; scripts link no extra runtime.

34.2 Client and model profiles

The first step is always the model profile — a non-secret capability contract: context window, input/output caps, capability bits (tool calls, streaming, image input…). The GLM family ships built in, borrowed by pointer, immutable:

xllm_error tErr;
const xllm_model_profile* pProfile;
xllm_client_config tConf;
xllm_client* pClient;

xllmErrorInit(&tErr);
pProfile = xllmModelProfileBuiltin("glm-5.2");      /* borrowed, not copied */
if ( pProfile == NULL ) { /* wrong name: NULL */ ... }

xllmClientConfigInit(&tConf);                      /* all defaults */
tConf.sBaseUrl      = "https://open.bigmodel.cn/api/paas/v4";
tConf.sApiKey       = getenv("XLLM_KEY");           /* borrowed pointer! production: xs.json Custom */
tConf.sModel        = "glm-5.2";
tConf.eProvider     = XLLM_PROVIDER_GLM;
tConf.pModelProfile = pProfile;                     /* with a profile: validated + snapshotted */
pClient = xllmClientCreate(&tConf, &tErr);          /* engine owned privately by default */

Three points: all strings are borrowed (the library does not copy them; their lifetime must cover the client); the engine can be shared (fill pNetEngine with the host/app engine and multiple clients share it; NULL means the library owns one); profiles pay off three ways — config consistency is checked at creation, capabilities are checked before each request (a vision request against a text-only model is rejected), and xllmModelProfileSupports(pProfile, XLLM_CAP_*) drives business branching.

34.3 Requests and calls

A request is a stack value; construction touches no network. Messages come in four roles — system / user / assistant / tool; text is one API call, multimodal goes through parts:

xllm_request tReq;
char* sJson;

xllmRequestInit(&tReq);
xllmRequestSetModel(&tReq, "glm-5.2");
xllmRequestAddTextMessage(&tReq, XLLM_ROLE_SYSTEM, "You are a rigorous assistant.");
xllmRequestAddTextMessage(&tReq, XLLM_ROLE_USER, "Describe XServer in one sentence.");
printf("tokens: %llu\n",
    (unsigned long long)xllmEstimateTextTokens("Describe XServer in one sentence."));

sJson = xllmClientBuildRequestJson(pClient, &tReq, &tErr);
/* {"model":"glm-5.2","messages":[{"role":"system",...},...]} */
xllmFree(sJson);                                   /* anything the library allocated: xllmFree */
xllmRequestUnit(&tReq);

Two calling paths: the one-shot xllmClientComplete (retries and stream reassembly inside; returns the full response); async xllmClientStart → a borrowed xfuture* → xllmCallWait / xllmCallCancel. Tool calls: the request side declares tools with xllmRequestAddTool; the response side hands tool calls to your business code, and results flow back via xllmRequestAddToolResult — execution always stays in your hands.

The typical split inside xs: an HTTP route callback receives the request → Complete runs on a background thread (the threading discipline of ch. 41) → the result is written back. Never run blocking Complete inside an engine worker callback.

34.4 Dialects, streaming & retries

Streaming: both xllmClientComplete/Start accept xllm_stream_callbacks (text deltas, reasoning deltas, tool-call deltas, finish); NULL means non-streaming. Retries are governed by uMaxAttempts / uRetryBaseDelayMs / uRetryMaxDelayMs; rate-limited responses honor the server's Retry-After. The xllm_diagnostics in the result (attempt count, final retryability) plus xllmErrorRetryable(&tErr) decide whether the business side should wait and try again. Cancellation comes in two flavors: per request via xllmRequestSetDeadline / SetCancel (xcancel), and per call via xllmCallCancel.

34.5 The contract

Threading: a client is usable across threads (connection pool and locks inside), but a single xllm_call is not concurrent; the future is borrowed — valid until Wait/Destroy. Variant isolation follows the sqlite/xtp2 law: the default variant has no <xllm.h> (a compile-time missing-header error), scripts probe with #ifdef XS_USE_XLLM, and nested TCC environments see the same.

Common mistakes

Exercises