Request bodies: incremental reads, fixed and chunked

Overview

16.1 Intro

Bodies arrive three ways: fixed-length (Content-Length), chunked, and ends-at-connection-close (no length; rare and dangerous). They share one read interface — the pReq->body state machine plus xrtHttp1BodyRead. You don't care about the encoding: the machine unifies all three into "how much, and is there more".

16.2 The framing state machine

Key insight: the HTTP/1. parser rejects conflicting length declarations (Content-Length together with Transfer-Encoding is a protocol error) — one family among 35 fine-grained error codes; malformed input is stopped before reaching your business.

16.3 The read-loop pattern

/* Read the whole body into a dynamic buffer (small bodies).
 * Large bodies should be processed as they arrive — see the challenge. */
static bool ReadBodyAll(XS_HttpReq* pReq, xbuffer** ppOut)
{
    xbuffer* pBuf = xrtBufferCreate(...);
    char arrChunk[8192];

    for ( ;; ) {
        xhttp1bodystatus st = xrtHttp1BodyRead(
            pReq->body, arrChunk, sizeof(arrChunk), &iGot);
        if ( st == XHTTP1_BODY_ERROR ) {
            xrtBufferDestroy(pBuf);
            return false;          /* malformed input */
        }
        if ( iGot > 0 ) xrtBufferAppend(pBuf, arrChunk, iGot);
        if ( st == XHTTP1_BODY_DONE ) break;
        if ( iGot == 0 ) {
            /* nothing new this round: real code re-enters on events */
        }
    }
    *ppOut = pBuf;
    return true;
}

What just happened: reads are incremental — each call hands you "what's there now"; if data hasn't fully arrived you keep waiting for events (re-entering the callback on the same connection's later driver rounds). That's why large uploads never need the whole body in memory: process chunk by chunk as they come.

Now chapter 4's /echo can be honored: POST → read the whole body → return it verbatim with chapter 15's ReplyRaw (echoing the request's Content-Type).

16.4 Completeness probes & 413

• body_limit (chapter 9's Custom knob): exceeding it yields 413 — the defense sits in the driver layer; your callback never even fires
• Completeness probe: the driver offers a check that consumes nothing from the transport buffer — telling whether the body has fully arrived, so you never respond on a half-body
• Malformed chunked: bad chunk lines, illegal trailers, oversized chunk lines are parser error codes; the connection closes with 400
• Slow-loris: idle_timeout is the time defense; byte-dribbling connections close on expiry

16.5 Contract

• Reading: bodies are consumed only through BodyRead; the driver cross-checks your consumption against the wire — a mismatch severs the connection.
• Completion: before XS_OK, either finish the body or explicitly hand it back for draining — returning XS_OK half-read breaks keep-alive.
• Memory: process large bodies as they arrive; don't default to ReadBodyAll.

Common mistakes

Exercises