http
Package http.
HTTP/1.1 over link / wire / until / fault. Parse a request from bytes, or read from a connection into a caller-sized wire (the max request size). read takes the unconsumed prefix length and returns Incoming with leftover compacted to the front of the wire, so one connection can carry many requests. write serializes a Response through an arena; write_head answers a HEAD request with the head write would send -- Content-Length the length of the body GET would carry, as RFC 9110 has it -- and no body. A request keeps its header block as it arrived; header(req, name) looks it up case-insensitively. A body is framed by Content-Length or by Transfer-Encoding: chunked (chunk extensions ignored, trailers read and discarded). wants_close follows HTTP/1.1 keep-alive (and HTTP/1.0 close-by-default). A request with no header lines at all (GET / HTTP/1.0 and a blank line) is accepted.
read is the hot path, so it allocates only what the request it returns holds: the path, one copy of the header block, the body when there is one, and the Request/Incoming around them. The common methods (GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS) and both versions come back as literals, and wants_close decides from the header block in place. A plain GET costs 6 GC allocations; tests/run_tests.sh holds it to that.
Framing decides where a request ENDS, so it is a security boundary: if a proxy in front and this server frame the same bytes differently, the leftover is read as a second request the proxy never saw (request smuggling). read and parse therefore refuse, rather than guess at:
Transfer-Encodingtogether withContent-Length, or in an HTTP/1.0 request;- any coding but exactly
chunked(no lists such asgzip, chunked); - a repeated
Content-LengthorTransfer-Encodingheader; - a
Content-Lengthlonger than 18 digits, or not all digits; - chunk-size lines over 1KB, sizes over 15 hex digits, bare LFs, chunk data not followed by CRLF, or trailers over 8KB.
- a method that is not an RFC 9110 token, so a quote, a backslash or a control byte such as ESC never reaches a handler or a log line inside the method.
After read returns an error, close the connection (the examples all return). The error means this server could not tell where the request ended, so it cannot tell where the next one begins either.
The writing side holds the same line. A CR or LF inside a header field would end the line early, so what follows would become a header of its own or the start of the body: response splitting, when a handler copies request data into a header. write and serialize therefore write every field (a with_header value, a raw with_headers line, location, the content type, the status text) with CR and LF turned into spaces, the rule Go's net/http applies. A clean field is written as it is, at no extra cost. httpc does the same for the method and the request headers it sends, and percent-encodes CR, LF, space and tab in the request target. http2's header encoder does the same for HTTP/2 fields.
Every response carries Date (RFC 9110 6.6.1): the current second, kept by the runtime for all threads and copied into the response with no clock read and no allocation; a handler's own Date is the one sent. A 1xx, 204 or 304 response is written with no Content-Length and no body, whatever the Response holds (8.6 forbids the header on 1xx and 204, and on a 304 unless it equals the 200's; body bytes there would be read as the start of the next response). http2 follows the same rules. A prebuilt static rendering (static_render) ends its header block with static_date_line(), whose date write_static refreshes on every send; a rendering built elsewhere gets the same by ending its headers with that line.
import "http";
fn serve(c: link) {
let ra = arena_new(16384);
let sa = arena_new(16384);
let buf = ra.wire(8192);
let filled = 0;
while true {
let rr = http.read(&mut c, buf, filled, until_never());
guard let got = rr else { return; }
let wr = http.write(&mut c, http.ok_text(got.req.path), &mut sa,
until_never());
guard let _n = wr else { return; }
sa.reset();
if http.wants_close(got.req) { return; }
filled = got.filled;
}
}See examples/httpd/ for a listener loop on this package.
In a program that asks proc.shutdown_requested(), every thread starts with SIGTERM/SIGINT blocked, and one dedicated thread receives them with sigwait — so no signal lands on a worker thread mid-request, and whichever thread a blocked accept() is parked on, the runtime wakes it. There's a narrow startup race inherent to this: a signal that arrives in the brief window before main() blocks the signals gets the OS's default disposition (immediate termination) instead of graceful handling, same as any signal-handling program.
API#
raw_headers is exactly the CRLF-separated "name: value" lines this request's header block was made of, unparsed -- not a map[str]str. Building a map costs an allocation for the map plus roughly two more per header, on every request, whether or not anything ever reads a header; most requests never do. header/header_or scan this directly (strings.find_field, no allocation for the search itself); headers() builds a map from it on demand for a caller that genuinely wants one. Constructed only by parse/read (off the wire) or request (validated, from a map) -- never assign this field directly from unvalidated bytes; a value containing "\r\n" inside a hand-built block is a second header no map ever held.
read couples two things that only sometimes belong together: filling the caller's buffer and framing what's in it. serve_conn needs the split for frame dispatch: read_frame frames with the same scans/errors/rules as read, but returns the parsed head (method/path strs, header block, body) instead of a Request -- so the router matches on strs with no raw copy, no rescan, no second slice. Offsets stay for Frame callers and tests; the serve path does not use them.
The request WITHOUT materialising method/path/version strs: framing offsets only. read/serve use this when they only route on the raw path bytes (see path_matches below); parse keeps returning strs for callers that actually need them.
header()'s two real call sites (Ctx.header in zokor, the WebSocket handshake) are both header(...) ?? fallback already -- this is that, without the opt allocation ?? unwraps.
Every header a request carries, as a map -- built fresh on each call by scanning raw_headers once. Nothing in either repo iterates a request's headers today (confirmed by grep), which is what makes "built on demand" the right default over a cached field: a cache nothing reads is pure cost. Walks the same line shape scan_headers does, for a different reason (that one validates and extracts two offsets; this one assumes an already-valid block and extracts every name and value) -- kept as two functions rather than one parameterized by what to do with each line, which would obscure both to save repeating six lines.
fn request(method: str, path: str, version: str, headers: map[str]str, body: bytes) -> result[Request, str] stdlib/http/http.sl:1465#Builds a Request from a map the way the eager parser builds one off the wire: a header name or value may not contain CR or LF (an application- or test-constructed header gets no such check for free the way one read off the wire does -- this is where that rule lives for this path), and a name may not contain the colon that separates it from its value on the wire. Both are a real hazard, not a theoretical one, now that raw_headers is what a request's headers actually are: a value containing "\r\n" would be concatenated straight into the block below as a SECOND header line -- injection -- if it weren't rejected first.
Framing offsets for one request inside raw, with NOTHING materialised: no method/path/version strs, no header copies. The request line is validated (two spaces, HTTP/1.x token); the header block is validated by scan_headers; the body is framed by frame(). Callers that need strs (logging, tests) use parse(); the server path matches/routes directly on these offsets -- see path_matches and method_is below -- so the ~10 framing strs per request never exist.
Is this frame's method exactly want (e.g. "GET"), compared in place -- no method str is ever built. Takes a line_end int (not a Frame) so both Frame and WireFrame callers share it.
Is this frame's path exactly want (bytes compared in place, query string ignored -- matches path.strip_query semantics)? No path str. The two hot paths (/ and /users/-prefixed) compare inline with no to_bytes at all; anything else falls through to the general compare, which still costs one to_bytes of want. Route fpaths could precompute that bytes once at registration -- the next step if /users/:id still shows hot after this.
The :id in "/users/:id" for a frame already known to match that shape -- the ONE allocation the params map used to cost per route scanned. Returns "" when the shape does not match.
The method bytes as a str: the ONE framing str frame dispatch needs (to map to the Method enum). Everything else matches in place; this is 1 alloc where parse() spent ~10. Takes line_end so Frame and WireFrame share it.
The full Request for a frame, built ONCE for the route that runs: method/path/version strs, raw_headers slice, body slice. serve() callers already hold one; serve_frame builds one -- never both. Takes explicit offsets so Frame and WireFrame share it.
fn frame_request_at(raw: bytes, line_end: int, head_end: int, body: bytes) -> Request stdlib/http/http.sl:1786#The Request for a WireFrame: method/path already parsed, body already sliced -- no offsets, no raw, no second copy. headers rides along (one block copy, same as read -- the wire is compacted before return, so there is nothing to materialise from later). version comes from the flag (no version str is stored).
The old builder path, kept for the differential test: must stay byte-identical to serialize_sized() above for every shape.
Serialize with a handful of allocations, not ~40: size the response first (status line + headers + framing + body, all cheap integer arithmetic over lengths -- see emit_len below), allocate exactly that many bytes with strings.bytes_zero (one header plus one buffer -- no throwaway str), then fill by slice assignment. The old builder path stays below as serialize_builder for the differential test; this is what serialize and write's fallback call, so an oversized response costs a few allocations instead of ~40. The remaining handful is load-bearing, not waste: one to_bytes per fill_str piece (the str->bytes copy the compiler cannot fuse), one filtered-extra list, and the response_conn scan's own line strs. Counting them is what keeps this honest -- see the probe notes.
The escaped form of msg as bytes, for a caller that already holds bytes and wants the quoted payload without a str in between.
Whether the connection should close after answering r: HTTP/1.1 stays open unless the client sent Connection: close, HTTP/1.0 closes unless it sent Connection: keep-alive (either case-insensitive, OWS trimmed; with the header repeated, the one header() returns decides). Allocates nothing.
The int-flag twin, for callers holding a version str rather than a Head (read_frame's WireFrame.version). Same rules; the block is passed explicitly because there is no Head in scope.
fn read_frame(c: &mut link, buf: wire, filled: int, idle_deadline: until, header_deadline: until, body_deadline: until) -> result[WireFrame, str] stdlib/http/http.sl:2755#read_frame: read's framing, plus the parsed head the router matches on. Implemented INSIDE read's loop shape (not as read-plus-copy): read compacts the buffer before returning, so after read there is no head left to copy -- buf[0..filled] is the NEXT request. This duplicates read's loop deliberately (same scans, same errors, same close rules), and the duplication is the contract: any change to read's framing must land here too. What differs from read: on a complete frame the method/path strs and the header block + body are COPIED OUT of the wire ONCE -- the same strs + copies read already pays (hd.method/path, hd.headers, hd.body) -- and NO separate message copy is made. The old shape did to_bytes(buf[0..end]) PLUS parse_frame over the copy PLUS wants_close_scan over the copy: a full second framing pass and a whole-message copy per request. Now the WireFrame carries what the router needs directly: method/path strs for the enum + Request build, headers/body slices for the Ctx. Routing matches on strs (==), never on raw bytes -- no path_is_at rescan, no per-route to_bytes. The message copy is gone entirely. Chunked bodies: hd.body IS the reassembled copy (one alloc the old path also spent); the WireFrame body is that copy directly. Three deadlines, not one, because "how long should this wait" has three different honest answers depending on what the connection is doing: idle_deadline while nothing has arrived at all yet (a kept- alive connection between requests, or a fresh one that never sends anything -- this is the one that should be generous, since a real client legitimately sits idle between requests); header_deadline once bytes have started arriving but the request line and headers aren't complete yet; body_deadline once the headers are in but a declared body hasn't fully arrived. header_deadline and body_deadline are both the tight, slow-loris-shaped ones -- a client that has started a request and then trickles it in a byte at a time is not the same risk as one that hasn't sent anything yet, and idle_deadline being long must not become an amplifier for that. Recomputed fresh every loop iteration rather than tracked as a separate flag: frame_head_wire's own re-parse of the same (only ever growing) buffer already answers "is the header complete yet" idempotently, so re-deriving which phase this recv is in from that costs nothing extra and can't drift out of sync with it.
fn read(c: &mut link, buf: wire, filled: int, deadline: until) -> result[Incoming, str] stdlib/http/http.sl:2836#serialize()'s GC allocations (~40 of them for a typical response, see serialize()'s own comment) replaced with two passes over the caller's own arena: size, then fill. If the response is larger than what's left of the arena, falls back to serialize() + send_bytes rather than letting a.wire(need) past capacity kill the task -- a slow response stays a slow response instead of becoming a dropped connection. Responses with exactly the text_response shape (one content-type, nothing else) take the fixed fast path above: no map iteration, no integer str, no intermediate bytes. Anything else uses the general emit below, unchanged. A static route's answer: status, content type, and body -- no Response struct, no extra list, nothing to shape-check. serve_conn gets one from serve_static and hands it straight to write_static, which sends the PREBUILT keep-alive rendering (assembled once at registration -- see static_render below) with one wire alloc and one memcpy. The static snapshot lives on the Route (see router.sl); this is just the per-request view of it.
fn write_static(c: &mut link, b: StaticBody, a: &mut arena, close: bool, deadline: until) -> result[int, fault] stdlib/http/http.sl:3211#The static emit: the keep-alive rendering is prebuilt, not emitted. Hot path (HTTP/1.1 keep-alive, which is every wrk/ab request): one wire, one memcpy, one send -- no probe pass, no Response struct, no map, no integer str, no per-piece puts. The close variant (HTTP/1.0, Connection: close) is rare and pays the emit cost below; it never touches the hot path.
The keep-alive rendering, assembled ONCE at registration: status line, one content-type, content-length, connection, blank line, body -- the exact bytes write_static memcpys per request. status_text_of covers the statuses static routes return; the fallback ("OK") matches write_static's own close variant above, so the two can never disagree on a reason phrase.
The last header line of a prebuilt static rendering: date: and a date, refreshed in place on every send. A rendering is shared by every request, so the date cannot be part of it; a renderer outside this package (zokor's, with its security headers) ends its header block with this line to get the same. Any other rendering is sent as it is.
fn write(c: &mut link, r0: Response, a: &mut arena, deadline: until) -> result[int, fault] stdlib/http/http.sl:3393#fn write_head(c: &mut link, r0: Response, a: &mut arena, deadline: until) -> result[int, fault] stdlib/http/http.sl:3401#The answer to a HEAD request: the head write would send for r0 -- Content-Length the length of its body -- and no body (RFC 9110 9.3.2: HEAD's Content-Length is the one GET would carry). Give it the response GET would produce, body and all.
fn text_response_bytes(status: i32, status_text: str, content_type: str, body: bytes) -> Response stdlib/http/http.sl:3445#text_response with a BYTES body: same shape, no to_bytes copy. The hot path (zokor's JSON renderers, static bodies) already holds bytes; forcing them through str and back cost a full copy plus the literal's own allocation on every response.
fn text_response(status: i32, status_text: str, content_type: str, body: str) -> Response stdlib/http/http.sl:3458#bad_request with a BYTES message: same envelope, no to_bytes round-trip. The message still rides inside a JSON string, so it is escaped the same way -- see escape_json_into below.