How it works

The compiler pipeline, the collector, the scheduler.

How it works

main.sl ──loader──> packages ──lexer/parser──> ASTs ──codegen──> main.gen.c ──cc──> ./main

Loader (src/loader.c) — resolves imports (local directory, native package, stdlib, then slang.project pins), scans package directories for .sl files (in deterministic sorted order), merges them per package, and detects cycles via canonical paths.
Lexer (src/lexer.c) — tokenizes source into identifiers, keywords, literals, and operators.
Parser (src/parser.c) — recursive-descent parser producing an AST (src/ast.h).
Code generator (src/codegen/) — walks the ASTs, performs type inference and semantic checks (including pub enforcement), and emits readable C. The runtime in runtime/ (GC, scheduler, pool, containers, native packages) is real C, spliced into every generated file so the binary stays self-contained. Split by concern: core.c, infer.c, expr.c/stmt.c, program.c, liveness.c (GC safepoint roots), and native.c (NatSig dispatch). internal.h holds the shared CG struct.

Native-package signatures live under src/codegen/pkg_<name>/. Their C runtimes are runtime/sl_<name>.c. json uses dispatch.c because decode/encode are generic over the target type. Adding a fixed-signature package is a pkg_<name>/ directory, a runtime/sl_<name>.c file, and one line in loader.c.

Driver (src/main.c) — glues it together and shells out to cc. Because GCC/Clang compile the generated C, you get their full optimizer for free.

Large stack frames. A task starts on an 8KB stack that grows only at a safepoint, which runs after the current function's frame already exists. How big a frame is depends on the C compiler: clang gives every call site its own spill slot and inlines callees into their caller, so a function with a few hundred call sites can need more than the whole stack and used to die with SIGBUS before printing anything (gcc gave the same C a frame of a few hundred bytes). The driver therefore asks the compiler for the real frame of every function (-Wframe-larger-than, read from clang's and gcc's own reports). A slang function reported over the limit is regenerated behind a thin wrapper that grows the stack before the function is entered, and the program is compiled a second time. So is the C slangc generates for json.decode/json.encode, pg.rows_as and inspect, whose frames grow with the type: a decoder for a 200-field struct needs more stack than a safepoint leaves free, and without a guard the task ran into its guard page. The guards a build ends with are kept in the cache (~/.cache/slang/frames/, or $SLANG_CACHE/frames/, one small file per program) and applied up front the next time the same program is built, so a rebuild compiles once; every compile still checks the frames, and one that has outgrown its guard is regenerated as before. A guard is sized in whole 4KB pages, as the stack grows, so a frame that changes by a few bytes costs no compile. Programs with no large frame pay nothing, and --emit-c output has no guards. The limit is 1536 bytes; SLANG_FRAME_LIMIT overrides it (the tests set a tiny one to put a guard on nearly every function).

Development builds. --dev (or SLANG_DEV=1 in the environment) compiles the generated C at -O0 without LTO, which is most of a build's time: a 21k-line service rebuilds in about 16 s instead of 73 s. The program runs slower (about 1.6x the CPU per request on that service), so it is for the edit-and-rerun loop -- zokor watch builds this way -- and never for what you deploy. Unoptimized frames are several times bigger, so a dev build also starts tasks on a 32KB stack with a 16KB safepoint margin, guards frames over 8KB, and is cached apart from the release build of the same program.

Inspect what slang generates:

./slangc examples/hello/main.sl --emit-c && cat main.gen.c

Memory management

Compiled programs embed a precise mark-sweep collector (runtime/sl_gc.c). Allocations go through sl_gc_alloc. main() registers the thread, starts the worker pool, and switches into the main task. There is no libgc dependency.

What this means in practice:

No manual memory management in slang; no leaks from string churn.
Collection is tracing (mark-and-sweep), so reference cycles are collected — unlike refcounting.
Cost: stop-the-world pauses. The allocator still serializes on a mutex (batched); that is the current throughput ceiling.
Every pool worker is registered with the collector. A collection stops the world, walks safepoint roots, the run queue, parked tasks, and (for async-preempted tasks) a conservative stack scan.
Pacing follows the live heap, like Go's default (GOGC=100): the next collection comes after allocating as much as survived the last one, and never before 8MB. The heap peaks near twice what is live, however much garbage a program makes — streaming three million database rows runs in under 20MB.
Generational and non-moving: new objects start in a nursery, and a minor collection runs after every 512KB of allocation. The nursery grows, up to 2MB per worker (16MB at most), while minors take a large share of the time and keep finding much of it live -- a busy server re-marking its in-flight requests -- and shrinks back when they do not, so a program whose garbage dies young keeps the small footprint. It marks from the roots and the remembered set (old objects a write barrier saw gain a pointer to a young one), sweeps only the nursery, and promotes what survives two minors: data that is merely in use when a minor lands, like a request half way through, stays young and dies young instead of being left for a major. Old objects are never traced by a minor, so its cost follows the nursery, not the heap: about 0.8ms per minor whether a program keeps 20k or a million long-lived objects. Majors sweep everything and promote what they keep.
Both kinds sweep in parallel: the threads stopped for a collection take a share of the heap's pages each, instead of waiting while one thread sweeps them all. SLANG_GC_STAT counts the joins (sweep_helps).
A list or map whose elements hold no pointer -- numbers, bool, enums, and value structs made only of those -- is never traced at all: a table of a million ints costs a collection nothing to keep.
A list or map written since the last minor is traced from the first position written, not from the start, so filling a million-entry list or map costs each minor only what was added since the previous one. Updating an existing map key is the exception: its position is not recorded, so the next minor traces that whole map.
SLANG_GC_STAT=1 prints collection counts and pause times at exit (minor_pause_ns_total for the minors). SLANG_GC_THRESHOLD_KB=n collects every n KB instead, with no pacing: for tests, since a rooting bug only shows when a collection lands at the one safepoint where an object is unrooted. SLANG_GC_NURSERY_KB=n does the same for minors.
SLANG_GC_VERIFY_MINOR=1 checks every minor against a full mark and reports any live young object it missed, with the kind of old container holding it -- a store with no write barrier. It prints missed= at exit; the suite requires 0.

Project layout

src/
  common.h       allocation helpers, growable string buffer, file I/O
  loader.h/.c    package discovery, merging, cycle detection
  lexer.h/.c     tokenizer
  ast.h          AST node definitions
  parser.h/.c    recursive-descent parser
  rtpath.h/.c    locate runtime/ next to slangc
  codegen.h      public codegen API (codegen_program, and the frame guards)
  codegen/       type checking + C emission
  main.c         driver: flags, invokes cc
runtime/       real C runtime spliced into generated programs
  sl_core.c sl_gc.c sl_containers.c sl_sched.c sl_pool.c
  sl_time.c sl_net.c sl_tls.c sl_json.c sl_proc.c sl_fs.c
stdlib/        slang-source packages (`import "http"`, `import "byteutil"`;
             `import "log"` is a native package — no source files)
examples/      one directory per example program
tests/         language tests plus tests/runtime/ (no slangc)
Makefile       build/test/clean