v5 Capstone: full pipeline to executable program

You now have all major compiler stages connected:

• lexer
• parser
• semantic analyzer
• module code generator
• linker
• bytecode serializer/deserializer
• VM execution

This module is about pipeline correctness with typed, multi-module boundaries.

1) Ownership contract per stage

Each stage has one explicit responsibility:

• Lex: source text -> tokens
• Parse: tokens -> module-structured AST (Program -> Modules)
• Analyze: semantic + module graph + type rules -> diagnostics or valid decorated AST
• CompileModules: modules -> module bytecode artifacts
• Link: module artifacts -> one merged executable bytecode
• Encode/Decode: bytecode <-> stable artifact
• VM.Run: execute merged bytecode -> final value

No stage should short-circuit this ownership:

• parser must not inject linker assumptions
• analyzer must not emit bytecode
• compiler must not parse source
• VM should not read AST/source

That split is what prevents cross-stage regressions.

2) Deterministic failure map

In this pack, error ordering is part of the design:

1. lex/parse errors
2. semantic errors (analyze)
3. compile/link/encode/decode failures
4. VM/runtime failures

analyze may compute many diagnostics internally, but CompileAndRun returns one deterministic analyzer error for this pack.

This is intentionally strict so tests compare exact messages, and learners debug one stage at a time.

3) Why encode/decode is kept inside CompileAndRun

A compiler-only path can pass tests even when the artifact boundary is broken. CompileAndRun should therefore:

• compile modules
• link
• encode
• decode
• only then run

The decode step catches:

• malformed encoding
• contract mismatch between compiler and runtime readers
• portability issues if external tools consume artifacts

So this extra step is a correctness guardrail, not redundant work.

4) Multi-module trace and expected checkpoints

Example flow:

module math { export fn add(a, b) { return a + b; } } module main { import math; let x = math.add(1, 2); }

Execution trace should be deterministic:

1. lexer
2. parse -> Program{Modules:[math, main]}
3. analyze -> validates import, export, order, signatures
4. compileModules -> bytecode artifacts per module
5. link -> rebases closures/functions, merges MAINs
6. encode
7. decode
8. vm run -> final value

If behavior diverges, inspect the earliest differing stage before touching VM.

5) Determinism and artifact stability

CompileAndRun should preserve these to keep snapshots stable:

• module order
• export declaration order
• import and symbol resolution order
• linker rebasing sequence

Avoid unstable iteration over maps/slices in order-sensitive logic.

A stable pipeline means:

• same source -> same encode text
• same encode text -> same decode artifact
• same decode artifact -> same runtime result

6) Runtime contract and isolation

A fresh VM run should be deterministic per invocation:

• one entry point from linked main
• same stack behavior for same bytecode
• no leaked globals from prior runs

If results differ across runs, check:

• hidden global state in VM constructor
• stale module tables
• non-idempotent artifact side effects

7) Practice checkpoints (almost solved)

1) Stage precedence + Program with both parse and semantic errors should return parse error first.

2) Analyzer gate + If analyzer emits unknown export: math.x, capstone should return that and not compile/link/run.

3) Round-trip check + CompileModules -> Link -> Encode -> Decode failures should surface before VM execution.

4) Deterministic run shape + math -> main module order must remain stable; closure IDs should be rewritten deterministically.

5) Main as execution entry + main module main code and linked main sequence define the only top-level execution path.

6) Clean output expectation + Successful run returns only runtime value and optional runtime error.

7) Non-execution boundary failures + Unknown module, forward import, duplicate exports, or arity mismatch should be compile-time or linker errors, not runtime crashes.

This stage is the proving ground of architecture discipline, not isolated implementation tricks.