Parsing Programs: declarations, statements, and statement-form discrimination

v2 turns an expression parser into a program parser with real statement boundaries.

Problem model at this stage

At this point, parser has enough primitives for expressions and arithmetic. The gap is:

• decide what a top-level declaration is,
• distinguish assignment from expression statement,
• and support blocks and control flow at statement scope.

This stage is about grammar partitioning, not deep semantic validation.

Grammar

program    -> decl* EOF
decl       -> funDecl | stmt

funDecl    -> "fn" IDENT "(" params? ")" block
params     -> IDENT ("," IDENT)*

stmt       -> varDecl
           | ifStmt
           | whileStmt
           | returnStmt
           | block
           | assignStmt
           | exprStmt

varDecl    -> ("let" | "var" | "const") IDENT ("=" expr)? ";"
assignStmt -> IDENT "=" expr ";"
exprStmt   -> expr ";"
block      -> "{" stmt* "}"

ifStmt     -> "if" "(" expr ")" block ("else" block)?
whileStmt  -> "while" "(" expr ")" block
returnStmt -> "return" expr? ";"

Decision points in parseDecl / parseStmt

1) Top-level dispatch

• parseDecl checks for fn first.
• If token is fn, parse a function declaration and do not fall through to statement parsing.
• Every other token at top-level becomes a statement.

2) Statement dispatch inside blocks and top-level

• tokens let/var/const -> declaration
• if, while, return -> dedicated parsers
• { -> recursive block parser
• fn does not appear here for statements in this stage (functions are top-level only in v2)
• default path: treat as expression/assignment disambiguation

3) Assignment disambiguation trick

• if current token is identifier and next token is = => assignment
• otherwise parse as expression statement

This avoids ambiguous grammar and keeps one-pass parsing deterministic.

Expression-call shape at this stage

Call syntax is still limited to direct identifier calls:

• parser recognizes call form while parsing an identifier primary
• ExprCall.Value stores callee name and parsed arg list

Meaning:

• f() is parsed as call expression
• obj.f() and (f())() are not yet valid here (member/index support appears later)

Function declaration constraints

parseFnDecl order is strict:

1. consume fn
2. parse function name
3. parse params with parentheses
4. parse function body block

No return type or annotations yet in this stage.

Block parsing invariants

• parse block must start with {
• collect statements until } or EOF
• missing } should fail immediately with a clear error
• nested blocks are parsed recursively through parseStmt

Error hotspots

• declaration messages should point to declaration boundary:
  • expected function name
  • expected '(' after function name
  • expected ';' after declaration
• assignment shape messages:
  • expected '=' in assignment
• return/condition shape:
  • expected ')' after condition/expression
• block boundary messages:
  • expected '{'
  • expected '}' after block

Deep practice checkpoints (almost solved)

1) Top-level function dispatch

Input: `fn add(a, b) { return a + b; } let x = 1;`

Expected:

• first node is function declaration statement
• second node is declaration statement

2) if / while parse in statement form

Input: if (true) { while (x) { return; } }

Expected:

• nested control-flow nodes as statements

3) Return with and without expression

return 1 + 2; return;

Expected:

• both parse as return statements

4) Function placement restriction

x = fn() {}

Expected:

• parse expression statement/assignment path, not top-level function declaration
• if invalid as expression, should be caught in expression parsing

5) Assignment vs expression disambiguation

Input: a = b + 1;

Input: f(a, b);

Expected:

• first parsed as assignment statement
• second parsed as expression statement containing call expression

6) Block recursion

Input: { { return 1; } }

Expected:

• outer block contains inner block as a statement

7) Function parameter edge cases

Input: fn f(a, b, c) {}

Expected:

• 3 parameters in order

Input: fn f(,b) {}

Expected:

• parse error for parameter syntax

8) Missing semicolons

`let x = 1 let y = 2;`

Expected:

• parse error after first declaration due to missing ';'

9) Call argument lists

print(a, b, c + d);

Expected:

• parses call expression with three args, preserving nested expression in third arg