feat: add lambda expression support to BAML compiler#3302
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds end-to-end lambda/closure support (parsing, CST→AST lowering, HIR capture analysis, TIR inference/checking, MIR lowering, bytecode/runtime closure & cell support), Array/Map map* builtin declarations, formatter updates, extensive tests, and VM/GC/emitter integration for captures. Changes
Sequence Diagram(s)sequenceDiagram
participant Src as Source
participant Parser as Parser
participant CST as CST
participant Lower as Lowering/AST
participant HIR as HIR/Scopes
participant TIR as TIR/Infer
participant MIR as MIR/Lower
participant Codegen as Codegen/Emit
participant VM as VM/Runtime
Src->>Parser: parse lambda syntax (<...>(params) -> { ... })
Parser->>CST: produce LAMBDA_EXPR node
CST->>Lower: lower_lambda_expr -> Expr::Lambda(FunctionDef)
Lower->>HIR: create lambda scope, register params
HIR->>TIR: infer_lambda_body / type-check lambda
TIR->>MIR: lower lambda into child MirFunction + Rvalue::MakeClosure
MIR->>Codegen: collect/compile nested lambdas, emit MakeClosure/MakeCell
Codegen->>VM: runtime allocates Closure/Cell and wires captures
Estimated Code Review Effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 19
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
baml_language/crates/baml_compiler_parser/src/parser.rs (1)
3562-3599: 🧹 Nitpick | 🔵 TrivialAdd parser-level tests for the new lambda disambiguation branches.
parse_primary_exprnow relies on custom(/<lookahead to choose between lambdas and non-lambda expressions. A few unit tests in this module for(x) -> {},<T>(x) -> {},(x), and nested parameter types would make regressions much easier to localize than end-to-end fixtures alone.As per coding guidelines,
**/*.rs: Prefer writing Rust unit tests over integration tests where possible.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 06170aa1-cc8f-4342-8db1-3a9d22fdd0b7
⛔ Files ignored due to path filters (37)
baml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/comment_after_string_in_config/baml_tests__comment_after_string_in_config__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/comment_in_type/baml_tests__comment_in_type__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/config_dictionary/baml_tests__config_dictionary__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/config_model_string/baml_tests__config_model_string__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/control_flow/baml_tests__control_flow__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/format_checks/baml_tests__format_checks__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/header_in_llm_function/baml_tests__header_in_llm_function__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__01_lexer__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__02_parser__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__05_diagnostics.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__10_formatter__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__01_lexer__lambda_basic.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__02_parser__lambda_basic.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__05_diagnostics.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__10_formatter__lambda_basic.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__01_lexer__lambda_errors.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__02_parser__lambda_errors.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__05_diagnostics.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__10_formatter__lambda_errors.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/o1_allowed_roles/baml_tests__o1_allowed_roles__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/retry_policy/baml_tests__retry_policy__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/src/compiler2_tir/snapshots/baml_tests__compiler2_tir__phase5__snapshot_baml_package_items.snapis excluded by!**/*.snap
📒 Files selected for processing (22)
baml_language/crates/baml_builtins2/baml_std/baml/containers.bamlbaml_language/crates/baml_compiler2_ast/src/ast.rsbaml_language/crates/baml_compiler2_ast/src/lower_cst.rsbaml_language/crates/baml_compiler2_ast/src/lower_expr_body.rsbaml_language/crates/baml_compiler2_hir/src/builder.rsbaml_language/crates/baml_compiler2_mir/src/lower.rsbaml_language/crates/baml_compiler2_tir/src/builder.rsbaml_language/crates/baml_compiler2_tir/src/infer_context.rsbaml_language/crates/baml_compiler2_tir/src/normalize.rsbaml_language/crates/baml_compiler_parser/src/parser.rsbaml_language/crates/baml_compiler_syntax/src/syntax_kind.rsbaml_language/crates/baml_fmt/src/ast/expressions.rsbaml_language/crates/baml_fmt/src/ast/tokens.rsbaml_language/crates/baml_fmt/src/lib.rsbaml_language/crates/baml_lsp2_actions/src/check.rsbaml_language/crates/baml_tests/projects/lambda_advanced/lambda_advanced.bamlbaml_language/crates/baml_tests/projects/lambda_basic/lambda_basic.bamlbaml_language/crates/baml_tests/projects/lambda_errors/lambda_errors.bamlbaml_language/crates/baml_tests/src/compiler2_tir/mod.rsbaml_language/crates/bex_vm/src/package_baml/array.rsbaml_language/crates/bex_vm/src/package_baml/map.rsbaml_language/crates/tools_onionskin/src/compiler.rs
baml_language/crates/baml_tests/projects/lambda_advanced/lambda_advanced.baml
Show resolved
Hide resolved
Binary size checks passed✅ 7 passed
Generated by |
Merging this PR will not alter performance
|
There was a problem hiding this comment.
Actionable comments posted: 12
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
baml_language/crates/bex_vm_types/src/types.rs (1)
809-817:⚠️ Potential issue | 🟠 Major
Object::Variantis still classified asenum.While touching
ObjectType::of(), the variant arm below still returnsSelf::Enum. That keeps runtime type checks and diagnostics wrong for actual enum values.🩹 Proposed fix
match ob { Object::Function(func) => Self::Function(FunctionType::from(&func.kind)), Object::Closure(_) => Self::Closure, Object::Cell(_) => Self::Cell, Object::Class(_) => Self::Class, Object::Instance(_) => Self::Instance, Object::Enum(_) => Self::Enum, - Object::Variant(_) => Self::Enum, + Object::Variant(_) => Self::Variant, Object::String(_) => Self::String, Object::Array(_) => Self::Array, Object::Map(_) => Self::Map,
♻️ Duplicate comments (2)
baml_language/crates/baml_tests/src/compiler2_tir/mod.rs (2)
253-284:⚠️ Potential issue | 🟡 MinorRender lambda
throwsclauses in signatures.
FunctionDef::throwsis still ignored here, so throwing lambdas snapshot exactly like non-throwing ones in both the TIR and HIR renderers.🩹 Proposed fix
let ret = func_def .return_type .as_ref() .map(|te| format!(" {}", type_expr_to_string(&te.expr))) .unwrap_or_default(); + let throws = func_def + .throws + .as_ref() + .map(|te| format!(" throws {}", type_expr_to_string(&te.expr))) + .unwrap_or_default(); let generics = if func_def.generic_params.is_empty() { String::new() } else { format!( "<{}>", @@ - format!("{generics}({}) ->{ret} {{ ... }}", params.join(", ")) + format!("{generics}({}) ->{ret}{throws} {{ ... }}", params.join(", "))
402-410:⚠️ Potential issue | 🟠 MajorKeep lambda body snapshots typed.
This path still drops into
render_expr_body_untyped, so inferred parameter, capture, and return types inside lambdas stop being asserted by the TIR snapshots. Please plumb the lambda scope’s ownScopeInferencehere instead of using the untyped fallback.Also applies to: 447-497
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3ee3e52f-6c8a-40e2-9abe-9f140b3c3833
⛔ Files ignored due to path filters (10)
baml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__06_codegen.snapis excluded by!**/*.snap
📒 Files selected for processing (29)
baml_language/crates/baml_compiler2_emit/src/analysis.rsbaml_language/crates/baml_compiler2_emit/src/emit.rsbaml_language/crates/baml_compiler2_emit/src/lib.rsbaml_language/crates/baml_compiler2_emit/src/pull_semantics.rsbaml_language/crates/baml_compiler2_emit/src/stack_carry.rsbaml_language/crates/baml_compiler2_emit/src/verifier.rsbaml_language/crates/baml_compiler2_hir/src/builder.rsbaml_language/crates/baml_compiler2_hir/src/semantic_index.rsbaml_language/crates/baml_compiler2_mir/src/builder.rsbaml_language/crates/baml_compiler2_mir/src/cleanup.rsbaml_language/crates/baml_compiler2_mir/src/ir.rsbaml_language/crates/baml_compiler2_mir/src/lower.rsbaml_language/crates/baml_compiler2_mir/src/pretty.rsbaml_language/crates/baml_compiler2_tir/src/builder.rsbaml_language/crates/baml_compiler2_tir/src/inference.rsbaml_language/crates/baml_tests/src/compiler2_tir/mod.rsbaml_language/crates/baml_tests/tests/lambdas.rsbaml_language/crates/bex_engine/src/conversion.rsbaml_language/crates/bex_engine/src/lib.rsbaml_language/crates/bex_heap/src/accessor.rsbaml_language/crates/bex_heap/src/gc.rsbaml_language/crates/bex_heap/src/heap_debugger/real.rsbaml_language/crates/bex_vm/src/debug.rsbaml_language/crates/bex_vm/src/package_baml/root.rsbaml_language/crates/bex_vm/src/package_baml/unstable.rsbaml_language/crates/bex_vm/src/vm.rsbaml_language/crates/bex_vm_types/src/bytecode.rsbaml_language/crates/bex_vm_types/src/types.rsbaml_language/crates/tools_onionskin/src/compiler.rs
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
baml_language/crates/baml_compiler2_tir/src/builder.rs (1)
4287-4294:⚠️ Potential issue | 🟠 MajorDon't leak lambda-local
ExprIdstate back into the parent builder.
infer_lambda_bodyisolatesexpressionsandbindings, butresolutions,catch_residual_throws, andexhaustive_matchesstill accumulate lambda-local entries keyed by the lambda arena'sExprIds. Those ids can collide with the parent scope and corrupt member resolution, throw tracking, or match-exhaustiveness data after the lambda returns.Suggested fix
let saved_generic_params = self.generic_params.clone(); let saved_expressions = std::mem::take(&mut self.expressions); let saved_bindings = std::mem::take(&mut self.bindings); + let saved_resolutions = std::mem::take(&mut self.resolutions); + let saved_catch_residual_throws = std::mem::take(&mut self.catch_residual_throws); + let saved_exhaustive_matches = std::mem::take(&mut self.exhaustive_matches); @@ let lambda_expressions = std::mem::replace(&mut self.expressions, saved_expressions); self.bindings = saved_bindings; + self.resolutions = saved_resolutions; + self.catch_residual_throws = saved_catch_residual_throws; + self.exhaustive_matches = saved_exhaustive_matches; self.locals = saved_locals; self.declared_types = saved_declared; self.declared_return_ty = saved_return_ty;Also applies to: 4357-4363
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 9463ebcc-78a5-4553-ae90-aa12e8c44294
⛔ Files ignored due to path filters (21)
baml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/__baml_std__/baml_tests____baml_std____06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__01_lexer__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__02_parser__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__10_formatter__lambda_advanced.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__01_lexer__lambda_errors.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__02_parser__lambda_errors.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__03_hir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_tir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__05_diagnostics.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__06_codegen.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__10_formatter__lambda_errors.snapis excluded by!**/*.snap
📒 Files selected for processing (6)
baml_language/crates/baml_builtins2/baml_std/baml/containers.bamlbaml_language/crates/baml_compiler2_tir/src/builder.rsbaml_language/crates/baml_tests/projects/lambda_advanced/lambda_advanced.bamlbaml_language/crates/baml_tests/projects/lambda_errors/lambda_errors.bamlbaml_language/crates/baml_tests/src/compiler2_tir/mod.rsbaml_language/crates/bex_vm/src/package_baml/map.rs
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (3)
baml_language/crates/baml_tests/src/compiler2_tir/mod.rs (1)
253-292:⚠️ Potential issue | 🟡 MinorUse an HIR-aware formatter for lambda signatures.
This shared helper still hard-codes
type_expr_to_string(), so the HIR path prints lambda param/return/throwsannotations without the package qualification thatrender_hir2()preserves elsewhere. Please split this into TIR/HIR variants or pass a type-formatting callback.Also applies to: 1381-1395
baml_language/crates/baml_compiler2_tir/src/builder.rs (2)
693-739:⚠️ Potential issue | 🟠 MajorLambda-arg inference is still source-order dependent.
Pass 2 still visits lambda arguments once in source order. If an earlier lambda needs a type variable that only a later lambda binds, the earlier one still falls back to synthesis and can emit
CannotInferLambdaParamTypefor an otherwise-valid call. Defer unresolved lambda args and retry until the bindings map stops changing.
838-890:⚠️ Potential issue | 🟠 MajorCheck-mode lambdas still ignore local generics and can widen the expected return.
These annotation lowerings only see
self.generic_params, so lambda-local<T>annotations still becomeUnknownhere even though synthesis mode handles them.effective_retalso takes the explicit annotation without first checkingannotation <: expected_ret, which accepts wider return contracts than the expected function type.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7348f974-0193-405f-ac5d-ea1d55e42290
📒 Files selected for processing (7)
baml_language/crates/baml_compiler2_hir/src/builder.rsbaml_language/crates/baml_compiler2_hir/src/semantic_index.rsbaml_language/crates/baml_compiler2_mir/src/lower.rsbaml_language/crates/baml_compiler2_tir/src/builder.rsbaml_language/crates/baml_compiler2_tir/src/inference.rsbaml_language/crates/baml_tests/src/compiler2_tir/mod.rsbaml_language/crates/baml_tests/tests/lambdas.rs
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
baml_language/crates/baml_tests/src/compiler2_tir/mod.rs (1)
475-500:⚠️ Potential issue | 🟡 MinorIIFEs and nested lambda call sites still hide their bodies in TIR snapshots.
The typed
Expr::Callpath only expands compound arguments, not a compoundcallee, andrender_expr_body_untyped()has no call-specific recursion at all.(() -> { ... })()and higher-order calls inside lambda bodies therefore collapse back to the{ ... }placeholder, which underasserts some of the new closure cases.Also applies to: 511-558
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: c1ccefaa-582e-45d0-a29c-294f9358cf5f
⛔ Files ignored due to path filters (3)
baml_language/crates/baml_tests/snapshots/lambda_advanced/baml_tests__lambda_advanced__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_basic/baml_tests__lambda_basic__04_5_mir.snapis excluded by!**/*.snapbaml_language/crates/baml_tests/snapshots/lambda_errors/baml_tests__lambda_errors__04_5_mir.snapis excluded by!**/*.snap
📒 Files selected for processing (5)
baml_language/crates/baml_compiler2_mir/src/pretty.rsbaml_language/crates/baml_fmt/src/ast/expressions.rsbaml_language/crates/baml_tests/src/compiler2_tir/mod.rsbaml_language/crates/baml_tests/tests/lambdas.rsbaml_language/crates/bex_engine/src/lib.rs
Create lambda_basic and lambda_errors test projects with BAML files exercising lambda syntax (annotated params, inferred return types, captures, nested lambdas, generics, error cases). Generate baseline snapshots showing expected parse errors before parser support is added.
Add LAMBDA_EXPR to SyntaxKind, implement looks_like_lambda and looks_like_generic_lambda predicates for disambiguating parenthesized expressions from lambda expressions, and add parse_lambda_expr to handle the full lambda syntax including optional generics, params, return type, throws clause, and block body.
Add Lambda(Box<FunctionDef>) to the Expr enum with CST lowering that creates a fresh ExprBody for the lambda's block. Register lambda scopes in HIR with ScopeKind::Lambda and seed param bindings. Add MIR panic stub for lambda expressions. Wire up exhaustive match arms in TIR, test harness, and tools_onionskin.
Implement synthesis and checking modes for Expr::Lambda in the TIR. Lambdas in synthesis mode require annotated param types; in checking mode, unannotated params get their types from the expected function type context. Variable capture works via save/restore of locals. Fix partially-resolved function type checking so lambdas passed to generic higher-order functions get contextual param types even when the return type contains unresolved type vars.
Handle LAMBDA_EXPR CST nodes in the formatter with proper printing of optional generic params, parameter list, arrow, optional return type, optional throws clause, and block body. Add GenericParamList and ThrowsClause formatter types. Include idempotency unit tests.
Add parse_lambda_parameter_list/parse_lambda_parameter that allow
type annotations to be omitted (e.g. (x) -> { x * 2 }). This removes
the spurious E0010 parser error for unannotated lambda params.
Update HIR and TIR snapshot rendering to recursively print lambda
bodies instead of just showing "<lambda>", making it possible to
verify the internal expression structure.
The formatter expects if-conditions wrapped in parentheses (PAREN_EXPR), matching BAML's standard syntax. Change `if x < 0` to `if (x < 0)` in the lambda_basic test so the formatter can round-trip the file correctly.
Replace the catch-all <stmt> placeholder in render_stmt_untyped with proper rendering of throw, return, assign, break, continue, and assert statements so lambda body snapshots show full expression trees.
When a call expression has compound arguments (lambdas, blocks, if-expressions), expand them recursively below the call line so lambda bodies passed to higher-order functions are visible.
Add higher-order map methods to Array<T> and Map<K,V> with VM stubs. Array.map<U>(f: (T) -> U) -> U[] and Map.map/map_keys/map_values enable lambda expressions as arguments to container methods.
In resolve_builtin_method, add method-level generic params (e.g. U in map<U>) as TypeVar entries in the bindings map before lowering the method signature. This allows method generics to survive as TypeVars through lower_type_expr_with_generics, enabling call-site inference to bind them from argument types (e.g. U=int from a lambda returning int). Previously method generics became Unknown, causing map() to return unknown[] instead of int[].
When inferring types for a generic lambda like <T>(x: T) -> T { x },
combine the lambda's own generic_params with the parent scope's
generic_params before calling lower_type_expr_in_ns. Previously only
parent generics were passed, so T was unrecognized and became Unknown.
Identify captured variables at HIR time in SemanticIndexBuilder. For each lambda scope, walk the body's ExprBody to find single-segment Path names that resolve to ancestor scope bindings (up to the Function boundary). Record captures on ScopeBindings and mark defining scopes' captured_names for downstream MIR cell-wrapping decisions.
…tructions, MIR data structures Add Object::Closure and Object::Cell variants, 6 new bytecode instructions (MakeClosure, MakeCell, LoadDeref, StoreDeref, LoadCapture, StoreCapture), Rvalue::MakeClosure, MirFunction.lambdas, LocalDecl.is_captured, GC tracing, and VM execution handlers. All additive — no existing behavior changes.
…osures - Save/restore self.expressions and self.bindings in infer_lambda_body to prevent arena ID collisions between lambda and parent scopes - Change expr_types and pat_types keys from bare AstExprId/AstPatId to (FileScopeId, AstExprId/AstPatId) for proper scope isolation - Populate ScopeKind::Lambda with real inference in inference.rs - Add PullSink::make_closure, StackifyCodegen::make_closure, and StackCarryPullSink::make_closure for emit layer support
Replace MIR panic stub with real lambda lowering via save/restore on LoweringContext. Lambda bodies compile to separate MirFunctions stored in parent's lambdas vec. Pass 4 recursively compiles lambda children and maps lambda_idx to ObjectIndex for MakeClosure emission. Non-capturing lambdas and IIFEs now compile and execute correctly.
Add Place::Capture(usize) to MIR for captured variable access in lambda bodies. Wire HIR captured_names into LocalDecl.is_captured. Emit MakeCell preamble for captured locals, LoadDeref/StoreDeref in parent functions, LoadCapture/StoreCapture in lambda bodies. Captures are passed as operands to MakeClosure and shared via Cell objects for mutation semantics.
Add CaptureRef instruction for forwarding cell pointers to inner closures without dereferencing. Fix transitive capture propagation to avoid duplicate entries. Mark lambda params as captured when nested lambdas reference them. Add 7 execution tests: shared mutation counter, transitive captures across 3 scopes, IIFE returning closure, multiple closures sharing cells, deep 3-level nesting, and loop variable accumulation.
…snapshots, fix test types - Report type expression diagnostics in all 4 lambda `lower_type_expr_in_ns` callsites (synthesis param/return, checking param/return) instead of silently dropping them. Added `test_unknown_param_type` and `test_unknown_return_type` error test cases that now correctly produce `unresolved type: NonExistentType`. - Render `throws` clauses in `format_lambda_signature` for TIR/HIR snapshots. - Fix `test_nested_map` return type from `int[]` to `int[][]`. - Remove unused `map_keys`/`map_values` VM stubs (defined in BAML source, not `$rust_function`). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…hrough captures Fix D: When a lambda body assigns a function call result to a captured variable (e.g. `x = helper()`), the MIR builder panicked with "Call destination must be a local place". Now lower_call routes non-Local destinations through a temp local, then assigns to the real destination. Fix F: Thread `DefinitionSite` through HIR capture tracking for shadowing safety. `ScopeBindings.captures` is now `Vec<(Name, DefinitionSite)>` instead of `Vec<Name>`, uniquely identifying which declaration is captured. Mark `is_captured` at the capture-operand building site in MIR (where the exact Local is known) instead of relying solely on the name-based post-pass. Also adds 9 execution tests probing issues identified in PR review: - Issue A (virtual inlining): 2 tests — both pass - Issue B (@watch + cells): 1 test — passes - Issue D (capture call dest): 1 test — was crashing, now fixed - Issue E (resolution key collision): 2 tests — both pass - Issue F (shadowing captures): 2 tests — ignored (BAML disallows shadowing) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for Issue B: verify that `watch let` variables captured by lambdas work correctly with cell wrapping. Both `watch let x` with lambda-only mutation and mixed parent + lambda mutation produce correct values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t, HIR type formatting - Fix parked VMs missing frame-pointer forwarding after GC. The active VM path already called apply_frame_forwarding but parked VMs did not, leaving stale frame.function pointers after moving GC. - Add `// lambda[idx]` labels to MIR pretty printer so child lambdas are unambiguously identified by their MakeClosure slot index. - Propagate multi_lined from ThrowsClause inner type print so parent layout decisions get correct signal. - Add format_lambda_signature_hir for HIR-qualified type names in lambda signatures (uses prefix for local type names). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
21fa705 to
5abe7b2
Compare
What problems was I solving
BAML had no support for lambda expressions (anonymous functions / closures). Users couldn't write inline callbacks, pass functions to higher-order methods like
.map(), or use closures that capture variables from enclosing scopes. Every transformation required a named function definition, making code verbose and blocking a broad class of functional programming patterns — from simple transforms to closures returned from functions, IIFEs, and nested lambdas with shared mutable state.After this PR, lambda expressions are a first-class feature across the entire compiler stack: they parse, format, type-check with bidirectional inference, compile to bytecode with cell-based capture semantics, and execute in the VM with full GC support. Users can write lambdas with minimal annotation — parameter and return types are inferred from context when possible.
What user-facing changes did I ship
(params) -> [RetType] { body }with optional type annotations, optional return type, optional generic parameters (<T>(x: T) -> T { x }), and optionalthrowsclausesitems.map((x) -> { x * 2 })infersx: intfromArray<int>.map)Array.map<U>,Map.map<U>,Map.map_keys<U>,Map.map_values<U>— higher-order methods that accept lambda callbackslambda_basic(9 functions),lambda_advanced(~33 functions),lambda_errors(4 error cases) with full snapshot coverage across all 8 compiler phasesHow I implemented it
This was a full-stack feature spanning every compiler layer, implemented across 18 commits in two tasks: first the frontend (parser → AST → HIR → TIR + formatter), then the backend (MIR → emit → bytecode → VM → GC).
1. Syntax & Parser
syntax_kind.rs— AddedLAMBDA_EXPRnode kind to the CSTparser.rs— Addedlooks_like_lambda()andlooks_like_generic_lambda()disambiguation predicates with depth-aware paren scanning (up to 64 tokens lookahead). Addedparse_lambda_expr()withparse_lambda_parameter_list()(optional type annotations unlike regular function params). Hooked intoparse_primary_exprfor bothLParen(regular lambdas) andLess(generic lambdas)2. AST Lowering
ast.rs— AddedExpr::Lambda(Box<FunctionDef>)variant reusing the existingFunctionDefstruct with synthetic name"<anonymous function>"lower_expr_body.rs— Addedlower_lambda_exprwith a freshLoweringContextfor the lambda's ownExprBodyarena. Madelower_paramspublic vialower_cst.rs3. HIR Scope Registration & Capture Analysis
hir/builder.rs— Added pass that pushesScopeKind::Lambda, registers params, and recursively walks the lambda's ownExprBody.ScopeBindings.capturesrecords which parent-scope variables each lambda references.ScopeBindings.captured_nameson function scopes marks which locals are captured by any descendant lambda — the foundation for cell wrapping in MIR4. TIR Type Inference
tir/builder.rs— Three major additions:infer_expr(synthesis): Lambda with no expected type — requires annotated params, infers return from bodycheck_expr(checking): Lambda with expectedTy::Function— decomposes expected type for bidirectional param/return inference (the key toitems.map((x) -> { x * 2 }))infer_lambda_bodyhelper: Save/restore approach for locals, declared_types, return_ty, generic_params, andexpressions(to prevent ExprId collisions between lambda and parent arenas). Lambda params seeded on top of parent locals so captures work naturallyapply((x) -> { x * 2 }, 21)bindsT=intfrom21before checking the lambda)resolve_member_accessto include method-level generics in bindings soitems.map<U>(...)resolves correctlyinfer_context.rs— AddedCannotInferLambdaParamTypediagnostic variantnormalize.rs— FixedOptional/Unionsubtyping to work in both directions5. Formatter
expressions.rs— AddedLambdaExpr,GenericParamList,ThrowsClausestructs with fullFromCST/Printableimpls, dispatching onLAMBDA_EXPRCST nodetokens.rs/lib.rs— Supporting token and top-level dispatch additions6. Container Builtins
containers.baml— AddedArray.map<U>(self, f: (T) -> U) -> U[],Map.map<U>(self, f: (K, V) -> U) -> U[],Map.map_keys<U>,Map.map_values<U>array.rs/map.rs— VM implementations that invoke closures viacall_indirect7. VM Structural Foundation
types.rs—Object::Closure { function: HeapPtr, captures: Vec<Value> }andObject::Cell { value: Value }with fullDisplay,ObjectType, andvalue_type_tagsupportbytecode.rs— 7 new instructions:MakeClosure,MakeCell,LoadDeref,StoreDeref,LoadCapture,StoreCapture, andCaptureRef(7th not in original plan — needed for forwarding cell pointers to inner closures)gc.rs—add_references_to_worklistandfixup_object_referencestrace Closure/Cell objectsvm.rs—collect_frame_rootsandapply_frame_forwardingensure frame function pointers survive GC relocation. Execution handlers for all 7 instructions.resolve_callable_target,load_function,execute_call_from_locals_offset, andallocate_real_locals_for_frameall extended forObject::Closurelib.rs(engine) — Frame root collection and forwarding in GC cycle8. MIR IR & Lowering
ir.rs—Rvalue::MakeClosure { lambda_idx, captures },Place::Capture(idx),MirFunction.lambdas: Vec<MirFunction>,LocalDecl.is_captured: boollower.rs— Key migration:expr_typeskey changed fromAstExprIdto(FileScopeId, AstExprId)withcurrent_scopetracking, eliminating ExprId collisions. Addedlower_lambdamethod with full save/restore of parent state (builder, body, source_map, locals, exit_block, loop/catch context, pending_lambdas, capture_indices). HIRcaptured_names→is_capturedflag.capture_indices: Option<HashMap<Name, usize>>for lambda body variable resolution.transitive_captures_needed: Vec<Name>for nested lambda propagation9. Emit Layer
pull_semantics.rs—make_closure(),load_capture(),store_capture_value()trait methods +walk_rvalue_pullandwalk_place_pullarmsemit.rs—lambda_object_indices,lambda_names,captured_locals,loading_for_closure_capturefields onStackifyCodegen. Cell-wrapping preamble:LoadVar → MakeCell → StoreVarfor eachis_capturedlocal. Parent-side deref:LoadDeref/StoreDerefinstead ofLoadVar/StoreVar. Capture operand loading:LoadVar(bypassing deref) whenloading_for_closure_captureis set. Lambda body:Place::Capture(idx)→LoadCapture(idx)/StoreCapture(idx). Transitive forwarding:CaptureRef(idx)viaload_capturewhenloading_for_closure_captureis setlib.rs(emit) —compile_lambdas_flat()recursively compiles lambdaMirFunctions into bytecodeFunctionobjects, registering them inprogram.objects.lower_let_bodychanged to return lambdas alongside the body10. Tests
compiler2_tir/mod.rs— 216+ lines of TIR snapshot infrastructure:format_lambda_signature,render_expr_body_untyped, lambda capture annotations, compound argument expansionDeviations from the plans
This feature was implemented across two plans:
Implemented as planned
Expr::Lambda(Box<FunctionDef>), HIR lambda scope pass, formatterLambdaExpr— all match planDeviations/surprises
(FileScopeId, AstExprId)key migration)contains_typevarcheck. Implementation went further with a true two-pass approach (non-lambda args first, then lambda args) for better generic resolutionresolve_member_accessneeded "no changes." Implementation found and fixed a bug where method-level generic params (like<U>onArray.map<U>) weren't being added to bindingsnormalize.rsneeded "no changes." A bug was found and fixed whereOptional<T>wasn't recognized as a subtype ofUniontypes containingTandNullCaptureRef(Plan B): Plan B specified 6 instructions. Implementation added a 7th —CaptureRefpushes the raw cell pointer from a closure's captures without reading through the cell. Necessary for forwarding captured cells to inner closures during transitive capture propagationloading_for_closure_captureflag (Plan B): Emit layer uses a boolean onStackifyCodegento distinguish "load for closure capture" (cell pointer) vs "load for use" (cell value). Plan B mentioned the concept but didn't detail this mechanismlower_let_bodyreturns lambdas (Plan B): Plan B didn't account for lambdas inside let-binding initializers. Implementation changedlower_let_bodyto returnOption<(MirFunctionBody, Vec<MirFunction>)>Additions not in plans
lambda_advancedtest project: 334-line comprehensive test suite exercising generic inference, chained maps, nested generics, optional/union patterns, shadowing, and complex compositionmap/map_keys/map_valuesmethods: Added toArray<T>andMap<K, V>incontainers.bamlwith VM implementations — needed to test real lambda-as-argument patternscheck.rsandcompiler.rsneededExpr::LambdaarmsItems planned but not implemented
How to verify it
Setup
Automated Tests
Manual Testing
lambda_basic.baml— verify each lambda pattern (annotated, inferred, zero-param, capture, nested, generic, multi-param)lambda_advanced.baml— verify complex patterns (chained maps, nested captures, higher-order return, IIFEs, shadowing)lambda_errors.baml— verify each error case produces clear diagnostics[captured]annotations andmake_closurervalues with capture operandsLoadDeref/StoreDeref,LoadCapture/StoreCapture,CaptureRef,MakeClosureinstructionsDescription for the changelog
Add lambda expression support to BAML: anonymous functions with
(params) -> { body }syntax, bidirectional type inference, cell-based closure captures with shared mutation semantics, transitive capture propagation for nested lambdas, and higher-order container methods (Array.map,Map.map,Map.map_keys,Map.map_values).Summary by CodeRabbit
New Features
Bug Fixes / Diagnostics
Tests
Runtime / Tooling