Conversation
LPTK
force-pushed
the
hkmc2
branch
2 times, most recently
from
46c5ce1 to
414cf0c
Compare
# Conflicts: # hkmc2DiffTests/src/test/scala/hkmc2/MLsDiffMaker.scala
…into hkmc2-deforest-new-new
ychenfo
force-pushed
the
hkmc2-deforest-new-new
branch
from
21b6e91 to
a4b9fbf
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds a deforestation optimization pass to the hkmc2 compiler. Deforestation is an optimization that eliminates intermediate data structures by fusing producers (constructors) with consumers (pattern matches/selections). The PR supersedes PR #335.
Changes:
- Adds a complete deforestation pipeline: pre-analysis, constraint collection, constraint solving, and rewriting phases
- Integrates the deforestation pass into the lowering pipeline, controlled by a
:deforestcommand - Fixes determinism in the SCC partitioning algorithm and adds supporting infrastructure changes
Reviewed changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
hkmc2/shared/src/main/scala/hkmc2/Config.scala |
Adds Deforest config case class and deforest field to Config |
hkmc2DiffTests/src/test/scala/hkmc2/MLsDiffMaker.scala |
Adds :deforest command for test files |
hkmc2/shared/src/main/scala/hkmc2/Uid.scala |
Adds Result and StratVar UID handlers for deforestation |
hkmc2/shared/src/main/scala/hkmc2/codegen/Block.scala |
Sets tsym on BlockMemberSymbol in FunDefn.withFreshSymbol |
hkmc2/shared/src/main/scala/hkmc2/codegen/LambdaRewriter.scala |
Refactors to use FunDefn.withFreshSymbol |
hkmc2/shared/src/main/scala/utils/TraceLogger.scala |
Widens emitDbg visibility to protected[hkmc2] |
core/shared/main/scala/utils/algorithms.scala |
Fixes nondeterminism in SCC by using LinkedHashSet |
hkmc2/shared/src/main/scala/hkmc2/codegen/Lowering.scala |
Integrates deforestation pass into lowering pipeline |
hkmc2/shared/src/main/scala/hkmc2/codegen/deforest/Deforest.scala |
Core types, extractors, and entry point for deforestation |
hkmc2/shared/src/main/scala/hkmc2/codegen/deforest/Analyze.scala |
Pre-analysis, constraint collection, and constraint solving |
hkmc2/shared/src/main/scala/hkmc2/codegen/deforest/Rewrite.scala |
Rewriting pass that applies deforestation transformations |
hkmc2/shared/src/test/mlscript/deforest/*.mls |
Comprehensive test suite for deforestation (12 test files) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
LPTK
left a comment
LPTK
left a comment
There was a problem hiding this comment.
Ok, I don't have time to review everything in detail at this time, but I believe this is more or less ready to merge!
hkmc2/shared/src/main/scala/hkmc2/codegen/deforest/Analyze.scala
Outdated
Show resolved
Hide resolved
| end DeforestConstraintsCollector | ||
|
|
||
|
|
||
| class DeforestConstrainSolver(val collector: DeforestConstraintsCollector): |
There was a problem hiding this comment.
Doesn't this belong in its own file?
There was a problem hiding this comment.
I think maybe it's reasonable to keep them here for now, they are indeed both part of the analysis phase and the classes are not particularly large (~260 loc).
hkmc2/shared/src/main/scala/hkmc2/codegen/deforest/Rewrite.scala
Outdated
Show resolved
Hide resolved
| inline def exit() = indent -= 1 | ||
|
|
||
| protected def emitDbg(str: Str): Unit = scala.Predef.println(str) | ||
| protected[hkmc2] def emitDbg(str: Str): Unit = scala.Predef.println(str) |
There was a problem hiding this comment.
I don't think this change is needed?
There was a problem hiding this comment.
In Lowering.scala I make a new traceLogger to output fusion opportunities, and its emitDbg need to use the emitDbg from lowering traceLogger (not inherited), so I think I need protected[hkmc2] to widen access...
There was a problem hiding this comment.
Why not pass the same TraceLogger as for lowering?
There was a problem hiding this comment.
Oh, my intention is that deforest's tracelogger can be enabled independently of lowering tracing. If we reused the same tracelogger, we would have to enable lowering debug just to see deforest summary. The emitDbg calling the lowering tracelogger's emitDbg is just so the output still goes to the same place, prefixed by "deforest > ".
- use && - proper TODOs - `locally` style
Supersedes and closes #335