[OpenCLML] Reactor and introduce on chip memory and memory planner#14922
[OpenCLML] Reactor and introduce on chip memory and memory planner#14922echuraev merged 5 commits intoapache:mainfrom
Conversation
|
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.
Generated by tvm-bot |
srkreddy1238
changed the title
srkreddy1238
force-pushed
the
clml_rqueues_on_chip
branch
4 times, most recently
from
9da9bd2 to
676d4e6
Compare
Introduced thread context with CLMLWorkspace. Organized the code as runtime, utils and memory planners Introcuded recording queue support and on chip memory support. On chip memory allocation planner to acommodate multiple tensors at a time. DDR memory planner introduced to reuse the underlaying memory across multiple tensor descriptors. Dense layer support refactored to use GEMM. CLML binary operators doesn't support broadcasting. Hence introduced an explicite broadcast op as a work around. clml SDK codegen is enhanced accordingly.
srkreddy1238
force-pushed
the
clml_rqueues_on_chip
branch
from
676d4e6 to
3bf7e63
Compare
echuraev
left a comment
echuraev
left a comment
There was a problem hiding this comment.
Is it necessary to add new tests for memory planner?
We definitely need few test cases. Let me find a way of exposing the plan to verify externally. |
|
Probably you can take a look at the OpenCL tests: https://github.com/apache/tvm/blob/main/tests/cpp-runtime/opencl/opencl_texture_pool_test.cc |
|
@echuraev have you ever built gtests (opencl-cpptest bin) for Android ? |
srkreddy1238
force-pushed
the
clml_rqueues_on_chip
branch
from
3e02f6e to
ffb3f82
Compare
Yes, I did it. You can build |
| using namespace tvm::runtime; | ||
| using namespace tvm::runtime::cl; | ||
|
|
||
| void InitMemoryPlan(tvm::runtime::contrib::CachedLayer& layer) { |
There was a problem hiding this comment.
I saw that in class CLMLRuntime you have almost the same methods. Probably it is better to test them? You can change private to protected and inherit CLMLRuntime class in your test class. It should be like a test wrapper above CLMLRuntime class.
There was a problem hiding this comment.
Only InitMemoryPlan can be reused but PlanMemory has dependency with JSONRuntime nodes. initializing CLMLRuntime here requires Json graph and it's dependents. Hence I tried isolating the test environment within "CahcedLayer``` object. Let me see how much I can reuse from CLMLRuntime.
…pache#14922) * Reactor and introduce in chip memory and memory planner Introduced thread context with CLMLWorkspace. Organized the code as runtime, utils and memory planners Introcuded recording queue support and on chip memory support. On chip memory allocation planner to acommodate multiple tensors at a time. DDR memory planner introduced to reuse the underlaying memory across multiple tensor descriptors. Dense layer support refactored to use GEMM. CLML binary operators doesn't support broadcasting. Hence introduced an explicite broadcast op as a work around. clml SDK codegen is enhanced accordingly. * * review comments * * Memory planner cpp_runtime tests. * * gtest build rules while in android environments. * * review comments --------- Co-authored-by: Siva Rama Krishna Reddy B <sivb@blr-ubuntu-ripper.qualcomm.com>
3 participants
Introduced thread context with CLMLWorkspace.
Organized the code as runtime, utils and memory planners Introcuded recording queue support and on chip memory support. On chip memory allocation planner to acommodate multiple tensors at a time. DDR memory planner introduced to reuse the underlaying memory across multiple tensor descriptors.
Dense layer support refactored to use GEMM.
CLML binary operators doesn't support broadcasting. Hence introduced an explicite broadcast op as a work around.
clml SDK codegen is enhanced accordingly.