Auto-tuning scripts to maximize GPU kernel performance

[junrushao]

Depends on mlc-ai/relax#204, which has been decomposed into 5 and sent separately to mainline:

• [Bugfix][MetaSchedule] RPC port needs to be an integer apache/tvm#14781
• [Bugfix][Codegen][CUDA] Wrong casting in ASM apache/tvm#14782
• [MetaSchedule] Add an API to dump a pruned database apache/tvm#14783
• [MetaSchedule] Enhance tune_tir to tune IRModule of TIR Collections apache/tvm#14784
• [MetaSchedule] Clear screen only when specified apache/tvm#14785

[MarcelDelhez]

I would say more ... but, perhaps I am blind ... It would very interesting to have a documentation on how to train a model (ideally from scratch and using PDFs or a web site (html pages) like whitead/paper-qa does.
But, at least a documentation with a script to tune based on own data.

[funnbot]

I was able to use meta_schedule.relax_integration.extract_tasks() to get task weights, convert to tune context and tune_tasks (tune_tir calls this)

It’s also easy to load the cached pickle from the build script and filter out dynamic functions when looping over the modules functions, without a separate split step.

I still can’t figure out how the database in this repo has tunings for NT_matmul, maybe a custom schedule?

[yzh119]

Hi @MarcelDelhez, what do we mean by "tuning" here is tuning the kernel performance (to be faster) instead of fine-tuning weights.
Support fine-tuning in MLC-LLM is indeed very important but it's out of the scope of this PR, you can create an issue to discuss that :)

[MarcelDelhez]

I agree tuning was not the correct word. My concern was more about learning from own documents.
Such a bot is useful when it can answer to questions from own documents. So, my 'tunig' word must be replaced by fine tuning but into that case, there is also need for forgetting outdated information and ... a bundle of gpus to train a model and, I can easily understand it is out of your scope

[junrushao]

Agreed that "tuning" is a pretty overloaded term - in this particular case, I am referring to "auto-tuning compiler", which is the key to GPU performance. With TVM Unity auto tuning, MLC LLM is able to generate performant code on average phones for those 3b/7b models as fast as 10 tok/sec