AutoTVM: selecting tuning templates when extracting task

[zhenhuaw-me]

Make the procedure of trying new templates easier.

Would you please have a look at this @eqy @merrymercy @tmoreau89 @vinx13

[comaniac]

Thanks for the PR. This new optional argument indicates the template key when creating tasks, but what happens if I have more than one kinds of ops (e.g., dense, conv2d) in a model with different template keys? Specifically, in the following model:

conv2d
conv2d
dense

If I put template_key='winograd', then the semantic to dense is incorrect. On the other hand, I may only want to the first conv2d to use Winograd but keep the second one direct.

[zhenhuaw-me]

Thank you for the review @comaniac @tmoreau89 @vinx13 , really good points to have a dict holding the op to key mapping! I will update correspondingly.

[zhenhuaw-me]

Comments addressed, please have a look into this when you are available :) @comaniac @tmoreau89 @vinx13

For the template key dict part, now have a TOPI op to template key mapping. Relay ops may have many topi ops which are implemented with different template keys, so maybe not good enough to be the dict keys.

For the complicate case @comaniac mentioned in #4338 (review) , I guess it could be the user to handle as it is hard to generalize and the API (if there were such) could be annoying.

[comaniac]

I agree with you that the case uses different template keys for one kind of ops could be too complicate to be handled here so let's put it away first.

[zhenhuaw-me]

Thank you for the quick look @comaniac ! Now updated.

[comaniac]

Thanks. LGTM.
btw, would you update the tutorial as well?

[zhenhuaw-me]

Thank you @comaniac ! I will update the tutorials in another PR.

@tmoreau89 @vinx13 Would you like to have a look please :)

[vinx13]

Thanks @jackwish @comaniac @tmoreau89 this is merged