[FRONTEND][TFLITE] Add support for TFLite's regular NMS operator

[ilyag-grovety]

This PR adds support of regular NMS operator.

Open questions:

1. How to properly test added functionality?
   Other NMS implementations, e.g., fast NMS, use a TF frozen graph from TF official website to convert a model to TFLite and keep NMS operations only. In order to create a similar test, we need to find an archive on TF official website that contains a frozen graph of a model compiled with --use-regular-nms=True flag. We haven't found it yet, so any help is appreciated.
2. Regular NMS requires two sort operations:
   • Sorting the scores after selecting scores above nms_score_threshold. This PR implements this with a simple bubble sort in order to prove the algorithm's semantics. We tried to replace it with tvm.contrib.sort.argsort. It works well when testing the regular NMS with run_tvm_graph as fast NMS test does or building and running the regular NMS with llvm target. At the same time, it fails to build (error is provided below) when target=ethos-u,cmsis-nn,c. It seems that __tvm_module_ctx variable is only being initialized when cpp runtime is chosen.
     The error:
     error: '__tvm_module_ctx' undeclared (first use in this function) 203 | if (TVMBackendGetFuncFromEnv(__tvm_module_ctx, "tvm.contrib.sort.argsort", &tvm_contrib_sort_argsort_packed) != 0) {
   • Sorting the scores of previous and current NMS steps. There are two alternatives here:
     • implement some sorting algorithm as part of hybrid script (to replace current bubble sort)
     • save the result of each NMS step and use argsort after the hybrid script part. This approach has a drawback as it requires significant amount of memory to store the results of each NMS step.

[tvm-bot]

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

• cc @leandron _{See #10317 for details}

_{Generated by tvm-bot}

[ilyag-grovety]

@leandron @Aleksei-grovety

[sergio-grovety]

cc @ekalda

[ekalda]

Thanks for the patch @ilyag-grovety, looks all very neat and good to me. Hopefully the CI will go green as well once the model URL will be properly enabled.

Regarding to testing, from my research, I didn't find a way to create that operator through current TF or Keras APIs either, so I think using the model consisting of this operator is the best approach here.

I think using simple sorting algorithm is fine for now. I don't know what it would take to enable argsort for cpp runtime, but any effort of that kind should be a separate patch anyway. If bubble sort becomes a massive performance bottleneck in the future, then it is possible to further enhance this operator with enabling argsort or implementing a different sorting algorithm, but for now it should be fine.

[Aleksei-grovety]

Hi @mehrdadh! Could you please upload the model to S3 (workflow parameters url: url,
sha256: f35589f2153a33615ac38b86fc7a2fd7fa9e3e3897c4930c16c161e6b2cd3a4e,
upload_path: upload_path)? It's necessary for testing NMS.

[Aleksei-grovety]

Thanks @ilyag-grovety! Looks good just a couple comments.

[Aleksei-grovety]

Is it necessary to subtract zero here?

[ilyag-grovety]

No, I'll remove, thanks

[Aleksei-grovety]

Shouldn't there be out_num_detections_shape[0] = boxes_shape[0]? As further comments point to num_detections of size (batch_size,).

[ilyag-grovety]

You're right, fixed

[sergio-grovety]

@tvm-bot rerun

[ekalda]

LGTM, thanks for all the work @ilyag-grovety!