Garbage Collector for Indexify

[diptanu]

We need a new service which deletes intermediate results from Indexify requests.

The service should in its configuration take a list of Indexify endpoints that it needs to monitor continuously and delete the logs, and request context objects.

We could implement it as the service waking up every few minutes and listing all the requests in Indexify.

Indexify could have a new table which gets updated by the state machine when requests complete. Once the requests are completed it could mark the request id to be garbage collected. We need to expose a new internal api which lists all the requests that need to be GCed. The external system can read the list and delete the data from storage.