Allow specifying an S3 prefix for storage

[darintay]

It would be great to be able to specify a prefix for S3 storage, so that instead of everything living in s3://my-bucket/... it can live in s3://my-bucket/with/some/prefix-...

I'd like to use this for organizing data within the bucket so I can put other things in there too, and then also potentially pointing multiple independent instances at the same bucket, rather than having to create N separate buckets.

Describe alternatives you've considered
None as far as I know.

Additional context
I'm actually interested in this for loki, and the issue I found there (grafana/loki#1760) pointed me towards here - I couldn't find an existing issue for it, so I've made one.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

[darintay]

Still interested in this, and seems like so are a few others.

Any chance someone could let us know if this is at all feasible? Would the cortex team be open to a PR for this, if someone were motivated to do it?

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

[daswars]

why?