HTTP caching layer directly in the gateway

[hsanjuan]

Caching on top of rainbow means we cannot effectively integrate the content-blocking layer as the cache would bypass it.

If we can introduce an HTTP cache directly at the gateway handlers that performs moderately well then we could resolve this problem. Investigate what options exist in go.

[hsanjuan]

So, it seems most caching options out there focus on in-memory caching, or sending the data to redis. I mean, it's not like everyone stores their files on disk chunked into a key-value store and have to recompose files from it.

If we put a cache in there, perhaps we can make a flatfs datastore and store the re-assembled files with it. In any case we would have to write this somewhere inside the gateway code boxo so that it sits after blocking-checks and before hitting unixfs. I don't think caching makes sense for individual car blocks, we can go do disk for those already.