Prompted by nextstrain/ncov-ingest#446
Some ideas of speeding up upload-to-s3 proposed in related Slack thread
- Configure threads for compression since that is most likely the bottleneck as we are compressing using a single thread
- Update hashing to stop going through Python. Or we should compute the hash of the compressed version.
Prompted by nextstrain/ncov-ingest#446
Some ideas of speeding up
upload-to-s3proposed in related Slack thread