Problem: no compact storage for archive node

[yihuang]

Closes: #704

Solution:

• patch iavl to support extracting state changes
• implement versiondb using rocksdb user-defined timestamp feature
• now librocksdb is always required to build cronosd
  versiondb is only enabled if binary is built with rocksdb backend.
• implement cli commands to manage the migration process:

$ ./build/cronosd changeset
dump and manage change sets files and ingest into versiondb

Usage:
  cronosd changeset [command]

Available Commands:
  convert-to-sst Convert change set files to versiondb/rocksdb sst files, which can be ingested into versiondb later
  dump           Extract changesets from iavl versions, and save to plain file format
  ingest-sst     Ingest sst files into versiondb
  print          Pretty-print the content of change set file
  to-versiondb   Feed change set files into versiondb
  verify         Replay the input change set files in order to rebuild iavl tree in memory and output root hash, user can compare the root hash against the on chain hash

• Review Guides

• VersionDB Documentation

• The Future Plan for IAVL Tree Itself

• Performance Numbers in Testnet Run

👮🏻👮🏻👮🏻 !!!! REFERENCE THE PROBLEM YOUR ARE SOLVING IN THE PR TITLE AND DESCRIBE YOUR SOLUTION HERE !!!! DO NOT FORGET !!!! 👮🏻👮🏻👮🏻

PR Checklist:

• Have you read the CONTRIBUTING.md?
• Does your PR follow the C4 patch requirements?
• Have you rebased your work on top of the latest master?
• Have you checked your code compiles? (make)
• Have you included tests for any non-trivial functionality?
• Have you checked your code passes the unit tests? (make test)
• Have you checked your code formatting is correct? (go fmt)
• Have you checked your basic code style is fine? (golangci-lint run)
• If you added any dependencies, have you checked they do not contain any known vulnerabilities? (go list -json -m all | nancy sleuth)
• If your changes affect the client infrastructure, have you run the integration test?
• If your changes affect public APIs, does your PR follow the C4 evolution of public contracts?
• If your code changes public APIs, have you incremented the crate version numbers and documented your changes in the CHANGELOG.md?
• If you are contributing for the first time, please read the agreement in CONTRIBUTING.md now and add a comment to this pull request stating that your PR is in accordance with the Developer's Certificate of Origin.

Thank you for your code, it's appreciated! :)

[tomtau]

will there be any fallback for other db backends?

[yihuang]

will there be any fallback for other db backends?

that would be the original proposal, if there are any interests for that.
user can still specify different db backends for tm and iavl db though, just the versiondb will be always rocksdb for this implementation.

[codecov]

Codecov Report

Merging #791 (6193698) into release/v1.0.x (4e4b015) will increase coverage by 6.86%.
The diff coverage is 46.75%.

@@                Coverage Diff                 @@
##           release/v1.0.x     #791      +/-   ##
==================================================
+ Coverage           34.57%   41.43%   +6.86%     
==================================================
  Files                  28       52      +24     
  Lines                1481     3393    +1912     
==================================================
+ Hits                  512     1406     +894     
- Misses                916     1826     +910     
- Partials               53      161     +108     

Impacted Files	Coverage Δ
app/versiondb_placeholder.go	0.00% <0.00%> (ø)
versiondb/client/cmd.go	0.00% <0.00%> (ø)
versiondb/client/dump.go	0.00% <0.00%> (ø)
versiondb/client/ingest_sst.go	0.00% <0.00%> (ø)
versiondb/client/open_db.go	0.00% <0.00%> (ø)
versiondb/client/print.go	0.00% <0.00%> (ø)
versiondb/client/to_versiondb.go	0.00% <0.00%> (ø)
versiondb/client/verify.go	0.00% <0.00%> (ø)
versiondb/client/wrapreader.go	0.00% <0.00%> (ø)
versiondb/client/convert_to_sst.go	36.07% <36.07%> (ø)
... and 14 more

[yihuang]

it seems can't disable the typecheck linter in CI? it don't reproduce locally, will ignore for now.
related: golangci/golangci-lint-action#624

[yihuang]

it seems can't disable the typecheck linter in CI? it don't reproduce locally, will ignore for now. related: golangci/golangci-lint-action#624

finally figured out, typecheck error just means it fails to compile somewhere, it's CGO dependencies not setup in our case.