Implement LFU cache eviction strategy

[patricksanders]

All submissions

• Did you include the version part parameter, i.e. [major | minor | patch], to the beginning of the pull request title so that the version is bumped correctly?
  • Example pull request title: '[minor] Added a new parameter to the RefreshableSession object.'
  • Note: the version part parameter is only required for major and minor updates. Patches may exclude the part parameter from the pull request title, as the default is 'patch'.
• Did you verify that your changes pass pre-commit checks before opening this pull request?
  • The pre-commit checks are identical to required status checks for pull requests in this repository. Know that suppressing pre-commit checks via the --no-verify | -nv arguments will not help you avoid the status checks!
  • To ensure that pre-commit checks work on your branch before running git commit, run pre-commit install and pre-commit install-hooks beforehand.
• Have you checked that your changes don't relate to other open pull requests?

New feature submissions

• Does your new feature include documentation? If not, why not?
  • Does that documentation match the numpydoc guidelines?
  • Did you locally test your documentation changes using sphinx-build doc doc/_build from the root directory?
• Did you write unit tests for the new feature? If not, why not?
  • Did the unit tests pass?
  • Did you know that locally running unit tests requires an AWS account?
    • You must create a ROLE_ARN environment variable on your machine using export ROLE_ARN=<your role arn here>.

Submission details

Implements #109

In the true spirit of Cunningham, I present a change that technically works. At the time of writing, this PR adds an implementation of a Least Frequently Used cache eviction strategy. It does so in a way that adds a lot of implementation complexity, because I definitely didn't nail the abstraction.

Shall we dig in?

BaseCache

BaseCache is an ABC that implements a generalized version of the dunder methods from ClientCache. Notably, these dunder methods call the equivalent public methods instead of interacting directly with self._cache (for example, __getitem__(self, key) calls self.get(key)) so strategy-specific logic is contained in subclass implementations. The class also defines abstract methods for all of the public methods that will be implemented on subclasses. Lastly, it has a static new() method that takes max_size and strategy arguments and returns an instance of the appropriate subclass based on strategy.

The underlying data structure for the cache is a plain dict, previously an OrderedDict. I'm not sure if this was the right decision, but I'm fairly pleased with the outcome. The new LFU strategy requires more tracking than OrderedDict provides, and the LFU tracking data structure can also be used for LRU tracking. I decided to share the same components for both strategies. One significant complexity tradeoff is that this requires adding a layer of indirection to cache values to carry a reference to the value's "tracking node". More on that later.

LRUClientCache

LRUClientCache inherits from BaseCache and implements all of its abstract methods. These methods are (or should be) equivalent to the current ClientCache.

[to be continued...]