add compatibility for editable wheels and src layout

[thrau]

Motivation

When using an src layout project structure together with the common pip install -e . pattern to populate a virtual environment, plux plugins would not be resolved correctly. This is because the editable wheel creates a .dist-info directory within site-packages for the distribution, in which plux has a hard time creating the entry_points.txt file because of the way pip isolates the build context (details aren't so important).

This PR adds functionality for both build time and run time to resolve entry points from editable installs, even when a source layout is used.

Explanation

The main idea is to hook into the procedure that creates the editable install (in setuptools, this is the editable_wheel command), and in the resulting .dist-info directory that is part of the site-packages, leave behind a link to the .egg-info of the source that contains the real entry_points.txt.

In the case of localstack when using a src layout, the result of pip install -e . will moving forward look as follows (simplified):

thomas@om /tmp % tree -a localstack
localstack
├── localstack-core
│   ├── localstack
│   └── localstack_core.egg-info
│       ├── entry_points.txt
│       └── plux.json
├── pyproject.toml
├── tests
└── .venv
    └── lib
        └── python3.11
            └── site-packages
                └── localstack_core-3.4.1.dev0.dist-info
                    └── entry_points_editable.txt

where entry_points_editable.txt will contain one line: /tmp/localstack/localstack-core/localstack_core.egg-info/entry_points.txt.
This is both very easy to create during build time, and also fairly easy to resolve during runtime. However, getting this logic into stevedore was too complicated, so I re-wrote the entry points resolution mechanism natively in plux, but borrowed a lot of their ideas. It's vastly simplified but works well. Performance (cold start without cache, cold start reading a cache from file, and warm in-memory cache) is basically unchanged.

Testing

You can test this in localstack:

• check out the move-source-code branch
• clear out all existing .egg-info directories or anything else
• run make clean install
• run pip install -e ../plux (with this PR checked out)
• update pyproject.toml to requires = ['setuptools', 'wheel', 'plux @ file:///home/thomas/workspace/localstack/plux'] (update to your path accordingly
• run pip instal -e . again (this way your local plux version will be used to build the editable wheel, which will correctly create the entry_points_editable.txt
• python -m localstack.cli.plugins list --namespace "localstack.packages" should show all the packages

Changes

• added a new command python -m plux resolve to test resolution of entrypoints from the current path using a PluginManager - mostly for debugging
• Entry point resolution at runtime is now done via our own procedure, that largely implements the idea of stevedore (aggregate all entry points and cache the aggregation on disk)
• Removed stevedore requirement

[alexrashed]

Awesome! Really great to see this feature implemented, paving the way for the multi-distribution setup!
I chimed in with a review out of curiosity 😛
I tested the implementation with the current layout (and it works fine, i.e. is compatible with the non-src-layout), but I was having a hard time testing it in localstack/localstack.
But I didn't dig too deep. Maybe localstack/localstack#10800 is missing a commit still on your machine (last push was a few days ago)?