Skip to content

refactor(block-producer): simplify shared mempool#548

Merged
polydez merged 1 commit intonext-block-producerfrom
polydez-next-bp-simplify-shared-mempool
Nov 8, 2024
Merged

refactor(block-producer): simplify shared mempool#548
polydez merged 1 commit intonext-block-producerfrom
polydez-next-bp-simplify-shared-mempool

Conversation

@polydez
Copy link
Contributor

@polydez polydez commented Nov 8, 2024

I think, creation of SharedMempool as separated structure doesn't give us any benefits, but makes code more complex. Proposed refactoring in this PR.

By the way, @Mirko-von-Leipzig, do we need additional mutex here? https://github.com/0xPolygonMiden/miden-node/blob/9445ca2276b3ed5d02bf14928adb221fd274562b/crates/block-producer/src/server/mod.rs#L165-L171
According to comment, this can help with rate limiting and queuing of new transaction, but it seems to me that one mutex in SharedMempool is enough to do this job.

@polydez polydez added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label Nov 8, 2024
@Mirko-von-Leipzig
Copy link
Collaborator

By the way, @Mirko-von-Leipzig, do we need additional mutex here?

https://github.com/0xPolygonMiden/miden-node/blob/9445ca2276b3ed5d02bf14928adb221fd274562b/crates/block-producer/src/server/mod.rs#L165-L171

According to comment, this can help with rate limiting and queuing of new transaction, but it seems to me that one mutex in SharedMempool is enough to do this job.

I believe we do need it, my reasoning for this is that we need the batch and block building tasks to have priority access to the mempool.

To elaborate more: the mempool is shared between the block-producer's RPC, batch-builder and block-builder processes. The batch and block processes each have single threaded access to the mempool, however the rpc will likely have thousands of concurrent tasks trying to access the mempool. This would mean the batch and block access to the mempool would be competing with all of these.

This extra mutex forces all of these concurrent rpc connections into a single queue, meaning we effectively only have a single rpc access instead of thousands. Without this mutex the batch and block tasks would be mempool starved. We could instead try find a priority based locking mechanism, but I believe this approach is (1) simpler, and (2) probably sufficient since each has a 1/3 access chance under load.

Copy link
Collaborator

@Mirko-von-Leipzig Mirko-von-Leipzig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks I agree this is better, thank you!

@polydez
Copy link
Contributor Author

polydez commented Nov 8, 2024

@Mirko-von-Leipzig so, the main reason to add second mutex is giving priority access to mempool of batch/block builders. This makes sense, thank you for clarification!

@polydez polydez merged commit 16c3087 into next-block-producer Nov 8, 2024
@polydez polydez deleted the polydez-next-bp-simplify-shared-mempool branch November 8, 2024 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog This PR does not require an entry in the `CHANGELOG.md` file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants