Skip to content

tokio-style broadcast channels #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
goodboy merged 32 commits into from
Sep 3, 2021
Merged

tokio-style broadcast channels #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
3817b4f
Ultra naive broadcast channel prototype
goodboy Aug 8, 2021
6a2c3da
Simplify api around receive channel
goodboy Aug 8, 2021
1af7dbb
`Task` is hashable, so key on it
goodboy Aug 8, 2021
64358f6
Rename to broadcast mod, don't expect mem chan specifically
goodboy Aug 9, 2021
4ad75a3
Obviously keying on tasks isn't going to work
goodboy Aug 9, 2021
6e78bcf
Facepalm: use single `_subs` per clone set
goodboy Aug 10, 2021
ceed96a
Add common state delegate type for all consumers
goodboy Aug 10, 2021
a12b1fc
Drop optimization check, binance made its point
goodboy Aug 10, 2021
2d1c241
Add subscription support to message streams
goodboy Aug 15, 2021
6c17c73
Store handle to underlying channel's `.receive()`
goodboy Aug 16, 2021
346b5d2
Blade runner it
goodboy Aug 19, 2021
236ed0b
Initial broadcaster tests including one to test our `MsgStream.subscr…
goodboy Aug 19, 2021
a4cb0ef
Fix `.receive()` re-assignment, drop `.clone()`
goodboy Aug 19, 2021
2bad2ba
Don't enable debug mode..it borks CI
goodboy Aug 19, 2021
bec3f59
Drop uuid4 keys, raise closed error on subscription after close
goodboy Aug 20, 2021
d7ad898
Add subscribe after close test
goodboy Aug 20, 2021
093e7d9
Instance ids are ints
goodboy Aug 20, 2021
0d70e30
Add laggy parent stream tests
goodboy Aug 31, 2021
63ec740
Add some bcaster ref sanity asserts around subscriptions
goodboy Aug 31, 2021
39cf9af
Rename test module
goodboy Aug 31, 2021
5182ee7
Add a "faster task is cancelled" test
goodboy Aug 31, 2021
7857a9a
Add `shield: bool` kwarg to `Portal.open_stream_from()`
goodboy Aug 31, 2021
d9e793d
Can't use built-in generics till 3.9...
goodboy Aug 31, 2021
44ef26b
Shorten default feeder mem chan size to 64
goodboy Aug 31, 2021
5c63550
Shorten sequence length for test speedup
goodboy Aug 31, 2021
9258f79
Don't wake sibling bcast consumers on a cancelled call
goodboy Aug 31, 2021
d9bb52f
Store array `maxlen` in state singleton
goodboy Sep 1, 2021
b7b489d
Drop shielded stream api usage
goodboy Sep 1, 2021
5881a82
Add a first receiver is cancelled test
goodboy Sep 2, 2021
2745a2b
Solve first-recv-cancelled by recursive `.receive()` on wake
goodboy Sep 2, 2021
bcf5b9f
Add news fragment
goodboy Sep 2, 2021
1137a9e
Fix 404ed tokio urls
goodboy Sep 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions newsfragments/229.feature.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Add `tokio-style broadcast channels
<https://docs.rs/tokio/1.11.0/tokio/sync/broadcast/index.html>`_ as
a solution for `#204 <https://github.com/goodboy/tractor/pull/204>`_ and
discussed thoroughly in `trio/#987
<https://github.com/python-trio/trio/issues/987>`_.

This gives us local task broadcast functionality using a new
``BroadcastReceiver`` type which can wrap ``trio.ReceiveChannel`` and
provide fan-out copies of a stream of data to every subscribed consumer.
We use this new machinery to provide a ``ReceiveMsgStream.subscribe()``
async context manager which can be used by actor-local concumers tasks
to easily pull from a shared and dynamic IPC stream.
Loading