Repository navigation
Conversation
rpmsg_virtio ran its own kthread per device, waking it through readyrx/ freerx node lists for every received buffer. Move the rx scheduling into the rpmsg framework: each device gets CONFIG_RPMSG_WQUEUE_NUMBER work queues, and rpmsg_queue_rx_work() hands a received buffer to the queue picked by the endpoint priority. A bound RT endpoint is still processed in the interrupt; an unbound one and every other endpoint go to a work queue, and a name service message takes the priority of the endpoint it binds. rpmsg_wait() now runs the pending works of the calling queue while it waits, so a service that blocks inside a callback no longer stalls rx, with the depth bounded by RPMSG_RECURSIVE_LIMIT. The transport wait/post ops lose their semaphore argument, and rpmsg_register() takes the number of rx works to preallocate (0 for rpmsg_port and rpmsg_router_edge). CONFIG_RPMSG_VIRTIO_PRIORITY/STACKSIZE become CONFIG_RPMSG_WQUEUE_*. Signed-off-by: Yongrong Wang <wangyongrong@xiaomi.com> Signed-off-by: guanyi3 <guanyi3@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
With CONFIG_RPMSG_WQUEUE_GLOBAL, rpmsg_initialize() creates one set of CONFIG_RPMSG_WQUEUE_NUMBER work queues and every rpmsg device queues its works there, instead of each device creating its own set. This saves the thread stacks on a system with several remote cores. Signed-off-by: Yongrong Wang <wangyongrong@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
rpmsg_wait() run on an rpmsg work queue returned RPMSG_EOPNOTSUPP once the recursion reached RPMSG_RECURSIVE_LIMIT, even though the semaphore it waited for could still be posted. rpmsg_virtio_notify_wait() then could not finish waiting for a tx buffer. Keep looping until the semaphore is taken: at the limit, only wait instead of dispatching more works. Add rpmsg_tickwait() with a timeout, make rpmsg_wait() a wrapper with no timeout, and pass the remaining time to the transport wait op so that rpmsg_virtio_notify_wait() can give up after RPMSG_VIRTIO_TIMEOUT_MS. Signed-off-by: Yongrong Wang <wangyongrong@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
xiaoxiang781216
approved these changes
Nov 17, 2026
acassis
approved these changes
Nov 17, 2026
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rpmsg_virtio receives with its own kthread per device: the rx interrupt
moves every non-RT buffer to a readyrx list and wakes the thread, which
then runs the endpoint callbacks one by one. A callback that blocks (for
example in
rpmsg_wait()for a tx buffer) stalls the whole device, andother transports cannot reuse the scheme.
This PR moves rx scheduling into the rpmsg framework:
drivers/rpmsg: dispatch rx in per-device work queues: each devicecreates
CONFIG_RPMSG_WQUEUE_NUMBERwork queues; the rx interruptqueues a buffer with
rpmsg_queue_rx_work()on the queue picked by theendpoint priority (bound RT endpoints are still handled in the
interrupt).
rpmsg_wait()called from one of these queues runs thequeue's pending works while it waits, bounded by a recursion limit, so
a blocking callback no longer stalls rx. The kthread and the
readyrx/freerx lists in rpmsg_virtio are removed.
drivers/rpmsg: add an option to share the work queues between devices:CONFIG_RPMSG_WQUEUE_GLOBALcreates one set of queues inrpmsg_initialize()for all devices.drivers/rpmsg: fix rpmsg_wait() not returning in the recursive path:at the recursion limit
rpmsg_wait()returned an error although thesemaphore could still be posted; keep waiting instead. Adds
rpmsg_tickwait()with a timeout (rpmsg_wait()becomes a wrapper)and passes the remaining time to the transport wait op.
Impact
RPMSG_VIRTIO_PRIORITY/RPMSG_VIRTIO_STACKSIZEare replacedby
RPMSG_WQUEUE_PRIORITY/RPMSG_WQUEUE_STACKSIZE(same defaults),plus
RPMSG_WQUEUE_NUMBER(default 1) andRPMSG_WQUEUE_GLOBAL(default n). No in-tree defconfig sets the old options.
RPMSGnowselects
SCHED_WORKQUEUE.rpmsg-<cpu>-<n>instead ofrpmsg-virtio.rpmsg_tickwait();rpmsg_wait()keeps its signature(now a macro), so the existing callers are unchanged.
drivers/rpmsg/rpmsg.h: the transportwait/postopslose the semaphore argument, and
rpmsg_register()takes the number ofrx works to preallocate; rpmsg_port and rpmsg_router_edge pass 0.
Testing
Host: Ubuntu 22.04.5 LTS x86_64, QEMU version 10.2.1, sim: cc (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0, qemu-armv8a: aarch64-none-elf-gcc (Arm GNU Toolchain 12.2.Rel1 (Build arm-12.24)) 12.2.1 20221205
qemu-armv8a:rpserver + qemu-armv8a:rpproxy (rptun ivshmem; the pair also
talks over rpmsg_port_uart). A test-only patch (not in this PR) logs, once
per endpoint, whether the ping callback runs in the interrupt and in
which task.
rpmsg ping all 20 1024 3 0pings with the normal endpoint,... 11 0with the RT one:RPMSG_WQUEUE_NUMBER=1=3rpmsg-virtiorpmsg-<cpu>-0rpmsg-<cpu>-1(default priority maps to the middle queue)rpmsg-uart-rxWith
=3each device has three queues at priority 224..226 (ps), andwith
RPMSG_WQUEUE_GLOBAL=y,RPMSG_WQUEUE_NUMBER=2there is one sharedpair
rpmsg-glb-0/1; pings and rpmsgfs (cat /sp/versionfrom theproxy) pass there too.
ping path, this PR, RPMSG_WQUEUE_NUMBER=3
ping path, master
ps, RPMSG_WQUEUE_NUMBER=3 / RPMSG_WQUEUE_GLOBAL=y
Every commit builds with no warning on sim:rpserver,
qemu-armv8a:rpproxy, and sim:rpserver with
RPMSG_WQUEUE_GLOBAL=y.No regression in rpmsg: sim:rpserver/rpproxy and qemu-armv8a:rpserver/rpproxy boot and pass the fixed rpmsg cases (dump, ping, rpmsgfs; 15/15). qemu-armv8a was built with LIBC_ENVPATH and BINFMT_CHECKEXEC disabled, which current master needs for nxinit to start nsh there; unrelated to this change.