Repository navigation
Conversation
Add nuttx/ring.h implementing a unified lock-free ring queue supporting SPSC/MPSC/SPMC/MPMC modes using RTS (Relaxed Tail Sync) algorithm with single CAS operations. API uses explicit mode macros (ring_spsc_enqueue, ring_mpmc_dequeue, etc.) with compile-time flags and sizeof for optimal codegen. RTS tail CAS uses release/acquire ordering matching DPDK rte_ring. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Extract sp/sc/mp/mc start and finish as reusable building blocks, refactor existing enqueue/dequeue to use them. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add dequeue_skip variants that advance ring indices without touching data, built on the reserve + commit building blocks from the previous patch. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add ring_buf_s descriptor and two-phase zero-copy operations: - enqueue: start (get writable pointers) + copy + finish (publish) - dequeue: start (get readable pointers) + copy + finish (release) Unify enqueue/dequeue to use ring_buf_s internally via shared ring_buf_fill, ring_copy_from, ring_copy_to helpers, eliminating the old ring_copy_from/ring_copy_to that duplicated wrap-around logic. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add an atomic_t magic field to ring_s for detecting whether a ring has been previously initialized. This supports rings placed in memory that survives reboot (shared SRAM, battery-backed RAM). ring_init() now uses a three-state CAS protocol: - Uninitialized (magic != READY/INIT): fresh init, zero indices - RING_MAGIC_READY: warm restart, validate and recover indices - RING_MAGIC_INIT: another core is initializing, spin-wait Multi-core safe: concurrent ring_init() calls are serialized via CAS on the magic field. Loser cores spin until the winner sets RING_MAGIC_READY, ensuring no core uses the ring before init completes. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add ring_space(r) that returns the number of available slots for enqueue, complementing the existing ring_count/ring_capacity. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
When count is zero, ring_buf_init returns early without initializing ptr1/len1/ptr2/len2. GCC inlines the callers (ring_mp_enqueue, ring_sc_dequeue, etc.) and cannot prove these members are set, triggering -Werror=maybe-uninitialized. Initialize all ring_buf_s members on the zero-count path. Signed-off-by: yezhonghui <yezhonghui@xiaomi.com>
Add overwrite enqueue that discards oldest committed data when the ring is full, guaranteeing writes always succeed. Typical use case: sensor data streams and log buffers where latest data matters most. The discard path acts as a virtual multi-consumer (CAS on cons_head + mc_finish), so overwrite is only available for MC queues (SPMC/MPMC). SC queues cannot support overwrite because the producer must CAS cons_head, which conflicts with SC's plain-store protocol. Provides both copy and zero-copy (start/finish) variants: ring_spmc_enqueue_overwrite / ring_mpmc_enqueue_overwrite ring_spmc_enqueue_overwrite_start / ring_spmc_enqueue_overwrite_finish ring_mpmc_enqueue_overwrite_start / ring_mpmc_enqueue_overwrite_finish Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
SPSC mode has no concurrent writers on the same index - producer only writes prod_tail, consumer only writes cons_tail. Atomic operations are unnecessary overhead for this case. Replace atomic_read/atomic_set_release with volatile pointer access and explicit UP_WMB/UP_RMB memory barriers: - Producer: write data -> UP_WMB -> volatile write prod_tail - Consumer: volatile read prod_tail -> UP_RMB -> read data This avoids issues on platforms where atomic instructions are not available or have high overhead (e.g. AMP without shared atomic support), while maintaining correct memory ordering for SMP/BMP with coherent cache. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add comprehensive usage documentation at the top of ring.h covering declaration, initialization, mode selection (SPSC/MPSC/ SPMC/MPMC), overwrite mode, zero-copy pattern, and query APIs. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
When the ring is full (enqueue) or empty (dequeue), the start functions return buf->cnt == 0. Previously ring_copy_from and ring_copy_to would fall into the else branch and call memcpy with len1=0 and ptr1=NULL, which is undefined behavior per the C standard even when size is zero. Change the else branch to else-if (buf->cnt != 0) so that zero-count cases are safely skipped without adding an extra early-return branch. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add a mask field to ring_s: if capacity is power-of-2, mask stores capacity-1 for fast bitwise index; otherwise mask is 0 and ring_idx() falls back to modulo. This allows ring queues to use any positive capacity without wasting memory to round up. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add compile-time helpers for ring buffer size validation: - RING_SIZE_MAX: maximum allowed capacity (2^23, limited by 24-bit index) - RING_SIZE_VALID(sz): validates size is > 0, power-of-2, and <= RING_SIZE_MAX This allows users of ring.h to validate buffer sizes consistently without open-coding the constraints. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
The struct-level aligned_data(CACHELINE_SIZE) attribute on ring_s makes some compilers (e.g. TASKING) propagate the alignment to every member, inflating sizeof(struct ring_s) from 192 to 640 bytes and breaking the cross-core shared memory layout. Move the attribute from the type to each declaration/definition site (RING_DECLARE, RING_DEFINE_BMP) so it binds to the variable, and use lib_memalign in ring_alloc for the dynamic path. The cache-line isolation between regions is preserved by the explicit pad0/pad1/pad2 arrays. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add two peek functions that read data from the ring using a caller- managed index without advancing cons_tail: - ring_sc_peek: single-consumer peek, no retry, suitable for SC or panic-path scenarios where overwrite race is not a concern. - ring_mc_peek: multi-consumer peek with optimistic-read-then-verify loop, retries automatically if the read region was overwritten during copy. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
ring_*_enqueue_overwrite() guarantees the new element is admitted by silently evicting older committed elements when the ring is full. Until now that loss was invisible to callers: the function returned the number of bytes admitted (always == count * elem_size) and gave no way to learn how many bytes the eviction took with it. For users that care about end-to-end accounting (note backends in particular) this turned overwrite enqueue into a black box. Surface the eviction count by folding it into the return value: - ring_sp_enqueue_overwrite_start() / ring_mp_enqueue_overwrite_start() now return uint32_t, the number of elements evicted to make room for the request. The MP variant accumulates evictions across multiple CAS retries when more than one tail block has to be reclaimed before head can advance. Zero-copy callers can pair the start() return value with their own bookkeeping. - ring_sp_enqueue_overwrite() / ring_mp_enqueue_overwrite() capture the start() return value and now return buf.cnt - discard, i.e. the net change in ring occupancy rather than the gross number of elements written. When the ring is not full this still equals count; when it is full the caller can compute eviction loss as count - return. The macros ring_spmc_enqueue_overwrite / ring_mpmc_enqueue_overwrite are unchanged at the call site syntax level. This is a generic ring extension; the first consumer (notebuf folding the count into drop_bytes) is added in a separate patch. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
ring_reset() previously returned void and silently discarded all
elements still in the ring at the time of the call. Callers that
care about loss accounting (note backends in particular) had no way
to learn how many elements were dropped, forcing them to either
sample ring_count() before the reset (racy) or stop tracking drops
entirely on NOTE_CLEAR-style operations.
Convert the macro to a GCC statement expression that snapshots the
producer/consumer index delta before zeroing the four indices and
yields it to the caller. All existing void-context callers
(ring_init_magic warm-restart fallback, ring_init_magic first-time
init, ring_spsc_init, notebuf_ioctl NOTE_CLEAR) compile unchanged
because C permits discarding expression values.
Consumers that want to fold the discarded count into their own
"dropped" metric can now write:
dropped = ring_reset(buf->ring);
atomic_add(&stats.drop_bytes, dropped);
This is a generic ring extension; the first consumer (notebuf
folding the count into drop_bytes on NOTE_CLEAR) is added in a
separate patch.
Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
Add ring_sc_skipto()/ring_mc_skipto() that advances cons_tail to a target index directly, computing the skip count internally. This avoids exposing RTS_IDX encoding details to callers. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
Shorten ring public macro names by removing redundant verbs: - ring_xxxx_dequeue_skip -> ring_xxxx_skip Skip always operates on consumer side, "dequeue" is redundant. - ring_xxxx_enqueue_overwrite -> ring_xxxx_overwrite Overwrite always implies enqueue, "enqueue" is redundant. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Insert UP_WFE() in busy-wait loops to hint the processor to reduce power consumption during spinning. Add UP_SEV() at completion points to signal waiting cores. Signed-off-by: ganjing <ganjing@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Add DEBUGASSERT(r != NULL && cap > 0 && cap <= RING_IDX_MASK) at the entry of ring_init_magic() to catch invalid parameters early. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
Add RING_CAPACITY(bufsize) macro to extract data capacity from total buffer size (bufsize - sizeof(struct ring_s)). All ring APIs now accept total buffer size (header + payload) as the size parameter, and compute the usable data capacity internally. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Remove char ring[] from struct ring_s to fix Clang warning -Wgnu-variable-sized-type-not-at-end when ring_s is embedded as a non-terminal member via RING_DECLARE. Replace with ring_data(r) macro that computes the buffer address as (struct ring_s *)(r) + 1. Memory layout is unchanged. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
Add a static_assert(RING_CAPACITY(buflen) <= RING_SIZE_MAX) as the first member of the ring struct in RING_DECLARE and RING_DEFINE_BMP, so an over-sized buffer is rejected at compile time instead of silently overflowing the 24-bit ring index. Centralizing the check in the macros removes the need for callers to repeat it at each ring definition site. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
ring_is_valid_cons compared the raw 32-bit signed difference of two indices, but ring indices are 24-bit free-running counters (RTS_IDX_MASK) that wrap at 2^24. Near the wrap boundary the comparison is wrong: e.g. cons_index=0x000003 is actually ahead of cons_tail=0xfffffe, yet the plain signed difference is large and negative, so the macro misjudged live data as lapped. Shift the 24-bit difference left by 8 so bit 23 lines up with the 32-bit sign bit, turning the test into a correct modular "within half the 24-bit range" comparison. Since the live window is bounded by capacity <= RING_SIZE_MAX (2^23), the half-range split exactly separates ahead-of-cons from lapped-behind-cons. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
The overwrite helpers previously returned the net change in ring occupancy (inserted minus evicted), which hides the true number of elements discarded when the ring is full. Change ring_sp_overwrite() and ring_mp_overwrite() to return the eviction (discard) count directly so callers can account for dropped data precisely; the count is zero when the ring had enough free space. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
The external-index peek requires callers to handle absolute ring indices, which forces scanners that inspect buffer contents to reach into ring internals. Add ring_*_peek_at(r, items, count, offset): it reads count elements at a fixed offset from the oldest element (cons_tail) without advancing, expressing the read position relative to the consumer tail so the caller never sees an absolute index. An out-of-range offset yields zero. Implemented by delegating to the existing single/multi-consumer peek primitives. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
A reader that walks the ring by absolute index needs a way to start from the oldest readable position without a valid index in hand. Add the RING_IDX_OLDEST sentinel: a real index never exceeds RTS_IDX_MASK (24 bits), so this out-of-range value is unambiguous. When ring_mc_peek() sees it in *index it skips ring_clamp_cons() (whose signed compare would be meaningless on the sentinel) and starts from the live cons_tail. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
A reader that tracks an absolute position and wants to detect whether an overwrite lapped it needs to compare its expected next position against the index a peek reported. The peek returns a 24-bit index, while the expected position is a linear (unwrapped) value, so the two cannot be compared directly without masking. Add ring_wrap(pos) to wrap a linear position into the same 24-bit index space using the public RING_IDX_MASK, so callers avoid reaching into the internal packed layout. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
ring_mp_overwrite_start() bumps prod_head.cnt unconditionally once its reserve CAS succeeds, even when count is 0. Its commit counterpart ring_mp_finish() returns early when buf->cnt == 0 and never bumps prod_tail.cnt. A zero-count overwrite therefore adds a prod_head.cnt increment with no matching prod_tail.cnt increment, leaving a permanent head.cnt != tail.cnt gap. The RTS tail sync in ring_mp_finish() only advances prod_tail.idx when tail.cnt catches up to head.cnt. Once the gap is permanent that condition can never hold again, so prod_tail.idx freezes. With a frozen tail, ring_mp_overwrite_start() computes committed == 0, every subsequent overwrite finds discard > committed, and all producers spin forever in the eviction retry loop. Guard against count == 0 before the reserve loop and return an empty buf, mirroring the n == 0 break already present in ring_mp_start(). The copy wrappers stay correct because ring_buf_init() already zeroes buf on a zero count. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
User pass *init to judge whether this init is the first init of the ring to avoid race condition when user want to initialize their own data struct. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
…ings ring_sc_start()/ring_mc_start() derive the available element count from a non-atomic pair of index reads: prod_tail is loaded separately from the consumer index. In OVERWRITE mode the producer is allowed to advance the consumer-owned cons_head/cons_tail to evict the oldest entries, so a flush consumer can observe a torn snapshot where prod_tail is stale while the consumer index has already been pushed past it in 24-bit modular order. ring_dist() then computes (prod_stale - cons_new) & RING_IDX_MASK, which underflows to ~16M - far larger than the ring capacity. n = min(count, available) stays huge and ring_buf_init() faithfully splits it into len1 + len2 with no capacity clamp, producing an over-length segment. The downstream memcpy in the note/netpoll path then walks off the mapped ring and takes a bus trap. A consistent ring can never hold more committed entries than its capacity, so any available > capacity is by definition a torn read. Force available to 0 in that case: the round yields no data and the periodic watermark/timeout flush re-reads a coherent snapshot on the next pass with no entries lost. Clamping to 0 rather than capacity avoids advancing the consumer index past the real prod_tail, which would corrupt the ring. The same producer/consumer divergence also underflows the producer side. ring_mp_overwrite_start() computes free_space = capacity - ring_dist( prod_head, cons_head); once the producer laps the consumer by more than capacity in 24-bit modular order this subtraction wraps to ~4G. The "free_space < count" test is then never true, so the eviction branch that reclaims the oldest slots is never entered again: the producer advances prod_head without bound and the evicted-byte accounting silently stops, leaving megabytes of overwritten data uncounted. Guard the subtraction to yield 0 when the ring is already full, forcing the eviction path to run and restoring the invariant dist(prod_head, cons_head) <= capacity. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com>
…apacity The RTS ring packs a 24-bit byte index that wraps at 2^24. The physical mapping uses idx % capacity when capacity is not a power of two. Physical continuity across the wrap requires 2^24 to be a multiple of capacity, which only holds for power-of-two capacities. On non-power-of-two capacities the physical position jumps by (2^24 % capacity) bytes at every 2^24 boundary, tearing any record straddling the wrap and producing illegal bytes downstream. Introduce a wrap field equal to (2^24 / capacity) * capacity, the largest capacity-multiple not exceeding 2^24, and route all index arithmetic through ring_advance/ring_dist so indices wrap at a capacity multiple. Power-of-two capacities keep wrap == 2^24 (identical codegen, no regression); non-power-of-two capacities now stay continuous across the wrap. ring_is_valid_cons() takes the ring instead of the capacity, and the ring_wrap macro is replaced by ring_advance/ring_dist. sizeof(struct ring_s) is preserved at 192 bytes. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
since the caller alredy add the barrier insruction correctly Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
prepare for the head and data separation Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Split the ring into a handle and a blob. struct ring_blob_s is the memory that may be shared across cores or survive a reboot: magic, geometry, the producer/consumer indices and the data. struct ring_s is a per-user handle in ordinary RAM holding a blob pointer and a boot-scoped magic. The handle magic is cleared on every boot, so a second ring_xxx_init() on the same handle in the same boot returns early instead of resetting a ring already in use, while the blob magic still drives warm-restart recovery. The caller now provides the blob storage and its total size; RING_DECLARE/RING_DEFINE/RING_INIT are removed. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
ring_mpmc_init() takes a rewind count: on a warm restart it pulls cons_tail back that many elements, so data the previous run had already consumed becomes readable again. Only meaningful for a blob that survives reboot (a located .noinit section or shared SRAM); a cold start has nothing to replay and a blob with corrupted indices is reset, so neither path applies it. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
Reservation could push prod_head - cons_head past capacity, letting a
producer write over the oldest live element.
After evicting old data to make room, the loop refreshed old_prod before
the prod_head CAS. That defeated the CAS: the space just evicted may
already have been taken by another producer, and with a refreshed
expected value the CAS succeeded anyway, reserving on top of the
eviction instead of retrying.
Two producers A and B, both writing count = 32 bytes into a full ring of
capacity = 1856, starting at prod_head = P, cons_head = C, P - C = 1856:
# who action prod_head cons_head used
1 A old_prod = P (read at top of loop) P C 1856
2 A free_space = 0, so discard = 32 P C 1856
3 A CAS cons_head: C -> C+32, succeeds P C+32 1824
4 A ring_mc_finish(), cons_tail = C+32 P C+32 1824
5 B sees free_space = 32 >= 32, skips the
eviction branch entirely, and
CAS prod_head: P -> P+32 succeeds P+32 C+32 1856
6 A re-reads prod_head, gets P+32 P+32 C+32 1856
7 A CAS prod_head: P+32 -> P+64, expected
value was just read, so it succeeds P+64 C+32 1888
Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
ring_mpmc_init / ring_spsc_init reused a persisted blob for a warm restart whenever its magic already read READY, then immediately called ring_dist to recover the live element count. ring_dist divides by wrappoint, so a blob left in memory by a previous firmware whose bytes happen to decode as magic == READY but wrappoint == 0 divides by zero on the very first ring_dist. Even when wrappoint is non-zero, trusting magic alone lets stale indices from an incompatible layout be reused as if valid. The ramlog syslog device hits this: it keeps its blob in a located .noinit section with CONFIG_RAMLOG_RECOVER_SIZE, so the first log write after a warm boot faults and the DIVBYZERO escalates to a HardFault. Geometry (capacity and wrappoint) is a pure function of buflen and is identical on every boot of a given build. Compute it up front and treat the blob as a valid warm restart only when magic == READY and the stored capacity and wrappoint both match the freshly computed values; any mismatch (stale residue, a resized buffer, or a zeroed wrappoint) falls back to a full ring_reset. The geometry is stamped before any ring_dist, so the divisor is always valid and only a provably self-consistent blob keeps its indices. Signed-off-by: wangjianyu3 <wangjianyu3@xiaomi.com> Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
…apped ring_mp_overwrite_start() spun forever in an interrupt handler, wedging the CPU. When the ring is full the reservation has to evict old data first, bounded by the committed region: dist(prod_tail, cons_head). Slots between prod_tail and prod_head belong to producers that reserved them but have not called ring_mp_finish() yet, so evicting them would hand out memory another producer is writing into. The loop therefore retried until enough data had been committed. That retry assumes the in-flight producers can make progress. The note path is reentrant and does not disable interrupts, so they may not: a producer preempted between its prod_head CAS and ring_mp_finish() cannot run while an interrupt handler on the same CPU spins waiting for it. prod_tail stays frozen, committed only shrinks as later producers evict, and the retry never terminates. Observed on TriCore during boot, with a note of 20 bytes on a ring of capacity 1856: prod_head = 0x763cc545 cnt=118 idx=3982661 prod_tail = 0x753cbded cnt=117 idx=3980781 cons_head = 0xc83cbde9 cnt=200 idx=3980777 cons_tail = 0xc83cbde9 cnt=200 idx=3980777 free_space = 0, committed = dist(prod_tail, cons_head) = 4, discard = 20 > 4 The RTS counters are consistent here - prod skew is 1, matching the single in-flight producer - so nothing is corrupt; the loop simply has no exit. Drop the request instead. Trace records are discardable, and the whole count is reported as discarded so CONFIG_DRIVERS_NOTE_STATS accounts for it. Returning an empty buf makes ring_mp_finish() return early, leaving no prod_head reservation unmatched. Once the stalled producer completes, prod_tail catches up and the ring resumes accepting writes. ring_sp_overwrite_start() is single-producer, so prod_head never runs ahead of prod_tail there and it needs no equivalent change. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
None bluk is best-effort: it takes as much as fits and reports how much that was. That suits a byte stream, where a short write only shortens the data, but not a framed one, where the reader locates each record by a length field carried in the record before it. A fragment left in the ring desynchronises the reader from the frame boundaries, so every later record is misparsed - the ring is not one record short, its whole remaining content is lost. Callers can pre-check ring_space() and skip the enqueue themselves, but that reads the consumer index twice for what is one decision, and every caller with a framed payload has to repeat it. Add a bulk variant that reserves count slots or none at all, so buf.cnt comes back as count or 0 and never as a prefix. Signed-off-by: chenzihan1 <chenzihan1@xiaomi.com>
ring_*_skipto computed the skip count as ring_dist(target, cons), a modular forward distance. A target no longer ahead of the consumer (already consumed, or lapped by the writer) wraps that distance into a near-full-ring count, so the skip would evict almost all still-live data instead of doing nothing. Check ring_is_valid_cons against the same cons snapshot before taking the distance, and no-op when the target is not ahead. Signed-off-by: yezhonghui <yezhonghui@xiaomi.com>
The data area is a uint8_t array and capacity counts bytes, but every index is still scaled by elem_size when an address is formed. Any elem_size above one walks past data[]: sixteen uint32_t enqueued into a 16-byte ring write 64 bytes. Remove elem_size everywhere. Counts, lengths and ring_buf_s fields are bytes; the copy macros take a void pointer instead of sizing the element from it, and the zero-copy start macros lose their elem_size argument. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
ring_sc_start() and ring_sc_peek() read element data right after the prod_tail load with no read barrier in between, so a weakly ordered core can return bytes older than that prod_tail. ring_sc_finish() published cons_tail after UP_WMB(), which orders stores only, so the producer could reuse slots still being read. Move the UP_RMB() in ring_sc_start() and ring_sc_peek() after the prod_tail load, and use UP_DMB() before the cons_tail store. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
ring_spsc_init() must not run on both sides at once: two callers that both find the blob uninitialized both reset it, so a side stalled between its magic check and its reset wipes whatever the other side has already initialized and sent. Add a slave argument: a slave binds its handle only once the blob is initialized and never resets it, so the master can own init and the slave only bind, without atomic RMW on the shared memory. The return type becomes int so a slave gets -EAGAIN/-EINVAL to retry. Signed-off-by: Bowen Wang <wangbowen6@xiaomi.com>
|
GUIDINGLI
approved these changes
Nov 17, 2026
xiaoxiang781216
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
include/nuttx/ring.h, a header-only lock-free byte ring queue modelledon DPDK's
rte_ring. Producer and consumer each keep a head/tail pair packedwith a thread counter into one 32-bit word (
{cnt:8, idx:24}, DPDK's RTSscheme), so multi-producer and multi-consumer sides commit with a single CAS
and never spin waiting for another thread's tail.
also works on shared memory between cores without coherent atomics), MPSC,
SPMC, MPMC.
_bulk) enqueue/dequeue, zero-copy_start/_finish,skip/skipto,peek/peek_at, and overwrite mode(SPMC/MPMC) that evicts the oldest data when full.
may be shared across cores or survive a reboot) and a per-user handle in
ordinary RAM.
ring_*_init()detects a blob left by a previous boot andrecovers its contents (warm restart), with an optional rewind to replay
already-consumed data, and resets a blob whose indices or geometry do not
match.
ring_spsc_init()has a slave mode so only one side of across-core ring ever resets it.
There is no in-tree user yet. Internally it backs the note/notesnap, ramlog
and uart_rpmsg drivers; those conversions will follow as separate PRs per
driver.
Why 45 commits: the history is kept as developed by several authors
(Bowen Wang, chenzihan1, wangjianyu3, Xiang Xiao, yezhonghui, ganjing) so each
change stays attributed and reviewable on its own. Every commit only touches
include/nuttx/ring.h; changes to other files in the original commits (thein-tree users above) were dropped, and those commits carry an extra
Signed-off-by.
Depends on #21843 (
atomic_ptr_t,atomic_cmpxchg_release_acquire) and#21895 (
irq_save/irq_restore), both merged.Impact
to existing code.
testing/libc/ring).Testing
Host: Ubuntu 22.04.5 LTS x86_64; sim: gcc 13.1.0; qemu-armv8a: aarch64-none-elf-gcc 12.2.1, QEMU 10.2.1
The cmocka tests from apache/nuttx-apps#4075 (80 cases: 36 SPSC, 11 MPSC,
11 SPMC, 22 MPMC, including pthread producer/consumer stress, wrap-around on
non-power-of-two capacities, overwrite, warm restart/rewind, geometry
mismatch and SPSC slave init), with
CONFIG_TESTING_CMOCKA=y CONFIG_TESTING_RING=y(qemu-armv8a also needsCONFIG_ARCH_SETJMP_H=yfor cmocka):
sim:nshqemu-armv8a:nsh_smp(CONFIG_SMP_NCPUS=2)sim:nsh
qemu-armv8a:nsh_smp
No regression in rpmsg: sim:rpserver/rpproxy and qemu-armv8a:rpserver/rpproxy boot and pass the fixed rpmsg cases (dump, ping, rpmsgfs; 15/15). qemu-armv8a was built with LIBC_ENVPATH and BINFMT_CHECKEXEC disabled, which current master needs for nxinit to start nsh there; unrelated to this change.