Skip to content

RMT copy encoder reports COMPLETE+MEM_FULL simultaneously on exact-fit buffer case (IDFGH-16460) #17592

@avjui

Description

@avjui

Answers checklist.

  • I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
  • I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
  • I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v.5.5.0

Espressif SoC revision.

ESP32-S3 (QFN56) (revision v0.2)

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

esp32-s3-wroom-1-n16r8

Power Supply used.

USB

What is the expected behavior?

When the copy encoder exactly fills the RMT buffer (mem_have == mem_want), it should return a consistent state that allows the user encoder to advance correctly without duplication or corruption.

Either:

  • Return COMPLETE only (encoding finished cleanly), or
  • Return MEM_FULL only (caller yields, then resumes with COMPLETE later).
  • In either case, the state should not cause duplicated symbol output or force the caller into an ambiguous “COMPLETE+MEM_FULL” situation.

What is the actual behavior?

  • In ESP-IDF v5.5, the copy encoder sets both RMT_ENCODING_COMPLETE and RMT_ENCODING_MEM_FULL when mem_have == mem_want.
  • At the same time, it resets its internal last_symbol_index to 0.
  • This combination leads to problems in user encoders:
  • If the caller reacts to MEM_FULL first and yields, the COMPLETE is not processed, and on resume the encoder re-sends the same symbols → duplicated characters.
  • If the caller processes COMPLETE first, the MEM_FULL yield is skipped, and the driver may fail to switch buffers correctly → corrupted or truncated output.
  • In practice, this results in systematic data duplication/corruption on every “exact-fit” condition

Steps to reproduce.

#include <stdio.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/rmt_tx.h"
#include "driver/rmt_encoder.h"
#include "driver/uart.h"
#include "esp_log.h"
#include "esp_clk_tree.h"

static const char *TAG = "RMT_UART_STATEMACHINE";

#define TX_GPIO_NUM GPIO_NUM_4
#define RX_GPIO_NUM GPIO_NUM_5
#define UART_PORT UART_NUM_1
#define BAUD_RATE 9600
#define MEMORY_BLOCKS 48 * 4 // Small memory to test chunked transmission

typedef enum
{
    UART_ENCODER_INIT,
    UART_ENCODER_GENERATING_FRAME,
    UART_ENCODER_TRANSMITTING_FRAME,
    UART_ENCODER_DONE
} uart_encoder_state_t;

typedef struct
{
    rmt_encoder_t base;
    rmt_encoder_t *copy_encoder;
    uart_encoder_state_t state;
    size_t byte_index;
    rmt_symbol_word_t frame_symbols[10]; // Current frame being transmitted
    uint16_t bit_duration_ticks;
} uart_encoder_t;

static int encoder_call_count = 0;

static void setup_uart_receiver(void)
{
    uart_config_t uart_config = {
        .baud_rate = BAUD_RATE,
        .data_bits = UART_DATA_8_BITS,
        .parity = UART_PARITY_DISABLE,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
        .source_clk = UART_SCLK_DEFAULT,
    };

    uart_driver_install(UART_PORT, 2048, 0, 0, NULL, 0);
    uart_param_config(UART_PORT, &uart_config);
    uart_set_pin(UART_PORT, UART_PIN_NO_CHANGE, RX_GPIO_NUM, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);
}

static size_t uart_encode(rmt_encoder_t *encoder, rmt_channel_handle_t channel,
                          const void *primary_data, size_t data_size, rmt_encode_state_t *ret_state)
{
    uart_encoder_t *uart_encoder = __containerof(encoder, uart_encoder_t, base);
    const char *data = (const char *)primary_data;
    rmt_encode_state_t session_state = RMT_ENCODING_RESET;
    size_t encoded_symbols = 0;
    static int mem_full_count = 0;


    // Process characters until memory is full or all done
    while (uart_encoder->byte_index < data_size)
    {
        switch (uart_encoder->state)
        {
        case UART_ENCODER_INIT:
        {
            uart_encoder->byte_index = 0;
            uart_encoder->state = UART_ENCODER_GENERATING_FRAME;
            // Fall through
        }
        case UART_ENCODER_GENERATING_FRAME:
        {
            uint8_t current_char = data[uart_encoder->byte_index];
            uint16_t frame = (current_char << 1) | (1 << 9); // START + DATA + STOP

            // Generate 10 symbols for this frame
            for (int bit = 0; bit < 10; bit++)
            {
                uint8_t bit_value = (frame >> bit) & 0x01;
                uint8_t rmt_level = bit_value ? 0 : 1; // Invert for hardware
                uart_encoder->frame_symbols[bit] = (rmt_symbol_word_t){
                    .duration0 = (uint16_t)(uart_encoder->bit_duration_ticks / 2),
                    .level0 = rmt_level,
                    .duration1 = (uint16_t)(uart_encoder->bit_duration_ticks / 2),
                    .level1 = rmt_level};
            }
            uart_encoder->state = UART_ENCODER_TRANSMITTING_FRAME;
            // Fall through
        }
        case UART_ENCODER_TRANSMITTING_FRAME:
        {
            size_t frame_encoded = uart_encoder->copy_encoder->encode(
                uart_encoder->copy_encoder, channel,
                uart_encoder->frame_symbols,
                10 * sizeof(rmt_symbol_word_t),
                &session_state);

            encoded_symbols += frame_encoded;

            if (session_state & RMT_ENCODING_MEM_FULL)
            {
                mem_full_count++;
                // Only log critical MEM_FULL events (every 4th one where duplications occur)
                if (mem_full_count % 4 == 0) {
                    esp_rom_printf("MF%d@%d:%c\n", mem_full_count, 
                                   (int)uart_encoder->byte_index, data[uart_encoder->byte_index]);
                }
                *ret_state = RMT_ENCODING_MEM_FULL;
                return encoded_symbols;
            }

            if (session_state & RMT_ENCODING_COMPLETE)
            {
                // Frame complete, move to next character
                uart_encoder->byte_index++;
                uart_encoder->state = UART_ENCODER_GENERATING_FRAME;
                // Continue loop to process next character
                continue;
            }

            // If we get here, something unexpected happened
            *ret_state = session_state;
            return encoded_symbols;
        }
        case UART_ENCODER_DONE:
        {
            *ret_state = RMT_ENCODING_COMPLETE;
            return encoded_symbols;
        }
        }
    }

    // All characters processed
    uart_encoder->state = UART_ENCODER_DONE;
    *ret_state = RMT_ENCODING_COMPLETE;
    return encoded_symbols;
}

static esp_err_t uart_encoder_reset(rmt_encoder_t *encoder)
{
    uart_encoder_t *uart_encoder = __containerof(encoder, uart_encoder_t, base);
    ESP_LOGI(TAG, "Encoder reset called - state: %d, byte_index: %d",
             uart_encoder->state, (int)uart_encoder->byte_index);

    // Reset to initial state
    uart_encoder->state = UART_ENCODER_INIT;
    uart_encoder->byte_index = 0;

    // Reset copy encoder
    if (uart_encoder->copy_encoder)
    {
        rmt_encoder_reset(uart_encoder->copy_encoder);
    }
    return ESP_OK;
}

static esp_err_t uart_encoder_del(rmt_encoder_t *encoder)
{
    uart_encoder_t *uart_encoder = __containerof(encoder, uart_encoder_t, base);

    // No symbols array to free in the frame-by-frame approach
    // frame_symbols is a fixed array in the struct

    if (uart_encoder->copy_encoder)
    {
        rmt_del_encoder(uart_encoder->copy_encoder);
    }
    free(uart_encoder);
    return ESP_OK;
}

static esp_err_t new_uart_encoder(uint16_t bit_duration, rmt_encoder_handle_t *ret_encoder)
{
    uart_encoder_t *uart_encoder = (uart_encoder_t *)calloc(1, sizeof(uart_encoder_t));
    if (!uart_encoder)
    {
        return ESP_ERR_NO_MEM;
    }

    uart_encoder->base.encode = uart_encode;
    uart_encoder->base.del = uart_encoder_del;
    uart_encoder->base.reset = uart_encoder_reset;
    uart_encoder->bit_duration_ticks = bit_duration;
    uart_encoder->state = UART_ENCODER_INIT;

    // Create copy encoder
    rmt_copy_encoder_config_t copy_config = {};
    esp_err_t ret = rmt_new_copy_encoder(&copy_config, &uart_encoder->copy_encoder);
    if (ret != ESP_OK)
    {
        free(uart_encoder);
        return ret;
    }

    *ret_encoder = &uart_encoder->base;
    return ESP_OK;
}

static void uart_test_task(void *param)
{
    ESP_LOGI(TAG, "RMT UART Encoder Test");
    ESP_LOGI(TAG, "Connect GPIO %d (TX) to GPIO %d (RX)", TX_GPIO_NUM, RX_GPIO_NUM);

    setup_uart_receiver();

    // Use proven timing values
    uint32_t resolution_hz = 6666666;  // Working resolution
    uint16_t bit_duration_ticks = 694; // Working bit duration for 9600 baud

    ESP_LOGI(TAG, "Timing: RMT=%d Hz, bit_duration=%d ticks", (int)resolution_hz, bit_duration_ticks);

    // Create RMT TX channel
    rmt_tx_channel_config_t tx_config = {
        .gpio_num = TX_GPIO_NUM,
        .clk_src = RMT_CLK_SRC_DEFAULT,
        .resolution_hz = resolution_hz,
        .mem_block_symbols = MEMORY_BLOCKS,
        .trans_queue_depth = 1,
        .intr_priority = 0,
        .flags = {
            .invert_out = true, // Hardware inversion
            .with_dma = false,
            .io_loop_back = false,
            .io_od_mode = false,
            .allow_pd = false}};

    rmt_channel_handle_t tx_channel = NULL;
    ESP_ERROR_CHECK(rmt_new_tx_channel(&tx_config, &tx_channel));
    ESP_ERROR_CHECK(rmt_enable(tx_channel));

    // Create UART encoder
    rmt_encoder_handle_t uart_encoder = NULL;
    ESP_ERROR_CHECK(new_uart_encoder(bit_duration_ticks, &uart_encoder));

    const char *test_msg = "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.";
    size_t msg_len = strlen(test_msg);

    ESP_LOGI(TAG, "Test message: \"%s\" (%d chars = %d symbols)", test_msg, (int)msg_len, (int)(msg_len * 10));
    ESP_LOGI(TAG, "Memory blocks: %d, estimated chunks: %d",
             MEMORY_BLOCKS, (int)((msg_len * 10 + MEMORY_BLOCKS - 1) / MEMORY_BLOCKS));

    // Show expected MEM_FULL positions
    ESP_LOGI(TAG, "=== EXPECTED MEM_FULL POSITIONS (every ~19 chars due to 192 symbol buffer) ===");
    for (int i = 1; i <= 7; i++) {
        int expected_pos = (i * 192) / 10; // 192 symbols = ~19.2 chars
        if (expected_pos < msg_len) {
            ESP_LOGI(TAG, "MEM_FULL #%d expected around position %d: '%c'", 
                     i, expected_pos, test_msg[expected_pos]);
        }
    }

    // Clear UART buffer
    uart_flush(UART_PORT);
    vTaskDelay(pdMS_TO_TICKS(100));

    // Reset encoder call counter
    encoder_call_count = 0;

    // Transmit via RMT
    rmt_transmit_config_t transmit_config = {
        .loop_count = 0,
        .flags = {0}};

    ESP_LOGI(TAG, "Starting transmission...");
    ESP_ERROR_CHECK(rmt_transmit(tx_channel, uart_encoder, test_msg, msg_len, &transmit_config));
    ESP_ERROR_CHECK(rmt_tx_wait_all_done(tx_channel, pdMS_TO_TICKS(10000)));

    ESP_LOGI(TAG, "Transmission complete after %d encoder calls", encoder_call_count);

    // Read received data from UART
    uint8_t rx_buffer[1024];
    memset(rx_buffer, 0, sizeof(rx_buffer));

    // Give time for all data to be received
    vTaskDelay(pdMS_TO_TICKS(1000));

    int received = uart_read_bytes(UART_PORT, rx_buffer, sizeof(rx_buffer) - 1, pdMS_TO_TICKS(3000));

    if (received > 0)
    {
        rx_buffer[received] = '\0';
        ESP_LOGI(TAG, "Expected: \"%s\" (%d bytes)", test_msg, (int)msg_len);
        ESP_LOGI(TAG, "Received: \"%s\" (%d bytes)", rx_buffer, received);

        if (received == msg_len && memcmp(test_msg, rx_buffer, msg_len) == 0)
        {
            ESP_LOGI(TAG, "SUCCESS: Message transmitted correctly!");
        }
        else
        {
            ESP_LOGE(TAG, "FAILURE: Message mismatch detected");
            if (received != msg_len)
            {
                ESP_LOGE(TAG, "Length mismatch: expected %d, got %d", (int)msg_len, received);
                ESP_LOGE(TAG, "Extra bytes: %d (indicates character duplication)", received - (int)msg_len);
            }

            ESP_LOGI(TAG, "Analyzing received data for duplications:");

            // Check for duplications at predicted positions
            int duplication_positions[] = {48, 86, 123, 161, 198};
            char expected_chars[] = {'g', 't', 'd', 'd', 'r'};
            
            for (int i = 0; i < 5; i++) {
                int pos = duplication_positions[i];
                if (pos < received && pos < msg_len) {
                    ESP_LOGI(TAG, "Position %d: Expected '%c', Got '%c' %s", 
                             pos, expected_chars[i], rx_buffer[pos],
                             (rx_buffer[pos] != expected_chars[i]) ? "❌ CORRUPTED" : "✅ OK");
                    
                    // Check if there's a duplication (character appears twice)
                    if (pos + 1 < received) {
                        if (rx_buffer[pos] == rx_buffer[pos + 1]) {
                            ESP_LOGE(TAG, "DUPLICATION DETECTED: '%c' appears twice at positions %d and %d", 
                                     rx_buffer[pos], pos, pos + 1);
                        }
                    }
                }
            }

            // Show first difference for debugging
            int compare_len = (received < msg_len) ? received : msg_len;
            for (int i = 0; i < compare_len; i++)
            {
                if (rx_buffer[i] != test_msg[i])
                {
                    ESP_LOGE(TAG, "First difference at position %d: expected 0x%02X '%c', got 0x%02X '%c'",
                             i, test_msg[i], test_msg[i], rx_buffer[i], rx_buffer[i]);
                    
                    // Check if this matches a predicted corruption position
                    for (int j = 0; j < 5; j++) {
                        if (i == duplication_positions[j]) {
                            ESP_LOGE(TAG, "This matches predicted corruption after MEM_FULL #%d!", (j+1)*4);
                            break;
                        }
                    }
                    break;
                }
            }
        }
    }
    else
    {
        ESP_LOGE(TAG, "No data received from UART - check wiring");
    }

    // Cleanup
    rmt_disable(tx_channel);
    rmt_del_channel(tx_channel);
    rmt_del_encoder(uart_encoder);
    uart_driver_delete(UART_PORT);

    ESP_LOGI(TAG, "Test complete");
    vTaskDelete(NULL);
}
extern "C" void app_main(void)
{
    xTaskCreate(uart_test_task, "uart_test", 8192, NULL, 5, NULL);
}

Debug Logs.

ESP-ROM:esp32s3-20210327

Build:Mar 27 2021

rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)

SPIWP:0xee

mode:DIO, clock div:1

load:0x3fce2820,len:0x159c

load:0x403c8700,len:0xd24

load:0x403cb700,len:0x2f48

entry 0x403c891c

I (24) boot: ESP-IDF 5.5.0 2nd stage bootloader

I (25) boot: compile time Sep 13 2025 23:13:33

I (25) boot: Multicore bootloader

I (25) boot: chip revision: v0.2

I (28) boot: efuse block revision: v1.3

I (31) boot.esp32s3: Boot SPI Speed : 80MHz

I (35) boot.esp32s3: SPI Mode       : DIO

I (39) boot.esp32s3: SPI Flash Size : 8MB

I (43) boot: Enabling RNG early entropy source...

I (47) boot: Partition Table:

I (50) boot: ## Label            Usage          Type ST Offset   Length

I (56) boot:  0 nvs              WiFi data        01 02 00009000 00006000

I (62) boot:  1 phy_init         RF data          01 01 0000f000 00001000

I (69) boot:  2 factory          factory app      00 00 00010000 00100000

I (76) boot: End of partition table

I (79) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=0d960h ( 55648) map

I (96) esp_image: segment 1: paddr=0001d988 vaddr=3fc94c00 size=02690h (  9872) load

I (98) esp_image: segment 2: paddr=00020020 vaddr=42000020 size=1e380h (123776) map

I (123) esp_image: segment 3: paddr=0003e3a8 vaddr=3fc97290 size=00768h (  1896) load

I (124) esp_image: segment 4: paddr=0003eb18 vaddr=40374000 size=10be0h ( 68576) load

I (142) esp_image: segment 5: paddr=0004f700 vaddr=600fe000 size=00020h (    32) load

I (148) boot: Loaded app from partition at offset 0x10000

I (149) boot: Disabling RNG early entropy source...

I (159) cpu_start: Multicore app

I (168) cpu_start: Pro cpu start user code

I (168) cpu_start: cpu freq: 160000000 Hz

I (169) app_init: Application information:

I (169) app_init: Project name:     rmt_serial

I (173) app_init: App version:      4d99ecc-dirty

I (177) app_init: Compile time:     Sep 13 2025 23:13:10

I (182) app_init: ELF file SHA256:  5c51484e1...

I (187) app_init: ESP-IDF:          5.5.0

I (190) efuse_init: Min chip rev:     v0.0

I (194) efuse_init: Max chip rev:     v0.99 

I (198) efuse_init: Chip rev:         v0.2

I (202) heap_init: Initializing. RAM available for dynamic allocation:

I (208) heap_init: At 3FC982E8 len 00051428 (325 KiB): RAM

I (213) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM

I (218) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM

I (224) heap_init: At 600FE020 len 00001FC8 (7 KiB): RTCRAM

I (230) spi_flash: detected chip: boya

I (232) spi_flash: flash io: dio

W (235) spi_flash: Detected size(16384k) larger than the size in the binary image header(8192k). Using the size in the binary image header.

I (248) sleep_gpio: Configure to isolate all GPIO pins in sleep state

I (254) sleep_gpio: Enable automatic switching of GPIO sleep configuration

I (261) main_task: Started on CPU0

I (281) main_task: Calling app_main()

I (281) RMT_UART_STATEMACHINE: RMT UART Encoder Test (State Machine Version)

I (281) RMT_UART_STATEMACHINE: Connect GPIO 4 (TX) to GPIO 5 (RX)

I (291) RMT_UART_STATEMACHINE: Timing: RMT=6666666 Hz, bit_duration=694 ticks

D (291) rmt: new group(0) at 0x3fcecb44, occupy=ffffff00

D (301) rmt: group (0) clock resolution:80000000Hz

D (301) rmt: new tx channel(0,0) at 0x3fcec910, gpio=4, res=6666666Hz, hw_mem_base=0x60016800, dma_mem_base=0x0, dma_nodes=0x0, ping_pong_size=96, queue_depth=1

D (321) rmt: new copy encoder @0x3fcecc10

I (321) RMT_UART_STATEMACHINE: Test message: "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet." (295 chars = 2950 symbols)

I (351) RMT_UART_STATEMACHINE: Memory blocks: 192, estimated chunks: 260

I (411) main_task: Returned from app_main()

I (511) RMT_UART_STATEMACHINE: Starting transmission...

I (511) RMT_UART_STATEMACHINE: Encoder reset called - state: 0, byte_index: 0

I (821) RMT_UART_STATEMACHINE: Transmission complete after 31 encoder calls

I (4821) RMT_UART_STATEMACHINE: Expected: "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet." (295 bytes)

I (4831) RMT_UART_STATEMACHINE: Received: "Lorem ipsum dolor sit amet, consetetur sadipscinng elitr, sed diam nonumy eirmod tempor inviduntt ut labore et dolore magna aliquyam erat, sed ddiam voluptua. At vero eos et accusam et justo dduo dolores et ea rebum. Stet clita kasd gubergrren, no sea takimata sanctus est Lorem ipsum dollor sit amet." (301 bytes)

E (4871) RMT_UART_STATEMACHINE: FAILURE: Message mismatch detected

E (4871) RMT_UART_STATEMACHINE: Length mismatch: expected 295, got 301

E (4881) RMT_UART_STATEMACHINE: Extra bytes: 6 (indicates character duplication)

I (4891) RMT_UART_STATEMACHINE: MEM_FULL events logged during transmission:

I (4901) RMT_UART_STATEMACHINE: MF4@47:n  -> Next char at pos 48 should be 'g'

I (4901) RMT_UART_STATEMACHINE: MF8@85:r  -> Next char at pos 86 should be 't'

I (4911) RMT_UART_STATEMACHINE: MF12@122:a -> Next char at pos 123 should be 'd'

I (4921) RMT_UART_STATEMACHINE: MF16@160:e -> Next char at pos 161 should be 'd'

I (4931) RMT_UART_STATEMACHINE: MF20@197:e -> Next char at pos 198 should be 'r'

I (4931) RMT_UART_STATEMACHINE: Analyzing received data for duplications:

I (4941) RMT_UART_STATEMACHINE: Position 48: Expected 'g', Got 'n' ❌ CORRUPTED

I (4951) RMT_UART_STATEMACHINE: Position 86: Expected 't', Got 'r' ❌ CORRUPTED

I (4951) RMT_UART_STATEMACHINE: Position 123: Expected 'd', Got ' ' ❌ CORRUPTED

I (4961) RMT_UART_STATEMACHINE: Position 161: Expected 'd', Got ' ' ❌ CORRUPTED

I (4971) RMT_UART_STATEMACHINE: Position 198: Expected 'r', Got 'l' ❌ CORRUPTED

E (4981) RMT_UART_STATEMACHINE: First difference at position 48: expected 0x67 'g', got 0x6E 'n'

E (4981) RMT_UART_STATEMACHINE: This matches predicted corruption after MEM_FULL #4!

I (4991) RMT_UART_STATEMACHINE: === CONCLUSION ===

I (5001) RMT_UART_STATEMACHINE: Every 4th MEM_FULL event in ESP-IDF copy encoder causes

I (5001) RMT_UART_STATEMACHINE: the next character to be corrupted/duplicated.

D (5021) rmt: del tx channel(0,0)

D (5021) rmt: del group(0)

I (5021) RMT_UART_STATEMACHINE: Test complete

Diagnostic report archive.

No response

More Information.

if i change line

bool encoding_truncated = mem_have < mem_want;

from
bool encoding_truncated = mem_have < mem_want;
to
bool encoding_truncated = mem_have <= mem_want;
the error is gone.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions