Skip to content

Matter: bound SubscribeRequest data-version filter memory - #25160

Open
SoWizard wants to merge 1 commit into
arendst:developmentfrom
SoWizard:claude/matter-subscription-filter-memory
Open

SoWizard wants to merge 1 commit into
arendst:developmentfrom
SoWizard:claude/matter-subscription-filter-memory

Conversation

@SoWizard

Copy link
Copy Markdown
Contributor

Description:

Apple Home subscribes with a SubscribeRequest that carries many DataVersionFilters. The request recorded on an ESP32-C5 bridge was 1,166 bytes: 1 attribute path, 1 event path, 1 event filter and 80 data-version filters. process_incoming() parses the whole request into a generic TLV tree, then SubscribeRequestMessage.from_TLV() builds a typed list of every filter, and both stay alive while the subscription is set up. The responder never uses these filters when reporting. On a non-PSRAM ESP32-C5 this temporary peak left only a few KB free, and subscription setup hit BE_MALLOC_FAIL.

This change adds a fast-track path for opcode 0x03, next to the existing Read and Invoke ones. SubscribeRequestMessage.parse_raw() parses the request element by element, and parses then drops each data-version filter instead of keeping it (tag 8 stays an empty array). After the standalone ACK, validate_raw_filters() decodes each filter with the existing DataVersionFilterIB.from_TLV(), one at a time.

  • The order generic parse, then standalone ACK, then typed decode is unchanged, and so are the errors raised: malformed TLV is still rejected before the ACK, and an invalid filter after it.
  • Requests whose root is not a structure fall back to TLV.parse(). With a duplicate tag 8, only the first is used, as before.
  • SubscribeRequestMessage.from_TLV() still decodes and keeps every filter for other callers.
  • Only the SubscribeRequest path changes; ReadRequest is untouched. As before, filters are not used to suppress reports.

Filters are decoded a second time after the ACK only to keep that existing ACK and error order. On the host this adds about 0.5 ms per 80-filter request (about 1.06 ms to 1.6 ms). If you prefer, validation can be dropped or folded into the first pass.

Related issue: None; a reproducer test is included.

Validation:

  • Fresh host Berry interpreter built from this branch, rebased on current development (fb629ed).

  • New subscription_filter_memory_tests.be: 12 passed, 0 failed. With the base revision's Matter_IM.be and Matter_IM_Data.be passed as the first argument: 10 passed, 2 failed. The two failures are the memory cases: the tree handed to subscribe_request keeps 400 filter nodes, and one parse builds 414 TLV nodes. The test request is built with the recorded shape and size (1,166 bytes). The other cases check that setup, ACK order and errors match the base, including a frame header before the payload, filters before the other fields, a duplicate tag 8, an empty filter array and a non-struct root.

  • Host native allocation peak during setup (64-bit host, synthetic request with the recorded shape; these are not ESP32 byte counts):

    Data-version filters Before After
    50 90,400 16,824
    80 139,636 16,824
    160 268,716 16,824

    With an extra-allocation budget of 96 KiB, the base fails setup with BE_MALLOC_FAIL after the GC retry; this branch succeeds.

  • A local differential check of 119 malformed, duplicate, unknown-tag and truncated SubscribeRequests gives identical errors, ACK counts, subscription state and response bytes compared with the base. That harness is not included in this PR.

  • Existing subscription_setup_tests.be (18 passed), subscription_ack_tests.be (13 passed) and http_recovery_tests.be pass. run_tests.be and button_stability_tests.be stop at the same Wi-Fi diagnostics assertion on unchanged development as on this branch.

  • The exact branch builds for ESP32-C5 with no compiler warnings: Arduino 3.3.12, platform 2026.09.50, ESP-IDF 5.5.5. Static RAM is unchanged.

From the repository root:

make -C lib/libesp32/berry -j4
cd lib/libesp32/berry_matter/tests
../../berry/berry -g subscription_filter_memory_tests.be
../../berry/berry -g subscription_setup_tests.be
../../berry/berry -g subscription_ack_tests.be
../../berry/berry -g http_recovery_tests.be

To reproduce the original behaviour, export the base revision's Matter_IM.be and Matter_IM_Data.be to a separate directory and pass that directory as the first argument to subscription_filter_memory_tests.be.

The same parsing change has run since 17 October on an ESP32-C5 Apple Home bridge, in a combined custom build that also contained other local changes (an allocation-failure safety change and diagnostics). Six of six real 80-filter subscription setups completed, with 63–85 KB free when each setup finished, compared with 6–10 KB free and allocation failures before. This exact branch has not been flashed, and no long-term stability claim is made.

Checklist:

  • The pull request is done against the latest development branch
  • Only relevant files were touched
  • Only one feature/fix was added per PR and the code change compiles without warnings
  • The code change is tested and works with Tasmota core ESP8266 V.2.7.8 — not applicable to this ESP32 Matter implementation
  • The code change is tested and works with Tasmota core ESP32 V.3.3.12 from Platform 2026.09.50 — exact branch compilation and prior combined-build observation described above
  • I accept the CLA.

NOTE: The code change must pass CI tests. Your PR cannot be merged unless tests pass

🤖 Generated with Claude Code

Apple Home SubscribeRequests carry about 80 DataVersionFilters (1,166 bytes).
The generic TLV tree and the typed filter list were both kept while the
subscription was set up, although the filters are not used for reports. On a
non-PSRAM ESP32 this temporary peak could make setup fail with BE_MALLOC_FAIL.

Handle SubscribeRequest on the fast track: SubscribeRequestMessage.parse_raw()
parses the request element by element and drops each filter after parsing it,
and validate_raw_filters() decodes the filters one at a time after the
standalone ACK. Parse, ACK and typed-decode order and the raised errors are
unchanged; from_TLV() still decodes all filters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SoWizard SoWizard mentioned this pull request Oct 19, 2026
4 of 6 tasks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant