Skip to content

fix(autoware_behavior_velocity_traffic_light_module): shut down rclcpp after each fixture test - #13507

Merged
xmfcx merged 1 commit into
autowarefoundation:mainfrom
xmfcx:fix-traffic-light-test-shutdown
Oct 21, 2026
Merged

xmfcx merged 1 commit into
autowarefoundation:mainfrom
xmfcx:fix-traffic-light-test-shutdown

Conversation

@xmfcx

@xmfcx xmfcx commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Description

test_autoware_behavior_velocity_traffic_light_module sometimes passes all 13 tests and then exits with SIGSEGV (-11). CTest then marks it failed.

The cause is in TrafficLightModuleTest. Its SetUp calls rclcpp::init() when the context is not ok, but no test calls rclcpp::shutdown(). The tests in test_node_interface.cpp do shut down. So the default context stays valid until the process exits. Its static destructor then calls Context::shutdown() during exit, and that can crash. The likely reason is that other static objects that it needs are already destroyed at that point. The backtrace does not show which one.

The backtrace of a crash:

#0  rclcpp::Context::shutdown(std::string const&) () from /opt/ros/jazzy/lib/librclcpp.so
#1  rclcpp::Context::~Context() () from /opt/ros/jazzy/lib/librclcpp.so
#2  ?? () from /opt/ros/jazzy/lib/librclcpp.so
#3  __run_exit_handlers (...) at ./stdlib/exit.c:108
#4  __GI_exit (...) at ./stdlib/exit.c:138

This PR adds a TearDown() that calls rclcpp::shutdown(). SetUp already calls rclcpp::init() again when the context is not ok, so each test gets a fresh context.

Related links

The failure on main of autoware_core, in the build-and-test-jazzy-above job, which builds this package from autoware_universe main:

How was this PR tested?

See Verification below.

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None. Only a test changes.

AI usage

AI usage: Written with Claude Code in Breeze from the failed autoware_core CI runs. It reproduced the crash, read the core dumps, and wrote the fix.

Self-review: One review round by a separate AI session with a clean context, on ec58127e4f. It found no problem in the code. It found one contradiction in this description, which is now fixed. It also noted a side benefit: the fixture no longer leaves a context alive, so the unguarded rclcpp::init() in test_node_interface.cpp would no longer throw under --gtest_shuffle.

Verification: The test binary ran 240 times before and after the fix on ROS 2 Jazzy with Cyclone DDS: 27 crashes before, 0 after. colcon test of the package passes.

Repeat run before the fix

  • Local Jazzy workspace, RMW_IMPLEMENTATION=rmw_cyclonedds_cpp (the same as the CI job), autoware_universe at 4fb338fa76. The test file is the same as on main.
  • The test binary ran 240 times, 8 at a time, each with its own ROS_DOMAIN_ID: 27 of 240 exited with 139 (SIGSEGV). Each crash log ends with [ PASSED ] 13 tests., the same as in CI.
  • Under gdb, 120 runs gave no crash. The timing must be real to hit it.

Core dumps

  • 48 more runs with core dumps on: 5 crashed. Two cores read with gdb show the backtrace above, in the main thread.

Repeat run after the fix

  • colcon build --packages-select autoware_behavior_velocity_traffic_light_module, then the same 240 runs: 240 of 240 exited with 0.

colcon test

  • colcon test --packages-select autoware_behavior_velocity_traffic_light_module: 50 tests, 0 errors, 0 failures, 14 skipped (the skips are cppcheck).

  • Did not run: Humble, because the local workspace is Jazzy only.

  • Did not run: Fast DDS or Zenoh. I tested only Cyclone DDS, which the CI job uses.

  • Did not run: the CI of autoware_core or autoware_universe with this change.

…p after each fixture test

TrafficLightModuleTest::SetUp calls rclcpp::init() when the context is not
ok, but nothing calls rclcpp::shutdown(). The default context then stays
valid until exit, and its static destructor calls Context::shutdown() while
other statics are already gone. This segfaults the test binary after all
tests pass, in about 1 of 10 runs.

Signed-off-by: Mete Fatih Cırıt <mfc@autoware.org>
@github-actions github-actions Bot added the component:planning Route planning, decision-making, and navigation. (auto-assigned) label Oct 16, 2026
@github-actions

Copy link
Copy Markdown

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@xmfcx xmfcx added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Oct 16, 2026
@xmfcx xmfcx self-assigned this Oct 16, 2026
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 37.64%. Comparing base (f81a3bf) to head (ec58127).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #13507      +/-   ##
==========================================
- Coverage   37.68%   37.64%   -0.05%     
==========================================
  Files        2034     2037       +3     
  Lines      141948   142131     +183     
==========================================
+ Hits        53500    53505       +5     
- Misses      88448    88626     +178     
Flag Coverage Δ *Carryforward flag
daily ?
full-suite 37.64% <ø> (-0.05%) ⬇️ Carriedforward from fad5625

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@xmfcx
xmfcx marked this pull request as ready for review October 16, 2026 18:54
@xmfcx
xmfcx requested a review from mitsudome-r October 16, 2026 18:54
@xmfcx xmfcx added run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) and removed run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) labels Oct 20, 2026
@xmfcx
xmfcx merged commit 693a4b1 into autowarefoundation:main Oct 21, 2026
94 of 96 checks passed
@xmfcx
xmfcx deleted the fix-traffic-light-test-shutdown branch October 21, 2026 08:22
@github-project-automation github-project-automation Bot moved this from To Triage to Done in Software Working Group Oct 21, 2026
paulsohn-github-app Bot pushed a commit to paulsohn/autoware_universe_planning_2 that referenced this pull request Oct 21, 2026
…p after each fixture test (autowarefoundation/autoware_universe#13507)

TrafficLightModuleTest::SetUp calls rclcpp::init() when the context is not
ok, but nothing calls rclcpp::shutdown(). The default context then stays
valid until exit, and its static destructor calls Context::shutdown() while
other statics are already gone. This segfaults the test binary after all
tests pass, in about 1 of 10 runs.

Signed-off-by: Mete Fatih Cırıt <mfc@autoware.org>
shmpwk pushed a commit to tier4/pilot_auto_planning that referenced this pull request Nov 5, 2026
…p after each fixture test (autowarefoundation/autoware_universe#13507)

TrafficLightModuleTest::SetUp calls rclcpp::init() when the context is not
ok, but nothing calls rclcpp::shutdown(). The default context then stays
valid until exit, and its static destructor calls Context::shutdown() while
other statics are already gone. This segfaults the test binary after all
tests pass, in about 1 of 10 runs.

Signed-off-by: Mete Fatih Cırıt <mfc@autoware.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:planning Route planning, decision-making, and navigation. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Fix the SIGSEGV at exit of test_autoware_behavior_velocity_traffic_light_module

2 participants