Skip to content

feat(perception): apply new perception pipeline - #3008

Merged
badai-nguyen merged 15 commits into
feat/v0.64/e2efrom
cherry-pick/new-perception-pipeline
Jun 2, 2026
Merged

badai-nguyen merged 15 commits into
feat/v0.64/e2efrom
cherry-pick/new-perception-pipeline

Conversation

@ktro2828

@ktro2828 ktro2828 commented May 27, 2026 •

Copy link
Copy Markdown

What

This PR cherry-picks PRs from exp/pointcloud_semseg_v4.4.

The following PR is also required to merge:

This pull request introduces a new label-based Euclidean clustering node to the autoware_euclidean_cluster package, along with its configuration and launch files. It also implements the previously unimplemented clustering logic for the voxel grid-based method and adds a new object fusion merger node to the autoware_object_merger package. The most important changes are summarized below.

Label-Based Euclidean Clustering Node

  • Added a new LabelBasedEuclideanClusterNode implementation, including its header file (label_based_euclidean_cluster_node.hpp), CMake registration, and dependencies on autoware_shape_estimation for shape estimation capabilities. [1] [2] [3] [4]
  • Introduced a configuration file (label_based_euclidean_cluster.param.yaml) specifying clustering parameters, semantic label mappings, and shape estimation options.
  • Provided launch files (label_based_euclidean_cluster.launch.py and .launch.xml) for starting the new node with configurable arguments for input/output topics and parameters. [1] [2]
  • Updated the README with documentation for the new node, its parameters, and behavior regarding label mapping and probability handling. [1] [2]

Voxel Grid-Based Clustering Logic

  • Implemented the actual clustering logic for VoxelGridBasedEuclideanCluster::cluster, including voxel grid filtering, 2D centroid clustering, and output filtering based on cluster size.

Object Merger Enhancements

  • Added a new ObjectFusionMergerNode to the autoware_object_merger package, including CMake registration, source file addition, and a corresponding configuration YAML. Also included a test for this node. [1] [2] [3]

These changes collectively expand the perception capabilities by enabling label-aware clustering of semantic point clouds and improved object fusion.

badai-nguyen and others added 12 commits May 27, 2026 14:54
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
…oundation#12547)

* feat(autoware_ptv3): source cloud reconstruction & entropy

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): clarify reconstruction feature handling

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(autoware_ptv3): remove unnecessary buffers clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): simplify source reconstruction buffers

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* fix(autoware_ptv3): remove redundant memory clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): add __restrict__

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

* style(pre-commit): autofix

* feat(autoware_ptv3): add restrict

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): parallelizaiton

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

---------

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
@github-actions

Copy link
Copy Markdown

@ktro2828 Thank you for contributing to TIER IV E2E branch! Edit the checklist below and check the boxes as you complete the items.

Things to check before merging

Please check the following points before merging:

  • Write a brief description of this PR that can be understood by coding-agents.
  • Contact the branch manager on when the PR will be tested on vehicle.
  • Recommend to use Squash and Merge to merge the PR.
  • Report in #team-product-dev channel on the contents of the PR. @ branch manager.

After merging

Please check the following points after merging:

  • Wait for the automatic hash synchronization, which will automatically create a PR to update the hash, you should check the PR and merge it.

@tier4-autoware-public-bot tier4-autoware-public-bot Bot added beta/v0.64 Base branch: beta/v0.64 diffusion_planner Product: diffusion_planner labels May 27, 2026
@ktro2828
ktro2828 marked this pull request as ready for review May 27, 2026 08:23
@badai-nguyen

Copy link
Copy Markdown

@a-maumau we are going to merger ptv3 pipeline with e2e branch here

@badai-nguyen badai-nguyen left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed launch with "use_semantic_segmentation_ptv3" default="true"

2026-05-29_23-56 2026-05-29_23-59

@ktro2828 ktro2828 added the run:build-and-test-differential After beta/v0.41 label Jun 1, 2026
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
@sonarqubecloud

sonarqubecloud Bot commented Jun 1, 2026

Copy link
Copy Markdown

@badai-nguyen
badai-nguyen merged commit 4e7aaa1 into feat/v0.64/e2e Jun 2, 2026
25 of 27 checks passed
@badai-nguyen
badai-nguyen deleted the cherry-pick/new-perception-pipeline branch June 2, 2026 01:40
shanpeng-m pushed a commit that referenced this pull request Jun 9, 2026
* fix: update ptv3 filter classes

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat: add label-based clustering algorithm

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat(object_merger): add a new merger (#2906)

* fix: missign mapped_label

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* chore: move shape policy to launch

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: label_based_EU node name

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: using voxel_based EC

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat(autoware_ptv3): source cloud reconstruction & entropy (autowarefoundation#12547)

* feat(autoware_ptv3): source cloud reconstruction & entropy

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): clarify reconstruction feature handling

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(autoware_ptv3): remove unnecessary buffers clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): simplify source reconstruction buffers

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* fix(autoware_ptv3): remove redundant memory clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): add __restrict__

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

* style(pre-commit): autofix

* feat(autoware_ptv3): add restrict

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): parallelizaiton

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

---------

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>

* change to partial reconstruction

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: imu scale estimation covariance limit for stability

Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>

* chore: fix pre-commit

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

---------

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
Co-authored-by: badai-nguyen <dai.nguyen@tier4.jp>
Co-authored-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
(cherry picked from commit 4e7aaa1)
tier4-autoware-public-bot Bot pushed a commit to tier4/autoware_universe_sensing_perception that referenced this pull request Jul 16, 2026
…rse#3008)

* fix: update ptv3 filter classes

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat: add label-based clustering algorithm

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat(object_merger): add a new merger (tier4/autoware_universe#2906)

* fix: missign mapped_label

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* chore: move shape policy to launch

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: label_based_EU node name

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: using voxel_based EC

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat(autoware_ptv3): source cloud reconstruction & entropy (tier4/autoware_universe#12547)

* feat(autoware_ptv3): source cloud reconstruction & entropy

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): clarify reconstruction feature handling

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(autoware_ptv3): remove unnecessary buffers clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): simplify source reconstruction buffers

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* fix(autoware_ptv3): remove redundant memory clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): add __restrict__

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

* style(pre-commit): autofix

* feat(autoware_ptv3): add restrict

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): parallelizaiton

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

---------

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>

* change to partial reconstruction

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: imu scale estimation covariance limit for stability

Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>

* chore: fix pre-commit

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

---------

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
Co-authored-by: badai-nguyen <dai.nguyen@tier4.jp>
Co-authored-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
tier4-autoware-public-bot Bot pushed a commit to tier4/autoware_universe_sensing_perception that referenced this pull request Jul 16, 2026
…rse#3008)

* fix: update ptv3 filter classes

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat: add label-based clustering algorithm

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat: update cluster

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

* feat(object_merger): add a new merger (tier4/autoware_universe#2906)

* fix: missign mapped_label

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* chore: move shape policy to launch

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: label_based_EU node name

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: using voxel_based EC

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* feat(autoware_ptv3): source cloud reconstruction & entropy (tier4/autoware_universe#12547)

* feat(autoware_ptv3): source cloud reconstruction & entropy

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): clarify reconstruction feature handling

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(autoware_ptv3): remove unnecessary buffers clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* refactor(ptv3): simplify source reconstruction buffers

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* fix(autoware_ptv3): remove redundant memory clear

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): add __restrict__

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

* style(pre-commit): autofix

* feat(autoware_ptv3): add restrict

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* feat(autoware_ptv3): parallelizaiton

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

---------

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>

* change to partial reconstruction

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

* fix: imu scale estimation covariance limit for stability

Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>

* chore: fix pre-commit

Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>

---------

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Signed-off-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
Co-authored-by: badai-nguyen <dai.nguyen@tier4.jp>
Co-authored-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: Sergio Reyes Sanchez <sergioreyes.sanchez@tier4.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beta/v0.64 Base branch: beta/v0.64 diffusion_planner Product: diffusion_planner run:build-and-test-differential After beta/v0.41 type:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants