Repository navigation
Description
Description
package.xml files are wrong in both directions, and nothing checks them.
Measured on a jazzy workspace of 491 packages and 4829 dependency entries. The counts below cover only the 18 repositories that the Autoware Foundation owns, which hold 415 of those packages.
| Defect | Count |
|---|---|
| Dependencies declared with no evidence of use | 353 in 149 packages |
| Dependencies used but never declared | 1354 in 266 packages |
Declared <depend> but used only under test/ |
86 in 52 packages |
The second row matters most, and it counts three kinds of use. A package includes a header it never declares. A package names a type, and the header arrives through another dependency. A package uses a system library such as Boost or Eigen, in either way. All of them are correct today by luck. The build works only while another declared dependency re-exports the owner, or while some other package installs the system library. When that carrier goes away, the package no longer compiles, and its own CI never sees the change.
This issue supersedes the older one, which asked for a one-time cleanup and gave no way to hold the result.
- Previous issue: Clean unused dependencies to improve build time #3468
Purpose
The goal is accurate manifests that stay accurate. A count that someone fixes once grows back. The count in the previous issue went from 297 in 2023 to several hundred today.
Accurate manifests also give a smaller build graph. That part is real but modest: about 3.9% of include directories, and a p90 rebuild fan-out that falls from 41 packages to 35. This issue is not a build-time optimization. Do not present it as one.
Possible approaches
The checker, the CI jobs, and the correction order, in detail
1. Ship a checker anyone can run
The current checker is a 155-line shell script in autoware_tools. It reports 1133 findings on this tree, which is 26.6% of every <depend>, and most of them are wrong. Its main rule assumes that a package named autoware_* installs headers under autoware/<suffix>/. 38 of the 179 such packages do not use that layout.
A working checker needs three things the shell script does not have.
- A real header index. Map each header to the package that ships it, from the installed and source trees. Do not guess the path from the package name.
- Qualified name resolution. In most wrong findings the package writes
dep::Typebut never includes adep/header, because the header arrives through a third package. An index of the namespaces and type names that each package declares removes most of these wrong findings. On a hand-checked sample of 30, it moved the false positive rate from 25.0% to 4.5%. - System library resolution. A Boost or Eigen include resolves through the compiler search paths,
dpkg, and the rosdep cache. The suggested name is the rosdep key this workspace already prefers. Comments and string literals are blanked before the scan, because a Doxygen example is not a use. A library that needs its own artifact at link time, such asBoost.Serialization, gets its own key. - A maintainer allowlist, as data. Some entries are deliberate and no analysis can derive that.
autoware_internal_msgsis a meta package, and it depends on the interface packages under it on purpose. Without a file that records such rulings, every new run starts from raw output again. A reviewer asked for this file in 2023. Nobody built it.
The numbers in this issue come from a checker built this way, so the approach works. It can live in place of autoware_dependency_checker in autoware_tools. It can also live next to check-package-depends in autoware-guideline-check, which already owns a small part of the missing direction.
2. Run it in CI
Speed is not the problem. Measured on the full 491-package workspace:
- Full sweep, both directions: 29 s wall, with a warm cache.
- The header index and the symbol index take about 4 s of that.
colcon buildis not necessary. The checker builds both indexes from the source trees.
The cost is the workspace. A pull request in autoware_universe contains only that repository, and the checker cannot resolve a header that autoware_core owns. The CI containers already solve this: core-dependencies compiles the dependencies into /opt/autoware, and universe-dependencies starts from core-devel. A job therefore needs no import step.
The two directions have different accuracy, so give them different jobs.
- Missing dependencies: gate them. A package that includes a header that it does not declare is an exact finding, with no false positives. Fail the job. Start with the changed packages only, so that the existing 1354 do not block unrelated work.
- Unused dependencies: report them, do not gate them. A small error rate remains, and no static analysis can find one class of use. Run this direction on a schedule over the full workspace, and publish the result.
The checker must exit non-zero and print machine-readable output. The current one does neither.
3. Correct the existing entries, in order
- Add the missing dependencies first. 1354 entries. Adding a
<depend>cannot break a build. Until a package declares what it includes, no one can safely remove an upstream entry. Start with the 54 downstream packages that sit behind the 32 load-bearing edges. - Move the test-only entries. 86 entries in 52 packages, from
<depend>to<test_depend>. The risk is low. - Remove the unused entries last, with one pull request per top-level folder. A reviewer asked for that split in 2023. Nobody finished it. Every pull request must build the reverse-dependency closure, not only the changed packages, because a package can build alone and still break its consumers.
Work on autoware_core is complete. A child issue lists its 211 entries, each with a permalink to a line that needs the dependency. All 14 pull requests are merged, and the child issue is closed. Independent per-package reviews validated every entry, and the review rounds corrected the checker five times.
Definition of done
The checker is not reliable enough to ship or to gate CI. It runs locally for now, and each finding gets a manual review before a pull request. A CI gate can also wait. The cleanup can correct entries much faster than new wrong ones arrive, and the build CI catches a real break.
The work covers the three main repositories: autoware_core, autoware_universe, and autoware_tools.
- Add the missing dependencies, one repository at a time.
autoware_coreis complete.autoware_universeis in progress.autoware_toolsfollows.
- Move the test-only
<depend>entries to<test_depend>or<exec_depend>, in the same repository order. - Remove the unused entries last, with one pull request per top-level folder. Each pull request builds the reverse-dependency closure. The order is reversed, because a removal in a repository can break its consumers.
autoware_toolsfirst.autoware_universenext.autoware_corelast.
- Ship the checker, the maintainer rulings file, and the CI jobs, after the local rounds prove the output.
Written with Claude Code, from a fresh analysis of a local jazzy workspace and the thread of the previous issue.
Activity