Skip to content

Possibility to create separate JUnit XML files on retest #705

@heinemichael

Description

@heinemichael

For any retest of C++ test suite, the gtest output xml file gets overwritten. This is problematic, for example, if the test suite contains a flaky test. It could fail on the first run, but pass on the second run (given that --retest-until-pass is passed to colcon test). The generated XML output file contains only test results of the last test execution, any previous test exeuction gets overwritten. The only way to detect such cases would be to manually parse the output of colcon test, line by line.

I would suggest to generate separate XML files for each run, e.g.:

build/<pkg>/test_results/<pkg>/<test_suite_name>_run_1.gtest.xml
build/<pkg>/test_results/<pkg>/<test_suite_name>_run_2.gtest.xml
build/<pkg>/test_results/<pkg>/<test_suite_name>_run_3.gtest.xml
...

The problem is that all previous files must be deleted before calling the test suite, as there might be only one test run, while the previous one had multiple ones, and those previous XML files are then being copied to the Testing folder.

Ideally, the output files should be merged, but since gtest does not allow to merge output files, that operation requires manual XML parsing and generation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions