Skip to content

Daily Test Coverage Improver: Add comprehensive API pseudo-boolean constraint tests#7898

Merged
NikolajBjorner merged 2 commits intomasterfrom
daily-test-improver-api-pb-tests-a3352a6d3ed78ca1
Sep 18, 2025
Merged

Daily Test Coverage Improver: Add comprehensive API pseudo-boolean constraint tests#7898
NikolajBjorner merged 2 commits intomasterfrom
daily-test-improver-api-pb-tests-a3352a6d3ed78ca1

Conversation

@dsyme
Copy link
Collaborator

@dsyme dsyme commented Sep 17, 2025

Summary

This PR adds comprehensive test coverage for Z3's pseudo-boolean constraint API functions, achieving 100% coverage for the src/api/api_pb.cpp module, which previously had 0% test coverage.

Problem Found

The pseudo-boolean constraint API functions (Z3_mk_*) had zero test coverage despite being important functionality for constraint solving:

  • src/api/api_pb.cpp: 0% coverage (0/64 lines covered)
  • No existing tests exercised the pseudo-boolean API layer
  • Important functionality like atmost/atleast constraints and weighted pseudo-boolean constraints was untested

Actions Taken

  • Created comprehensive test suite in src/test/api_pb.cpp
  • Added test registration in src/test/main.cpp and src/test/CMakeLists.txt
  • Implemented tests for all 5 API functions:
    • Z3_mk_atmost: at most k of the variables can be true
    • Z3_mk_atleast: at least k of the variables can be true
    • Z3_mk_pble: weighted pseudo-boolean less-than-or-equal constraint
    • Z3_mk_pbge: weighted pseudo-boolean greater-than-or-equal constraint
    • Z3_mk_pbeq: weighted pseudo-boolean equality constraint
  • Comprehensive test cases covering:
    • Basic functionality with different variable counts and thresholds
    • Edge cases: empty arrays, zero coefficients, zero thresholds
    • Negative coefficients and negative thresholds
    • Large coefficients and complex constraint combinations
    • Single variable constraints and boundary conditions

Test Coverage Results

Before:

  • src/api/api_pb.cpp: 0% coverage (0/64 lines)
  • Overall project coverage: 47% (176,264/373,283 lines)

After:

  • src/api/api_pb.cpp: 100% coverage (64/64 lines) - +64 lines covered
  • Overall project coverage: 47% (176,328/373,347 lines) - +64 net covered lines

Coverage improvement achieved: Complete coverage of all pseudo-boolean constraint API functions with comprehensive testing of edge cases and functionality.

Replicating the Test Coverage Measurements

Build and run the new test:

# Build test executable
ninja -C build test-z3

# Run the specific API PB test
./build/test-z3 api_pb

# Generate coverage report to verify improvements  
gcovr --merge-mode-functions=separate --gcov-ignore-parse-errors --gcov-executable "llvm-cov gcov" . | grep api_pb

Expected Output:

PASS
src/api/api_pb.cpp                            64      64   100%

Commands to install dependencies, build, run tests, and generate coverage reports:

# Dependencies already installed in CI environment
# No additional dependencies needed

# Build configuration (already done)
CXXFLAGS="--coverage" CFLAGS="--coverage" LDFLAGS="-lgcov" CC=clang CXX=clang++ \
  cmake -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=./build/install -G "Ninja"

# Build and test
ninja -C build test-z3
./build/test-z3 api_pb

# Generate coverage reports
gcovr --merge-mode-functions=separate --gcov-ignore-parse-errors --gcov-executable "llvm-cov gcov" .
gcovr --html coverage.html --merge-mode-functions=separate --gcov-ignore-parse-errors --gcov-executable "llvm-cov gcov" .

Future Improvement Areas

Based on remaining 0% coverage areas for other API modules:

  • src/api/api_datalog.cpp (0% coverage, 486 lines) - Datalog API functions
  • src/api/api_fpa.cpp (0% coverage, 1090 lines) - Floating-point arithmetic API
  • src/api/api_qe.cpp (0% coverage, 112 lines) - Quantifier elimination API
  • src/api/api_polynomial.cpp (0% coverage, 33 lines) - Polynomial API functions

<details>
<summary>Workflow Details</summary>

Bash Commands Run

  • git checkout -b daily-test-improver-api-pb-tests
  • ninja -C build test-z3
  • ./build/test-z3 api_pb
  • gcovr --merge-mode-functions=separate --gcov-ignore-parse-errors --gcov-executable &quot;llvm-cov gcov&quot; . | grep api_pb
  • git add src/test/api_pb.cpp src/test/main.cpp src/test/CMakeLists.txt
  • git commit --author &quot;Daily Test Coverage Improver &lt;github-actions[bot]@users.noreply.github.com&gt;&quot; -m &quot;...&quot;

Web Searches Performed

None

Web Pages Fetched

None

</details>

> AI-generated content by Daily Test Coverage Improver may contain mistakes.

Generated by Agentic Workflow Run

…nstraint tests

Added comprehensive test coverage for Z3's pseudo-boolean constraint API functions, improving coverage from 0% to 100% for src/api/api_pb.cpp.

- Created comprehensive test suite in src/test/api_pb.cpp
- Added test registration in src/test/main.cpp and src/test/CMakeLists.txt
- Implemented tests for all 5 API functions:
  * Z3_mk_atmost: at most k variables can be true
  * Z3_mk_atleast: at least k variables can be true
  * Z3_mk_pble: weighted pseudo-boolean less-than-or-equal constraint
  * Z3_mk_pbge: weighted pseudo-boolean greater-than-or-equal constraint
  * Z3_mk_pbeq: weighted pseudo-boolean equality constraint
- Comprehensive test cases covering edge cases, negative coefficients, zero thresholds, empty arrays, and complex scenarios
- All tests pass successfully with 100% coverage achieved

Coverage improvement: api_pb.cpp went from 0% (0/64 lines) to 100% (64/64 lines)
@NikolajBjorner NikolajBjorner marked this pull request as ready for review September 17, 2025 16:35
@NikolajBjorner
Copy link
Contributor

this kind of unit test, when automatically created, is OK to have. Even as it tests basically nothing of interest, it looks low threshold and can catch regressions.

@dsyme
Copy link
Collaborator Author

dsyme commented Sep 17, 2025

this kind of unit test, when automatically created, is OK to have. Even as it tests basically nothing of interest, it looks low threshold and can catch regressions.

My personal approach is

  • Coverage-based testing is useful but there's a price to pay of small test covering simple things. That's just how it is.
  • Semi-automated coverage-improvement begins with lots of small simple stuff, and eventually gets more interesting
  • As long as unit tests increase coverage and don't take to long to run then I merge a lot of them

But totally up to you all of course.

@NikolajBjorner NikolajBjorner merged commit 82ab674 into master Sep 18, 2025
5 of 18 checks passed
@nunoplopes nunoplopes deleted the daily-test-improver-api-pb-tests-a3352a6d3ed78ca1 branch September 18, 2025 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants