Repository navigation
Autoharness: report bounded Rc/Arc reference-count models - #5070
acearyanarun wants to merge 5 commits into
Conversation
Kani models the reference counts of generated Rc/Arc values with only finitely many states (model-checking#4752), yet autoharness reported such harnesses as checked "against all possible inputs" with no caveat, since Rc/Arc arguments are unbounded in the pointee and need no --bounded-arguments. Add an independent, non-gating caveat, mirroring is_ctor_based: - HarnessMetadata::is_refcount_bounded, set when some generated value contains an Rc/Arc, found by walking the argument types (references, raw pointers, arrays, slices, tuples, pattern types, ADT fields and generic arguments; PhantomData skipped; over-approximating, so a missed caveat is not possible). Formatting-trait harnesses use the impl's self type. - A "(refcount)" marker in the summary table (it fits the Kind column, so other rows do not shift), a note after the table, and a qualified "all possible inputs" message. - is_refcount_bounded in the --export-json document, with the schema, fixtures, checker and RFC 0015 updated. ArgSupport::Bounded is deliberately not reused: it is gated behind --bounded-arguments, which would disable Rc/Arc support by default. The Rc/Arc generators are unchanged (see model-checking#4788). Tests: a json-handler test asserting the export field, marker, note and message per function (direct, AnyRc/AnyArc model paths, nested field, tuple, reference, Box; negative controls incl. PhantomData), with covers that must be SATISFIED for the generated unique state; updated smart-pointer expectations; driver unit tests. Docs: limitations of the finite model in the autoharness reference. Part of model-checking#4752.
|
@wodex1nhaoIeng @srivatsansamraj @Tianshu-Huang @CYJ904 Hi team! I've opened PR #5070 to address the bounded Rc/Arc reference-count reporting limitation in AutoHarness (#4752). The changes add a separate (refcount) marker to verification summaries and JSON metadata without modifying the existing Rc/Arc generators or depending on #4788. Local builds, unit tests, AutoHarness regression tests, formatting, and Clippy checks have all passed. Could you review the changes when you get a chance, particularly the recursive type detection, metadata compatibility, and reporting behavior? |
|
@acearyanarun could you resolve the branch conflicts? |
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
What changed in this PR
Adds explicit reporting that AutoHarness-generated Rc/Arc inputs have finitely modeled reference counts, surfacing this limitation in CLI output and JSON export without changing generation behavior.
Changes:
- Introduces
HarnessMetadata::is_refcount_boundedand propagates it through compiler metadata, driver output, and JSON export/schema. - Updates AutoHarness summary/progress messaging to mark affected harnesses with
(refcount)and print an explanatory note. - Adds regression tests/fixtures and documentation covering the new caveat and export field.
| File | Description |
|---|---|
| tests/script-based-pre/cargo_autoharness_smart_pointers/src/lib.rs | Updates test description to note (refcount) marking for Rc/Arc. |
| tests/script-based-pre/cargo_autoharness_smart_pointers/smart-pointers.expected | Updates expected AutoHarness summary rows and adds the (refcount) note. |
| tests/json-handler/schema-validation/kani_json_schema.json | Extends schema with is_refcount_bounded. |
| tests/json-handler/schema-validation/fixture-non-completed-outcomes.json | Adds is_refcount_bounded to fixtures. |
| tests/json-handler/schema-validation/fixture-completed-failure.json | Adds is_refcount_bounded to fixtures. |
| tests/json-handler/schema-validation/check_fixtures.py | Adds negative fixture mutations for is_refcount_bounded. |
| tests/json-handler/autoharness-refcount/test.sh | New end-to-end JSON/log assertions for (refcount) reporting. |
| tests/json-handler/autoharness-refcount/test.rs | New functions covering direct/nested Rc/Arc detection and non-vacuity. |
| tests/json-handler/autoharness-refcount/config.yml | Adds script-based test configuration. |
| rfc/src/rfcs/0015-export-json.md | Documents the new JSON field and semantics. |
| kani_metadata/src/harness.rs | Adds HarnessMetadata::is_refcount_bounded with serde default. |
| kani-driver/src/sarif.rs | Updates test harness metadata construction with new field. |
| kani-driver/src/metadata.rs | Updates test harness metadata construction with new field. |
| kani-driver/src/harness_runner.rs | Qualifies per-function “all possible inputs” messaging using the new caveat. |
| kani-driver/src/frontend/schema_utils.rs | Exports is_refcount_bounded to JSON and adds test assertions. |
| kani-driver/src/autoharness/mod.rs | Adds (refcount) marker to summary rows and prints explanatory note. |
| kani-compiler/src/kani_middle/mod.rs | Adds type-walk helper contains_refcounted_ptr for Rc/Arc detection. |
| kani-compiler/src/kani_middle/metadata.rs | Threads is_refcount_bounded through automatic harness metadata generation. |
| kani-compiler/src/kani_middle/codegen_units.rs | Computes is_refcount_bounded for autoharness caveats during partitioning. |
| docs/src/reference/experimental/autoharness.md | Documents the finite reference-count modeling caveat and reporting. |
Review effort: Lite
| /// Identify the worker on every line of a rendered verification result. | ||
| /// What an automatic harness checks its function against, for the "Autoharness: Checking | ||
| /// function ..." message. A harness with a caveat does not check *all* possible inputs, so | ||
| /// qualify the message accordingly so it does not masquerade as exhaustive (c.f. the | ||
| /// "(bounded)" and "(refcount)" markers in the summary). | ||
| fn autoharness_inputs_description(harness: &HarnessMetadata) -> &'static str { | ||
| match (harness.is_bounded, harness.is_refcount_bounded) { | ||
| (false, false) => "all possible inputs", | ||
| (true, false) => "all possible inputs (bounded for some arguments)", | ||
| (false, true) => "all possible inputs (with finitely modeled Rc/Arc reference counts)", | ||
| (true, true) => { | ||
| "all possible inputs (bounded for some arguments, with finitely modeled Rc/Arc reference counts)" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| fn prefix_thread_lines(output: &str, thread_index: usize) -> String { |
There was a problem hiding this comment.
Fixed in d02e67b: the doc comment is back on prefix_thread_lines, and autoharness_inputs_description has its own.
| /// representatives. Its verification result therefore does not cover every reference | ||
| /// count a caller could produce, even though `Rc`/`Arc` arguments need no | ||
| /// `--bounded-arguments` and do not set `is_bounded` | ||
| /// (<https://github.com/model-checking/kani/issues/4752>). | ||
| #[serde(default)] | ||
| pub is_refcount_bounded: bool, |
There was a problem hiding this comment.
Renamed to has_refcount_caveat in d02e67b, everywhere it appears: HarnessMetadata, the --export-json field, the JSON schema and its fixtures, RFC 0015, and the tests. I didn't keep the old JSON name with #[serde(rename)], since the field hasn't been released yet.
| /// that the only failure mode, the walk reports `true` if it exceeds its budget (e.g. for | ||
| /// polymorphically recursive types). Values not visible in the type (behind `dyn Trait`, or | ||
| /// function pointers) cannot be generated by automatic harnesses and are not considered. | ||
| pub fn contains_refcounted_ptr(tcx: TyCtxt, ty: Ty, visited: &mut FxHashSet<Ty>) -> bool { |
There was a problem hiding this comment.
Now pub(crate) (d02e67b). It's only used inside kani-compiler.
feliperodri
left a comment
There was a problem hiding this comment.
Thanks for the detailed description. The reporting change is clear and the tests cover the cases it claims. Needs changes before merge:
- Merge conflict. The PR is not mergeable (state
dirty). Please rebase onmain. - Draft note. The description starts with "Draft — local validation complete", but the PR isn't a draft. Remove the note, or convert the PR to a draft.
- Copilot's points are valid. Please fix the doc comment at
harness_runner.rs:100(it now sits on the wrong function). Consider renamingis_refcount_boundedto something likehas_refcount_caveat, so it isn't read as related to--bounded-arguments. Makecontains_refcounted_ptrpub(crate)unless it's meant as public API. - Checks. Three macOS jobs are
cancelled. Please re-run them.
The [F] Soundness label fits: the PR makes the overclaim visible (a bounded model reported as "all possible inputs").
The autoharness reference page was restructured on main (model-checking#4991). Move this branch's reference-count documentation into the new structure: a sentence under "Box, Rc and Arc", a "(refcount)" entry under "Summary table and markers", and a "Reference counts" subsection under "Soundness caveats".
- Rename `is_refcount_bounded` to `has_refcount_caveat` in the metadata, the `--export-json` schema, fixtures, RFC 0015 and tests, so the field is not read as related to `--bounded-arguments`. - Move the `prefix_thread_lines` doc comment back onto that function. - Make `contains_refcounted_ptr` `pub(crate)`. - Add the field to `fixture-completed-success.json`, which model-checking#5154 added on `main`; the schema requires it.
…aveat The relocated bullets described a model with shared representatives (from the closed model-checking#4788) and the overflow trade-off of that design. Generated Rc/Arc values are always uniquely owned, so say that only the counts 1 and 0 are checked and that no other holder exists.
1994a45 to
ca831cc
Compare
|
Thanks for the review, @feliperodri. I've addressed each point:
Local results (macOS, GitHub CI: the new workflow runs are waiting for maintainer approval, so they haven't run yet. Could you approve them when you have a moment? |


Summary
AutoHarness-generated
Rc/Arcvalues have finitely modeled reference counts, but current AutoHarness output describes these harnesses as verifying "all possible inputs" without identifying that limitation. This PR makes the limitation visible without disabling defaultRc/Arcsupport.HarnessMetadata::is_refcount_boundedcaveat.(refcount)in the AutoHarness summary, adds an explanatory note, and qualifies the per-function progress message.is_refcount_boundedin--export-json, including schema, fixtures, and RFC documentation updates.This PR does not change any
Rc/Arcgenerator.Why this matters
Today, generated
Rc/Arcvalues are uniquely owned (strong_count == 1,weak_count == 0). PR #4788 proposes additional shared-ownership states (strong_countin{1, 2}andweak_countin{0, 1}). Neither model covers all exact count values a caller may supply, so a successful verification result needs to communicate that limitation. A function checking, for example,Rc::strong_count(&x) > 2can still depend on unmodeled states.The existing
(bounded)marker is gated by--bounded-arguments; reusing it would prevent default generation ofRc/Arcarguments. The new(refcount)caveat is separate and non-gating.Implementation
PhantomData. It deliberately errs toward warning on uncertain nesting rather than silently omitting a caveat.falsewhen absent in older serialized metadata.AnyRc/AnyArcreporting paths, negative controls, and coverage expectations for uniquely owned states.The type-based detection may flag types whose generation cannot actually produce an
Rc/Arc. The warning does not claim complete modeling. It also does not cover manual harnesses usingkani::any().Difference from #4788
Rc/Arcownership statesRc/Arcarbitrary-value generatorsShared-state regression cases that depend on #4788 are intentionally excluded from this PR; they can be contributed when the generator change lands.
Testing
Passed locally on macOS using nightly-2026-10-23:
cargo build-dev— passed, includingkani-compiler.cargo test -p kani-driver -p kani_metadata— 214 driver tests and 4 metadata tests passed; 2 documentation tests passed, 1 ignored.cargo run -p compiletest -- --suite json-handler --mode exec autoharness-refcount— 1 passed, 0 failed.cargo run -p compiletest -- --suite script-based-pre --mode exec --force-rerun cargo_autoharness_smart_pointers— 1 passed, 0 failed.cargo run -p compiletest -- --suite script-based-pre --mode exec --force-rerun autoharness— 42 passed, 0 failed../scripts/kani-fmt.sh --check— passed.cargo clippy --workspace --tests -- -D warnings— passed.RUSTFLAGS="--cfg=kani_sysroot" cargo clippy --workspace -- -D warnings— passed.The broader AutoHarness suite includes the smart-pointer test, so these counts overlap.
Pending: GitHub CI and maintainer review. The shared-ownership regression tests depending on #4788 are intentionally excluded.
Related work
Part of #4752. Related: #4788 (generator), #4958 (aliasing metadata), and #4991 (AutoHarness documentation restructure; may require rebase/conflict resolution).
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WddFow1MppHhSFwg9HdZCz