Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 16 additions & 9 deletions projects/_template/release-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,25 @@ Example shape:

## Binary-exclude list

TODO: list any binary content the source artefact must NOT contain
(per `release-verify-rc`'s no-prohibited-binaries check). The
default list is conservative, `.class`, `.jar`, `.so`, `.dylib`,
`.dll`, `.exe`, pre-built minified JS bundles checked into the
source tree. Project-specific exclusions go here.
TODO: configure `release-verify-rc` Step 6's prohibited-binary check.

The skill always scans a **fixed baseline**: `.class`, `.jar`,
`.so`, `.dylib`, `.dll`, `.exe`, `.pyc`, and `__pycache__`
directories. List here:

1. **Additional prohibited globs** the source artefact must also not
contain (appended to the Step 6 `find` beyond the baseline).
2. **Known-accepted exceptions** — specific paths that match a
prohibited pattern but must ship; Step 6 classifies those as
`EXPECTED-BINARY` rather than `PROHIBITED-BINARY`.

Example shape:

> - `*.class`, `*.jar`, Java compiled output never ships in source.
> - `assets/vendor/**/*.min.js`, vendored minified JS that has a
> source-checked counterpart; flagged on every source-release
> verification.
> - `assets/vendor/**/*.min.js` — additional prohibited glob
> (vendored minified JS; flagged on every source-release
> verification).
> - `third-party/some-native.so` — known-accepted exception (ships
> with a documented source counterpart).

## Apache RAT configuration

Expand Down
6 changes: 4 additions & 2 deletions projects/magpie/release-build.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,5 +69,7 @@ and are never emitted.
## Binary-exclude list

The source artefact must contain no compiled or opaque binary content.
Conservative default denylist for `release-verify-rc`:
`.class`, `.jar`, `.so`, `.dylib`, `.dll`, `.exe`, `.pyc`.
`release-verify-rc` Step 6 always scans its fixed baseline
(`.class`, `.jar`, `.so`, `.dylib`, `.dll`, `.exe`, `.pyc`,
`__pycache__`). Magpie adds no project-specific globs and names no
known-accepted exceptions.
51 changes: 32 additions & 19 deletions skills/release-verify-rc/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,44 +428,57 @@ small (version-string-only changes).

## Step 6 — Binary exclusion check

Scan the unpacked source artefact for files matching the
binary-exclude list from `release-build.md`. Emit the paste-ready
scan command:
Scan the unpacked source artefact for prohibited binaries. The
paste-ready `find` always starts from a **fixed baseline** of
patterns; it is not generated wholesale from adopter config.

Using the Binary-exclude list from `release-build.md` (heading
**Binary-exclude list** — same file Step 4 reads for RAT
configuration), append any additional globs that list names beyond
the baseline, then emit the recipe:

**Baseline (always scanned):** `.class`, `.jar`, `.so`, `.dylib`,
`.dll`, `.exe`, `.pyc`, and `__pycache__` directories.

**Additions from `release-build.md`:** any extra globs under
**Binary-exclude list** that the baseline does not already cover
(for example `*.min.js` or `assets/vendor/**/*.min.js`). Translate
each into a `-name` or `-path` predicate and OR it into the `find`
below before emitting.

```bash
# Compiled Java class files, native shared libraries, AND compiled
# Python bytecode. `.pyc` / `__pycache__` must NEVER appear in a
# Fixed baseline. `.pyc` / `__pycache__` must NEVER appear in a
# source release — their presence proves the artefact was zipped from
# a working tree that had run tests rather than exported clean from
# the tag (build via `git archive <tag>`, never `zip -r`).
# Append -name / -path OR-predicates for any extra globs named under
# <project-config>/release-build.md § Binary-exclude list that the
# baseline does not already cover.
find <unpacked-dir> \( -type f \( -name "*.class" -o -name "*.jar" \
-o -name "*.so" -o -name "*.dylib" -o -name "*.dll" -o -name "*.exe" \
-o -name "*.pyc" \) -o -type d -name "__pycache__" \) -print
```

Emit the bare `find` with no `grep` post-filtering: the recipe must
surface every matching file so nothing is hidden from the voter. The
binary-exclude list from `release-build.md` is applied in the JSON
classification below (matching files become `expected_binaries`, the
rest `prohibited_found`), not by filtering the command.
surface every matching file so nothing is hidden from the voter. Do
not drop baseline predicates when the Binary-exclude list is empty or
only restates the baseline — the baseline is mandatory.

`<unpacked-dir>` is the source artefact filename with its archive
extension removed: `apache-airflow-2.11.0-source-release.tar.gz` unpacks
to `apache-airflow-2.11.0-source-release`. Do not drop the
`-source-release` suffix or substitute a shortened name. Resolve
`<unpacked-dir>` to this concrete directory before emitting the recipe.

The default prohibited extensions are `.class`, `.jar`, `.so`,
`.dylib`, `.dll`, `.exe`. The `release-build.md` binary-exclude list
may add project-specific extensions or glob patterns.

A file that appears in the binary-exclude list of `release-build.md`
is a known-and-accepted binary; surface it as `EXPECTED-BINARY` with
a note.
The same Binary-exclude list is then applied in the JSON
classification below. A found path the list marks as a
known-and-accepted binary is `EXPECTED-BINARY` (`expected_binaries`);
any other baseline or addition hit is `PROHIBITED-BINARY`
(`prohibited_found`). Classification does not filter the command.

A file that matches a prohibited extension but is NOT in the
binary-exclude list is classified `PROHIBITED-BINARY` and causes a
hard `FAIL`.
A file that matches a prohibited pattern but is NOT marked
known-and-accepted in the Binary-exclude list is classified
`PROHIBITED-BINARY` and causes a hard `FAIL`.

Return ONLY valid JSON with this structure:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
"status": "PASS",
"prohibited_found": [],
"expected_binaries": [],
"paste_recipe": "find apache-airflow-2.11.0-source-release -type f \\( -name '*.class' -o -name '*.jar' -o -name '*.so' -o -name '*.dylib' -o -name '*.dll' -o -name '*.exe' \\)"
"paste_recipe": "find apache-airflow-2.11.0-source-release \\( -type f \\( -name '*.class' -o -name '*.jar' -o -name '*.so' -o -name '*.dylib' -o -name '*.dll' -o -name '*.exe' -o -name '*.pyc' \\) -o -type d -name '__pycache__' \\) -print"
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->

release-build.md binary-exclude list: *.class, *.jar
release-build.md Binary-exclude list (known-accepted): *.class, *.jar
(no additional scan globs beyond the fixed baseline)

Scan of unpacked apache-airflow-2.11.0-source-release.tar.gz:
Files with prohibited extensions found: none
Files listed as EXPECTED-BINARY in release-build.md: none
No .class, .jar, .so, .dylib, .dll, or .exe files detected in the source artefact.
No .class, .jar, .so, .dylib, .dll, .exe, .pyc, or __pycache__ detected in the source artefact.
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
"status": "FAIL",
"prohibited_found": ["airflow/vendor/some-lib/some-lib-1.0.jar"],
"expected_binaries": [],
"paste_recipe": "find apache-airflow-2.11.0-source-release -type f \\( -name '*.class' -o -name '*.jar' -o -name '*.so' -o -name '*.dylib' -o -name '*.dll' -o -name '*.exe' \\)"
"paste_recipe": "find apache-airflow-2.11.0-source-release \\( -type f \\( -name '*.class' -o -name '*.jar' -o -name '*.so' -o -name '*.dylib' -o -name '*.dll' -o -name '*.exe' -o -name '*.pyc' \\) -o -type d -name '__pycache__' \\) -print"
}
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->

release-build.md binary-exclude list: *.class (listed as expected/known binary)
release-build.md Binary-exclude list (known-accepted): *.class
(no additional scan globs beyond the fixed baseline)

Note: *.jar is NOT in the binary-exclude list — jar files are prohibited in this source artefact.
Note: *.jar is NOT marked known-and-accepted — jar files matching the
fixed baseline are prohibited in this source artefact.

Scan of unpacked apache-airflow-2.11.0-source-release.tar.gz:
Files with prohibited extensions found:
airflow/vendor/some-lib/some-lib-1.0.jar (*.jar — prohibited, not in binary-exclude list)
airflow/vendor/some-lib/some-lib-1.0.jar (*.jar — prohibited, not known-accepted)

Files listed as EXPECTED-BINARY in release-build.md:
(none with *.class extension found either)
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ The model must return ONLY valid JSON matching this schema:
Grading rules:
- `status` must be `"FAIL"` if `prohibited_found` is non-empty.
- `status` must be `"PASS"` if `prohibited_found` is empty.
- `expected_binaries` lists files that match a prohibited extension pattern but are
explicitly listed in the `release-build.md` binary-exclude list; these are
EXPECTED-BINARY, not PROHIBITED.
- `paste_recipe` must be a non-empty string with the find command to scan the
unpacked artefact.
- `expected_binaries` lists files that match a prohibited pattern but are
marked known-and-accepted in the `release-build.md` Binary-exclude list;
these are EXPECTED-BINARY, not PROHIBITED.
- `paste_recipe` must be a non-empty string with a `find` that includes the
fixed baseline (`.class`, `.jar`, `.so`, `.dylib`, `.dll`, `.exe`, `.pyc`,
`__pycache__`) and any extra globs from `release-build.md` beyond that
baseline.
- No extra keys are permitted in the response.