Note about `actions/upload-artifact@v4`

[LecrisUT]

Uploading to the same dist artifact is no longer supported ¹. The documentation would need to be updated accordingly

Footnotes

1. https://github.com/actions/upload-artifact/tree/v4#not-uploading-to-the-same-artifact ↩

[henryiii]

Ouch, that affects a lot of things. The ability to simply aggregate was very useful. Is there a nice way to recombine artifacts? If you get all artifacts, you might get things you don't need, and they go into directories. I think the aggregation job will now require at least an extra step.

I'm surprised a very useful feature was axed but no clear replacement was worked out first.

[LecrisUT]

Agreed, I did not find a clean way other than sticking to v3 or downloading all and using the artifact name to merge it back.

I didn't check the git blame to see the discussion on when this was removed, but there was this old issue I stumbled upon: actions/upload-artifact#24

[Czaki]

It will be a big pain. In many projects, logs are combined into a single target to simplify download.

[henryiii]

If there was just something like "combine: true" (and ideally a way to list artifacts, like in actions/download-artifact#202, possibly even with globs!), then this would be much easier, and the promise of faster downloads is nice. But currently it seems really ugly to workaround for projects like cibuildwheel users (and logging, etc).

[LecrisUT]

Download wise, there is an approach by not specifying the name in download-artifact ¹, which will extract them in their corresponding name directory. It is not ideal because it will download all artifacts instead of allowing for a glob pattern.

Footnotes

1. https://github.com/actions/download-artifact/tree/v4#download-all-artifacts ↩

[henryiii]

Something like this, perhaps?

    - uses: actions/download-artifact@v4
      with:
        path: all

    - name: Merge files
      run: |
        mkdir dist
        mv all/*/* .
        rmdir all/*
        rmdir all

Any way to make that cleaner? (also not sure if shell: bash is required)

[LecrisUT]

Considering that this will most likely be done in a standalone job, that should be ok as being runs-on: ubuntu-latest. But I think some sanity paths are in order:

    - uses: actions/download-artifact@v4
      with:
        path: all
    - name: Merge files
      run: |
        mkdir dist
        mv all/*/dist/* ./dist

The download-artifact creates an extra directory between all and dist

[henryiii]

that should be ok as being runs-on: ubuntu-latest

🤦

The download-artifact creates an extra directory between all and dist

It shouldn't create a dist at all. The structure should be all/<name>/thing.tar.gz, etc. There is no "dist" (or "wheelhouse", for cibuildwheel) in the archives.

So this should be fine, I think:

    - uses: actions/download-artifact@v4
      with:
        path: all

    - name: Merge files
      run: |
        mkdir dist
        mv all/*/* .

(all still untested)

[LecrisUT]

It shouldn't create a dist at all.

This would depend on how you uploaded the artifact dist/* vs dist. The downloaded artifacts are automatically extracted in those folders. Granted I've only tested the later statement, but the former one has been like that since the beginning, hasn't it?

[henryiii]

cibuildwheel artifacts are not in dist. They are in wheelhouse. So I’m quite sure the directory is already not included by the upload action using the expression we already have.

[LecrisUT]

Oh, yes indeed:

      - name: Build wheels
        run: pipx run cibuildwheel==2.16.2
      - uses: actions/upload-artifact@v3
        with:
          path: ./wheelhouse/*.whl

I was remembering these snippets in the documentation:

    - name: Build SDist
      run: pipx run build --sdist
    - uses: actions/upload-artifact@v3
      with:
        path: dist/*.tar.gz
...
    - uses: actions/download-artifact@v3
      with:
        name: artifact
        path: dist
    - uses: pypa/gh-action-pypi-publish@release/v1