-
-
Notifications
You must be signed in to change notification settings - Fork 810
Convert test workflow to container-based testing #1208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
23e7e8c to
d19c0ec
Compare
|
This version of the container CI setup runs on all of the expected platforms. The failures on Fedora 43 and 44 amd64 are expected... they are probably the result of glibc changes, and illustrating those failures is largely the point of this change. The failures on arm are not expected. It looks like the tests that 1840407 marked skipped aren't being skipped in the arm container. I'll have to look into that later. |
|
Very cool @gordonmessmer - funny that we both got to work on that almost at the time 😄 Can you mark the failing tests as skipped on CI then ? So we can still have a 🟢 build for this PR, and tackle the known failures in individual followup issues/PRs ? |
|
FWIW in my own tests in #1206 I remember Arch was also not playing well for the test suite. |
4059975 to
ef33e72
Compare
I suppose I should make a habit of looking at open PRs to see if the thing I want to do is already in progress, but.... -_-
If you'd like, I can squash these commits and then work on disabling some tests. Or you can squash these and merge in its current condition, and then disable tests in another PR. Either way, I think it makes sense to disable tests in a separate commit so that it's easier to revert later. I think that the PR is now working as expected. All of the Ubuntu jobs are green. Fedora 42 on amd64 is green. Fedora 43 and rawhide fail, but that's expected. Not entirely sure why f42:arm64 fails. :-/ |
Yeah, that's fine. As long as we track the disabled tests in an issue or somewhere so we don't forget they fail (and point it if someone comes and asks why it fails) I can de-dust my arm64 dev box and install a fedora43 vm over the weekend and have a look. But in the meantime I see no reason not to merge this PR as-is when all lights come back green |
|
I'll send a commit to disable some tests shortly... Are you able to reopen and rebase #1197 ? It fixes some of the tests currently failing. |
Refactor the CI workflow to build and run tests inside Docker containers instead of directly on GitHub runners. This adds support for testing on Fedora releases in addition to Ubuntu 22.04 and 24.04.
d6febdd to
e28dc31
Compare
e28dc31 to
049d3dd
Compare
049d3dd to
1a098de
Compare
Refactor the CI workflow to build and run tests inside Docker containers instead of directly on GitHub runners. This adds support for testing on Fedora releases in addition to Ubuntu 22.04 and 24.04.
Description
This change uses container images for tests so that platforms other than the ones provided directly by GitHub can be tested.
Testing in container images makes it easier to support a broader range of platforms, but more importantly it allows gef to test and adapt to changes in glibc, gcc, and Python as they are released, rather than re-synchronizing with those projects every two years.
Checklist