Skip to content

Switch Inflator from Mono to dotnet#4387

Merged
HebaruSan merged 8 commits into
KSP-CKAN:masterfrom
HebaruSan:feature/dotnet-inflator
Jun 7, 2025
Merged

Switch Inflator from Mono to dotnet#4387
HebaruSan merged 8 commits into
KSP-CKAN:masterfrom
HebaruSan:feature/dotnet-inflator

Conversation

@HebaruSan
Copy link
Copy Markdown
Member

Motivation

While auditing Netkan code to try to make progress on the array bounds errors that occasionally show up in the status page, I noticed that:

  • netkan.exe --releases all wouldn't work because all was translated into null which later was defaulted to 1 in the chain of processing of that option.
  • It would be nice to be able to pass multiple netkans on the command line of netkan.exe.
  • [Feature]: Remove the Mono dependency on macOS #4353's idea of switching from Mono to dotnet, while problematic for the Mac build because I don't have a Mac on which to test it, could be piloted in the Inflator container much more easily.

Problem

See KSP-CKAN/NetKAN#10588, the progress bar for removing HideEmptyTechNodes looks weird, showing negative bytes remaining and greater than 100% completion:

image

Reported by Discord user Shadlock.

Cause

If a module's install_size doesn't match up with what's on disk, then we end up removing a different number of bytes than expected. Bytes remaining is calculated naïvely by subtracting bytes completed from install_size, which can be negative if there are too many bytes. The percent completed works similarly.

Changes

  • Now the Inflator container uses dotnet instead of Mono
    • The build scripts now run dotnet publish for Netkan to generate a self-contained app for net8.0 and linux-x64
    • Dockerfile.netkan now uses ubuntu:latest again because it no longer installs Mono, it installs the entire net8.0 publish folder for Netkan into /home/netkan, and its ENTRYPOINT now runs the dotnet build of CKAN-NetKAN
    • The workflow that builds the container now retrieves the artifact containing _build/out instead of _build/repack, and runs the Docker build in _build/out/CKAN-NetKAN/Release/bin/net8.0/linux-x64/publish instead of _build/repack/Release
    • One potential advantage of dotnet is "trimming", which can shrink the container by excluding assemblies and even parts of assemblies that aren't needed (including system assemblies). I experimented with this, but it consistently broke the app. Instead, a .dockerignore file contains all the DLLs that I was able to confirm are not needed, as a halfway measure until we are able to get real trimming to work.
  • Now the metadata tester container installs a config file at /usr/local/etc/metadata.ini as specified in Get netkan and ckan commands from config file xKAN-meta_testing#101, which we can use later to make it use dotnet instead of Mono (which is not attempted here).
  • Now the --releases all option will properly inflate all releases.
  • Now you can pass as many netkans as you like to netkan.exe, and all will be inflated.
  • Now ModuleInstaller sanity checks the numbers that it sends out for status updates, so negative bytes remaining will no longer be seen for mods with bigger than expected files.
  • The Netkan error about "Generated 2 modules but only 1 requested" will now append "(pre-release)" after versions which are pre-releases
  • TLS1.3 is now force-allowed in addition to TLS1.2, just in case it becomes necessary at some point in the future (see Force-allow TLS 1.2 on .NET 4.5 #2297).

Note that we may not get a new dev build after merging this, depending on whether the SignPath action continues to error out as in SignPath/github-action-submit-signing-request#9.

@HebaruSan HebaruSan added Bug Something is not working as intended Enhancement New features or functionality Core (ckan.dll) Issues affecting the core part of CKAN Build Issues affecting the build system Netkan Issues affecting the netkan data Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Cake Issues affecting Cake Network Issues affecting internet connections of CKAN labels Jun 7, 2025
@HebaruSan HebaruSan force-pushed the feature/dotnet-inflator branch from 306c078 to 26640c7 Compare June 7, 2025 00:47
@coveralls
Copy link
Copy Markdown

coveralls commented Jun 7, 2025

Pull Request Test Coverage Report for Build 15503475519

Details

  • 14 of 67 (20.9%) changed or added relevant lines in 8 files are covered.
  • 3 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.003%) to 30.325%

Changes Missing Coverage Covered Lines Changed/Added Lines %
Cmdline/Main.cs 0 1 0.0%
Netkan/Transformers/StagingLinksTransformer.cs 0 1 0.0%
Tests/Core/Net/AutoUpdateTests.cs 0 1 0.0%
Netkan/Processors/Inflator.cs 0 9 0.0%
Netkan/CmdLineOptions.cs 0 14 0.0%
Netkan/Program.cs 0 27 0.0%
Files with Coverage Reduction New Missed Lines %
Netkan/Program.cs 1 0.0%
Netkan/Processors/Inflator.cs 2 0.0%
Totals Coverage Status
Change from base Build 15494123722: 0.003%
Covered Lines: 13729
Relevant Lines: 44305

💛 - Coveralls

@HebaruSan HebaruSan force-pushed the feature/dotnet-inflator branch 4 times, most recently from ed1eeaf to 7d6ca5d Compare June 7, 2025 01:46
@HebaruSan HebaruSan force-pushed the feature/dotnet-inflator branch 2 times, most recently from bbc7147 to 31f7824 Compare June 7, 2025 02:42
@HebaruSan HebaruSan force-pushed the feature/dotnet-inflator branch from 31f7824 to 3ca2b7a Compare June 7, 2025 02:49
@HebaruSan HebaruSan merged commit 50e7965 into KSP-CKAN:master Jun 7, 2025
5 checks passed
@HebaruSan HebaruSan deleted the feature/dotnet-inflator branch June 7, 2025 02:59
@HebaruSan
Copy link
Copy Markdown
Member Author

All errors are being filed as "Permission denied" because the log4net.xml file is there despite being in the .dockerignore file, which seems to be completely missing in action somehow when built on GitHub, but very much present when I try it locally. 🤦

image

@HebaruSan
Copy link
Copy Markdown
Member Author

... and this is why.

https://github.com/actions/upload-artifact/blob/v4/docs/MIGRATION.md#hidden-files

(In Unix, files that begin with . are considered "hidden" because ls won't display them without the -a or -A flags.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something is not working as intended Build Issues affecting the build system Cake Issues affecting Cake Core (ckan.dll) Issues affecting the core part of CKAN Enhancement New features or functionality Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Netkan Issues affecting the netkan data Network Issues affecting internet connections of CKAN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants