-
Notifications
You must be signed in to change notification settings - Fork 775
Open
Description
Feature Request
Problem
When integrating nextflow lint with tools like pre-commit, all matching filenames are passed as CLI arguments. On large repositories (e.g. nf-core/modules with 1700+ .nf files), this exceeds the OS ARG_MAX limit (~2MB on Linux, shared between argv + envp), causing E2BIG errors:
nextflow: line 287: /usr/bin/dirname: Argument list too long
nextflow: line 323: /usr/bin/sed: Argument list too long
nextflow: line 327: java: Argument list too long
The current workaround is to batch files into multiple nextflow lint invocations (see seqeralabs/nf-lint-pre-commit#4), but this means multiple JVM startups and requires capturing output to present a clean summary.
Proposed Solution
Support reading file paths from a file or stdin, e.g.:
# Option A: response file (like javac @argfile)
nextflow lint @filelist.txt
# Option B: --files-from flag
nextflow lint --files-from filelist.txt
# Option C: stdin
find . -name '*.nf' | nextflow lint --stdinAny of these would allow a single nextflow lint invocation regardless of file count, avoiding the ARG_MAX limit entirely.
Context
- Reported in CI: https://github.com/nf-core/modules/actions/runs/22347821806/job/64668726513?pr=10241#step:4:156
- Related (CI-friendly logging):
nextflow lintignores--quietflag #6856 - Current workaround: fix: batch files in lint hook to avoid ARG_MAX on large repos seqeralabs/nf-lint-pre-commit#4
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels