Skip to content

nextflow lint: support reading file paths from a file or stdin to avoid ARG_MAX limits #6857

@adamrtalbot

Description

@adamrtalbot

Feature Request

Problem

When integrating nextflow lint with tools like pre-commit, all matching filenames are passed as CLI arguments. On large repositories (e.g. nf-core/modules with 1700+ .nf files), this exceeds the OS ARG_MAX limit (~2MB on Linux, shared between argv + envp), causing E2BIG errors:

nextflow: line 287: /usr/bin/dirname: Argument list too long
nextflow: line 323: /usr/bin/sed: Argument list too long
nextflow: line 327: java: Argument list too long

The current workaround is to batch files into multiple nextflow lint invocations (see seqeralabs/nf-lint-pre-commit#4), but this means multiple JVM startups and requires capturing output to present a clean summary.

Proposed Solution

Support reading file paths from a file or stdin, e.g.:

# Option A: response file (like javac @argfile)
nextflow lint @filelist.txt

# Option B: --files-from flag
nextflow lint --files-from filelist.txt

# Option C: stdin
find . -name '*.nf' | nextflow lint --stdin

Any of these would allow a single nextflow lint invocation regardless of file count, avoiding the ARG_MAX limit entirely.

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions