Skip to content

Explore parallelizing transform-gisaid #448

Description

@joverlee521

Originally proposed by @corneliusroemer in #446 (comment)

How hard is it to parallelize ~4h - transform-gisaid?

As this operates on ndjson lines, it might be parallelizable, or at least some part of it.

The obvious way to do so would be to have a split rule to divide input files into N chunks, run transform, and merge back.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions