Skip to content

Make scancode parallelism configurable#612

Merged
elrayle merged 4 commits intomasterfrom
roman/scancode_parallelism
Oct 24, 2024
Merged

Make scancode parallelism configurable#612
elrayle merged 4 commits intomasterfrom
roman/scancode_parallelism

Conversation

@RomanIakovlev
Copy link
Copy Markdown
Contributor

@RomanIakovlev RomanIakovlev commented Oct 23, 2024

Fixes #609
Replaces #611

Copy link
Copy Markdown
Contributor

@ljones140 ljones140 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work

Copy link
Copy Markdown
Collaborator

@elrayle elrayle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change requested, then this looks good.

Co-authored-by: E. Lynette Rayle <elrayle@users.noreply.github.com>
Copy link
Copy Markdown
Collaborator

@elrayle elrayle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this configurable. This will really help with long term maintenance and scalability as needed.

Copy link
Copy Markdown
Collaborator

@elrayle elrayle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs quotes around config name.

],
timeout: 1000,
processes: 2,
processes: Number(config.get(CRAWLER_SCANCODE_PARALLELISM) || process.env.CRAWLER_SCANCODE_PARALLELISM) || 2,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
processes: Number(config.get(CRAWLER_SCANCODE_PARALLELISM) || process.env.CRAWLER_SCANCODE_PARALLELISM) || 2,
processes: Number(config.get('CRAWLER_SCANCODE_PARALLELISM') || process.env.CRAWLER_SCANCODE_PARALLELISM) || 2,

@elrayle elrayle merged commit 8772e68 into master Oct 24, 2024
@elrayle elrayle deleted the roman/scancode_parallelism branch October 24, 2024 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scancode parallelism is hardcoded to 2 processes

3 participants