Skip to content

Collect pypi PURL ondemand #468#592

Merged
JonoYang merged 12 commits intomainfrom
468_data_collection_for_PyPI
Mar 31, 2025
Merged

Collect pypi PURL ondemand #468#592
JonoYang merged 12 commits intomainfrom
468_data_collection_for_PyPI

Conversation

@chinyeungli
Copy link
Contributor

  • Add data collection support for pypi
  • Fetch all versions if no version is provided in the purl
  • Generate multiple purls with the file_name qualifier when multiple packages are available for a single version.

It's now able to fetch JSON from PyPI API with inputing thr pypi purl with version and pass it to scan and return the scan result.

ToDo item is the second part of the issue

Signed-off-by: Chin Yeung Li <tli@nexb.com>
…he PURL.

Signed-off-by: Chin Yeung Li <tli@nexb.com>
…ackages are available for a single version.

Signed-off-by: Chin Yeung Li <tli@nexb.com>
@chinyeungli chinyeungli requested a review from JonoYang March 25, 2025 08:46
Copy link
Member

@JonoYang JonoYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chinyeungli Code looks good so far, just needs some tests

packages = build_packages(package_json, package_url)

for package in packages:
# package.extra_data["package_content"] = PackageContentType.SOURCE_ARCHIVE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove line if not needed

@pombredanne pombredanne changed the title 468 data collection for pypi Collect pypi PURL ondemand #468 Mar 27, 2025
chinyeungli and others added 4 commits March 28, 2025 07:00
Correct typo

Signed-off-by: Chin Yeung Li <tli@nexb.com>

Co-authored-by: Jono Yang <JonoYang@users.noreply.github.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
if not url:
continue

packagetype = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
@JonoYang
Copy link
Member

@chinyeungli LGTM! merging

@JonoYang JonoYang merged commit 7891773 into main Mar 31, 2025
7 checks passed
@chinyeungli chinyeungli deleted the 468_data_collection_for_PyPI branch April 2, 2025 07:31
JonoYang added a commit that referenced this pull request Apr 14, 2025
@pombredanne pombredanne moved this from Done to Validated in 00-AboutCodePlanner Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants