Store additional license fields #63

[tdruez]

• Added 2 new license related fields on the Component and Package models:

  • declared_license_expression
  • other_license_expression

• Added 2 properties on the Component and Package models:

  • declared_license_expression_spdx (computed from declared_license_expression)
  • other_license_expression_spdx (computed from other_license_expression)

• Removed 2 fields:

  • Package.declared_license
  • Component.concluded_license

---

• The new fields are available in Component and Package:

  • Add/Edit form, under the "Concluded license expression" field
  • "License" tab (only if a value is defined)
  • Admin form
  • API
  • Import
  • Reporting

• Set "Concluded license expression" for the license_expression fields verbose name on Component, Package, ProductComponent, and ProductPackage fields. (Not applied to Product and Subcomponent).

• Update all legacy license_expression help texts

• The new fields are now displayed (when a value is available) in "Scan" tab "DETECTED PACKAGE" section and the directly related "Set value to Package" form. Note that if the declared_license_expression is available in the Scan data, and the license_expression is empty on the Package instance, the license_expression (concluded) will be set with the declared_license_expression value.

[tdruez]

The latest addition as of June 19th:

• On the Package and Component form, the license builder (autocomplete) is now displayed for the 3 license_expression fields.

• The SCAN SUMMARY section "Set values to Package" feature was refactored to support the new fields:

  • The selected values (checkboxes) in "Declared license" are combined and put in the "Concluded license expression" and "Declared
    license expression" inputs of the form.
  • The selected values (checkboxes) in "Other licenses" are combined and put in the "Other license expression" input of the form.
  • In both "Set values to Package" forms, DETECTED PACKAGE and SCAN SUMMARY section, all the license expression fields are rendered with a license builder (autocomplete)

[DennisClark]

@tdruez the latest features look good on Staging, no problems found.

[tdruez]

The latest addition as of June 27th:

• The new license fields are automatically populated from the Package scan "Update packages automatically from scan".
• The new license fields are pre-filled in the Package form when using the "Add Package" from a PurlDB entry.
• The new license fields are pre-filled in the Component form when using the "Add Component from Package data".
• The license expression values provided in the form for the new field is now properly checked and return a validation error when incorrect.

@DennisClark that addition should complete the implementation. The last thing no implemented is the ability to render those new fields as SPDX. I've started to work on it but it will require a refactor of the license expression system.
Since this PR is large enough and the SPDX part is not a blocker, I'd like to merge and deploy the current progress and we can add the SPDX properties in another PR.

[DennisClark]

@tdruez Question please: how does a value get populated into "Other license expression"? I did a rather complicated create-package-scan from https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz in Staging Starship and it seemed to work just fine, except that there is nothing in the "Other license expression" field even though there are lots of "other_license_expressions" returned in the scan results. Shouldn't there be a process to combine and de-dupe those into a single expression? this is another one that we can defer so that you can go ahead and merge/deploy progress-to-date but we should give it some consideration soon.

nifi-rel-nifi-2.0.0-M3.tar.gz_scan.zip

[DennisClark]

@tdruez regarding my previous comment, I created a new package in Staging Starship from the PurlDB using
pkg:maven/org.apache.lucene/[email protected]?classifier=sources
and it worked just fine and populated the "Other license expression" field, which it may have taken directly from the PurlDB which has a value there.

[DennisClark]

@tdruez I successfully tested all the new features you mention (except the new-component-from-package problem mentioned above) but I have one concern about the UI. Is it possible, and practical, to present the individual licenses in "Declared license expression" and "Other license expression" with links to the DejaCode license definitions? Also is it possible to show the Usage Policy icon? See attached screenshot.

[tdruez]

Question please: how does a value get populated into "Other license expression"? I did a rather complicated create-package-scan from https://github.com/apache/nifi/archive/refs/tags/rel/nifi-2.0.0-M3.tar.gz in Staging Starship and it seemed to work just fine, except that there is nothing in the "Other license expression" field even though there are lots of "other_license_expressions" returned in the scan results.

other_license_expressions is a summary field containing multiple entries (list) whereas other_license_expression is a single flat field on the package.
The other_license_expression can be used when it is available on a "DETECTED PACKAGE" but the value available on the summary is not the same field type.

it worked just fine and populated the "Other license expression" field, which it may have taken directly from the PurlDB which has a value there.

Correct, it was taken from the PurlDB.

except when I did an Add to Component/New Component from Package Data, I got the attached error screen. (I got the same error in Staging Starship when I tried to do it there.)

That error is now fixed.

Is it possible, and practical, to present the individual licenses in "Declared license expression" and "Other license expression" with links to the DejaCode license definitions? Also is it possible to show the Usage Policy icon? See attached screenshot

Yes, I've added proper rendering, link, and policy, for the new fields.

---

Also, I've managed to implement the SPDX rendering for all license_expression fields.

In Reporting, there are 3 new properties available on Component and Package: concluded_license_expression_spdx, declared_license_expression_spdx, other_license_expression_spdx

[DennisClark]

@tdruez all my concerns are addressed in the latest Staging Starship. No problems found. Everything looks good and ready to go, thanks.