Improve "scan_package" or "scan_codebase" pipelines to improve the analysis of a single Java JAR

[pombredanne]

Here are some specifics about JARs:

1. They have a well defined structure with a META-INF root directory, MANIFEST.MF Java properties file
2. They may contain other conventionally located manifest such as a POM in /META-INF/maven///pom.xml and pom.properties but these may depend on other POMs (parent and property files)
3. MANIFEST.MF can contain extra details for OSGi (and an OSGI-INF directory)
4. They often lack details on origin and license
5. The Maven source JAR of a binary JAR is often not the exact corresponding source code, meaning that the source is often elsewhere in a version control repository and this may demand to flow scan details (origin, license, dependency) from the source repo to the binary, but there is not enforced convention to always include the source repo addresses.
6. There is no well defined license documentation convention such as using SPDX expressions
7. There is no well defined place to document dependencies, as this is often handled by plugins instead, meaning that evaluating dependencies is difficult short of using a Java package management tool
8. They are mostly written in Java, but there are many other popular languages that run on the JVM

[pombredanne]

This likely needs to be considered in this updated context, given the work on the new package set that @JonoYang is working on in the PurlDB where we can combine the metadata and scans from source and binary JARs.

[AyanSinhaMahapatra]

Btw, as part of this, we now have an updated package assembly with aboutcode-org/scancode-toolkit#3447 that addresses points 1-4.