ZFIN-9917: Remove jars from source tree by rtaylorzfin · Pull Request #1797 · ZFIN/zfin

rtaylorzfin · 2026-04-02T01:56:01Z

Removed solr distribution (unused as it is pulled via the docker hub image)
Upgraded the PostgreSQL JDBC driver to version 42.7.8 in both the Solr and Tomcat Docker images, now fetched directly from the official source with a SHA256 checksum for integrity.
Updated the Gradle dependency for org.postgresql:postgresql to use a variable (postgresqlVersion) instead of a hardcoded version so we can match the same version everywhere more easily
Replace the 41 MB fat JAR (GWT 2.8.2 with javax->jakarta transformation and all bundled dependencies) with Maven-resolved gwt-dev 2.11.0 which natively supports jakarta.servlet.
Update GWT compilation classpath in both compileGwtModule and gwtCompile tasks to use configurations.compileClasspath with proper ordering
Add org.w3c.css:sac:1.3 dependency for UiBinder CSS processing
Filter jackson-annotations sources JAR from Maven (incompatible with GWT JRE emulation); local eclipse-transformed version is used instead
Remove unused server-side imports from GWT-visible source files: LookupStrings.java (Spring Model), RelatedEntityDTO.java (Publication), ExpressionCurationService.java (@PathVariable), FeatureAddView.java (javassist)
Inline LookupStrings.idNotFound() into its 2 callers and remove the method
Add analysis document in documentation (lib/Java/gwt/GWT-JACKSON-NOTES.md)

The bin/, server/, and solr.xml were from an ancient ~6.x era Solr distribution, completely superseded by the solr:8.11.3 Docker image. The dataimporthandler jars in site_index/lib/ were version 8.6.3 and already loaded from the Docker image's /opt/solr/dist/ via solrconfig.xml. The old postgresql JDBC driver (9.4.1210) is replaced by a Gradle-managed dependency in the next commit.

Add deploySolrJdbc Gradle task that copies the postgresql jar from runtimeClasspath to $SOLR_HOME/lib/, called by ant deploy-solr. Also aligns test postgresql dependency (42.2.18 -> 42.7.8) with prod.

…ation Download the jar at build time instead of checking it into the repo.

Use the same ADD --checksum approach as the Tomcat Dockerfile. This removes the need for the deploySolrJdbc Gradle task and Ant wiring.

Define postgresqlVersion once in buildscript.ext and reference it in all Gradle dependency declarations. Upgrade SchemaSpy from broken 5.0.0 (jar was missing) to 6.2.4 via Gradle dependency, use -vizjs for built-in diagram rendering, and fix DB user to use PGUSER.

These jars are already declared as Gradle dependencies and get copied from runtimeClasspath during deployment, overwriting these checked-in copies. The remaining 9 jars in WEB-INF/lib match the localFileDependencies list — jars not available via Maven that must be checked in.

JSObject.class and JSException.class are ancient Netscape plugin API artifacts with zero references anywhere in the codebase.

The tomcat-juli-extra.jar (v7.0.32) and tomcat-juli-adapters.jar (v7.0.30) were used to route Tomcat JULI logging to log4j 1.x. These jars are from Tomcat 7 but the Docker image runs Tomcat 10, which dropped JULI extras entirely. Tomcat 10's built-in JULI logging already writes to catalina.out and console by default. Also removes the now-unnecessary bin/ copy from the Ant create-mutant-instance target.

Smoke test targets (smokeTests, productionSmokeTests, smokeTestsReadWrite, smoke) now delegate to Gradle via `gradle test -Psuite=<SuiteName>`, matching the pattern already used by the `test` target. The email notification on failure (handleSmokeFailure) is preserved for productionSmokeTests. This allows removing the 18 checked-in jwebunit jars from lib/Java/jwebunit/ since Gradle already declares equivalent dependencies (jwebunit-core 2.4/3.3, htmlunit 2.70.0, etc.) as testImplementation. Also removes SmokeTestsReadWrite.java from Gradle's test source exclusion list so it can be compiled and run via Gradle.

Removes standard Maven jars from lib/Java/gwt/ that are already available via Gradle dependencies: jackson-annotations-2.15.2, jakarta.ws.rs-api-3.1.0, jakarta.xml.bind-api-3.0.1, and validation-api-1.0.0.GA (+sources). The 5 eclipse-transformed jars (gwt-dev, gwt-jackson, jsinterop, jackson-annotations-2.8.2-sources) and gwt-user are kept because they contain javax->jakarta namespace transformations needed for Tomcat 10 compatibility, or have no Maven equivalent. Also updates the gwtCompile task to pull standard deps from Gradle's compileClasspath instead of the fileTree, and fixes a broken localFileDependencies reference (jsinterop filename mismatch). Remove unused zip files from lib/Java

@PathVariable

… 2.11.0 Replace the 41 MB fat JAR (GWT 2.8.2 with javax->jakarta transformation and all bundled dependencies) with Maven-resolved gwt-dev 2.11.0 which natively supports jakarta.servlet. - Update GWT compilation classpath in both compileGwtModule and gwtCompile tasks to use configurations.compileClasspath with proper ordering - Add org.w3c.css:sac:1.3 dependency for UiBinder CSS processing - Filter jackson-annotations sources JAR from Maven (incompatible with GWT JRE emulation); local eclipse-transformed version is used instead - Remove unused server-side imports from GWT-visible source files: LookupStrings.java (Spring Model), RelatedEntityDTO.java (Publication), ExpressionCurationService.java (@PathVariable), FeatureAddView.java (javassist) - Inline LookupStrings.idNotFound() into its 2 callers and remove the method - Add analysis document in docs/ Replace 7 checked-in GWT jars with Maven dependencies Use full compileClasspath (denylist filter) instead of cherry-picking individual jars. Only 2 jars remain in lib/Java/gwt/ that can't come from Maven: jackson-annotations-2.8.2-sources-eclipse-transformed.jar (GWT JRE emulation incompatibility) and gwt-jackson-0.15.0-eclipse-transformed.jar (contains .java sources needed for GWT client compilation). Add some documentation around the remaining jar files that are needed for gwt Eliminate last 2 checked-in GWT jars, replace with Maven dependencies - Remove gwt-jackson-0.15.0-eclipse-transformed.jar (Maven 0.15.4 includes sources) - Remove jackson-annotations-2.8.2-sources-eclipse-transformed.jar - Add gwtSources configuration to pull jackson-annotations 2.11.4 sources (newest GWT-compatible version; 2.12+ uses String.format unsupported by GWT) - Exclude gwtSources from Jackson version resolution strategy to prevent Gradle from upgrading it to the GWT-incompatible 2.15.2 - Add GWT-JACKSON-NOTES.md documenting the constraints lib/Java/gwt/ now contains only documentation, no jars.

rtaylorzfin · 2026-04-02T20:07:19Z

More information on the refactorings to remove some of the gwt-related jars:

Analysis: gwt-dev-eclipse-transformed.jar

What it is

A fat/uber JAR (41 MB, 22,672 entries, 17,622 .class files) containing the GWT 2.8.2 development compiler plus all transitive dependencies, with javax.servlet references renamed to jakarta.servlet using an Eclipse transformation tool.

GWT version: 2.8.2 (git rev faff18e7a)
Built with: JDK 1.8.0_112-google, Ant 1.9.3
Location: lib/Java/gwt/gwt-dev-eclipse-transformed.jar

Bundled dependencies

Library	Version
Eclipse JDT Compiler	~1,296 classes
Jetty (all modules)	9.2.14.v20151106
HtmlUnit	2.19
Google Guava (shaded)	`com.google.gwt.thirdparty.guava`
Protobuf Java	2.5.0
Gson	2.6.2
Apache BCEL	bundled
Apache Ant (taskdefs)	bundled
ICU4J	bundled
CERN Colt	bundled
Commons Codec	1.10
Commons Collections	3.2.2
Commons IO	2.4
Commons Lang3	3.4
Commons Logging	1.2
HttpClient	4.5.1
CSS Parser	0.9.18
Apache Jasper	8.0.9.M3
Rhino JS (via htmlunit-core-js)	bundled

How it's used

In `build.gradle`

The gwtCompile task (line ~1441) compiles all three GWT modules:

org.zfin.gwt.lookup.Lookup
org.zfin.gwt.marker.Marker
org.zfin.gwt.curation.Curation

Its classpath is:

classpath = files('home/WEB-INF/classes') +
            fileTree(dir: 'lib/Java/gwt', include: '*.jar') +  // <-- picks up the fat JAR
            files('home/WEB-INF/lib/restygwt-2.2.7-eclipse-transformed.jar') +
            files('source')

The fileTree(dir: 'lib/Java/gwt', include: '*.jar') glob picks up this JAR, which provides the com.google.gwt.dev.Compiler main class and all its runtime dependencies.

Version mismatch is intentional

compileOnly "org.gwtproject:gwt-dev:2.11.0" — used for Java IDE compilation and type checking only
gwt-dev-eclipse-transformed.jar (GWT 2.8.2) — used for actual GWT→JavaScript compilation

The compileGwtModule task (line ~1418) uses configurations.compileClasspath (Maven-resolved 2.11.0), but it's a single-module convenience task. The gwtCompile task that actually runs during builds uses local file JARs including the fat 2.8.2 JAR.

Why the fat JAR exists

GWT 2.8.2's gwt-dev uses javax.servlet internally
The ZFIN project migrated to Jakarta EE (jakarta.servlet)
The JAR was created by running an Eclipse transformation tool to rename all javax.servlet → jakarta.servlet references in the bytecode
No Maven artifact exists for this — it's a custom one-off repackaging
The stock gwt-dev-2.11.0.jar from Maven is a thin JAR (16 MB, 6,466 classes) that expects transitive deps to be provided separately, unlike this fat JAR

Attempt to replace with Maven-resolved gwt-dev 2.11.0

What we tried

Changed the gwtCompile classpath from local file JARs to:

classpath = configurations.compileClasspath +
            fileTree(dir: 'lib/Java/gwt', include: '*.jar') +
            files('source')

This successfully resolved all transitive dependencies (cern/colt, ASM, Eclipse JDT, etc.) via Maven.

What happened

GWT compilation failed with errors like:

[ERROR] Line 6: The import org.zfin.framework.presentation.LookupStrings cannot be resolved
[ERROR] Line 53: LookupStrings cannot be resolved to a variable

And javax.validation source errors (fixed by keeping fileTree for validation-api-1.0.0.GA-sources.jar).

Root cause: GWT 2.11.0 source resolution behavior change

GWT 2.8.2 behavior: When both .java source and .class bytecode are on the classpath for a class, the compiler uses the bytecode for non-GWT-module classes.

GWT 2.11.0 behavior: The compiler prefers the .java source and tries to GWT-compile it.

The source/ directory is on the GWT classpath (required for GWT module source). It contains source/org/zfin/framework/presentation/LookupStrings.java, which:

Is not inside any GWT <source> path (GWT modules only include ui, dto, event, util sub-packages)
Imports org.springframework.ui.Model, which GWT cannot compile
Was previously resolved from home/WEB-INF/classes/ as bytecode by GWT 2.8.2

The LookupStrings class is used by GWT client code (e.g., source/org/zfin/gwt/lookup/ui/Lookup.java line 6 imports it). It's a constants class that holds string values shared between GWT client and server code.

GWT module structure

source/org/zfin/gwt/lookup/Lookup.gwt.xml  → <source path='ui'/>
source/org/zfin/gwt/marker/Marker.gwt.xml  → (inherits Root)
source/org/zfin/gwt/curation/Curation.gwt.xml → (inherits Root)
source/org/zfin/gwt/root/Root.gwt.xml      → <source path='dto'/> <source path='event'/> <source path='ui'/> <source path='util'/>

Other "eclipse-transformed" JARs in the project

These JARs also underwent javax→jakarta transformation:

home/WEB-INF/lib/blast-serialization-1.0-eclipse-transformed.jar
home/WEB-INF/lib/jakarta.servlet.jsp.jstl-api-1.2.7-eclipse-transformed.jar
home/WEB-INF/lib/rescu-2.1.0-eclipse-transformed.jar
home/WEB-INF/lib/restygwt-2.2.7-eclipse-transformed.jar
home/WEB-INF/lib/taglibs-standard-impl-1.2.5-eclipse-transformed.jar
lib/Java/gwt/gwt-jackson-0.15.0-eclipse-transformed.jar
lib/Java/gwt/jackson-annotations-2.8.2-sources-eclipse-transformed.jar
lib/Java/gwt/jsinterop-annotations-2.0.0-eclipse-transformed.jar

Resolution: removed the fat JAR and switched to Maven-resolved gwt-dev 2.11.0

Issues encountered and fixes applied

1. Unused server-side imports in GWT-visible source files

GWT 2.11.0 is stricter about resolving imports in .java files found on the classpath. Three files had unused imports that caused compilation failures:

File	Removed import	Reason
`LookupStrings.java`	`org.springframework.ui.Model`	Used only in `idNotFound()` method, which was server-side only. Method inlined into the 2 callers (`ShowPublicationController`, `OrthologyController`) and removed.
`RelatedEntityDTO.java`	`org.zfin.publication.Publication`	Completely unused import (Publication has Hibernate annotations GWT can't compile).
`ExpressionCurationService.java`	`org.springframework.web.bind.annotation.PathVariable`	Unused import (code uses `@PathParam` from Jakarta WS-RS instead).
`FeatureAddView.java`	`javassist.compiler.ast.StringL`	Completely unused import.

2. Missing CSS SAC dependency

GWT's UiBinder needs org.w3c.css.sac.CSSException for processing CSS in .ui.xml templates. The old fat JAR bundled cssparser-0.9.18 which included this. GWT 2.11.0's transitive dependencies don't include it.

Fix: Added compileOnly "org.w3c.css:sac:1.3" to build.gradle.

3. Jackson annotations sources JAR conflict

gwt-jackson:0.15.4 declares a dependency on jackson-annotations with the sources classifier (standard for GWT libraries). Maven resolves this to jackson-annotations-2.15.2-sources.jar, which uses String.format(String, Set<String>) — not supported by GWT's JRE emulation.

Fix: Filtered jackson-annotations-*-sources.jar from the GWT compilation classpath. The local lib/Java/gwt/jackson-annotations-2.8.2-sources-eclipse-transformed.jar provides a compatible version.

4. Stale UiBinder XML templates in build output

sourceSets.main.output includes build/resources/main/ which contained outdated copies of .ui.xml template files (missing ui:field attributes that exist in the current source). When placed first on the classpath, GWT found the stale templates instead of the current ones.

Fix: Changed classpath ordering to put files('source') first, then files('home/WEB-INF/classes') for compiled bytecode, then Maven dependencies.

5. Compiled classes not on GWT classpath

configurations.compileClasspath does not include the compiled output directory (home/WEB-INF/classes). Classes like LookupStrings and View that are referenced by GWT client code but live outside GWT <source> paths need to be available as bytecode.

Fix: Explicitly added files('home/WEB-INF/classes') to the GWT compilation classpath.

Final classpath configuration

// Filter out jackson-annotations-*-sources.jar from Maven;
// the local eclipse-transformed sources JAR in lib/Java/gwt/ is used instead.
def filteredCompileClasspath = configurations.compileClasspath.filter { file ->
    !file.name.matches('jackson-annotations-.*-sources\\.jar')
}
classpath = files('source') +
            files('home/WEB-INF/classes') +
            filteredCompileClasspath +
            fileTree(dir: 'lib/Java/gwt', include: '*.jar')

Classpath ordering matters:

source/ — GWT module source files and .ui.xml templates (must be first to avoid stale copies)
home/WEB-INF/classes/ — compiled bytecode for non-GWT classes referenced by GWT code
configurations.compileClasspath — Maven-resolved dependencies including gwt-dev:2.11.0 and transitive deps
lib/Java/gwt/*.jar — local GWT JARs (gwt-user, validation-api-sources, jackson-annotations-sources-eclipse-transformed, etc.)

Files changed

build.gradle — updated GWT compilation classpath, added org.w3c.css:sac:1.3 dependency
lib/Java/gwt/gwt-dev-eclipse-transformed.jar — deleted (41 MB)
source/org/zfin/framework/presentation/LookupStrings.java — removed Spring import and idNotFound() method
source/org/zfin/gwt/root/dto/RelatedEntityDTO.java — removed unused Publication import
source/org/zfin/gwt/curation/ui/ExpressionCurationService.java — removed unused @PathVariable import
source/org/zfin/gwt/curation/ui/feature/FeatureAddView.java — removed unused javassist import
source/org/zfin/publication/presentation/ShowPublicationController.java — inlined idNotFound() logic
source/org/zfin/orthology/presentation/OrthologyController.java — inlined idNotFound() logic

rtaylorzfin · 2026-04-03T17:45:42Z

docker/tomcat/Dockerfile

+# and the build.gradle file (for the web app) as well. (Also update the checksum.)
+ADD --checksum=sha256:2a32a9dcbc42d67a50ad3a0de5efd102c8d2be46720045f2cbd6689f160ab7c7 \
+    https://jdbc.postgresql.org/download/postgresql-42.7.8.jar \
+    /usr/local/tomcat/lib/postgresql-42.7.8.jar


Can this be part of the deploy process instead of the image definition? That would keep the version numbers in sync

@cmpich after looking into this a bit, I realized the problem with deploying it from our compile container is that the compile container and the tomcat container do not share the /usr/local/tomcat/ path

rtaylorzfin · 2026-04-03T17:46:52Z

docker/solr/Dockerfile

+ADD --checksum=sha256:2a32a9dcbc42d67a50ad3a0de5efd102c8d2be46720045f2cbd6689f160ab7c7 \
+    https://jdbc.postgresql.org/download/postgresql-42.7.8.jar \
+    /var/solr/data/site_index/lib/postgresql-42.7.8.jar
+


Same as comment on tomcat Dockerfile: Can we deploy this file using our standard build process?

rtaylorzfin and others added 11 commits March 31, 2026 14:13

Deploy Solr JDBC driver from Gradle dependency instead of checked-in jar

342c549

Add deploySolrJdbc Gradle task that copies the postgresql jar from runtimeClasspath to $SOLR_HOME/lib/, called by ant deploy-solr. Also aligns test postgresql dependency (42.2.18 -> 42.7.8) with prod.

Upgrade Tomcat postgresql JDBC driver to 42.7.8 with checksum verific…

673ab50

…ation Download the jar at build time instead of checking it into the repo.

Download postgresql JDBC driver in Solr Docker image instead of Gradle

9108409

Use the same ADD --checksum approach as the Tomcat Dockerfile. This removes the need for the deploySolrJdbc Gradle task and Ant wiring.

Remove dead Netscape plugin API class files from lib/Java

770d6df

JSObject.class and JSException.class are ancient Netscape plugin API artifacts with zero references anywhere in the codebase.

rtaylorzfin commented Apr 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ZFIN-9917: Remove jars from source tree#1797

ZFIN-9917: Remove jars from source tree#1797
rtaylorzfin wants to merge 11 commits intoZFIN:mainfrom
rtaylorzfin:zfin-9917-part-3

rtaylorzfin commented Apr 2, 2026 •

edited

Loading

Uh oh!

rtaylorzfin commented Apr 2, 2026

Uh oh!

rtaylorzfin Apr 3, 2026

Uh oh!

rtaylorzfin Apr 3, 2026

Uh oh!

rtaylorzfin Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

rtaylorzfin commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rtaylorzfin commented Apr 2, 2026

Analysis: gwt-dev-eclipse-transformed.jar

What it is

Bundled dependencies

How it's used

In build.gradle

Version mismatch is intentional

Why the fat JAR exists

Attempt to replace with Maven-resolved gwt-dev 2.11.0

What we tried

What happened

Root cause: GWT 2.11.0 source resolution behavior change

GWT module structure

Other "eclipse-transformed" JARs in the project

Resolution: removed the fat JAR and switched to Maven-resolved gwt-dev 2.11.0

Issues encountered and fixes applied

1. Unused server-side imports in GWT-visible source files

2. Missing CSS SAC dependency

3. Jackson annotations sources JAR conflict

4. Stale UiBinder XML templates in build output

5. Compiled classes not on GWT classpath

Final classpath configuration

Files changed

Uh oh!

rtaylorzfin Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

rtaylorzfin Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

rtaylorzfin Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rtaylorzfin commented Apr 2, 2026 •

edited

Loading

In `build.gradle`