Proj experiments #1110

jdroenner · 2026-01-13T16:39:01Z

I added an entry to CHANGELOG.md if knowledge of this change could be valuable to users.

Here is a brief summary of what I did:

…e/geoengine into proj_experiments

michaelmattig · 2026-01-19T16:36:50Z

Cargo.toml

 clap = { version = "4.5", features = ["derive"] }
 clap_derive = "4.5"
 config = "0.15"
+crs-definitions = { git = "https://github.com/jdroenner/crs-definitions.git", branch = "area_of_use_and_extent"}


würdest du das so machen wollen, also die definitionen ins binary legen?

das ding was du geforkt hast hat keine Lizenz, können wir so nicht benutzen oder?

die Alternative wäre ja die irgendwie in eine Map zu legen. Das sind aber ja eigentlich Konstanten...

im Zweifel können wir das auch nachbauen oder eben anpassen. Das ist ja nichts magisches. Aber vielleicht fragen wir einfach nach ner Lizenz. Die Entries kann man alle mit Proj generieren.

michaelmattig · 2026-01-19T16:43:54Z

datatypes/src/util/mixed_projector.rs

+    util::{crs_definitions::StaticAreaofUseProvider, proj_projector::ProjAreaOfUseProvider},
+};
+
+pub enum MixedCoordinateProjector {


confusing name. why not a common trait?

There is a common trait. This enum simply mixes Geodesy and Proj in one type since there are cases where we can not create a Proj Projector and need to fallback.

michaelmattig · 2026-01-19T16:48:14Z

datatypes/src/util/crs_definitions.rs

+    def: Def,
+}
+
+impl AreaOfUseProvider for StaticAreaofUseProvider {


why not implement the trait on Def?

I want to keep external types wrapped so we do not need to use them everywhere. It was hard enough to find and change all places where Proj was used.

michaelmattig · 2026-01-19T16:48:33Z

datatypes/src/util/geodesy_projector.rs

+impl Transformer<geodesy::ctx::Minimal> {
+    pub fn pro4_string_modifications(code: u16, proj4: &str) -> String {
+        match (code, proj4) {
+            (3857, _) => proj4.replace("merc", "webmerc"),


guess this should be documented

also: are there any other cases that require this?

would it make sense to curate this in the crs definitions crate?

yes. This is a very complex topic. I do not know if other projections need this. Also the crs_definitions crate is not bound to Geodesy and as i see it this is something only geodesy requires. We should upstream this to the geo-geodesy crate but there are other things that needs to be done there first.

michaelmattig · 2026-01-19T16:49:46Z

datatypes/src/util/geodesy_projector.rs

+            });
+        }
+
+        if source.code() > u32::from(u16::MAX) {


isnt GOOGLE crs larger than u16?

yes but it is not a real EPSG code. This would need something handling "alias" names

michaelmattig · 2026-01-19T16:50:35Z

datatypes/src/util/geodesy_projector.rs

+    }
+}
+
+impl<C: geodesy::ctx::Context> Transformer<C> {


are there even multiple geodesy context implementations?

yes there are two and they plan a "ParallelContext" upstream

michaelmattig · 2026-01-19T16:56:02Z

datatypes/src/util/mixed_projector.rs

+};
+
+pub enum MixedCoordinateProjector {
+    ProjProjector(super::proj_projector::ProjCoordinateProjector),


unrelated to this line: in general we need to think about whether the approach to use geodesy because of performance and fall back to proj only when geodesy does not support a projection suits all our use cases. can we be reasonably certain that we will not need some crs in the future that geodesy does not support and where we need good performance? because then we would need to implement proj context reusage anyway.

well this is a very interesting question. One "projection" not implemented in Godesy is the GEOS projection. However, we could create a PR if we need it (it is not very complex). Basically all non-EPSG projections are not supported but this is not really Geodesys fault. Geodesy should add a WKT parser... Currently it requires the proj4 strings and we get them from crs-definitions and there are only EPSG codes listed.

currently, all projections we use should work. However, if someone uses DHDN then we have a problem.

michaelmattig · 2026-01-19T16:58:29Z

datatypes/src/util/geodesy_projector.rs

+pub struct Transformer<C: geodesy::ctx::Context = geodesy::ctx::Minimal> {
+    ctx: C,
+    source_crs: u16,
+    source: geodesy::ctx::OpHandle,


why store the ophandle?

this is what you use to call the "fwd" and "rew" projection

michaelmattig · 2026-01-19T16:58:48Z

datatypes/src/util/geodesy_projector.rs

+    ctx: C,
+    source_crs: u16,
+    source: geodesy::ctx::OpHandle,
+    source_latlon: bool,


is this not derivable fro the source/target crs?

well i derive it by looking for "latlong" in the proj4 string. I do not think there is a better way at the moment. Since we use this for every coordinate we should remember this instead of the proj string. However, if you see a better way please let me know :)

michaelmattig · 2026-01-19T16:59:08Z

datatypes/src/util/geodesy_projector.rs

+#[derive(Debug, Clone)]
+pub struct Transformer<C: geodesy::ctx::Context = geodesy::ctx::Minimal> {
+    ctx: C,
+    source_crs: u16,


is this always epsg code? are all relevant projections describable by this?

Geodesy currently parses the proj4 strings to create objects. We need to get the proj4 strings somewhere and the easiest way is to get them from the crs-definitions crate (this is what geo-geodesy does). The crs-definitions crate only has EPSG codes at the moment. We would need to add more e.g. ESRI if we need them

michaelmattig · 2026-01-19T17:00:00Z

datatypes/src/util/geodesy_projector.rs

+        let source_latlon = source_def.proj4.starts_with("+proj=longlat");
+        let target_latlon = target_def.proj4.starts_with("+proj=longlat");
+
+        let source_proj_string = Self::pro4_string_modifications(source_crs, source_def.proj4);


can't we store the definitions "correctly" in the first place instead of modifying them each time?

of the crs-definitions should be useable by other crates then no. If we want them only for geodesy then yes.

michaelmattig · 2026-01-19T17:01:04Z

datatypes/src/util/geodesy_projector.rs

+
+    pub fn transform_coord(&self, coord: Coordinate2D) -> Result<Coordinate2D, Error> {
+        let mut geodesy_coord = if self.source_latlon {
+            [geodesy::coord::Coor2D::gis(coord.x, coord.y)]


so we need to copy all data. does proj do this too?

i think we can also implement a trait. We need to look that up.

michaelmattig · 2026-01-19T17:02:13Z

datatypes/src/util/geodesy_projector.rs

+            [geodesy::coord::Coor2D::raw(coord.x, coord.y)]
+        };
+        self.ctx
+            .apply(self.source, geodesy::Direction::Inv, &mut geodesy_coord)?;


why does geodesy look up the op in the context each time? can we prevent this? we always use the same op but it does a registry lookup each time

it is a very strange design. We would need to look into this.

michaelmattig · 2026-01-19T17:03:21Z

datatypes/src/util/geodesy_projector.rs

+    */
+}
+
+pub fn geodesy_ctx() -> geodesy::ctx::Minimal {


thie Minimal context is also not Send or Sync or is it?

nope but it is fast. There is an issue for a ParallelContext. Maybe this would be Send or Sync.

michaelmattig · 2026-01-19T17:04:00Z

datatypes/src/util/proj_projector.rs

+};
+
+pub struct ProjCoordinateProjector {
+    pub from: SpatialReference,


can this not be derived from the Proj instance?

not in a way that would be halpfull

michaelmattig · 2026-01-19T17:06:13Z

operators/src/processing/reprojection.rs

 ) -> Result<SpatialGridDescriptor> {
    let proj_from_to =
-        CoordinateProjector::from_known_srs(in_srs, params.target_spatial_reference)?;
+        MixedCoordinateProjector::from_known_srs(in_srs, params.target_spatial_reference)?;


wouldn't it still make sense to try to reuse the projector? did you measure the overhead of instantiation?

i created a small benchmark. It is much faster then proj.. You can add more benches if you have ideas how to measure that.

…e/geoengine into proj_experiments

jdroenner added 5 commits January 8, 2026 17:56

add crs-definitions and tracing-flame

4aa98d7

add geodesy and crs-definitions projector and area of use derive

8ee6db6

remove debug symbols

b14154d

Merge branch 'new-pixel-based-queries-rebase' of github.com:geo-engin…

7a04f5d

…e/geoengine into proj_experiments

add MixedAreaProvider, fix Arrow version

a5650e9

michaelmattig reviewed Jan 19, 2026

View reviewed changes

jdroenner added 4 commits January 21, 2026 13:24

Merge branch 'new-pixel-based-queries-rebase' of github.com:geo-engin…

8687262

…e/geoengine into proj_experiments

Merge branch 'new-pixel-based-queries-rebase' of github.com:geo-engin…

298812a

…e/geoengine into proj_experiments

bring back release_max_level_debug

8640bdd

Merge branch 'new-pixel-based-queries-rebase' of github.com:geo-engin…

e46b67b

…e/geoengine into proj_experiments

Proj experiments #1110

Are you sure you want to change the base?

Proj experiments #1110

Uh oh!

Conversation

jdroenner commented Jan 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelmattig Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jdroenner Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

michaelmattig Jan 19, 2026 •

edited

Loading

jdroenner Jan 21, 2026 •

edited

Loading