[SEDONA-703] Add utils for converting between RDD[Row] and SpatialRdd

[james-willis]

Did you read the Contributor Guide?

• Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

• Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-703. The PR name follows the format [SEDONA-XXX] my subject.

What changes were proposed in this PR?

Added utils for converting between RDD[Row] and SpatialRDD. These are helpful for usecases that back physical nodes or functions that take in and return RDDs.

How was this patch tested?

Unit Tests

Did this PR include necessary documentation updates?

• Yes, I am adding a new API. I am using the current SNAPSHOT version number in vX.Y.Z format.

Docstrings are available.

[james-willis]

@Kontinuation @jiayuasu is this logic duplicated anywhere else? Maybe somewhere in the Join logic?

[jiayuasu]

@james-willis This is where Sedona join does the conversion:

DF to SpatialRDD:

sedona/spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/strategy/join/TraitJoinQueryBase.scala

Line 46 in 3151f7d

def toSpatialRDD(rdd: RDD[UnsafeRow], shapeExpression: Expression): SpatialRDD[Geometry] = {

RDD to DF:

sedona/spark/common/src/main/scala/org/apache/spark/sql/sedona_sql/strategy/join/TraitJoinQueryExec.scala

Line 125 in 3151f7d

matchesRDD.mapPartitions { iter =>

Please consider reusing the logic if possible,

[james-willis]

Please consider reusing the logic if possible

Thanks Jia.

I was thinking that these functions I created would be useful for Physical Nodes but the linked code reminds me that InternalRows/UnsafeRows are used in these Physical Nodes, whereas the functions in this PR are for Rows. While you can convert between Rows and InternalRows, its an unnecessary conversion. It is probably best to keep distinct implementations for these two interfaces.

The fact that the implementations here are only useful for RDD[Row] but not RDD[InternalRow] makes me wonder if these are useful enough to introduce into the Sedona API.

[jiayuasu]

Close in favor of #1780