[AURON #1888] Implement spark_partition_id() function

[ShreyeshArangath]

Which issue does this PR close?

Closes ##1888

Rationale for this change

This is part of the effort to add non-deterministic expressions to Auron.

What changes are included in this PR?

This PR adds support for spark_partition_id using native

Are there any user-facing changes?

N/A

How was this patch tested?

Unit tests

[Copilot]

Pull request overview

This PR implements support for the spark_partition_id() function using native execution, enabling Auron to handle this non-deterministic expression natively rather than falling back to Spark's implementation.

Changes:

• Added Rust implementation of SparkPartitionIdExpr that returns the partition ID as a constant Int32 array
• Integrated the expression into the planner, protobuf schema, and Scala conversion layers
• Included comprehensive unit tests for the Rust implementation

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
native-engine/datafusion-ext-exprs/src/spark_partition_id.rs	New Rust implementation of SparkPartitionIdExpr with unit tests
native-engine/datafusion-ext-exprs/src/lib.rs	Registers the new spark_partition_id module
native-engine/auron-planner/src/planner.rs	Integrates SparkPartitionIdExpr into the planner, passing partition_id from PhysicalPlanner
native-engine/auron-planner/proto/auron.proto	Adds SparkPartitionIdExprNode protobuf message definition
spark-extension/src/main/scala/org/apache/spark/sql/auron/NativeConverters.scala	Adds conversion for StubExpr("SparkPartitionID") to protobuf
spark-extension-shims-spark/src/main/scala/org/apache/spark/sql/auron/ShimsImpl.scala	Adds conversion for SparkPartitionID expression to protobuf and imports the catalyst expression

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.