Skip to content

Add AuronQueryTest: compare results with vanilla Spark and check native operators #1654

@yew1eb

Description

@yew1eb

Is your feature request related to a problem? Please describe.

  • Manually writing expected results is error-prone, hard to maintain, and often breaks across spark versions duto to behavioral differences.
  • Many existing tests never verify the physical plan; in practice, a lot of cases silently fallback to vanilla spark instead of executing with native operators.

Describe the solution you'd like

  • Add AuronQueryTest extending org.apache.spark.sql.QueryTest
  • Provide helpers to:
    • Run a query once with native disabled to collect the baseline(vanilla spark), then run with native enabled ans assert results match vanilla spark.
    • Optionally enforce that the physical plan contains only native operators (ignoring stage/adapter nodes), failing fast otherwise.

Describe alternatives you've considered

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions