Sort EXEC NOT SUPPORTED MAP #1024
Description
Describe the bug
SQL:
select /*+repartition(100) */ ret_map from pub_dw.pub_dwv_flow_actv_chnl_label_info_di where dt = '2025-05-28' limit 100
ret_map为Map结构的字段
EXECPTION:
org.apache.kyuubi.KyuubiSQLException: Error operating ExecuteStatement: task panics: Arrow error: Not yet implemented: not yet implemented: Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false) org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 15) (fs-hiido-dn-XX-XX-X.hiido.host.yydevops.com executor 1): java.lang.RuntimeException: task panics: Arrow error: Not yet implemented: not yet implemented: Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false)
经过排查发现 /*+repartition(100) */ 会触发 RoundRobinPartitioning ， blaze 则会先创建一个Sort_exec 节点
在Sort_exec节点的PruneSortKeysFromBatch 的try_new 方法 会进行RowConverter 构建
Codec 的 new 方法不支持Map结构。