diff --git a/docs/lakehouse/catalogs/iceberg-catalog.mdx b/docs/lakehouse/catalogs/iceberg-catalog.mdx index e714a423a1694..9633ccd5d41b0 100644 --- a/docs/lakehouse/catalogs/iceberg-catalog.mdx +++ b/docs/lakehouse/catalogs/iceberg-catalog.mdx @@ -1406,7 +1406,9 @@ For example, to view the table's history, you can execute: SELECT * FROM iceberg_table$history; ``` -> Currently, the `all_manifests` and `position_deletes` system tables are not yet supported and are planned to be supported in future versions. +> The `all_manifests` system table is supported starting from version 4.0.4. +> +> The `position_deletes` system table is not yet supported and is planned to be supported in future versions. ### entries @@ -1511,6 +1513,20 @@ Result: +---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+ ``` +### all_manifests + +> This feature is supported starting from version 4.0.4 + +Shows manifest file info of all valid snapshots of the table: + +`all_manifests` and `manifests` have the same structure. The difference is that `all_manifests` includes manifest files from all valid snapshots, while `manifests` only includes manifest files from the current snapshot. + +```sql +SELECT * FROM iceberg_table$all_manifests; +``` + +The result format is the same as the `manifests` system table. + ### metadata_log_entries Shows meta logs of the table: diff --git a/docs/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/docs/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index b769f76ac9de7..e47108ae3d2df 100644 --- a/docs/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/docs/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -22,10 +22,10 @@ ICEBERG_META( ## Required Parameters Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` pair. -| Field | Description | -|--------------|---------------------------------------------------------------------------------------------------------------------------------------| -| `` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | -| `` | The type of metadata you want to view. Currently, only `snapshots` is supported. | +| Field | Description | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | +| `` | The type of metadata you want to view. Supported types:
`snapshots`: Snapshot information
`manifests`: Manifest files of current snapshot
`all_manifests`: Manifest files of all valid snapshots (supported from version 4.0.4)
`files`: File information of current snapshot
`data_files`: Data files of current snapshot
`delete_files`: Delete files of current snapshot
`partitions`: Partition information
`refs`: Reference information (branches and tags)
`history`: History records
`metadata_log_entries`: Metadata log entries | ## Examples @@ -68,3 +68,61 @@ Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` p | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- View manifests of the iceberg table (manifest files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- View all_manifests of the iceberg table (manifest files of all valid snapshots, supported from version 4.0.4) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- View files of the iceberg table (file information of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- View data_files of the iceberg table (data files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- View delete_files of the iceberg table (delete files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- View partitions of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- View refs of the iceberg table (reference information) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- View history of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- View metadata_log_entries of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## Related + +For more detailed information about Iceberg system tables, please refer to [Iceberg Catalog System Tables](../../../lakehouse/catalogs/iceberg-catalog.mdx#system-tables). diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/iceberg-catalog.mdx b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/iceberg-catalog.mdx index bf2553497154b..7914b97455262 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/iceberg-catalog.mdx +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/iceberg-catalog.mdx @@ -1422,7 +1422,9 @@ SELECT * FROM iceberg_table$system_table_name; SELECT * FROM iceberg_table$history; ``` -> 目前 `all_manifests` 和 `position_deletes` 系统表尚未支持,计划在以后版本中支持。 +> `all_manifests` 系统表从 4.0.4 版本开始支持。 +> +> `position_deletes` 系统表尚未支持,计划在以后版本中支持。 ### entries @@ -1526,6 +1528,20 @@ SELECT * FROM iceberg_table$manifests; +---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+ ``` +### all_manifests + +> 该功能自 4.0.4 版本支持 + +显示表的所有有效快照的 manifest 文件信息: + +`all_manifests` 和 `manifests` 的结构相同,区别在于 `all_manifests` 包含了所有有效快照的 manifest 文件,而 `manifests` 只包含当前快照的 manifest 文件。 + +```sql +SELECT * FROM iceberg_table$all_manifests; +``` + +结果格式与 `manifests` 系统表相同。 + ### metadata_log_entries 显示表的元数据日志条目: diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index f91a2ca44a609..0177a414991d9 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -21,10 +21,10 @@ ICEBERG_META( ## 必填参数 iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 -| Field | Description | -|--------------|-----------------------------------------------------------------------------| -| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | -| `` | 元数据类型,目前仅支持 `snapshots`。 | +| Field | Description | +|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | +| `` | 元数据类型,支持以下类型:
`snapshots`:快照信息
`manifests`:当前快照的清单文件
`all_manifests`:所有有效快照的清单文件(从 4.0.4 版本开始支持)
`files`:当前快照的文件信息
`data_files`:当前快照的数据文件
`delete_files`:当前快照的删除文件
`partitions`:分区信息
`refs`:引用信息(分支和标签)
`history`:历史记录
`metadata_log_entries`:元数据日志条目 | ## 示例(Examples) @@ -68,3 +68,61 @@ iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- 查看 iceberg 表的 manifests(当前快照的清单文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- 查看 iceberg 表的 all_manifests(所有有效快照的清单文件,从 4.0.4 版本开始支持) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- 查看 iceberg 表的 files(当前快照的文件信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- 查看 iceberg 表的 data_files(当前快照的数据文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- 查看 iceberg 表的 delete_files(当前快照的删除文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- 查看 iceberg 表的 partitions(分区信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- 查看 iceberg 表的 refs(引用信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- 查看 iceberg 表的 history(历史记录) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- 查看 iceberg 表的 metadata_log_entries(元数据日志条目) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## 相关说明 + +更多关于 Iceberg 系统表的详细信息,请参阅 [Iceberg Catalog 系统表](../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index f91a2ca44a609..0177a414991d9 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -21,10 +21,10 @@ ICEBERG_META( ## 必填参数 iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 -| Field | Description | -|--------------|-----------------------------------------------------------------------------| -| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | -| `` | 元数据类型,目前仅支持 `snapshots`。 | +| Field | Description | +|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | +| `` | 元数据类型,支持以下类型:
`snapshots`:快照信息
`manifests`:当前快照的清单文件
`all_manifests`:所有有效快照的清单文件(从 4.0.4 版本开始支持)
`files`:当前快照的文件信息
`data_files`:当前快照的数据文件
`delete_files`:当前快照的删除文件
`partitions`:分区信息
`refs`:引用信息(分支和标签)
`history`:历史记录
`metadata_log_entries`:元数据日志条目 | ## 示例(Examples) @@ -68,3 +68,61 @@ iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- 查看 iceberg 表的 manifests(当前快照的清单文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- 查看 iceberg 表的 all_manifests(所有有效快照的清单文件,从 4.0.4 版本开始支持) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- 查看 iceberg 表的 files(当前快照的文件信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- 查看 iceberg 表的 data_files(当前快照的数据文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- 查看 iceberg 表的 delete_files(当前快照的删除文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- 查看 iceberg 表的 partitions(分区信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- 查看 iceberg 表的 refs(引用信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- 查看 iceberg 表的 history(历史记录) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- 查看 iceberg 表的 metadata_log_entries(元数据日志条目) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## 相关说明 + +更多关于 Iceberg 系统表的详细信息,请参阅 [Iceberg Catalog 系统表](../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx index bf2553497154b..7914b97455262 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx @@ -1422,7 +1422,9 @@ SELECT * FROM iceberg_table$system_table_name; SELECT * FROM iceberg_table$history; ``` -> 目前 `all_manifests` 和 `position_deletes` 系统表尚未支持,计划在以后版本中支持。 +> `all_manifests` 系统表从 4.0.4 版本开始支持。 +> +> `position_deletes` 系统表尚未支持,计划在以后版本中支持。 ### entries @@ -1526,6 +1528,20 @@ SELECT * FROM iceberg_table$manifests; +---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+ ``` +### all_manifests + +> 该功能自 4.0.4 版本支持 + +显示表的所有有效快照的 manifest 文件信息: + +`all_manifests` 和 `manifests` 的结构相同,区别在于 `all_manifests` 包含了所有有效快照的 manifest 文件,而 `manifests` 只包含当前快照的 manifest 文件。 + +```sql +SELECT * FROM iceberg_table$all_manifests; +``` + +结果格式与 `manifests` 系统表相同。 + ### metadata_log_entries 显示表的元数据日志条目: diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index f91a2ca44a609..0177a414991d9 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -21,10 +21,10 @@ ICEBERG_META( ## 必填参数 iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 -| Field | Description | -|--------------|-----------------------------------------------------------------------------| -| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | -| `` | 元数据类型,目前仅支持 `snapshots`。 | +| Field | Description | +|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 | +| `` | 元数据类型,支持以下类型:
`snapshots`:快照信息
`manifests`:当前快照的清单文件
`all_manifests`:所有有效快照的清单文件(从 4.0.4 版本开始支持)
`files`:当前快照的文件信息
`data_files`:当前快照的数据文件
`delete_files`:当前快照的删除文件
`partitions`:分区信息
`refs`:引用信息(分支和标签)
`history`:历史记录
`metadata_log_entries`:元数据日志条目 | ## 示例(Examples) @@ -68,3 +68,61 @@ iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对 | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- 查看 iceberg 表的 manifests(当前快照的清单文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- 查看 iceberg 表的 all_manifests(所有有效快照的清单文件,从 4.0.4 版本开始支持) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- 查看 iceberg 表的 files(当前快照的文件信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- 查看 iceberg 表的 data_files(当前快照的数据文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- 查看 iceberg 表的 delete_files(当前快照的删除文件) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- 查看 iceberg 表的 partitions(分区信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- 查看 iceberg 表的 refs(引用信息) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- 查看 iceberg 表的 history(历史记录) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- 查看 iceberg 表的 metadata_log_entries(元数据日志条目) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## 相关说明 + +更多关于 Iceberg 系统表的详细信息,请参阅 [Iceberg Catalog 系统表](../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)。 diff --git a/versioned_docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/versioned_docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index b769f76ac9de7..e47108ae3d2df 100644 --- a/versioned_docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/versioned_docs/version-3.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -22,10 +22,10 @@ ICEBERG_META( ## Required Parameters Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` pair. -| Field | Description | -|--------------|---------------------------------------------------------------------------------------------------------------------------------------| -| `
` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | -| `` | The type of metadata you want to view. Currently, only `snapshots` is supported. | +| Field | Description | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | +| `` | The type of metadata you want to view. Supported types:
`snapshots`: Snapshot information
`manifests`: Manifest files of current snapshot
`all_manifests`: Manifest files of all valid snapshots (supported from version 4.0.4)
`files`: File information of current snapshot
`data_files`: Data files of current snapshot
`delete_files`: Delete files of current snapshot
`partitions`: Partition information
`refs`: Reference information (branches and tags)
`history`: History records
`metadata_log_entries`: Metadata log entries | ## Examples @@ -68,3 +68,61 @@ Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` p | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- View manifests of the iceberg table (manifest files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- View all_manifests of the iceberg table (manifest files of all valid snapshots, supported from version 4.0.4) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- View files of the iceberg table (file information of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- View data_files of the iceberg table (data files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- View delete_files of the iceberg table (delete files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- View partitions of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- View refs of the iceberg table (reference information) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- View history of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- View metadata_log_entries of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## Related + +For more detailed information about Iceberg system tables, please refer to [Iceberg Catalog System Tables](../../../lakehouse/catalogs/iceberg-catalog.mdx#system-tables). diff --git a/versioned_docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx b/versioned_docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx index e714a423a1694..9633ccd5d41b0 100644 --- a/versioned_docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx +++ b/versioned_docs/version-4.x/lakehouse/catalogs/iceberg-catalog.mdx @@ -1406,7 +1406,9 @@ For example, to view the table's history, you can execute: SELECT * FROM iceberg_table$history; ``` -> Currently, the `all_manifests` and `position_deletes` system tables are not yet supported and are planned to be supported in future versions. +> The `all_manifests` system table is supported starting from version 4.0.4. +> +> The `position_deletes` system table is not yet supported and is planned to be supported in future versions. ### entries @@ -1511,6 +1513,20 @@ Result: +---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+ ``` +### all_manifests + +> This feature is supported starting from version 4.0.4 + +Shows manifest file info of all valid snapshots of the table: + +`all_manifests` and `manifests` have the same structure. The difference is that `all_manifests` includes manifest files from all valid snapshots, while `manifests` only includes manifest files from the current snapshot. + +```sql +SELECT * FROM iceberg_table$all_manifests; +``` + +The result format is the same as the `manifests` system table. + ### metadata_log_entries Shows meta logs of the table: diff --git a/versioned_docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md b/versioned_docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md index b769f76ac9de7..e47108ae3d2df 100644 --- a/versioned_docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md +++ b/versioned_docs/version-4.x/sql-manual/sql-functions/table-valued-functions/iceberg-meta.md @@ -22,10 +22,10 @@ ICEBERG_META( ## Required Parameters Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` pair. -| Field | Description | -|--------------|---------------------------------------------------------------------------------------------------------------------------------------| -| `
` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | -| `` | The type of metadata you want to view. Currently, only `snapshots` is supported. | +| Field | Description | +|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `
` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. | +| `` | The type of metadata you want to view. Supported types:
`snapshots`: Snapshot information
`manifests`: Manifest files of current snapshot
`all_manifests`: Manifest files of all valid snapshots (supported from version 4.0.4)
`files`: File information of current snapshot
`data_files`: Data files of current snapshot
`delete_files`: Delete files of current snapshot
`partitions`: Partition information
`refs`: Reference information (branches and tags)
`history`: History records
`metadata_log_entries`: Metadata log entries | ## Examples @@ -68,3 +68,61 @@ Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` p | 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} | +------------------------+----------------+---------------+-----------+-------------------+------------------------------+ ``` + +- View manifests of the iceberg table (manifest files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests"); + ``` + +- View all_manifests of the iceberg table (manifest files of all valid snapshots, supported from version 4.0.4) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests"); + ``` + +- View files of the iceberg table (file information of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files"); + ``` + +- View data_files of the iceberg table (data files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files"); + ``` + +- View delete_files of the iceberg table (delete files of current snapshot) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files"); + ``` + +- View partitions of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions"); + ``` + +- View refs of the iceberg table (reference information) + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs"); + ``` + +- View history of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history"); + ``` + +- View metadata_log_entries of the iceberg table + + ```sql + select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries"); + ``` + +## Related + +For more detailed information about Iceberg system tables, please refer to [Iceberg Catalog System Tables](../../../lakehouse/catalogs/iceberg-catalog.mdx#system-tables).