Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion docs/lakehouse/catalogs/iceberg-catalog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1406,7 +1406,9 @@ For example, to view the table's history, you can execute:
SELECT * FROM iceberg_table$history;
```

> Currently, the `all_manifests` and `position_deletes` system tables are not yet supported and are planned to be supported in future versions.
> The `all_manifests` system table is supported starting from version 4.0.4.
>
> The `position_deletes` system table is not yet supported and is planned to be supported in future versions.

### entries

Expand Down Expand Up @@ -1511,6 +1513,20 @@ Result:
+---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+
```

### all_manifests

> This feature is supported starting from version 4.0.4

Shows manifest file info of all valid snapshots of the table:

`all_manifests` and `manifests` have the same structure. The difference is that `all_manifests` includes manifest files from all valid snapshots, while `manifests` only includes manifest files from the current snapshot.

```sql
SELECT * FROM iceberg_table$all_manifests;
```

The result format is the same as the `manifests` system table.

### metadata_log_entries

Shows meta logs of the table:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ ICEBERG_META(
## Required Parameters
Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` pair.

| Field | Description |
|--------------|---------------------------------------------------------------------------------------------------------------------------------------|
| `<table>` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. |
| `<query_type>` | The type of metadata you want to view. Currently, only `snapshots` is supported. |
| Field | Description |
|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `<table>` | The full table name, which must be specified in the format of `database_name.table_name` for the Iceberg table that you want to view. |
| `<query_type>` | The type of metadata you want to view. Supported types:<br/>`snapshots`: Snapshot information<br/>`manifests`: Manifest files of current snapshot<br/>`all_manifests`: Manifest files of all valid snapshots (supported from version 4.0.4)<br/>`files`: File information of current snapshot<br/>`data_files`: Data files of current snapshot<br/>`delete_files`: Delete files of current snapshot<br/>`partitions`: Partition information<br/>`refs`: Reference information (branches and tags)<br/>`history`: History records<br/>`metadata_log_entries`: Metadata log entries |


## Examples
Expand Down Expand Up @@ -68,3 +68,61 @@ Each parameter in the `iceberg_meta` table function (tvf) is a `"key"="value"` p
| 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} |
+------------------------+----------------+---------------+-----------+-------------------+------------------------------+
```

- View manifests of the iceberg table (manifest files of current snapshot)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests");
```

- View all_manifests of the iceberg table (manifest files of all valid snapshots, supported from version 4.0.4)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests");
```

- View files of the iceberg table (file information of current snapshot)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files");
```

- View data_files of the iceberg table (data files of current snapshot)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files");
```

- View delete_files of the iceberg table (delete files of current snapshot)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files");
```

- View partitions of the iceberg table

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions");
```

- View refs of the iceberg table (reference information)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs");
```

- View history of the iceberg table

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history");
```

- View metadata_log_entries of the iceberg table

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries");
```

## Related

For more detailed information about Iceberg system tables, please refer to [Iceberg Catalog System Tables](../../../lakehouse/catalogs/iceberg-catalog.mdx#system-tables).
Original file line number Diff line number Diff line change
Expand Up @@ -1422,7 +1422,9 @@ SELECT * FROM iceberg_table$system_table_name;
SELECT * FROM iceberg_table$history;
```

> 目前 `all_manifests` 和 `position_deletes` 系统表尚未支持,计划在以后版本中支持。
> `all_manifests` 系统表从 4.0.4 版本开始支持。
>
> `position_deletes` 系统表尚未支持,计划在以后版本中支持。

### entries

Expand Down Expand Up @@ -1526,6 +1528,20 @@ SELECT * FROM iceberg_table$manifests;
+---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+
```

### all_manifests

> 该功能自 4.0.4 版本支持

显示表的所有有效快照的 manifest 文件信息:

`all_manifests` 和 `manifests` 的结构相同,区别在于 `all_manifests` 包含了所有有效快照的 manifest 文件,而 `manifests` 只包含当前快照的 manifest 文件。

```sql
SELECT * FROM iceberg_table$all_manifests;
```

结果格式与 `manifests` 系统表相同。

### metadata_log_entries

显示表的元数据日志条目:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ ICEBERG_META(
## 必填参数
iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对

| Field | Description |
|--------------|-----------------------------------------------------------------------------|
| `<table>` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 |
| `<query_type>` | 元数据类型,目前仅支持 `snapshots` |
| Field | Description |
|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `<table>` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 |
| `<query_type>` | 元数据类型,支持以下类型:<br/>`snapshots`:快照信息<br/>`manifests`:当前快照的清单文件<br/>`all_manifests`:所有有效快照的清单文件(从 4.0.4 版本开始支持)<br/>`files`:当前快照的文件信息<br/>`data_files`:当前快照的数据文件<br/>`delete_files`:当前快照的删除文件<br/>`partitions`:分区信息<br/>`refs`:引用信息(分支和标签)<br/>`history`:历史记录<br/>`metadata_log_entries`:元数据日志条目 |


## 示例(Examples)
Expand Down Expand Up @@ -68,3 +68,61 @@ iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对
| 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} |
+------------------------+----------------+---------------+-----------+-------------------+------------------------------+
```

- 查看 iceberg 表的 manifests(当前快照的清单文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests");
```

- 查看 iceberg 表的 all_manifests(所有有效快照的清单文件,从 4.0.4 版本开始支持)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests");
```

- 查看 iceberg 表的 files(当前快照的文件信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files");
```

- 查看 iceberg 表的 data_files(当前快照的数据文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files");
```

- 查看 iceberg 表的 delete_files(当前快照的删除文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files");
```

- 查看 iceberg 表的 partitions(分区信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions");
```

- 查看 iceberg 表的 refs(引用信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs");
```

- 查看 iceberg 表的 history(历史记录)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history");
```

- 查看 iceberg 表的 metadata_log_entries(元数据日志条目)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries");
```

## 相关说明

更多关于 Iceberg 系统表的详细信息,请参阅 [Iceberg Catalog 系统表](../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)。
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ ICEBERG_META(
## 必填参数
iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对

| Field | Description |
|--------------|-----------------------------------------------------------------------------|
| `<table>` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 |
| `<query_type>` | 元数据类型,目前仅支持 `snapshots` |
| Field | Description |
|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `<table>` | 完整的表名,需要按照目录名。库名.表名的格式,填写需要查看的 iceberg 表名。 |
| `<query_type>` | 元数据类型,支持以下类型:<br/>`snapshots`:快照信息<br/>`manifests`:当前快照的清单文件<br/>`all_manifests`:所有有效快照的清单文件(从 4.0.4 版本开始支持)<br/>`files`:当前快照的文件信息<br/>`data_files`:当前快照的数据文件<br/>`delete_files`:当前快照的删除文件<br/>`partitions`:分区信息<br/>`refs`:引用信息(分支和标签)<br/>`history`:历史记录<br/>`metadata_log_entries`:元数据日志条目 |


## 示例(Examples)
Expand Down Expand Up @@ -68,3 +68,61 @@ iceberg_meta 表函数 tvf 中的每一个参数都是一个 `"key"="value"` 对
| 2022-09-21 10:36:35 | 98865735822 | 64123452344 | overwrite | hdfs:/path/to/m2 | {"flink.job-id":"xxm2", ...} |
+------------------------+----------------+---------------+-----------+-------------------+------------------------------+
```

- 查看 iceberg 表的 manifests(当前快照的清单文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "manifests");
```

- 查看 iceberg 表的 all_manifests(所有有效快照的清单文件,从 4.0.4 版本开始支持)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "all_manifests");
```

- 查看 iceberg 表的 files(当前快照的文件信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "files");
```

- 查看 iceberg 表的 data_files(当前快照的数据文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "data_files");
```

- 查看 iceberg 表的 delete_files(当前快照的删除文件)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "delete_files");
```

- 查看 iceberg 表的 partitions(分区信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "partitions");
```

- 查看 iceberg 表的 refs(引用信息)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "refs");
```

- 查看 iceberg 表的 history(历史记录)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "history");
```

- 查看 iceberg 表的 metadata_log_entries(元数据日志条目)

```sql
select * from iceberg_meta("table" = "iceberg_ctl.test_db.test_tbl", "query_type" = "metadata_log_entries");
```

## 相关说明

更多关于 Iceberg 系统表的详细信息,请参阅 [Iceberg Catalog 系统表](../../../lakehouse/catalogs/iceberg-catalog.mdx#系统表)。
Original file line number Diff line number Diff line change
Expand Up @@ -1422,7 +1422,9 @@ SELECT * FROM iceberg_table$system_table_name;
SELECT * FROM iceberg_table$history;
```

> 目前 `all_manifests` 和 `position_deletes` 系统表尚未支持,计划在以后版本中支持。
> `all_manifests` 系统表从 4.0.4 版本开始支持。
>
> `position_deletes` 系统表尚未支持,计划在以后版本中支持。

### entries

Expand Down Expand Up @@ -1526,6 +1528,20 @@ SELECT * FROM iceberg_table$manifests;
+---------+------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------------------+---------------------+------------------------+---------------------------+--------------------------+--------------------------+-----------------------------+----------------------------+--------------------------------------------------------------------------------+
```

### all_manifests

> 该功能自 4.0.4 版本支持

显示表的所有有效快照的 manifest 文件信息:

`all_manifests` 和 `manifests` 的结构相同,区别在于 `all_manifests` 包含了所有有效快照的 manifest 文件,而 `manifests` 只包含当前快照的 manifest 文件。

```sql
SELECT * FROM iceberg_table$all_manifests;
```

结果格式与 `manifests` 系统表相同。

### metadata_log_entries

显示表的元数据日志条目:
Expand Down
Loading