Currently ingestion uses Apache Beam V2.5.0 which is compatible with Flink runner 1.4.x.
This version of flink doesn't have REST API to access metrics, e.g: /jobs/<jobid>/metrics which is added in Flink 1.5.x.
In order to use it we have to upgrade ingestion's BEAM SDK to either 2.8.0, 2.7.0 or 2.6.0
Currently ingestion uses Apache Beam V2.5.0 which is compatible with Flink runner 1.4.x.
This version of flink doesn't have REST API to access metrics, e.g:
/jobs/<jobid>/metricswhich is added in Flink 1.5.x.In order to use it we have to upgrade ingestion's BEAM SDK to either 2.8.0, 2.7.0 or 2.6.0