Hi aws-samples maintainers & community, thank you for the examples, they help a lot.
I found a small issue in one of the examples, particularly amazon-kinesis-data-analytics-java-examples/Beam. Actually, the code is fine, but correspondent documentation is not.
Page https://docs.aws.amazon.com/kinesisanalytics/latest/java/examples-beam.html#examples-beam-resources recommends to:
- use Apache Flink 1.11 cluster version
- and build fat jar using this command:
mvn package -Dflink.version=1.11.1 -Dflink.version.minor=1.8
If you build the example, upload .jar file and try to run your Kinesis Data Analytics app you will get this error:
Type 'org/apache/flink/streaming/api/transformations/TwoInputTransformation' (current frame, stack[4]) is not assignable to 'org/apache/flink/streaming/api/transformations/StreamTransformation'
Flink documentation (https://beam.apache.org/documentation/runners/flink/#flink-version-compatibility) states though:
The Flink cluster version has to match the minor version used by the FlinkRunner. The minor version is the first two numbers in the version string, e.g. in 1.8.0 the minor version is 1.8.
Thus flink.version.minor must be 1.11, and not 1.8. Full set of POM properties that work for me with the latest version in master branch:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.11</java.version>
<scala.binary.version>2.12</scala.binary.version>
<kda-runtime.version>1.2.0</kda-runtime.version>
<beam.version>2.28.0</beam.version>
<jackson.version>2.10.2</jackson.version>
<flink.version>1.11.3</flink.version>
<flink.version.minor>1.11</flink.version.minor>
</properties>
Hi aws-samples maintainers & community, thank you for the examples, they help a lot.
I found a small issue in one of the examples, particularly amazon-kinesis-data-analytics-java-examples/Beam. Actually, the code is fine, but correspondent documentation is not.
Page https://docs.aws.amazon.com/kinesisanalytics/latest/java/examples-beam.html#examples-beam-resources recommends to:
mvn package -Dflink.version=1.11.1 -Dflink.version.minor=1.8If you build the example, upload .jar file and try to run your Kinesis Data Analytics app you will get this error:
Flink documentation (https://beam.apache.org/documentation/runners/flink/#flink-version-compatibility) states though:
Thus
flink.version.minormust be 1.11, and not 1.8. Full set of POM properties that work for me with the latest version in master branch: