Feedback on the benchmark

[vd-rd]

Great work on this benchmark and thanks for taking the time to do this. It paints a clear image and explains a lot of things, so kudos for that and I'm looking forward to the updated version.

Two things i want to mention:

• i see you are contributing to istio, and i would suggest a disclaimer can help anybody looking not get a wrong picture of bias or something
• i see a few of the implementations rely on envoy proxy, including istio, so i am curious how istio avoids the issues that envoy has, if you know about that

Once again, great work

[howardjohn]

Hey @vd-rd , thanks for the issue! I do have a disclaimer in the first paragraph of https://github.com/howardjohn/gateway-api-bench.

Of the 4 Envoy-based implementations tested, I would say Istio and Kgateway are fairly similar in architecture, and Envoy Gateway and Cilium are both quite different.

If you look at the envoy configuration generated by EG compared to Istio, you'll see its totally different. So to achieve the same/similar behavior, EG takes very different approach that has different tradeoffs in terms of behavior (like the errors on route changes), scale, etc. The control plane architectures are also very different. The control plane architecture of Istio (and kgateway) follows is built around a library (https://blog.howardjohn.info/posts/krt-recap/) designed for high scale control planes. Istio has been very optimized over 8 years; because it is typically deployed as a service mesh (which has 10000s of Envoy instance), high scale is a hard requirement that may be less important for gateways (which maybe serve only 1-10 instances), which has forced it to be pretty fine tuned. I don't know much about the internals of EG control plane so I cannot speak much to it.

FWIW, neither Istio nor Envoy Gateway are an optimal approach to a Scalable gateway API control plane IMO. I posted a part 2 that showcases a new implementation that is 40-100x better than these in some tests (https://github.com/howardjohn/gateway-api-bench/blob/main/README-v2.md) building on the philosophy based I discussed in https://blog.howardjohn.info/posts/scaling-control-plane/

[vd-rd]

Thanks for the reply, what you pointed out makes sense, i will check out the resources. One more suggestion on my side would be to activate the discussions feature of the repo, as i think some of the topics are not really issues (like this one).
Once again, great work, I'll stay tuned for future versions.