Provide a example for integrating Aibrix with Envoy AI Gateway (using Gateway API + InferencePool)

[googs1025]

🚀 Feature Description and Motivation

I’m currently trying to integrate a mock Aibrix-deployed model (e.g., llama2-7b) with Envoy AI Gateway using the Kubernetes Gateway API and the InferencePool / AIGatewayRoute pattern (similar to the official envoyproxy/ai-gateway ai-gateway).

Use Case

We want to expose multiple Aibrix-managed LLMs (e.g., llama2-7b, qwen-14b) through a single, secure, OpenAI-compatible API endpoint powered by Envoy AI Gateway and gateway-api-inference-extension

Proposed Solution

• Helm does not require the gateway to be installed; the gateway and ai-gateway are installed externally.
• add example and README.md

[googs1025]

cc @Jeffwan

[googs1025]

FYI: https://gateway-api-inference-extension.sigs.k8s.io/concepts/api-overview/

FYI: https://aigateway.envoyproxy.io/docs/capabilities/inference/inferencepool-support

[googs1025]

Currently, it seems that supporting an EPP for the separation of StormService (PD) in Aibrix is ​​still under discussion. llm-d/llm-d-inference-scheduler#430 Direct integration with llm-d-inference-scheduler would be ideal. Alternatively, would it be necessary to implement an Aibrix EPP ourselves?

[googs1025]

Currently, it seems that supporting an EPP for the separation of StormService (PD) in Aibrix is ​​still under discussion. llm-d/llm-d-inference-scheduler#430 Direct integration with llm-d-inference-scheduler would be ideal. Alternatively, would it be necessary to implement an Aibrix EPP ourselves?

done