Introduce KubeAI for OPEA

[poussa]

Description

KubeAI is an open-source AI inferencing operator. This PR contains documentation, installation instructions and deployment files for running KubeAI with OPEA inference services.

For now, OPEA enables a subset of the KubeAI features. In the future more KubeAI service will be added.

Features

The following features are enabled at the moment.

• Installing KubeAI using Helm and using OPEA configuration.
• OPEA configuration includes OPEA images for vLLM CPU and Gaudi, Persistent Volume Claims (PVC) for model caching and resource profile for Gaudi.
• Text generation model (llama-3.1-8b) for vLLM (CPU and Gaudi) using PVC
• Text embedding model (BAII/BGE) for vLLM (CPU) using PVC
• Text generation model (qwen-2.5-0.5b for OLlama (CPU)

Issues

OIM Operator RFC

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)

Dependencies

n/a

Tests

Manually tested on CPU and Gaudi clusters.

[eero-t]

Otherwise looks fine, but I wonder whether this should go under helm-charts/ instead?

[poussa]

Otherwise looks fine, but I wonder whether this should go under helm-charts/ instead?

I don't think so. This is not a Helm chart but a solution which uses external Helm chart. Those are two different things, and would not be good to put them into same place.

[mkbhanda]

Otherwise looks fine, but I wonder whether this should go under helm-charts/ instead?

I don't think so. This is not a Helm chart but a solution which uses external Helm chart. Those are two different things, and would not be good to put them into same place.

I concur with @poussa

[poussa]

KubeAI requires resources.limits so putting it back.

[eero-t]

Approved.

PS. before force-pushing updates, please pull the pre-ci commit first. This avoids re-introducing trailing white-space + table alignment issues, that pre-ci needs to fix again.

[eero-t]

@chensuyue What's this toctree thing that CI complains about / why linking new README into an already existing README is not enough?
/home/runner/work/GenAIInfra/GenAIInfra/docs/_build/rst/GenAIInfra/kubeai/README.md: WARNING: document isn't included in any toctree

[poussa]

@chensuyue What's this toctree thing that CI complains about / why linking new README into an already existing README is not enough? /home/runner/work/GenAIInfra/GenAIInfra/docs/_build/rst/GenAIInfra/kubeai/README.md: WARNING: document isn't included in any toctree

I need to add entry to the docs index.rst file first to make infra ci/cd happy. I'll do that next.

[mkbhanda]

Minor typo, can approve