Popular repositories Loading

1. genai-bench genai-bench Public

   Forked from sgl-project/genai-bench

   Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

   Python

2. ome ome Public

   Forked from ome-projects/ome

   OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)

   Go

3. oci-openai-java oci-openai-java Public

   Forked from oracle/oci-openai-java

   OCI OpenAI Client for Java simplifies integration between OpenAI’s Java SDK and Oracle Cloud Infrastructure (OCI) GenAI service by providing robust authentication and authorization utilities.

   Java

4. oci-openai oci-openai Public

   Forked from oracle-samples/oci-openai

   The OCI-OpenAI simplifies integration between OpenAI’s Python SDK and Oracle Cloud Infrastructure (OCI) GenAI service by providing robust authentication and authorization utilities.

   Python

5. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

6. smg smg Public

   Forked from lightseekorg/smg

   Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat histor…

   Rust