[Agent] Add Model Router system for intelligent model selection#1090
Draft
wachterjohannes wants to merge 1 commit intosymfony:mainfrom
Draft
[Agent] Add Model Router system for intelligent model selection#1090wachterjohannes wants to merge 1 commit intosymfony:mainfrom
wachterjohannes wants to merge 1 commit intosymfony:mainfrom
Conversation
Implements Phase 1 of the Model Router architecture, enabling intelligent model selection based on input characteristics. This allows agents to automatically route requests to appropriate models (e.g., vision models for images, speech models for audio). Core Infrastructure: - RouterInterface: Universal contract for all router types - RoutingResult: Result with model name, optional transformer, reason, and confidence - TransformerInterface: Contract for input transformation - RouterContext: Provides platform access, model catalog, and metadata - SimpleRouter: Callable-based router for flexible routing logic - ChainRouter: Composite router for trying multiple routers in sequence - ModelRouterInputProcessor: Integration with Agent's input processing pipeline Input Enhancement: - Add optional platform property to Input class (backward compatible) - Update Agent to pass platform when creating Input - Platform available to input processors via Input.getPlatform() Includes 6 examples demonstrating different routing patterns and comprehensive test coverage (14 router tests, all 187 agent tests passing). Key Features: - No breaking changes - all changes are backward compatible - Extensible design - custom routers in <20 lines - Multi-provider support - works with any Platform implementation - Transformation support - routers can transform input before invocation - Default model fallback - routers can access agent's default model - Capability-based routing - find models supporting specific capabilities Architecture Decisions: - Single RouterInterface for all router types enables composition - Transformer in RoutingResult (router decides both model and transformation) - Platform passed via Input (no breaking changes to interfaces) - RouterContext provides metadata including default model - Start simple with SimpleRouter, add complexity gradually
Contributor
Author
|
@chr-hertel this is my first protoype of the router - don't look to deep into the code - just the architecture stuff and there will be more as soon as we merged the first part. following things i have planned:
Another time - i have to refactor this code as soon as we are OK with the patterns behind it :) |
chr-hertel
added a commit
that referenced
this pull request
Apr 20, 2026
…ting layer (chr-hertel) This PR was squashed before being merged into the main branch. Discussion ---------- [Platform] Introduce Provider abstraction and model routing layer | Q | A | ------------- | --- | Bug fix? | no | New feature? | yes | Docs? | no | Issues | | License | MIT ## Summary This PR introduces a **Provider** abstraction layer between `PlatformInterface` and the actual inference backends, along with a **ModelRouter** for routing. This is the foundational first step toward solving several long-standing architectural pain points. ### The Problem The Platform component has a fundamental 1:1 coupling: one `Platform` instance = one inference backend. This causes: - **No unified entry point for multi-provider setups** — users needing OpenAI + Anthropic must manage two separate `Platform` instances - **Static ModelCatalogs are a maintenance burden** (#1692) — 38+ hardcoded catalogs need constant updates as models change faster than library releases - **Model-level failover/routing has no natural home** — `FailoverPlatform` only fails over between whole platforms; #1864 wants model-level failover, #1808 wants model override in failover - **Input-based model routing belongs at Platform level** — #1090 puts it at Agent level, but it's a Platform concern - **Connector concept** (#646) — old sketch to introduce a layer below Platform for connection concerns never materialized ### The Solution ``` User Code | PlatformInterface (UNCHANGED) | Platform (refactored: router over providers) |-- ModelRouterInterface (routes model name → provider) |-- CompositeModelCatalog (merges provider catalogs) |-- ModelRoutingEvent (pre-routing hook, can set provider to skip router) | ProviderInterface (NEW) | Provider (extracted from current Platform logic) |-- ModelClientInterface[] / ResultConverterInterface[] / Contract / ModelCatalog ``` **`ProviderInterface`** encapsulates what `PlatformFactory::create()` currently produces — a complete, self-contained connection to a single inference backend (model clients, result converters, contract, catalog). **`Platform`** becomes a thin routing layer over multiple providers, using a `ModelRouterInterface` to determine which provider handles each request. **`ModelRoutingEvent`** is dispatched before routing, allowing listeners to modify the model/input/options or set a provider directly to skip the router entirely. **`PlatformInterface`** remains completely unchanged — all downstream code (Agent, Chat, Store, user code) continues to work without modification. ### What's Included **Core layer:** - `ProviderInterface` + `Provider` (extracted from current Platform logic) - `ModelRouterInterface` + `CatalogBasedModelRouter` (default: routes by catalog lookup) - `CompositeModelCatalog` (merges catalogs from multiple providers) - `ModelRoutingEvent` (Platform-level event with optional provider to skip routing) - Refactored `Platform` as router over providers **Reference bridge implementation:** - `OpenAi\ProviderFactory` (replaces `OpenAi\PlatformFactory`) **30 new tests**, all 510 tests pass (including existing). ### User-Facing DX ```php // Single provider $provider = OpenAi\ProviderFactory::create(apiKey: 'sk-...'); $platform = new Platform([$provider]); // Multi-provider with auto-routing $openai = OpenAi\ProviderFactory::create(apiKey: 'sk-...'); $anthropic = Anthropic\ProviderFactory::create(apiKey: 'sk-...'); $platform = new Platform([$openai, $anthropic]); $platform->invoke('gpt-4o', $messages); // → OpenAI $platform->invoke('claude-3-5-sonnet', $messages); // → Anthropic ``` ### How This Addresses Existing Work | PR/Issue | How it's addressed | |----------|--------------------| | #646 (Connectors) | `Provider` is the connector concept — encapsulates connection concerns | | #1692 (ModelCatalog burden) | `CompositeModelCatalog` + dynamic catalogs become first-class; static catalogs per bridge become optional | | #1753 (BC for model IDs) | Model ID resolution is now a pluggable strategy, not hardcoded | | #1808 (Model override in failover) | Enables `FailoverModelRouter` with model name translation (follow-up) | | #1864 (ModelFailoverPlatform) | Enables model-level failover natively via router (follow-up) | | #1090 (Model Router) | Enables input-based routing at Platform level via `CallableModelRouter` (follow-up) | ## Use-Case: Load Balancing The `ModelRouterInterface` enables load balancing between providers. For example, round-robin between two OpenAI providers with different API keys: ```php $openai1 = OpenAi\ProviderFactory::create(apiKey: 'sk-key1'); $openai2 = OpenAi\ProviderFactory::create(apiKey: 'sk-key2'); $platform = new Platform( [$openai1, $openai2], new RoundRobinModelRouter(), ); $platform->invoke('gpt-4o', $messages); // → openai1 $platform->invoke('gpt-4o', $messages); // → openai2 $platform->invoke('gpt-4o', $messages); // → openai1 ``` The router implementation: ```php final class RoundRobinModelRouter implements ModelRouterInterface { private int $counter = 0; public function resolve(string $model, iterable $providers, array|string|object $input, array $options = []): ProviderInterface { $candidates = []; foreach ($providers as $provider) { if ($provider->supports($model)) { $candidates[] = $provider; } } if ([] === $candidates) { throw new ModelNotFoundException(\sprintf('No provider found for model "%s".', $model)); } return $candidates[$this->counter++ % \count($candidates)]; } } ``` The same pattern works for weighted or random distribution — just a different `ModelRouterInterface` implementation. Composes with `ChainModelRouter` too (e.g. explicit routes for some models, load-balanced for the rest). ## Use-Case: Failover / Rate Limit Handling Failover is a provider-level concern, implemented as a `ProviderInterface` decorator — not a router concern (since routing happens before invocation, it can't react to runtime failures). **Cross-provider failover** (OpenAI rate-limited → fall back to Anthropic): ```php $failover = new FailoverProvider('main', [ OpenAi\ProviderFactory::create(apiKey: 'sk-...'), Anthropic\ProviderFactory::create(apiKey: 'sk-...'), ], $rateLimiterFactory); $platform = new Platform([$failover]); ``` **Nested composition** (load-balanced OpenAI pool with Anthropic as final fallback): ```php $openaiPool = new FailoverProvider('openai', [$openai1, $openai2], $rateLimiterFactory); $main = new FailoverProvider('main', [$openaiPool, $anthropic], $rateLimiterFactory); $platform = new Platform([$main]); ``` This subsumes the existing `FailoverPlatform` entirely — same pattern, but as a `ProviderInterface` decorator instead of a `PlatformInterface` decorator. More composable since it nests with other providers and works seamlessly with the routing layer. ## Use-Case: ModelsDev as Universal Catalog The Provider/ModelRouter rework shifts the role of the ModelsDev bridge. Its core value becomes clear — it's a **universal, dynamic model catalog** backed by the `symfony/models-dev` database. No hardcoded model lists, no API calls. New models appear with `composer update symfony/models-dev`. Routing and provider construction move to where they belong: `Platform` and bridge-specific `ProviderFactory` classes. ```php use Symfony\AI\Platform\Bridge\ModelsDev\ModelCatalog; use Symfony\AI\Platform\Bridge\OpenAi\ProviderFactory as OpenAiProviderFactory; use Symfony\AI\Platform\Bridge\Anthropic\ProviderFactory as AnthropicProviderFactory; use Symfony\AI\Platform\Platform; // Each bridge handles its own connection concerns (API keys, HTTP, contracts) // ModelsDev provides the catalog — always up-to-date, no static model lists $openai = OpenAiProviderFactory::create( apiKey: $_ENV['OPENAI_API_KEY'], modelCatalog: new ModelCatalog('openai'), ); $anthropic = AnthropicProviderFactory::create( apiKey: $_ENV['ANTHROPIC_API_KEY'], modelCatalog: new ModelCatalog('anthropic'), ); // Platform routes automatically — CatalogBasedModelRouter checks each provider's catalog $platform = new Platform([$openai, $anthropic]); $platform->invoke('gpt-4o', $messages); // → openai (catalog knows it) $platform->invoke('claude-3-5-sonnet', $messages); // → anthropic (catalog knows it) ``` All `ProviderFactory` signatures already accept `ModelCatalogInterface` — no new abstraction needed. The static catalogs in each bridge become the fallback default, ModelsDev becomes the dynamic option. **What remains valuable in ModelsDev:** - `ModelCatalog` — the universal catalog (core value) - `CapabilityMapper` — maps models-dev metadata to Capability enum - `DataLoader` — loads/caches the JSON - `ProviderRegistry` — discovery ("what providers exist?") **What shrinks or goes away:** - `PlatformFactory` — Platform + router handles routing natively now - `BridgeResolver` — routing is Platform's job, not the catalog's - `ModelResolver` with `"provider::model"` syntax — catalog-based routing makes this redundant for most cases The ModelsDev bridge becomes a focused, lean package: **a catalog, not a platform**. ### Follow-Up PRs This is scoped to the core layer + one bridge. Planned follow-ups: - Convert remaining 37 bridges from `PlatformFactory` to `ProviderFactory` - Add `ExplicitModelRouter`, `FailoverModelRouter`, `ChainModelRouter`, `CallableModelRouter` - `FailoverProvider` decorator (subsumes `FailoverPlatform`) - AI Bundle integration (DI wiring, routing config section) - Refocus ModelsDev bridge as universal catalog Commits ------- 94168f2 [Platform] Introduce Provider abstraction and model routing layer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the Model Router architecture, enabling intelligent model selection based on input characteristics. This allows agents to automatically route requests to appropriate models (e.g., vision models for images, speech models for audio).
Core Infrastructure:
Input Enhancement:
Includes 6 examples demonstrating different routing patterns and comprehensive test coverage (14 router tests, all 187 agent tests passing).
Key Features:
Architecture Decisions: