[Platform] Add ModelFailoverPlatform bridge by tony-stark-eth · Pull Request #1864 · symfony/ai

tony-stark-eth · 2026-04-05T10:48:08Z

Q	A
Bug fix?	no
New feature?	yes
Docs?	no
Issues	–
License	MIT

Summary

FailoverPlatform chains multiple platforms — when one provider fails, it tries the next provider. But there's no native support for chaining multiple models on a single platform.

ModelFailoverPlatform fills this gap: a PlatformInterface decorator that tries multiple models in sequence on the same underlying platform. When the requested model fails, it tries each fallback model in order.

Use cases:

OpenRouter free models (openrouter/free → minimax:free → glm:free)
Cost optimization (try gpt-4o-mini before gpt-4o on the same OpenAI platform)
Gradual migration (try new model, fall back to old if it fails)

Usage

// Bundle configuration
symfony_ai:
    platform:
        model_failover:
            openrouter_resilient:
                platform: ai.platform.openrouter
                models:
                    - 'minimax/minimax-m2.5:free'
                    - 'z-ai/glm-4.5-air:free'
                    - 'openai/gpt-oss-120b:free'

// Programmatic usage
$platform = ModelFailoverPlatformFactory::create(
    platform: $openRouterPlatform,
    models: ['minimax:free', 'glm:free', 'qwen:free'],
    logger: $logger,
);

// Tries 'gpt-4o' first, then each fallback model
$result = $platform->invoke('gpt-4o', $input);

Composes naturally with FailoverPlatform:

$platform = new FailoverPlatform([
    ModelFailoverPlatformFactory::create($openRouter, ['model-a', 'model-b']),
    ModelFailoverPlatformFactory::create($ollama, ['llama3', 'mistral']),
], $rateLimiter);

What's included

ModelFailoverPlatform — PlatformInterface decorator (same patterns as FailoverPlatform)
ModelFailoverPlatformFactory — static factory
Bundle config (model_failover platform type)
Deptrac layer + ruleset
splitsh.json entry
7 tests covering: construction validation, success path, single fallback, all-fail, deduplication, catalog delegation, options passthrough

Relationship to FailoverPlatform

	FailoverPlatform	ModelFailoverPlatform
Chains	Platforms	Models
Use case	Provider redundancy	Model redundancy
Dependencies	Rate limiter, clock	Logger only
Scope	Multiple providers	Single provider

Add a new bridge that complements FailoverPlatform (which chains platforms) by chaining models on a single platform. When the requested model fails, it tries each fallback model in order on the same underlying platform. This is useful when a provider offers multiple models and some may be temporarily unavailable or rate-limited (e.g. OpenRouter free models, or trying GPT-4o before falling back to GPT-4o-mini on the same OpenAI platform). The bridge includes: - ModelFailoverPlatform (PlatformInterface decorator) - ModelFailoverPlatformFactory - Bundle configuration (model_failover platform type) - Deptrac layer definition - Comprehensive test suite (7 tests)

Guikingone · 2026-04-05T15:12:08Z

Could we work toward a fusion between this PR and #1808?

tony-stark-eth · 2026-04-05T17:22:35Z

Could we work toward a fusion between this PR and #1808?

I think they solve two different use cases, no?

Guikingone · 2026-04-05T17:42:44Z

From my POV, you're trying to solve an issue on FailoverPlatform by introducing a new one that loops over models, I bet we can work on bringing the loop in the FailoverPlatform without creating a new platform, especially regarding the fact that my PR is already defining a model key per platform, adding support for an array of models is not a lot of work and could prevent repeating the "failover mechanism" on two platforms 🙂

tony-stark-eth · 2026-04-07T14:36:58Z

From my POV, you're trying to solve an issue on FailoverPlatform by introducing a new one that loops over models, I bet we can work on bringing the loop in the FailoverPlatform without creating a new platform, especially regarding the fact that my PR is already defining a model key per platform, adding support for an array of models is not a lot of work and could prevent repeating the "failover mechanism" on two platforms 🙂

I made it to draft for now as I'm still refining the functionality for my project, generally speaking I like your approach more but for my usecase this seems the way to go.

Guikingone · 2026-04-08T08:12:57Z

I pushed a new commit that contains the model override on the current implementation: da951e5, not sure if it will lend on main but I used your approach and improve the current one to solve the "model cascade".

…ting layer (chr-hertel) This PR was squashed before being merged into the main branch. Discussion ---------- [Platform] Introduce Provider abstraction and model routing layer | Q | A | ------------- | --- | Bug fix? | no | New feature? | yes | Docs? | no | Issues | | License | MIT ## Summary This PR introduces a **Provider** abstraction layer between `PlatformInterface` and the actual inference backends, along with a **ModelRouter** for routing. This is the foundational first step toward solving several long-standing architectural pain points. ### The Problem The Platform component has a fundamental 1:1 coupling: one `Platform` instance = one inference backend. This causes: - **No unified entry point for multi-provider setups** — users needing OpenAI + Anthropic must manage two separate `Platform` instances - **Static ModelCatalogs are a maintenance burden** (#1692) — 38+ hardcoded catalogs need constant updates as models change faster than library releases - **Model-level failover/routing has no natural home** — `FailoverPlatform` only fails over between whole platforms; #1864 wants model-level failover, #1808 wants model override in failover - **Input-based model routing belongs at Platform level** — #1090 puts it at Agent level, but it's a Platform concern - **Connector concept** (#646) — old sketch to introduce a layer below Platform for connection concerns never materialized ### The Solution ``` User Code | PlatformInterface (UNCHANGED) | Platform (refactored: router over providers) |-- ModelRouterInterface (routes model name → provider) |-- CompositeModelCatalog (merges provider catalogs) |-- ModelRoutingEvent (pre-routing hook, can set provider to skip router) | ProviderInterface (NEW) | Provider (extracted from current Platform logic) |-- ModelClientInterface[] / ResultConverterInterface[] / Contract / ModelCatalog ``` **`ProviderInterface`** encapsulates what `PlatformFactory::create()` currently produces — a complete, self-contained connection to a single inference backend (model clients, result converters, contract, catalog). **`Platform`** becomes a thin routing layer over multiple providers, using a `ModelRouterInterface` to determine which provider handles each request. **`ModelRoutingEvent`** is dispatched before routing, allowing listeners to modify the model/input/options or set a provider directly to skip the router entirely. **`PlatformInterface`** remains completely unchanged — all downstream code (Agent, Chat, Store, user code) continues to work without modification. ### What's Included **Core layer:** - `ProviderInterface` + `Provider` (extracted from current Platform logic) - `ModelRouterInterface` + `CatalogBasedModelRouter` (default: routes by catalog lookup) - `CompositeModelCatalog` (merges catalogs from multiple providers) - `ModelRoutingEvent` (Platform-level event with optional provider to skip routing) - Refactored `Platform` as router over providers **Reference bridge implementation:** - `OpenAi\ProviderFactory` (replaces `OpenAi\PlatformFactory`) **30 new tests**, all 510 tests pass (including existing). ### User-Facing DX ```php // Single provider $provider = OpenAi\ProviderFactory::create(apiKey: 'sk-...'); $platform = new Platform([$provider]); // Multi-provider with auto-routing $openai = OpenAi\ProviderFactory::create(apiKey: 'sk-...'); $anthropic = Anthropic\ProviderFactory::create(apiKey: 'sk-...'); $platform = new Platform([$openai, $anthropic]); $platform->invoke('gpt-4o', $messages); // → OpenAI $platform->invoke('claude-3-5-sonnet', $messages); // → Anthropic ``` ### How This Addresses Existing Work | PR/Issue | How it's addressed | |----------|--------------------| | #646 (Connectors) | `Provider` is the connector concept — encapsulates connection concerns | | #1692 (ModelCatalog burden) | `CompositeModelCatalog` + dynamic catalogs become first-class; static catalogs per bridge become optional | | #1753 (BC for model IDs) | Model ID resolution is now a pluggable strategy, not hardcoded | | #1808 (Model override in failover) | Enables `FailoverModelRouter` with model name translation (follow-up) | | #1864 (ModelFailoverPlatform) | Enables model-level failover natively via router (follow-up) | | #1090 (Model Router) | Enables input-based routing at Platform level via `CallableModelRouter` (follow-up) | ## Use-Case: Load Balancing The `ModelRouterInterface` enables load balancing between providers. For example, round-robin between two OpenAI providers with different API keys: ```php $openai1 = OpenAi\ProviderFactory::create(apiKey: 'sk-key1'); $openai2 = OpenAi\ProviderFactory::create(apiKey: 'sk-key2'); $platform = new Platform( [$openai1, $openai2], new RoundRobinModelRouter(), ); $platform->invoke('gpt-4o', $messages); // → openai1 $platform->invoke('gpt-4o', $messages); // → openai2 $platform->invoke('gpt-4o', $messages); // → openai1 ``` The router implementation: ```php final class RoundRobinModelRouter implements ModelRouterInterface { private int $counter = 0; public function resolve(string $model, iterable $providers, array|string|object $input, array $options = []): ProviderInterface { $candidates = []; foreach ($providers as $provider) { if ($provider->supports($model)) { $candidates[] = $provider; } } if ([] === $candidates) { throw new ModelNotFoundException(\sprintf('No provider found for model "%s".', $model)); } return $candidates[$this->counter++ % \count($candidates)]; } } ``` The same pattern works for weighted or random distribution — just a different `ModelRouterInterface` implementation. Composes with `ChainModelRouter` too (e.g. explicit routes for some models, load-balanced for the rest). ## Use-Case: Failover / Rate Limit Handling Failover is a provider-level concern, implemented as a `ProviderInterface` decorator — not a router concern (since routing happens before invocation, it can't react to runtime failures). **Cross-provider failover** (OpenAI rate-limited → fall back to Anthropic): ```php $failover = new FailoverProvider('main', [ OpenAi\ProviderFactory::create(apiKey: 'sk-...'), Anthropic\ProviderFactory::create(apiKey: 'sk-...'), ], $rateLimiterFactory); $platform = new Platform([$failover]); ``` **Nested composition** (load-balanced OpenAI pool with Anthropic as final fallback): ```php $openaiPool = new FailoverProvider('openai', [$openai1, $openai2], $rateLimiterFactory); $main = new FailoverProvider('main', [$openaiPool, $anthropic], $rateLimiterFactory); $platform = new Platform([$main]); ``` This subsumes the existing `FailoverPlatform` entirely — same pattern, but as a `ProviderInterface` decorator instead of a `PlatformInterface` decorator. More composable since it nests with other providers and works seamlessly with the routing layer. ## Use-Case: ModelsDev as Universal Catalog The Provider/ModelRouter rework shifts the role of the ModelsDev bridge. Its core value becomes clear — it's a **universal, dynamic model catalog** backed by the `symfony/models-dev` database. No hardcoded model lists, no API calls. New models appear with `composer update symfony/models-dev`. Routing and provider construction move to where they belong: `Platform` and bridge-specific `ProviderFactory` classes. ```php use Symfony\AI\Platform\Bridge\ModelsDev\ModelCatalog; use Symfony\AI\Platform\Bridge\OpenAi\ProviderFactory as OpenAiProviderFactory; use Symfony\AI\Platform\Bridge\Anthropic\ProviderFactory as AnthropicProviderFactory; use Symfony\AI\Platform\Platform; // Each bridge handles its own connection concerns (API keys, HTTP, contracts) // ModelsDev provides the catalog — always up-to-date, no static model lists $openai = OpenAiProviderFactory::create( apiKey: $_ENV['OPENAI_API_KEY'], modelCatalog: new ModelCatalog('openai'), ); $anthropic = AnthropicProviderFactory::create( apiKey: $_ENV['ANTHROPIC_API_KEY'], modelCatalog: new ModelCatalog('anthropic'), ); // Platform routes automatically — CatalogBasedModelRouter checks each provider's catalog $platform = new Platform([$openai, $anthropic]); $platform->invoke('gpt-4o', $messages); // → openai (catalog knows it) $platform->invoke('claude-3-5-sonnet', $messages); // → anthropic (catalog knows it) ``` All `ProviderFactory` signatures already accept `ModelCatalogInterface` — no new abstraction needed. The static catalogs in each bridge become the fallback default, ModelsDev becomes the dynamic option. **What remains valuable in ModelsDev:** - `ModelCatalog` — the universal catalog (core value) - `CapabilityMapper` — maps models-dev metadata to Capability enum - `DataLoader` — loads/caches the JSON - `ProviderRegistry` — discovery ("what providers exist?") **What shrinks or goes away:** - `PlatformFactory` — Platform + router handles routing natively now - `BridgeResolver` — routing is Platform's job, not the catalog's - `ModelResolver` with `"provider::model"` syntax — catalog-based routing makes this redundant for most cases The ModelsDev bridge becomes a focused, lean package: **a catalog, not a platform**. ### Follow-Up PRs This is scoped to the core layer + one bridge. Planned follow-ups: - Convert remaining 37 bridges from `PlatformFactory` to `ProviderFactory` - Add `ExplicitModelRouter`, `FailoverModelRouter`, `ChainModelRouter`, `CallableModelRouter` - `FailoverProvider` decorator (subsumes `FailoverPlatform`) - AI Bundle integration (DI wiring, routing config section) - Refocus ModelsDev bridge as universal catalog Commits ------- 94168f2 [Platform] Introduce Provider abstraction and model routing layer

tony-stark-eth requested review from OskarStark and chr-hertel as code owners April 5, 2026 10:48

carsonbot added Platform Issues & PRs about the AI Platform component Status: Needs Review labels Apr 5, 2026

tony-stark-eth force-pushed the feat/model-failover-platform branch from 6e00a10 to 01539f3 Compare April 5, 2026 10:53

tony-stark-eth changed the title ~~[Platform] Add ModelFailoverPlatform bridge~~ Add ModelFailoverPlatform bridge Apr 5, 2026

tony-stark-eth changed the title ~~Add ModelFailoverPlatform bridge~~ [Platform] Add ModelFailoverPlatform bridge Apr 5, 2026

tony-stark-eth force-pushed the feat/model-failover-platform branch 2 times, most recently from a48bcec to fba33db Compare April 5, 2026 11:14

tony-stark-eth force-pushed the feat/model-failover-platform branch from fba33db to 8f1ba28 Compare April 5, 2026 11:19

tony-stark-eth marked this pull request as draft April 7, 2026 14:35

chr-hertel mentioned this pull request Apr 10, 2026

[Platform] Introduce Provider abstraction and model routing layer #1923

Merged

chr-hertel mentioned this pull request Apr 17, 2026

[Platform] Add ModelRouter implementations for explicit, failover, chain, and callable routing #1952

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[Platform] Add ModelFailoverPlatform bridge#1864

[Platform] Add ModelFailoverPlatform bridge#1864
tony-stark-eth wants to merge 1 commit into
symfony:mainfrom
tony-stark-eth:feat/model-failover-platform

tony-stark-eth commented Apr 5, 2026 •

edited

Loading

Uh oh!

Guikingone commented Apr 5, 2026

Uh oh!

tony-stark-eth commented Apr 5, 2026

Uh oh!

Guikingone commented Apr 5, 2026

Uh oh!

tony-stark-eth commented Apr 7, 2026

Uh oh!

Guikingone commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

tony-stark-eth commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Usage

What's included

Relationship to FailoverPlatform

Uh oh!

Guikingone commented Apr 5, 2026

Uh oh!

tony-stark-eth commented Apr 5, 2026

Uh oh!

Guikingone commented Apr 5, 2026

Uh oh!

tony-stark-eth commented Apr 7, 2026

Uh oh!

Guikingone commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tony-stark-eth commented Apr 5, 2026 •

edited

Loading