Skip to content

Custom endpoints: per-model routing with APIM prefix preservation #1518

Open

Description

@Bionic711

Summary

Preserve APIM/gateway API path prefixes and support explicit per-model routing under one Custom endpoint. Track the staged implementation in the approved Custom Endpoints: Per-Model Routing and GenAI.mil specification.

Current Problem

With provider set to Azure OpenAI, selecting a Claude model can invoke the legacy Messages URL fallback. A URL such as https://gateway.example/aoai-global-team is reduced to https://gateway.example/anthropic/v1/messages, losing the configured gateway prefix. Endpoint-wide Custom API type and normalization also cannot represent mixed protocols and independent paths/versions under one endpoint.

Proposed Behavior

  • Insert the model API path immediately after the origin and before the endpoint's existing path.
  • Require explicit per-model Custom API type and Auto/Exact URL Handling. Keep protocol selection independent of model vendor/name.
  • Distinguish Azure deployment APIs from Azure/Foundry Chat Completions; reuse the OpenAI-compatible wire adapter for the latter.
  • Preserve model-local identifiers/versions, stable IDs, credential references, scope authorization, and protected outbound transports.
  • Provide conservative dry-run migration and guarded rollback rather than guessing at ambiguous legacy URLs or changing excluded consumers.
  • Add global/personal/group editor controls and shared resolution for model tests, model chat, and audited non-agent helpers.
  • Implement the separately staged, bounded GenAI.mil Chat Completions integration.

Phase Status (2026-09-22)

  • Phase 0: source selection and baseline repair complete locally in 0.261.040.
  • Phase 1: backend contract, pure resolver, model-local normalization/validation, migration planner/rollback, scoped context and cache/capability contracts complete locally in 0.261.041.
  • Phase 2: endpoint editors, model-library integration, previews, round trips, and browser coverage.
  • Phase 3: actual model-test/chat/non-agent helper dispatch integration and captured-request parity.
  • Phase 4: bounded GenAI.mil profile, discovery, payload/error handling, and mocked validation.
  • Phase 5: remaining release, browser, security, documentation, and rollout gates.

The checked phases describe local, uncommitted work, not released functionality. Draft PR #1514 currently contains the original specification; the Phase 0/1 implementation has not been pushed. Existing UI and legacy runtime dispatch remain unchanged. The APIM path regression is covered by the explicit resolver, but its live consumer integration remains Phase 3.

Acceptance Criteria

  • Mixed Custom models preserve independent protocol, identifier, path, URL Handling, and version choices through save/edit/export/import.
  • Gateway prefixes survive resolution and actual outbound requests; native Foundry and legacy Exact/Gemini behavior retain parity.
  • Test-model, chat, and in-scope helpers use the same authorized saved-state routing contract and reject stale/removed/disabled selections.
  • Migration is idempotent and reversible; stable IDs/secrets survive; ambiguous records and unsafe downgrades are rejected.
  • New routing does not weaken scope, SSRF/DNS, redirect, TLS/CA/mTLS, auth, identity-header, or Key Vault protections.
  • Unsupported capabilities/types and arbitrary request-template/state fields are rejected. Agents, Responses, and custom calls remain out of scope.
  • Complete specification acceptance matrix CE-01 through CE-13 with phase-appropriate regression/browser coverage and explicit live-validation limits.

Validation Completed

Phase 1: 21/21 guarded baseline scripts, four forward/reverse cold-import checks under normal/optimized Python, and 13/13 documentation checks passed. Focused editor/syntax diagnostics and existing-call signature checks passed. Tests blocked environment-file reads and external provider access; the docs inventory check allowed only its inspected read-only local generator command.

No full repository suite, browser suite, live authorization/provider validation, saved configuration migration, cloud deployment, or GenAI.mil call has been performed. Phase 1's agreed exit gates are satisfied; Phase 2 is the next implementation phase, not started by creating this issue.

References and Triage

Activity

  1. self-assigned this
    on Sep 22, 2026
  2. moved this from Pending Evaluation to In progress in Simple Chat Roadmapon Sep 22, 2026
  3. Bionic711 commented on Sep 25, 2026

    @Bionic711
    CollaboratorAuthor

    New-instance multi-endpoint default (0.261.043)

    • New app settings documents default enable_multi_model_endpoints to true.
    • Existing saved values remain unchanged; older documents missing the setting retain the historical disabled behavior.
    • Updated the admin reference and release notes. The existing one-way enablement guard is unchanged.
    • Validation: 34 relevant settings regression tests passed; documentation quality checks passed 6/6.
    • No Azure deployment was performed. The implementation remains local and uncommitted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions