Skip to content

fix(pi): exclude extension-only models from text generation pickers - #33337

Closed
StiensWout wants to merge 1 commit into
pingdotgg:mainfrom
StiensWout:t3/pi-text-generation-compat
Closed

StiensWout wants to merge 1 commit into
pingdotgg:mainfrom
StiensWout:t3/pi-text-generation-compat

Conversation

@StiensWout

Copy link
Copy Markdown
Contributor

Pi discovery includes extension and virtual models, but text generation intentionally runs with --no-extensions. The text generation pickers offered models that could never load, so selecting one left titles as the raw prompt.

Probe the model inventory with text generation's own launch and mark incompatible models. Existing text generation pickers and their cross-environment checks exclude those models after custom options are reconstructed. Normal turn pickers keep them. Built-in, models.json, default and compatible custom choices retain the existing picker behavior.

Verification

  • Deterministic red/green tests cover extension exclusion, custom aliases, compatible and stale selections, failed probes and a hanging probe using Deferred/TestClock. Focused status, text generation and selector tests: 72 passed.
  • Mutation proof: removing exclusion fails the status case and seven selector rows; inverting unknown-availability fallback fails the failed and hung probe cases.
  • pnpm exec vp run check:changed passes format, lint, knip, affected-package typechecks, focused tests and React compiler checks.
  • Real Pi 1.5.0 with isolated configuration, no paid turn: normal inventory lists local/qwen-coder and virtual auto/gpt-6-luna; --no-extensions --no-tools lists only local/qwen-coder. Public status marks the virtual model incompatible, and selectors exclude it only for text generation, including a duplicate custom entry.
  • Integrated isolated client: the same Pi instance lists the virtual model in the normal turn picker and omits it in the text generation picker.
  • Independent proof audit: PASS, no blocking gaps. Local cubic: no P0/P1; one P2 and two P3 findings, with comment wording corrected and follow-ups below.

Scope and limits

Standard change: model capability contract plus discovery lifecycle and existing selector consumers. No new UI flow or copy. The compatibility probe adds one short-lived Pi process per refresh, runs concurrently, and is capped at 10 seconds. A failed or timed-out probe preserves unknown availability and may still offer extension models.

Existing saved settings are not migrated: a previously saved extension-only selection can still fail server-side until changed or reset. The server reads the persisted setting even when the picker displays its normal fallback. Follow-up: validate per-model supportsTextGeneration at TextGeneration.generate before dispatch, independently of this picker fix.

No macOS or Windows runtime verification, paid generation, or live credentialed built-in model run; built-in compatibility is covered by the fake-CLI table. Probe caching, extra logging, a shared purpose alias and fallback-helper extraction were declined as unnecessary to this fix.

Council review

Rounds: 2 code reviews, then a scope-only recheck on the unchanged second-round head.

Delegate Round 1 Round 2 Scope recheck
GPT 6.1 Sol 8/10 9/10 Not rerun, same head
Claude Opus 5.5 8/10 8/10 9/10
Grok 4.7 9/10 8/10 9/10

Round 1's cross-environment selector finding was fixed. The later objection concerned saved-setting server fallback, accepted as the explicit follow-up above. Final in-scope verdict: all three reviewers 9/10, no blocker or should open.

Created with GPT 6.1 Sol (lead, fixes, review and audit), Claude Opus 5.5 (build, trim and review), and Grok 4.7 (review and evidence) in T3 Code using the Codex, Claude and Grok harnesses.

Closes #30360

Discover text generation compatibility with the same extension-free launch used for generation. Preserve normal turn models and filter known-incompatible choices after custom options are reconstructed, including scoped settings checks.

Co-authored-by: GPT 6.1 Sol <codex@openai.com>

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Co-authored-by: Grok 4.7 <noreply@x.ai>
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Nov 9, 2026
@StiensWout

Copy link
Copy Markdown
Contributor Author

Note

🤖 GPT 6.1 Sol responding on behalf of StiensWout

The saved-setting mismatch is part of this fix’s behavior, not an unrelated migration: the picker now displays a compatible fallback for a persisted extension-only selection, while apps/server/src/textGeneration/TextGeneration.ts can still dispatch the incompatible saved model. This leaves the original failed-title behavior behind a misleading selection. Before marking this PR ready, generation dispatch and the displayed compatible selection need to agree, with focused proof for an existing incompatible saved selection. No broad migration is required by this request. Collect this with the hosted-review round for one fix push.

@macroscopeapp

macroscopeapp Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — The change excludes extension-only Pi models from text-generation pickers while preserving normal turn choices. It also changes the effective automatic fallback when a default model is incompatible, requiring confirmation of that default-selection behavior.

You can add or adjust custom eligibility rules. Learn more.

Copy link
Copy Markdown
Member

Thanks, @StiensWout, for the fix and detailed verification! Closing this as a duplicate of #30426 by @juliusmarminge, which addresses the same extension-only Pi models in text-generation pickers. Please share any additional findings on that PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pi text generation picker offers extension-only models (provider packages, virtual models), so titles silently keep the raw prompt

2 participants