Skip to content

Introduce turn-level orchestration across chat capabilities #1021

Open

Description

@paullizer

Summary

Introduce a turn-level orchestration layer that coordinates SimpleChat's selected and available chat capabilities, gathers relevant evidence, executes bounded multi-step plans, and synthesizes one coherent final response. Grounded image generation will be the first end-to-end proving scenario, not the core abstraction.

User Value

Today, chat capabilities can execute along separate paths without reliably influencing the final response. Users need selected agents, documents, search modes, prior citations, conversation context, actions, and generated artifacts to participate in one intentional workflow that completes the requested outcome without unsupported claims.

Proposed Behavior

  • Snapshot the user message, selected chat controls, relevant compacted history, prior citation/artifact lineage, authorization context, and available governed capabilities for each submitted turn.
  • Create a lightweight plan for every turn; simple requests use a direct one-step plan without an extra planning model call.
  • Treat selected capabilities and sources as required attempts unless they are impossible or unauthorized, and report skips or failures.
  • Allow governed read-only capabilities to run automatically when needed; require approval for side effects, sensitive/consequential access, or budget overflow.
  • Execute bounded plans with status, dependencies, cancellation, partial failure, and dynamic read-only replanning within the original goal and budget.
  • Normalize results into a provenance-aware ledger that preserves conflicts, missing evidence, citations, and source lineage.
  • Use a central finalizer to combine executor outputs into the final answer or artifact plan.
  • Show concise progress and persist compact plan, status, provenance, and evidence metadata.

Acceptance Criteria

  • Every submitted turn has an immutable orchestration context and either a direct or coordinated plan.
  • Selected controls are represented as required attempts for that turn, while authorized read-only discovery follows governance and budget policy.
  • Relevant conversation history, prior citations, cited source documents, selected documents, agents/actions, search modes, and other chat capabilities can contribute to execution and final synthesis.
  • Plans support dependencies, bounded adaptive replanning, cancellation, explicit failure states, and clearly partial outcomes.
  • A central finalizer receives normalized outputs and provenance from all executed steps.
  • Conflicting or missing evidence is surfaced rather than silently merged or invented.
  • Risky, consequential, write, or over-budget steps require user approval.
  • Grounded image generation passes as the first end-to-end orchestration scenario without making the core contracts image-specific.
  • Functional and security tests cover direct, selected-source, auto-discovery, partial-failure, authorization, and synthesis behavior.

Notes

The implementation should be phased and reuse the existing chat route, agents/actions, workspace and selected-document search, web/deep research, source review, conversation citations, progress UI, artifact persistence, and cancellation mechanisms. Early planning should be deterministic where possible and avoid adding latency to simple turns.

Activity

  1. moved this from Pending Evaluation to In progress in Simple Chat Roadmapon Jul 14, 2026
  2. paullizer commented on Jul 14, 2026

    @paullizer
    CollaboratorAuthor

    Phase 1 implementation checkpoint

    Work has started on feature/orchestration-foundation-phase-1, based directly on Development.

    Completed

    • Reframed orchestration as a generic per-turn chat coordinator; grounded image generation is the first proving profile, not the core architecture.
    • Added deterministic one-step direct plans and coordinated plans without adding a planning-model call to simple turns.
    • Added compact submitted-control snapshots for selected agents, document actions, documents/scopes, retrieval modes, conversation-linked documents, assigned knowledge, model/reasoning selection, and prompt identity.
    • Added explicit evidence requirements, origin-aware required source attempts, dependency-aware action/finalizer steps, and policy defaults for governed read-only discovery, bounded replanning, central finalization, and approval classes.
    • Persisted orchestration metadata on standard streaming and Analyze/Compare user/assistant messages, including cancellation and partial-error terminal metadata.
    • Aligned image proposal guidance with the orchestration image-intent detector and added prompt-level evidence-before-proposal guidance for grounded image requests.
    • Added feature documentation and a 0.250.058 release note.

    Validation

    • Orchestration functional suite: 14/14 passed.
    • Image proposal pipeline: 3/3 passed.
    • Streaming SSE metadata: 3/3 passed.
    • Route policy contracts: 12/12 passed.
    • Python compilation and editor diagnostics: clean.
    • Scoped broken-access-control scan: passed for all modified backend modules.
    • git diff --check: clean.

    Known next phases

    • Normalize capability outputs into the shared result/evidence ledger.
    • Adapt history, prior citations/cited documents, workspace, web/research, agents, and actions to the executor contract.
    • Move selected agents beneath a true central finalizer instead of allowing them to own coordinated response streaming.
    • Add generic scheduling, per-step reconciliation, automatic governed capability discovery, retry/cancellation, bounded replanning, budget enforcement, and durable resume.

    No commit or pull request has been created yet.

  3. paullizer commented on Jul 14, 2026

    @paullizer
    CollaboratorAuthor

    Final Phase 1 review hardening

    An independent review and payload audit were completed after the initial checkpoint. The branch now also:

    • Reuses one image-intent detector for orchestration and production simpleimage proposal guidance, while keeping chart/analytical requests out of image generation.
    • Separates inferred evidence requirements from deduplicated source attempts so selected Web Search/documents can satisfy public-web/workspace requirements without duplicate root steps.
    • Makes selected document actions depend on their agent/document/conversation/assigned-knowledge inputs.
    • Uses nested document_action.document_ids as the authoritative submitted snapshot when present.
    • Returns orchestration metadata consistently in cancellation and partial-error terminal events.
    • Creates an evidence-discovery planning step when grounding is requested without a named source.
    • Documents that the current live chat payload has no dedicated selected-image-reference list; selected workspace images arrive as documents and explicit reference intent is currently inferred from the message.

    Final focused validation is now 15/15 orchestration scenarios plus 3/3 image proposal scenarios. Compilation, editor diagnostics, route policy contracts, streaming SSE metadata, scoped broken-access-control scanning, and git diff --check are clean.

  4. paullizer commented on Jul 14, 2026

    @paullizer
    CollaboratorAuthor

    Phase 2 implementation checkpoint

    Phase 2 is implemented in PR #1023, targeting feature/orchestration.

    Completed

    • Added a generic, output-neutral result/evidence ledger for requirements, permission-aware sources, supported and unsupported facts, computed outputs, citations, artifacts, conflicts, missing evidence, and execution failures.
    • Initialized each turn ledger from the immutable Phase 1 plan, including equivalent source/requirement linkage and unspecified-grounding evidence discovery.
    • Added provenance validation, denied-source protections, explicit requirement/source state reconciliation, and safe convenience APIs for normalized records.
    • Added bounded valid-JSON compaction that cannot retain dangling provenance references and excludes raw payloads, binary data, credential-like metadata, and signed URL query strings/fragments.
    • Persisted one plan-linked ledger across standard streaming user/success/cancel/partial-error metadata and Analyze/Compare user/success metadata.
    • Updated orchestration documentation, release notes, and version 0.250.059.

    Validation

    • Evidence ledger: 9/9 passed.
    • Phase 1 orchestration plan: 15/15 passed.
    • Image proposal pipeline: 3/3 passed.
    • Streaming SSE metadata: 3/3 passed.
    • Route policy contracts: 12/12 passed.
    • Python compilation, VS Code diagnostics, Broken Access Control scanning, XSS scanning, and staged diff checks are clean.

    Phase boundary

    Phase 2 establishes and persists the contract. Existing conversation, document, workspace, web/research, source-review, selected-image, agent, and action outputs remain on their current paths until the Phase 3 and Phase 4 adapters populate the ledger. Finalizer prompt injection waits for populated normalized evidence so an empty planned ledger cannot compete with existing context.

    Refs #1023

  5. paullizer commented on Jul 14, 2026

    @paullizer
    CollaboratorAuthor

    Phase 3 is implemented in PR #1028: #1028

    What changed:

    • Added generic collectors for authorized conversation facts, prior citation/artifact lineage, selected and conversation-upload documents, workspace search, Web Search, Source Review/Deep Research, and selected-image vision metadata.
    • Populated the shared evidence ledger before streaming finalization with explicit no-result, failure, skipped, partial, and unauthorized states.
    • Revalidated selected-document access at the collector boundary and omitted prompt-injection-flagged page excerpts from supported facts.
    • Updated version and release documentation to 0.250.060.

    Validation:

    • Phase 3 collectors: 15/15
    • Evidence ledger: 9/9
    • Turn planning: 15/15
    • Image proposal: 3/3
    • Streaming metadata: 3/3
    • Route policy: 12/12
    • Python compile, BAC/XSS guardrails, and whitespace checks passed.

    Phase 4 remains responsible for normalizing selected agent and action executor outputs beneath the central finalizer.

  6. paullizer commented on Jul 14, 2026

    @paullizer
    CollaboratorAuthor

    Phase 4 implementation checkpoint

    Phase 4 is implemented in PR #1030: #1030

    Completed

    • Added a connector-neutral evidence_collection task and response contract for selected agents and document actions.
    • Passed unresolved evidence requirements, optional capability tags/evidence types/permission metadata, delegated source attempts, and a structured output schema to executors.
    • Normalized current-turn tool/action facts, computed results, citations, artifacts, not_found, not_available, failure, and authorization states into the shared ledger.
    • Bound private evidence collection to authenticated request context, used per-turn invocation baselines, redacted credential-bearing output, and kept unsupported executor claims out of supported facts.
    • Buffered grounded-image selected-agent and document-action output so executors cannot emit a final simpleimage proposal before evidence collection completes.
    • Updated orchestration documentation, release notes, and version 0.250.061.

    Validation

    • Phase 4 contract: 10/10
    • Source collectors: 15/15
    • Evidence ledger: 9/9
    • Turn planning: 15/15
    • Image proposal: 3/3
    • Streaming metadata: 3/3
    • Tabular document actions: 3/3
    • Route policy: 12/12
    • Python compilation, editor diagnostics, full-file BAC/XSS scans, and staged whitespace checks passed.

    Phase boundary

    Phase 4 ends with normalized executor evidence and a deterministic status handoff. Phase 5 remains responsible for the separate central synthesis call that turns the completed ledger into evidence-backed image proposals and later output types.

    Refs #1021

  7. paullizer commented on Jul 15, 2026

    @paullizer
    CollaboratorAuthor

    Phase 5 central synthesis is implemented in #1032.

    What changed:

    • Added a generic, output-neutral central synthesis contract over the completed compact evidence ledger.
    • Proved the contract with grounded image proposals for model-only retrieval and selected-agent evidence collection.
    • Added explicit missing/conflict handling, unsupported-fact omission, fail-closed readiness checks, synthesis lifecycle metadata, and approval-time validation of evidence/reference-image IDs against the authorized source message ledger.
    • Updated browser proposal metadata handling, feature documentation, release notes, and version 0.250.062.

    Validation:

    • Orchestration contracts: 58/58 passed.
    • Image proposal, streaming metadata, tabular action, and collaboration compatibility: 12/12 passed.
    • Route policy contracts: 12/12 passed.
    • Python/JavaScript syntax, editor diagnostics, full-file BAC/XSS scans, and diff checks passed.
    • Desktop/mobile Playwright coverage collected but skipped locally because SIMPLECHAT_PLAYWRIGHT_CHAT_URL is not configured.

    Remaining scope: Analyze/Compare central-finalizer integration and resilient scheduling/retry/cancellation/durable resume remain later-phase work.

  8. paullizer commented on Jul 15, 2026

    @paullizer
    CollaboratorAuthor

    Phase 6 implementation checkpoint

    Phase 6 is implemented in PR #1034: #1034

    Completed

    • Added a request-scoped, in-process OrchestrationRun execution graph shared by direct and coordinated turns.
    • Added DAG validation, terminal node lifecycle, required/optional failure policy, finalizer gating, bounded read-only retries, opt-in safe parallelism, and bounded read-only replanning.
    • Reconciled existing authorized conversation, document, image, workspace, web, source-review, selected-agent, and document-action outputs into the runtime without moving Flask-scoped authorization work into worker threads.
    • Added runtime-node evidence provenance, concise progress events, Application Insights lifecycle telemetry, selected-agent required-attempt enforcement, fail-closed evidence discovery, cancellation, and terminal cleanup.
    • Persisted compact runtime metadata on standard streaming and Analyze/Compare user/assistant paths.
    • Updated orchestration documentation, release notes, and version 0.250.063.

    Validation

    • Phase 1-6 orchestration and compatibility checks: 84/84 passed.
    • Stream lifecycle and cancellation checks: 5/5 passed.
    • Route policy contracts: 12/12 passed.
    • Python compilation and VS Code diagnostics: clean.
    • Full-file Broken Access Control and XSS scans: passed.
    • Whitespace validation: clean.

    Remaining boundaries

    • Runtime is in-process and request-scoped; durable queueing and process-loss resume remain later phases.
    • Synchronous SDK cancellation is best effort, while pending finalization is blocked and late results are discarded where possible.
    • Phase 7 still owns evidence review, approval, intervention, and expanded progress UX.
    • Non-grounded selected-agent and Analyze/Compare paths retain compatibility finalizers until central synthesis expands to those output profiles.

    Refs #1021

  9. paullizer commented on Jul 15, 2026

    @paullizer
    CollaboratorAuthor

    Phase 7 implementation checkpoint

    Phase 7 is implemented in PR #1035: #1035

    Completed

    • Added ordered, monotonic orchestration progress for planning, selected images/documents, workspace and web evidence, agents/actions, source review, finalization, and image approval handoff.
    • Added accessible evidence review on grounded-image proposal cards with canonical source badges, explicit missing evidence, proposal-linked authorized reference previews, expandable details, and responsive desktop/mobile controls.
    • Added ready, confirmation-required, and blocked approval states. Terminal partial evidence requires an explicit acknowledgment; active, cancelled, failed, or supportless evidence remains blocked.
    • Revalidated personal conversation ownership and the exact source assistant message at generation time, constrained forged and cross-turn evidence/reference IDs, and preserved legacy no-ledger proposal compatibility.
    • Updated orchestration documentation, release notes, and version 0.250.064.

    Validation

    • Phase 1-7 orchestration and approval contracts: 85/85 passed.
    • Stream lifecycle, compatibility, collaboration, tabular, and route-policy contracts: 28/28 passed.
    • Live desktop/mobile Playwright coverage: 4/4 passed.
    • Python compilation, JavaScript syntax, editor diagnostics, whitespace, Broken Access Control, and XSS checks passed.

    Remaining boundaries

    • Approval review currently proves the grounded-image profile; generalized consequential/write approval remains later scope.
    • Runtime persistence and process-loss resume remain Phase 8 work.
    • Non-grounded selected-agent and Analyze/Compare paths retain their current compatibility finalizers.

    Refs #1021

  10. paullizer commented on Jul 15, 2026

    @paullizer
    CollaboratorAuthor

    Phase 8A capability discovery planning checkpoint

    The private orchestration roadmap now includes a Phase 8A plan for governed capability discovery and turn-scoped user choice.

    Diagnosis

    • The current planner primarily receives capabilities already selected in the chat payload. It does not receive a complete server-authoritative inventory of capabilities that are enabled, authorized, and available but unselected.
    • As a result, a current local-law request can remain a direct model-only answer even when Web Search or Deep Research could materially improve it.
    • The existing Phase 7 approval path is image-proposal specific; it is not yet a generalized pause, choice, and resume mechanism.

    Agreed capability semantics

    • selected: required attempt for the submitted turn.
    • unselected: still discoverable and eligible for recommendation or policy-governed read-only use.
    • approved: planner-proposed capability accepted for this turn.
    • declined: prohibited for this turn and not proposed repeatedly.
    • unavailable, unauthorized, or policy_blocked: never offered or executed.

    An inactive toolbar control is therefore not equivalent to a user denial.

    Phase 8A scope

    • Build a server-authoritative, governance-filtered capability inventory.
    • Use one generic contract across Workspace Search, Analyze, Compare, Image, Web Search, URL Access, and Deep Research.
    • Preserve original selection, planner proposal, user decision, and effective execution as separate provenance.
    • Present one structured choice when an unselected capability could materially improve freshness, completeness, confidence, or output quality and policy requires consent.
    • Persist an awaiting_user_choice checkpoint, end the current stream, and resume through an authenticated, idempotent decision request.
    • Reauthorize conversation access and capability availability before execution; reject forged, stale, expired, cross-turn, foreign-conversation, or newly unauthorized choices.
    • Minimize external queries. General research should omit unnecessary addresses, personal identifiers, conversation history, and workspace content; sensitive lookup inputs require explicit approval.
    • Keep simple timeless questions on the direct path without an extra planning-model call.

    For a current Virginia/Fairfax property-rules request, the intended UX is one choice: faster official Web Search, recommended Deep Research, or continue with a clearly limited model-only answer.

    Phase boundary

    Phase 8A proves discovery and choice with built-in chat tools first. Phase 8B will add governed recommendation of unselected agents on the same inventory, proposal, decision, authorization, and resume contracts. Raw agent instructions, secrets, and inaccessible catalog entries must not enter planner context.

    Planning validation

    • Required capability-state and security concepts are present.
    • All private roadmap links resolve.
    • No application code was changed for this planning checkpoint.

    Refs #1021

  11. paullizer commented on Jul 15, 2026

    @paullizer
    CollaboratorAuthor

    Source-status guidance follow-up

    PR #1036 implements the post-Phase 7 response cleanup: #1036

    • Suppresses success-only source-status notes in coordinated final responses.
    • Preserves disclosure for skipped, unavailable, unauthorized, failed, empty, and partial required sources.
    • Adds regression assertions and updates the application version to 0.250.065.

    Validation:

    • Turn orchestration plan tests: 15/15 passed.
    • Python compilation, VS Code diagnostics, and whitespace checks passed.

    Refs #1021

  12. 8 remaining items

  13. paullizer commented on Jul 16, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10A hardening complete: live planner contract aligned

    Controlled shadow evaluation exposed a production schema/validator mismatch before Phase 10B activation. The provider schema described requirement, candidate, capability, and evidence IDs too generically, while the server validator correctly required bounded aliases and exact authorized inventory values.

    Hardening implementation

    • Local branch: feature/orchestration-phase-10a-planner-contract-hardening
    • Baseline: 86378b8a83cda63f7e6c3be0bb09eb4099048df7
    • Application version: 0.250.071
    • Associated fix: docs/explanation/fixes/PHASE_10A_PLANNER_SCHEMA_ALIGNMENT_FIX.md

    Changes:

    • One request-specific schema now drives both the prompt and provider response_format.
    • Requirement/candidate aliases, eligible unselected capability IDs, safe evidence types, and request budgets are exact enums.
    • Selected mandates remain server-owned context and cannot be repeated as model-authored additions.
    • Input-not-ready capabilities remain absent; Analyze and Compare now have separate positive live coverage.
    • Reasoning-model fallback retains strict JSON schema when temperature is unsupported.
    • Default completion budget increased from 300 to 600 tokens based on controlled reasoning-model results.
    • Added a non-executing 19-scenario live runner with fail-closed semantic, end-to-end, strict-schema, provider-failure, timeout, invalid-output, false-proposal, leakage, execution-surface, and latency thresholds.
    • The privacy-safe report is bound to the exact manifest and evaluation source hashes and omits prompts, raw responses, model/deployment names, endpoint hosts, entity text, credentials, canonical IDs, and evidence.

    Validation

    Deterministic combined gate:

    • 236 passed, 5 expected environment-gated skips
    • Full-file Broken Access Control guardrail: passed for all 8 gate files
    • Full-file XSS sink guardrail: passed for all 8 gate files
    • Python compilation, editor diagnostics, CRLF-aware whitespace, and diff secret checks: clean

    Controlled gpt-5.6-terra gate:

    • 57/57 end-to-end samples passed across 19 realistic scenarios and three repetitions
    • Semantic accuracy: 100% in every category
    • Strict JSON-schema use: 100%
    • Timeout, operational-failure, invalid-output, false-proposal, and capability-leakage rates: 0%
    • Prohibited execution-surface imports: 0
    • Latency: p50 1.67s, p95 2.65s under the 5-second planner deadline
    • Evidence: artifacts/phase10a_controlled_shadow_report.json
    • Manifest hash: 0afbeebf616ae3b1
    • Evaluation-source hash: 14edd8d66bf02784

    The controlled Phase 10A preconditions are now acceptable. Phase 10B should begin only after this hardening branch is committed, reviewed, and merged into feature/orchestration; its revised implementation version should be 0.250.072.

    Refs #1021

  14. paullizer commented on Jul 16, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10A hardening PR opened

    The PR contains the strict live-schema alignment, selected/input-ready governance fixes, 600-token planner budget, source-bound non-executing 19-scenario controlled gate, accepted 57/57 evidence report, tests, feature/fix documentation, and release notes.

    Validation remains:

    • deterministic combined gate: 236 passed, 5 expected skips
    • BAC/XSS guardrails: passed for all 8 gate files
    • controlled gpt-5.6-terra: 57/57, 100% strict schema, zero failures/leakage/prohibited execution surfaces, p50 1.67s and p95 2.65s

    GitHub currently reports the PR mergeable; CLA passed and branch-flow is queued. Phase 10B should start only after PR #1044 merges, from the new integration baseline at version 0.250.072.

    Refs #1021

  15. paullizer commented on Jul 16, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10A hardening merged; Phase 10B ready

    PR #1044 is merged into feature/orchestration.

    • Merge commit: 3b2b2b9dc46d0e00248338296fb58d5c8c365418
    • Hardening implementation commit: 159e1ce93b06800d977ab276d4a61067e9843d11
    • Starting application version: 0.250.071
    • Phase 10B implementation version: 0.250.072
    • Phase 10B branch: feature/orchestration-phase-10b-governed-additive-plan-activation
    • PR base: feature/orchestration

    The Phase 10B branch name remains unused locally, on origin, and on the fork. The private execution plan and durable repository handoff now use the merged hardening baseline and accepted 57/57 controlled-shadow evidence. The obsolete 86378b8a / 0.250.070 handoff must not be used.

    Phase 10B may now begin with conservative activation: high-confidence validated proposals only, selected mandates preserved, deterministic conflict precedence, server-authored options/bundles, existing durable choice/resume lifecycle, exact decision/resume/execution reauthorization, and current-turn-only external queries. clarify remains observational until Phase 10C.

    Refs #1021

  16. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B governed additive plan activation is implemented locally and ready for commit/PR.

    • Branch: feature/orchestration-phase-10b-governed-additive-plan-activation
    • Baseline: 3b2b2b9dc46d0e00248338296fb58d5c8c365418
    • Application version: 0.250.072

    Implemented:

    • Added administrator-controlled assist mode while retaining off as default and shadow for evaluation.
    • Materialized only validated high-confidence read-only planner candidates into server-authored additive options, with deterministic precedence and fail-closed fallback.
    • Added recursive selected, automatic, and deterministic bundle handling; provenance v2 binds automatic roots and exact effective closure while preserving selection/discovery origins.
    • Reused the durable proposal, ETag decision, exact reauthorization, resume lease, child-run, process-loss reconciliation, evidence-ledger, and central-finalization lifecycle.
    • Hardened HTTP boundaries so browser payloads cannot inject server-only capability context, discovery references, minimized queries, leases, or execution identity.
    • Hardened exact resume ownership across post-claim reconstruction, route/stream handoff, cancellation, safety output, partial output, document-action reconciliation, and transient completion-write failures.
    • Added additive choice-card badges/context, Admin Settings rollout controls, privacy-safe activation/revalidation telemetry, feature docs, and release notes.
    • clarify remains observational and prior-turn external-query context remains deferred to Phase 10C.

    Validation:

    • ./.venv/Scripts/python.exe scripts/run_phase9_orchestration_quality_gates.py
    • 290 passed, 7 expected environment-gated skips.
    • Broken Access Control scan passed for 11 application files.
    • XSS scan passed for 12 backend/browser files.
    • Python compilation, editor diagnostics, and git diff --check are clean.
    • Final independent high/medium review found no blocking, high, or medium findings.

    Refs #1021

  17. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B PR submitted: #1046

    • Commit: d3dfa712bae28d9887b7d5574f689169c3193f5f
    • Base: feature/orchestration
    • Ready for review and currently mergeable
    • Branch-flow and CLA checks passed
    • Final local gate: 290 passed, 7 expected environment-gated skips; BAC/XSS, diagnostics, and whitespace checks clean

    PR: #1046

  18. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B activation follow-up applied to PR #1046.

    Root cause of the reported tool-blind response: the environment had chat_capability_planner_mode=off, so the planner LLM was never called. A 5-second planner timeout could also fall through to the base answer model during transient latency.

    Changes:

    • Active environment set to assist, 10-second timeout, 600 completion tokens.
    • Shipped defaults changed to governed assist and a bounded 10-second planner deadline.
    • Explicit off and shadow controls remain available; all approval, authorization, and deterministic conflict gates remain unchanged.
    • No JPMorgan/company/prompt-specific heuristic was added.

    Evidence:

    • Exact public_archive_three_year real planner probe returned strict-schema propose in 3.536 seconds.
    • Recommended capability: deep_research; alternative: web_search.
    • Full gate: 290 passed, 7 expected environment-gated skips; BAC/XSS clean.
    • Commit: 06eabd97

    PR: #1046

  19. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Planner Admin UX enhancement pushed to PR #1046.

    • Commit: 56b872811a794507663dee83a9057b51be74a320
    • Version: 0.250.073
    • Added visible and tooltip definitions for Off, Shadow, Assist, model source, endpoint/model IDs, timeout, completion budget, candidate plans, and capabilities per plan.
    • Assist is identified as the recommended operating mode; Shadow is evaluation-only; Off bypasses planning.
    • Added sliders for timeout (1-20s), completion budget (64-1200), and capabilities per plan (1-8).
    • Added candidate-plan dropdown (1-6).
    • Backend/schema bounds match the UI; defaults remain 10s / 600 / 3 / 4.
    • Full gate: 291 passed, 7 expected skips. Changed-line XSS/BAC checks passed.

    PR: #1046

  20. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B capability-choice UX checkpoint (d46e1f90, version 0.250.074, PR #1046): equal-width plan cards now replace the badge-heavy actions, with a corner Recommended ribbon, concise descriptions, time/cost summaries, and an Includes checklist for server-authored multi-capability plans. One click still submits only one persisted option_id and immediately resumes; the browser cannot compose arbitrary capability bundles.

    Validation: real desktop/mobile Playwright 6 passed; cumulative orchestration gate 296 passed, 3 skipped; full BAC/XSS, syntax, diagnostics, local-asset, and whitespace checks passed. PR remains mergeable with branch-flow and CLA passing.

  21. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B compact choice-state checkpoint (dd763318, version 0.250.075, PR #1046): after a capability decision is saved, the expanded selection UI now condenses immediately to one selected-plan row showing included capabilities and Saved/Running/Completed state. Resume and Retry remain available compactly when needed; decision failures leave the original choices visible.

    Validated with the real desktop/mobile module: 6 passed, completed height below 100px and below 40% of the expanded chooser. Cumulative gate: 296 passed, 3 skipped; BAC/XSS, syntax, diagnostics, asset/sink, and whitespace checks passed. PR remains mergeable.

  22. paullizer commented on Jul 17, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10B is merged through PR #1046 into feature/orchestration at 1aefa44b799659854c8e94a5c1c5c91f0f25bbb0, application version 0.250.075. Final cumulative validation was 296 passed, 3 skipped, with full-gate BAC/XSS, syntax, diagnostics, local-asset/sink, and whitespace checks passing.

    Phase 10C is ready for planning from that exact merged baseline. Recommended branch: feature/orchestration-phase-10c-contextual-goals-clarification; implementation version: 0.250.076; PR base remains feature/orchestration. The first checkpoint is an acceptance-matrix refresh and explicit security review for bounded prior-user-turn egress plus the durable clarification lifecycle. Existing server-authored option IDs, exact source-turn reauthorization, provenance, current-turn-only fallback, and the prohibition on assistant/tool/workspace text in external queries remain mandatory.

  23. paullizer commented on Jul 18, 2026

    @paullizer
    CollaboratorAuthor

    Phase 10C contextual goals and structured clarification is implemented in draft PR #1049 from commit b9443d24.

    What changed:

    • bounded active user-only contextual goal refs with exact source/hash/thread reconstruction;
    • explicit option-scoped approval before prior user text can enter external retrieval;
    • decline-aware external execution closure and repeated source/query reauthorization;
    • durable server-authored clarification with ETag leases, claim-generation fencing, exact Retry/Edit recovery, and process-loss reconciliation;
    • accessible prior-goal/clarification UI and privacy projection across history, collaboration, SSE, metadata, and telemetry;
    • application version and release notes updated to 0.250.076.

    Validation:

    • focused Phase 10C suite: 119 passed;
    • cumulative Phase 9/10A/10B/10C gate: 349 passed, 11 expected environment-gated skips;
    • full-file and added-line BAC/XSS scans passed;
    • route policy, JavaScript syntax, diagnostics, and whitespace passed;
    • final targeted independent review found no remaining medium-or-higher issue in the reviewed defect areas.

    PR: #1049

  24. paullizer commented on Jul 20, 2026

    @paullizer
    CollaboratorAuthor

    Planner-first Phase 10C follow-up is complete in PR #1049, commit ed5294cf.

    • Assist now uses one bounded non-executing model call for direct, propose, or clarify; deterministic requirement heuristics no longer suggest or auto-activate capabilities in Assist.
    • Incomplete global planner configuration uses the exact model selected for the chat turn. Complete configured selections remain global-only and exact.
    • Azure/OpenAI transport uses modern completion tokens, minimal reasoning, strict schema first, HTTP-400-only compatibility fallback, one shared deadline, and privacy-safe diagnostics.
    • Admin Settings now uses dependent enabled endpoint/model selectors and preserves both valid unsaved and intentionally cleared state.
    • Release notes and planner/orchestration/fix documentation are updated for 0.250.077.

    Validation: exact selected-model non-executing probe returned a valid Deep Research proposal with Web Search alternative in 4.017 seconds; cumulative gate 352 passed / 11 skipped; BAC 17 files passed; XSS 19 files passed; JavaScript syntax and diagnostics passed.

  25. paullizer commented on Aug 6, 2026

    @paullizer
    CollaboratorAuthor

    Post-10C orchestration architecture checkpoint

    Completed an end-to-end review of the orchestration roadmap beyond the merged Phase 10C baseline (0.250.077). The local private planning set now extends the architecture through output planning, user control, model routing, workflow tasks, governed memory/context, integration journeys, and consequential actions.

    Key decisions:

    • Capability planning, model routing, execution, evidence, deliverable planning, and output publication remain separate contracts.
    • Every workflow task attempt is its own orchestration run family; the workflow controller preserves the exact user-authored task order and passes typed, source-bound handoffs.
    • Instruction memory is now defined as a generic governed planning input, not only response guidance. Verified instructions normalize into allowlisted directives for capabilities, sources, models, approvals, deliverables, response behavior, and workflow tasks.
    • Directive effects include prefer, avoid, require, explicit_only, ask_first, forbid, and automatic_when_useful.
    • Raw memory text never directly controls execution. The planner sees bounded descriptors; server code applies precedence and revalidates directives before and after planning, at resume/execution, during deliverable materialization, and before publication.
    • Precedence is security/authorization, admin policy, explicit current-turn or task intent, confirmed per-turn overrides, conversation/workflow settings, verified instruction directives, then defaults.
    • Memory revision drift invalidates stale choices/plans and is fenced into conditional publication.
    • Facts, instructions, prior citation/artifact lineage, and authorized other-conversation evidence remain distinct source classes with separate provenance and egress rules.
    • Generated outputs use a two-stage deliverable_intent and materialized_deliverable_plan; prompt markers and character thresholds become hints/fallbacks rather than primary product behavior.
    • Mixed-scope inline responses and artifacts require exact audience compatibility. General write/consequential actions remain behind a separate immutable effect/approval/idempotency/receipt contract.

    New/refined roadmap slices:

    • Phase 11A: Deliverable Planning and Output Composition
    • Phase 12: Execution Modes, Review Visibility, and User Control
    • Phase 13: Simplified Composer and Tool Drawer
    • Phase 14: Advanced Plan Review, Revision, and Retry
    • Phase 15: Capability-Aware Model Routing
    • Phase 16: Task-Level Orchestration for Workflows
    • Phase 17: Governed Memory and Cross-Conversation Evidence
    • Phase 18: End-to-End Reference Journeys and Integration Gate
    • Phase 19: Governed Write and Consequential Actions

    Validation of the private planning docs:

    • local Markdown links and code fences clean across 27 orchestration documents;
    • changed-document whitespace checks clean;
    • basic/complex chat and workflow reference journeys plus parameterized integration coverage are present;
    • final independent architecture review found no medium-or-higher blockers.

    This checkpoint updates planning only. It does not claim Phases 11A-19 are implemented, does not change application code/version, and creates no tracked worktree changes because the planning folder is intentionally private/ignored.

    Refs #1021. Related generated-output work: #1071.

  26. added a commit that references this issue on Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions