[BREAKING] Python: Fix handoff workflow context management and improve AG-UI demo

[moonbox3]

Motivation and Context

Supersedes #4509

Addresses #4695, #4053, #4357, #5147

Handoff workflows had several context management bugs:

• Duplicate messages accumulating in _full_conversation across turns (Python: [Bug]: Duplicate messages in Handoff workflow #4695)
• Agents losing conversation context after handoffs — stale previous_response_id and partial cache (Python: [Bug]: HandoffBuilder with AzureOpenAIResponsesClient fails with stale previous_response_id and loses conversation context on handoff #4053)
• Server-assigned item IDs (rs_*, fc_*) replayed under store=False, causing API errors (Python: [Bug]: Handoff workflow with store=False sends server-assigned item IDs (rs_*, fc_*) in input, causing "Item not found" API errors #4357)

Important

HandoffBuilder.build() now requires all participant agents to set require_per_service_call_history_persistence=True. Existing code that constructs handoff workflows without this flag will raise a ValueError at build time with a message identifying which agents need updating. This is a one-line fix per agent constructor. This breaking change only affects the agent-framework-orchestrations package, which is still in beta.

Description

Handoff context management (_handoff.py):

• Simplified _run_agent_and_emit — removed cache manipulation, stale session clearing, and runtime tool message tracking
• _full_conversation now only stores cleaned (text-only) responses via clean_conversation_for_handoff, preventing tool content with server-assigned IDs from being replayed
• Removed forced store=False on cloned agents — users now control storage behavior
• Removed _persist_pending_approval_function_calls and _persist_missing_approved_function_results (no longer needed)
• Updated _is_handoff_requested to return the handoff message so it can be appended to the cache for call/result pairing
• Added build-time validation requiring require_per_service_call_history_persistence=True on all handoff participants

Core workflow improvements:

• Added empty message cache warnings in AgentExecutor._run_agent and _run_agent_streaming
• Moved _deliver_messages_for_edge_runner before early return in _runner.py
• Added Args/Returns docstring and TODO to clean_conversation_for_handoff

AG-UI handoff demo (samples/05-end-to-end/ag_ui_workflow_handoff):

• Updated to use require_per_service_call_history_persistence=True on all agents

Samples and tests:

• Updated all handoff samples (7 files) with require_per_service_call_history_persistence=True
• Updated integration tests to use FoundryChatClient instead of removed AzureOpenAIResponsesClient
• Added validation test for the new require_per_service_call_history_persistence requirement
• Removed tests for deleted functionality (store=False forcing, stale session clearing, persist methods)

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

[moonbox3]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework/_workflows
_agent_executor.py	191	16	91%	109, 133, 174, 200–201, 281–283, 285, 295–296, 343, 415–416, 488, 494
_runner.py	173	2	98%	304–305
packages/orchestrations/agent_framework_orchestrations
_handoff.py	339	53	84%	104–105, 107, 160–170, 172, 174, 176, 181, 312, 337, 364, 390, 453–454, 497, 505, 509–510, 548–550, 555–557, 675, 678, 691, 753, 758, 765, 775, 777, 796, 798, 880–881, 913–914, 1015, 1022, 1094–1095, 1097
_orchestrator_helpers.py	15	2	86%	73–74
TOTAL	27016	3185	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
5371	20 💤	0 ❌	0 🔥	1m 23s ⏱️

[Copilot]

Pull request overview

Fixes multiple Python handoff-workflow context management issues (duplicate history, stale/partial context after handoff, and replay of tool artifacts/IDs), and updates the AG-UI handoff demo + samples/tests to align with the new handoff persistence requirements.

Changes:

• Refactors HandoffAgentExecutor context handling and adds a build-time validation requiring require_per_service_call_history_persistence=True for all handoff participants.
• Improves core workflow runtime behavior (message delivery ordering helper placement; warnings when running agents with an empty cache).
• Updates handoff samples + AG-UI demo UI/backend to reflect new behavior and provide clearer demo lifecycle semantics.

Reviewed changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
python/uv.lock	Adjusts dependency markers (notably for pythonnet/cffi) for Python version gating.
python/packages/orchestrations/agent_framework_orchestrations/_handoff.py	Refactors handoff executor behavior, removes forced store=False, adds build-time validation for per-service-call history persistence.
python/packages/orchestrations/agent_framework_orchestrations/_orchestrator_helpers.py	Expands docs/notes for clean_conversation_for_handoff.
python/packages/core/agent_framework/_workflows/_runner.py	Refactors message delivery helper placement (no behavior change intended beyond flow ordering).
python/packages/core/agent_framework/_workflows/_agent_executor.py	Adds warnings when running an agent with an empty message cache (streaming + non-streaming).
python/packages/orchestrations/tests/test_handoff.py	Updates unit/integration tests for new handoff requirements and client usage; adds validation test.
python/packages/a2a/tests/test_a2a_agent.py	Formatting-only adjustments in test setup arrays.
python/samples/03-workflows/orchestrations/README.md	Documents the new require_per_service_call_history_persistence handoff requirement.
python/samples/03-workflows/orchestrations/handoff_simple.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/03-workflows/orchestrations/handoff_autonomous.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/03-workflows/orchestrations/handoff_with_tool_approval_checkpoint_resume.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/03-workflows/orchestrations/handoff_with_code_interpreter_file.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/03-workflows/agents/handoff_workflow_as_agent.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/autogen-migration/orchestrations/03_swarm.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/semantic-kernel-migration/orchestrations/handoff.py	Sets require_per_service_call_history_persistence=True on all participants.
python/samples/05-end-to-end/ag_ui_workflow_handoff/README.md	Adds demo lifecycle notes and documents the per-service-call persistence requirement.
python/samples/05-end-to-end/ag_ui_workflow_handoff/frontend/src/styles.css	Adds styling for “Start New Case” and pending empty state.
python/samples/05-end-to-end/ag_ui_workflow_handoff/frontend/src/App.tsx	Adds “case closed” behavior and a UI reset path for starting a new thread/case.
python/samples/05-end-to-end/ag_ui_workflow_handoff/backend/server.py	Wraps the workflow to block new top-level input on completed threads; adds closed-case notice workflow.