Skip to content

Commit 8869cea

Browse files
fix: discard follow-up passes that made no tool calls (#179)
Follow-up responses that contain only narration, refusals, or re-statements of the original answer are now discarded before concatenation. This catches the "split personality" pattern where the follow-up evaluator finds an opportunity but the LLM refuses or scope-polices instead of acting — producing contradictory content appended to an otherwise clean response. The check counts FunctionCallContent (native path) and [Tool result for ...] messages (text-based path) added during the follow-up loop. Zero tool calls means the follow-up added no new information and is dropped. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent c65b262 commit 8869cea

1 file changed

Lines changed: 30 additions & 2 deletions

File tree

src/RockBot.Host/AgentLoopRunner.cs

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1147,16 +1147,44 @@ await EnrichContextForRepromptAsync(
11471147
"look up contacts, etc. Do not claim you lack access without trying. " +
11481148
"Report what you found concisely."));
11491149

1150-
// Run one more pass through the tool loop.
1150+
// Run one more pass through the tool loop. Track message count so we can
1151+
// detect whether any tool calls were actually made during the pass.
1152+
var preFollowUpMessageCount = chatMessages.Count;
1153+
11511154
var result = modelBehavior.UseTextBasedToolCalling
11521155
? await RunTextBasedLoopAsync(
11531156
chatMessages, chatOptions, sessionId, null, tier,
11541157
onPreToolCall, onProgress, onToolTimeout, cancellationToken)
11551158
: await RunNativeLoopAsync(
11561159
chatMessages, chatOptions, null, tier, cancellationToken);
11571160

1161+
// Native path: FunctionCallContent in response messages.
1162+
// Text-based path: tool results appear as "[Tool result for ...]" user messages.
1163+
var addedMessages = chatMessages.Skip(preFollowUpMessageCount);
1164+
var followUpToolCalls = addedMessages
1165+
.SelectMany(m => m.Contents.OfType<FunctionCallContent>())
1166+
.Count();
1167+
if (followUpToolCalls == 0)
1168+
{
1169+
followUpToolCalls = addedMessages
1170+
.Count(m => m.Role == ChatRole.User
1171+
&& m.Text?.StartsWith("[Tool result for ", StringComparison.Ordinal) == true);
1172+
}
1173+
11581174
logger.LogInformation(
1159-
"Follow-up pass complete — {TextLen} chars", result.Response.Length);
1175+
"Follow-up pass complete — {TextLen} chars, {ToolCalls} tool call(s)",
1176+
result.Response.Length, followUpToolCalls);
1177+
1178+
// Discard follow-up passes that didn't actually invoke any tools — these are
1179+
// pure narration, refusals, or re-statements of the original answer. A useful
1180+
// follow-up should have called at least one tool to gather new information.
1181+
if (followUpToolCalls == 0)
1182+
{
1183+
logger.LogWarning(
1184+
"Follow-up pass made no tool calls ({TextLen} chars); discarding as commentary",
1185+
result.Response.Length);
1186+
return null;
1187+
}
11601188

11611189
// Discard follow-up responses that are capability denials, refusals, or
11621190
// meta-commentary about the agent's own rules/instructions rather than

0 commit comments

Comments
 (0)