Skip to content

fix(agent): give withheld final answers one bounded text-only retry - #2136

Merged
he-yufeng merged 1 commit into
HKUDS:mainfrom
remote-controlled-man:codex/prflow-5319305246d6
Nov 1, 2026
Merged

he-yufeng merged 1 commit into
HKUDS:mainfrom
remote-controlled-man:codex/prflow-5319305246d6

Conversation

@remote-controlled-man

Copy link
Copy Markdown
Contributor

Summary

  • Give a final answer rejected for tool-call markup one additional text-only turn after the normal iteration cap.
  • Keep that turn out of tool execution and goal continuation, clear rejected streamed text, and retain the existing grounding, cancellation, token-budget and usage paths.
  • Add eight regression cases through AgentLoop.run() and describe the bounded retry in the README.

Why

Refs #2122. The existing plain-text retry requires another iteration, but a forced-text answer normally arrives on the last permitted iteration. A valid re-emission was therefore unreachable after the cap.

The extra turn is only scheduled for a withheld tool-syntax answer. It is counted in reported iterations and usage. A second markup response or an unoffered structured tool call uses the existing degraded fallback; it cannot reopen research. This PR addresses the retry path; CLI/Web degraded-status presentation is unchanged.

Test Plan

  • 56 relevant agent-loop, streaming/interruption, outcome, compaction, grounding and goal/context test modules: 1665 passed, no skips (9 dependency deprecation warnings).
  • The two cap-recovery cases fail on the original base; all eight new cases pass with the fix. The complete DSML module has 11 passing cases.
  • Changed-file Ruff, Black changed-line checks, ci_private_files_gate.py, all seven ci_grep_gates.sh gates, and git diff --check pass.
  • Black checks cover the changed line ranges; unrelated whole-file formatting was not included.

The tests use offline scripted providers and synthetic tools. The full repository suite, frontend build, real providers and live/broker flows were not run.

Scope and risk

This changes the protected agent loop already discussed in #2122. One answer-only model turn can occur beyond the configured iteration cap; cancellation and an exhausted goal token budget still stop it. Broker, credential, MCP and network configuration are unchanged. Roll back by reverting the fix commit.

  • DCO sign-off included.
  • README updated.
  • No secrets or hardcoded runtime paths.

Signed-off-by: Steven Huang <134844631+remote-controlled-man@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 31, 2026 17:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@he-yufeng he-yufeng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the full diff and ran it in an isolated checkout of 1a7969a4484663ee68b35f380c97128e8b92b891.

The bounded retry is wired to the right failure mode. A withheld final answer gets one text-only turn only after the normal cap is exhausted; that turn cannot execute tools even if the provider returns structured calls, it cannot continue a goal, and it still stops on cancellation and the goal token budget. The new tests cover the cap case, the unoffered-call case, budget accounting, cancellation, grounding, and the correction path.

Verification: the focused loop tests pass 75/75, and the full non-E2E suite passes 21,273 with 111 skips. tools/ci_private_files_gate.py and git diff --check also pass. CI is green on all ten checks.

This is a good fix for #2122. Approving.

@he-yufeng
he-yufeng merged commit d51e651 into HKUDS:main Nov 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants