Repo-finalize step commits untracked scratch tests the verifier excluded ("chore: finalize repo for handoff"), breaking CI on the build PR #162

Summary

The repo-finalize agent (swe_af/prompts/repo_finalize.py) is told to make git status clean and then "Commit any cleanup: chore: finalize repo for handoff". It is also told not to modify or remove tests. When earlier agents leave untracked test files in the workspace, the only way to satisfy both rules is to git add them. It does that, so the build PR ships tests that the verifier deliberately excluded.

What we saw (two consecutive build() runs, Python node, main at d53b9fb)

• Build 1: the verifier reported "7 untracked, uncommitted scratch files … not in git, and 11 of their 181 tests fail against code that changed later … stale leftovers, not part of the delivered codebase." Then the finalize commit added exactly those 7 files to the PR, and CI failed with 11 tests.
• Build 2: the verifier said "For this run I ignored 11 untracked test files that were left uncommitted in tests/unit … One of them fails." The finalize commit again added all 11 (+2,082 lines), and CI failed.

In both builds the verified tree (before the finalize commit) passed the full suite. Reverting only the finalize commit's test files made CI green.

Why it happens

• SYSTEM_PROMPT: "A clean git status with no untracked debris" and "Do NOT modify source code, tests…"; "Do NOT remove files you're uncertain about".
• Task prompt step 4: "Check git status — ensure the working tree is clean."
• Step 5: "Commit any cleanup."

Untracked tests are not artifacts, so the agent won't delete them, and a "clean working tree" then means committing them. The verifier's findings about those files are not passed to the finalize agent.

Suggested fix

• Narrow what the finalize agent may stage. It should stage only its own cleanup (artifact deletions and the .gitignore change) and never git add untracked source, test or doc files. Leave them untracked and report them in the JSON, e.g. a new untracked_left list.
• Treat a dirty tree as acceptable. Replace "ensure the working tree is clean" with "ensure no artifacts remain" for this case.
• Optionally, feed the verifier's untracked-file findings to the finalize step, or fail the build's hygiene check when untracked tests exist, so earlier agents' leftovers get resolved before handoff rather than shipped.

Happy to test a fix against our setup.