Repository navigation
Conversation
A table flattened to prose (no Markdown pipes) states its cells as bare integers. The ingest only recorded numbers carrying a mark of measurement, so a document's comma-grouped figure like 220,308 never reached the ledger, and a report quoting the table's own cells was redacted as unsourced even though the document stated them (HKUDS#2126). A comma-grouped number is unambiguously a quantity, never a page number or year, so it is now admitted alongside measured figures. Plain ungrouped integers stay out to keep an invented small figure from matching one by chance. Test plan: ``` uv run --with pytest pytest agent/tests/test_grounding_document_figures.py -q uv run --with pytest pytest agent/tests/ -k grounding -q ``` Red-green: the new test fails before the change (grouped figures absent from the ledger) and passes after; the full document-figures file and the grounding suite stay green, including the page-number, year and ungrouped-int exclusions. Prepared with an AI assistant and reviewed line by line before posting. Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
4 tasks done
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A table flattened to prose (the common PDF-extraction shape, no Markdown pipes) states its cells as bare integers.
_ingest_document_textonly recorded numbers carrying a mark of measurement, so a document's comma-grouped figure like220,308never reached the evidence ledger. A report that faithfully reproduced the table's own cells was then redacted asnumeric_claim_conflict / not_in_referenced_calleven though the document stated them. Only a cell that also appeared elsewhere with a unit (e.g. the total restated as63.34 亿元) survived. Closes #2126.Fix
A comma-grouped number is unambiguously a stated quantity, never a page number, section number or year, so it is now admitted to the ledger alongside measured figures. Plain ungrouped integers stay excluded to keep an invented small figure from matching one by chance, preserving the invariant pinned in the existing tests.
Validation
Red-green on the repro:
test_document_prose_grouped_figures_reach_the_ledgerfails before the change (grouped figures absent from the ledger) and passes after.The full document-figures file and grounding suite stay green, including
test_a_page_number_and_a_year_are_not_findings(page12, year2025, ungrouped3400remain excluded).Prepared with an AI assistant and reviewed line by line before posting.