feat(hindsight): add recall_min_scores relevance floor to recall path#71122
Open
eulahwu917 wants to merge 1 commit into
Open
feat(hindsight): add recall_min_scores relevance floor to recall path#71122eulahwu917 wants to merge 1 commit into
eulahwu917 wants to merge 1 commit into
Conversation
Add opt-in recall_min_scores config key that passes a relevance floor through to Hindsight's recall calls, fixing the 'no relevance floor, backfills to token budget regardless of fit' bug that injects stale/ irrelevant facts into every turn. Changes: - Upgrade hindsight-client to 0.8.5 in all 3 version-pin locations: __init__.py (_MIN_CLIENT_VERSION), tools/lazy_deps.py, plugin.yaml - Add _normalize_min_scores() helper with key whitelist (semantic, keyword, reranker, final) and float coercion, fail-open on invalid input - Wire _recall_min_scores in initialize() next to other recall controls - Pass min_scores through at both arecall call sites: prefetch recall and hindsight_recall tool - Add recall_min_scores entry to get_config_schema() - 16 new unit tests: _normalize_min_scores validation + arecall wiring at both sites, including malformed config fail-open verification Reflect call sites intentionally untouched — server 0.8.4's ReflectRequest has no min_scores field (server-side limitation). Co-Authored-By: Planner (t_babb0d63, APPROVED WITH CORRECTIONS)
Collaborator
Related to #64914: both wire Hindsight min_scores into the same recall paths, but this PR changes the dependency floor and config contract. Please choose or consolidate the intended configuration/compatibility policy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add opt-in
recall_min_scoresconfig key that passes a relevance floor through to Hindsight's recall calls, fixing the "no relevance floor, backfills to token budget regardless of fit" bug that injects stale/irrelevant facts into every turn.Per Planner spec review (t_babb0d63, APPROVED WITH CORRECTIONS).
Changes
Version pin (B1 — atomic across 3 files)
plugins/memory/hindsight/__init__.py:_MIN_CLIENT_VERSION0.6.1 → 0.8.5tools/lazy_deps.py:hindsight-client==0.6.1→==0.8.5(exact pin)plugin.yaml:hindsight-client>=0.6.1→>=0.8.5New config + validation (B2)
recall_min_scoresinget_config_schema()— opt-in dict e.g.{"reranker": 0.01}_normalize_min_scores()helper with key whitelist{semantic, keyword, reranker, final}, float coercion, fail-open on invalid inputinitialize()next to other recall controlsCall-site wiring (2 sites)
queue_prefetch()— prefetch recall: passesmin_scorestoarecall(**recall_kwargs)hindsight_recalltool — passesmin_scorestoarecall(**recall_kwargs)min_scoresfield)Tests (16 new)
_normalize_min_scoresvalidation: None, valid dict, all keys, unknown key dropped, non-numeric dropped, all-invalid returns None, type error, not-a-dict, int→float coercionVerification
All 132 tests pass (116 existing + 16 new):
Files changed
plugins/memory/hindsight/__init__.py(+61 lines)plugins/memory/hindsight/plugin.yaml(+1/-1)tools/lazy_deps.py(+1/-1)tests/plugins/memory/test_hindsight_provider.py(+120 lines)