Skip to content

feat(hindsight): add recall_min_scores relevance floor to recall path#71122

Open
eulahwu917 wants to merge 1 commit into
NousResearch:mainfrom
eulahwu917:wt/t_8df589cb-recall-min-scores
Open

feat(hindsight): add recall_min_scores relevance floor to recall path#71122
eulahwu917 wants to merge 1 commit into
NousResearch:mainfrom
eulahwu917:wt/t_8df589cb-recall-min-scores

Conversation

@eulahwu917

Copy link
Copy Markdown

Summary

Add opt-in recall_min_scores config key that passes a relevance floor through to Hindsight's recall calls, fixing the "no relevance floor, backfills to token budget regardless of fit" bug that injects stale/irrelevant facts into every turn.

Per Planner spec review (t_babb0d63, APPROVED WITH CORRECTIONS).

Changes

Version pin (B1 — atomic across 3 files)

  • plugins/memory/hindsight/__init__.py: _MIN_CLIENT_VERSION 0.6.1 → 0.8.5
  • tools/lazy_deps.py: hindsight-client==0.6.1==0.8.5 (exact pin)
  • plugin.yaml: hindsight-client>=0.6.1>=0.8.5

New config + validation (B2)

  • recall_min_scores in get_config_schema() — opt-in dict e.g. {"reranker": 0.01}
  • _normalize_min_scores() helper with key whitelist {semantic, keyword, reranker, final}, float coercion, fail-open on invalid input
  • Wired in initialize() next to other recall controls

Call-site wiring (2 sites)

  • queue_prefetch() — prefetch recall: passes min_scores to arecall(**recall_kwargs)
  • hindsight_recall tool — passes min_scores to arecall(**recall_kwargs)
  • Reflect call sites intentionally untouched (server 0.8.4 ReflectRequest has no min_scores field)
  • No numeric default — ship unset/opt-in only

Tests (16 new)

  • _normalize_min_scores validation: None, valid dict, all keys, unknown key dropped, non-numeric dropped, all-invalid returns None, type error, not-a-dict, int→float coercion
  • Wiring: prefetch passes/omits min_scores, hindsight_recall tool passes/omits min_scores, malformed config drops with warning and does NOT pass to arecall (fail-open confirmed)

Verification

All 132 tests pass (116 existing + 16 new):

pytest tests/plugins/memory/test_hindsight_provider.py -v
# ============================= 132 passed in 3.15s ==============================

Files changed

  • plugins/memory/hindsight/__init__.py (+61 lines)
  • plugins/memory/hindsight/plugin.yaml (+1/-1)
  • tools/lazy_deps.py (+1/-1)
  • tests/plugins/memory/test_hindsight_provider.py (+120 lines)

Add opt-in recall_min_scores config key that passes a relevance floor
through to Hindsight's recall calls, fixing the 'no relevance floor,
backfills to token budget regardless of fit' bug that injects stale/
irrelevant facts into every turn.

Changes:
- Upgrade hindsight-client to 0.8.5 in all 3 version-pin locations:
  __init__.py (_MIN_CLIENT_VERSION), tools/lazy_deps.py, plugin.yaml
- Add _normalize_min_scores() helper with key whitelist (semantic,
  keyword, reranker, final) and float coercion, fail-open on invalid input
- Wire _recall_min_scores in initialize() next to other recall controls
- Pass min_scores through at both arecall call sites: prefetch recall
  and hindsight_recall tool
- Add recall_min_scores entry to get_config_schema()
- 16 new unit tests: _normalize_min_scores validation + arecall wiring
  at both sites, including malformed config fail-open verification

Reflect call sites intentionally untouched — server 0.8.4's
ReflectRequest has no min_scores field (server-side limitation).

Co-Authored-By: Planner (t_babb0d63, APPROVED WITH CORRECTIONS)
@alt-glitch alt-glitch added type/feature New feature or request comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers P3 Low — cosmetic, nice to have needs-decision Awaiting maintainer decision before any implementation sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades area/memory Memory subsystem: store, providers, sync, background reviews labels Jul 25, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #64914: both wire Hindsight min_scores into the same recall paths, but this PR changes the dependency floor and config contract. Please choose or consolidate the intended configuration/compatibility policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins needs-decision Awaiting maintainer decision before any implementation P3 Low — cosmetic, nice to have sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/memory Memory tool and memory providers type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants