Commit a105c44
committed
studio: exempt offers of help, and add a measured accuracy floor
Offering to help hands control back exactly like the existing "let me know"
exemption. On a corpus of real model turns, "I'll do my best to help" and
"allow me to assist" close a clarification request and never precede a tool
call, but they were read as intent and re-prompted. "help you" keeps its plan
reading when an action verb follows it.
The new test scores the classifier against 300 turns captured from three local
GGUF models, each one a finished answer: the turn called no tool, and three
regenerations behind the production nudge produced no tool call either. Over
those turns, wasted nudges go from 36 (12.0%) on main to 5 (1.7%), and retries
whose text would be discarded from 60 (20.2%) to 1 (0.3%).
Until now these patterns were tuned on hand-written example sentences, which
cannot show how often the classifier is right on real output.1 parent 20fe689 commit a105c44
3 files changed
Lines changed: 405 additions & 2 deletions
File tree
- studio/backend
- core/inference
- tests
- data
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
176 | 184 | | |
177 | 185 | | |
178 | 186 | | |
179 | 187 | | |
180 | | - | |
| 188 | + | |
| 189 | + | |
181 | 190 | | |
182 | 191 | | |
183 | | - | |
| 192 | + | |
184 | 193 | | |
185 | 194 | | |
186 | 195 | | |
| |||
0 commit comments