Repository navigation
Conversation
… event-wait holds, slow 1000-frame stretches) Each run's summary now counts frames of 1 s or more (long_frames), the event-wait holds in its log (holds) and the median of each stretch of --stretch frames (stretches, worst_stretch); results.md shows long_frames, worst_stretch and holds. --verdict judges every run CLEAN or BAD on them: a stretch is slow when its median is over --slow-factor (1.5) times the run's frame p50, or over --slow-ms. results.md gets a verdict line; exit 1 when any run is BAD. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Frame-time percentiles over a long window hide a slow stretch. In a 6-minute run, two 1000-frame stretches at 70–90 ms move p50 and p95 only a little, while the player sees about a minute of stutter. This adds what a long run needs, for any game:
long_frames(frames of 1 s or more),holds(a GPU queue held by an event wait, the host'san event wait has held itline), and the median of each stretch of--stretchframes (default 1000), withworst_stretch. results.md shows long_frames, worst_stretch and holds.--verdict: judges each run CLEAN or BAD on those counts. A stretch is slow when its median is over--slow-factor(1.5) times the run's own frame p50, or over--slow-mswhen given. The relative default works across games and frame targets.--verdict, the exit code and the existing columns are unchanged.bench.summary()keeps its signature (a new optionalstretchargument), so scripts/regress.py's call is unchanged.What it catches. I applied the branch's
summaryandjudgeto the frames.tsv and log of long runs of Ocarina of Time on an M2 Air (a 12,000-frame Kokiri walk, judged from frame 1000):These agree with the absolute rule we used for those runs (no 1000-frame median over 40 ms);
--slow-ms 40gives that rule.Tests (dm/tools/tests/test_bench.py):
test_stretches_and_judge: stretches, the half-stretch tail, the factor and absolute limits, holds, a 1.2 s frame, no frames.test_verdict_cli: a fake host writes a slow stretch and a hold line → BAD in results.md and exit 1; a clean run → exit 0.shutil.rmtree(onexc=)). test_docs.py OK.Not included: the GPU watchdog line (#1550, not on main). When it lands, add its line to the holds count.
Regress: bench.py is a measuring tool. It changes no chain, host or rewriter, and regress.py's call into bench.summary is unchanged and tested, so no game run is needed.
🤖 Generated with Claude Code