Skip to content

Remove pipeline archive files Metal refuses, so the prebuild runs again - #2539

Open
devMlGUE wants to merge 1 commit into
seedds:mainfrom
devMlGUE:fix-refused-archive-files
Open

devMlGUE wants to merge 1 commit into
seedds:mainfrom
devMlGUE:fix-refused-archive-files

Conversation

@devMlGUE

Copy link
Copy Markdown

What and why

After updating my Mac from macOS 26.6.2 to 27.2.1, Ocarina of Time started freezing for one to several seconds in places it had been smooth (talking to the Goron boss, bombs going off), and it kept happening on later starts.

The log explains it: every start prints one [RED] pipeline archive file … not usable (deployment target … is not compatible with current OS …): its bases are built again line per file written on 26 (57 of them here), and there's no pipeline prebuild: line at all. Metal only opens an archive on the major OS version it was made for (same thing as #673, but for the files the host writes itself). The start skipped those files but left them on disk, and since prebuilt.tsv was still there and still matched, the prebuild was treated as done. So every base that lived in the old prebuild files went to the draws, roughly 0.3 s each on the submit thread, on every start.

The change: a file Metal refuses gets removed together with its index (same as old indexes already are), and if it was a prebuild file, prebuilt.tsv goes too, so the next start runs the prebuild again. Reference entry added under Pipelines and caches in docs/reference/shaders.md.

Checked

  • Tests: t_pso_archive with a new case for this (a prebuild file Metal won't open, plus its index and the stamp): fails without the change, all 8 pass with it. docs_symptoms.py --check and test_docs.py pass.
  • Games run: Ocarina of Time (0400486024DEC000, v131072), on a copy of my real archive from after the update.
before after
refused files left on disk 57 (1.68 GB of 3.1 GB) 0
prebuild on start skipped runs again, once
[RED] lines per start 57, every start 57 once (the files are gone after)

The 24-minute session that showed it (before the change) had 53 stalls of 100 ms or more on new pipelines, 39 s in total.

  • Devices and OS: M1 Pro 16 GB, macOS 27.2.1, Xcode 27.2.
  • regress.py: not run, the change only touches the archive's startup cleanup, no shader or draw path.
  • Trade-off: the first start after a major update now spends time in the prebuild instead of handing those bases to gameplay forever. Worst case measured on main with an empty archive: 13.4 min for the 34,649 pairs (10 workers) plus 2.7 min for the prewarm lists.

Docs

  • The finding is in its docs/reference/<subsystem>.md entry, and python3 dm/tools/docs_symptoms.py was run
  • No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

https://claude.ai/code/session_01JkGASzyCKNRT9hFiZr9PnN

Symptom: after a major macOS update (26.6.2 to 27.2.1, M1 Pro), every
start logged one "pipeline archive file ... not usable (deployment
target ... is not compatible with current OS)" line per old file and no
"pipeline prebuild:" line, and scenes stalled for one to several seconds
on bases they had before, on every start.

Cause: pso4_init skipped a file Metal could not open but left it on
disk. The prebuild's files went with it, prebuilt.tsv still matched (it
does not hold the system version), so the prebuild was skipped as done
and its bases were left to the draws (about 0.3 s each on the submit
thread).

Fix: remove a refused file and its index, as old indexes already are;
a refused prebuild file also removes prebuilt.tsv.

Measured on Ocarina of Time, on a copy of the archive after the update:
57 refused files (1.68 GB of 3.1 GB, 7 of them prebuild files) were
removed and the prebuild ran again. With an empty archive on main the
whole prebuild took 13.4 min for 34,649 pairs (10 workers) plus 2.7 min
for the prewarm lists, once. The session before had 53 stalls of 100 ms
or more on new pipelines (39 s). t_pso_archive: new
test_unusable_file_removed fails without the fix; the 8 tests pass with
it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JkGASzyCKNRT9hFiZr9PnN
@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation labels Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant