An encoder's end barrier flushes to the device only; the start barriers carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each by capibawa · Pull Request #1556 · seedds/nxnative

capibawa · 2026-11-06T19:47:55Z

What and why

Symptom: CPU-visible Metal memory runs to several hundred MB, spread over thousands of 32–48 KB regions (the IOAccelerator (graphics) row of footprint). Most of it belongs to the main submission's 8 command allocators: DM_NVN_ALLOCPROF=1 shows them at 36–80 MB each in Ocarina of Time (0400486024DEC000) before main's Ocarina fixes.

Cause: while overlapping buffers or aliasing textures exist (large-pool chunks, image family heaps), resource_vis() adds MTL4VisibilityOptionResourceAlias. m4_end put that flag on every encoder's end barrier (barrierAfterStages:beforeQueueStages:) as well as its start barrier.

On the end barrier, the flag costs the command allocator about 100 KB per encoder. On the start barrier its cost is too small to measure.
An allocator keeps the size of the largest command buffer it has held, even after a reset. Submissions of 600–1,400 encoders therefore left all 8 allocators at that size.

Change:

m4_end now uses MTL4VisibilityOptionDevice.
Every encoder the host opens on the main command buffer already starts with a barrier that waits for all earlier work and carries resource_vis(). That covers the draw passes, enc_r_begin, enc_c_begin, blit_open, and the scaler's barrier before MetalFX. So an encoder's writes are still flushed for aliases before any later encoder reads them.
A copy encoder that wrote through an aliased buffer address (g_blit_alias) keeps the flush on its end barrier, as Buffer copies through aliased Metal addresses needs.
DM_NVN_END_ALIAS=1 brings the end-barrier flush back. DM_NVN_END_ALIAS=ab alternates it with the DM_AB phases, for comparing the two.
Diagnostic: DM_NVN_ALLOCPROF=2 gives each main submission a fresh allocator. It then reports command memory by operation, by encoder and by submission size, plus the indirect-count multi-draws encoded.

Docs: a new reference entry, The alias flush on every encoder's end barrier (frame-rate.md). Also the symptom index, env_switches.tsv, and a Found on line in Ocarina of Time's note.

Nothing in the open PRs touches these barriers. #1557, the resolution families change, is separate. Both PRs add a line at the end of Ocarina of Time's "Found on this game", so whichever merges second has a one-line conflict.

Checked

Tests:
- make -C dm/gfx/tests GAME=0400486024DEC000 check_t_copy_alias: 126 checks pass, with direct and deferred encoding. The new case has a shader write through one no-copy buffer and the next encoder's shader read through another buffer over the same memory, while overlapping buffers exist. It fails with device-only barriers on both sides and passes with the flush on the start barrier alone.
- The rest of make -C dm/gfx/tests GAME=0400486024DEC000 run, each check run separately, matches main at b2d46b8:
  - t_alpha, t_logic and t_fuse_check fail on main too. Their Python uses X | None, which the system Python 3.9 can't parse.
  - t_nvn and t_gfx_suspend don't link on main (_dm_vi_frame_queued).
- dm/tools/tests/test_docs.py passes.
Games run:
- Ocarina of Time v131072: a new game played to Kokiri Forest, picture correct. This run used this change and Resolution families take every GPU-written compressible 2D image, not only COMPRESSIBLE|IMAGE: Ocarina of Time's dynamic-resolution depth buffer (COMPRESSIBLE alone, in a virtual pool) kept 15-16 live sizes at one offset as separate 14.5 MB Metal textures #1557 together, on main at 2eb7182. I haven't rerun the game on this base (b2d46b8).
Not checked:
- regress.py was not run (the user asked me not to), and no other game was played.
- This change reaches every game while aliases exist. A game that reads aliased memory in a way the start barriers don't cover would show it as stale data. Running regress.py check --games on a few games with large pools or image families before merging would be worthwhile.
Device and OS: M3 Pro (18 GB), macOS 27.0.1.
Measurements (M3 Pro, 18 GB):
- Test program: 600 render passes of 4 draws took 27.7 MB of command memory with the flush on start barriers only, against 88.5 MB with it on end barriers too.
- Ocarina of Time, Kokiri Forest, before main's Ocarina fixes:
  - CPU-visible Metal memory fell from about 800 to 548 MB.
  - In one run with DM_NVN_END_ALIAS=ab DM_AB=600, scenes of about 150 passes took 18–30 ms a frame at p50 without the end flush, against 31–45 ms with it.
- Ocarina of Time on main at 2eb7182 (about 95 passes a frame):
  - CPU-visible Metal memory fell from 392 to 298 MB.
  - The 8 allocators fell from 181 to 116 MB in all.
  - Frame time showed no difference: the A/B phase pairs were within ±2 ms.

Docs

The finding is in its docs/reference/frame-rate.md entry and the game note, and python3 dm/tools/docs_symptoms.py was run (the symptom index)
No game files, keys or data extracted from a game in the change or the attachments

🤖 Generated with Claude Code

…rs carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each - Symptom: CPU-visible Metal memory (footprint's IOAccelerator (graphics)) of several hundred MB in thousands of 32-48 KB regions, most of it the 8 command allocators of the main submission. On Ocarina of Time (0400486024DEC000) before main's Ocarina fixes, scenes of about 150 passes a frame were slow while the GPU was not the limit. - Cause: while overlapping buffers or aliasing textures exist (large-pool chunks, image family heaps), resource_vis() adds MTL4VisibilityOptionResourceAlias, and m4_end put it on every encoder's end barrier (barrierAfterStages:beforeQueueStages:) as well as on its start barrier. On the end barrier it costs the command allocator about 100 KB per encoder; on the start barrier nothing measurable. An allocator keeps the size of the largest command buffer it held after a reset, so submissions of 600-1,400 encoders left all 8 at that size. - Fix: m4_end uses MTL4VisibilityOptionDevice. Every encoder the host opens on the main command buffer starts with a barrier carrying resource_vis() after all earlier work (draw passes, enc_r_begin, enc_c_begin, blit_open, the scaler's before MetalFX), so an encoder's writes are flushed for aliases before a later encoder reads them. A copy encoder that wrote through an aliased buffer address (g_blit_alias) keeps the flush on its end barrier. DM_NVN_END_ALIAS=1 brings the end flush back (ab alternates it with DM_AB). - DM_NVN_ALLOCPROF=2 gives each main submission a fresh allocator and reports command memory by operation, by encoder, by submission size, and the indirect-count multi-draws encoded. - Measured on an M3 Pro (18 GB): a test program's 600 render passes of 4 draws took 27.7 MB of command memory with the flush on start barriers only, 88.5 MB with it on end barriers too; device-only barriers on both sides read stale data through a second buffer over the same memory. Ocarina of Time in Kokiri Forest before main's Ocarina fixes: CPU-visible Metal memory about 800 -> 548 MB, frames 18-30 ms at p50 without the end flush against 31-45 ms with it (DM_NVN_END_ALIAS=ab). On main at 2eb7182: 392 -> 298 MB, the 8 allocators 181 -> 116 MB, no frame-time difference (A/B phase pairs within 2 ms). - Checked: check_t_copy_alias (126 checks, both encoding modes) with a new case, a shader writing through one no-copy buffer and the next encoder's shader reading through another over the same memory, which fails with device-only barriers; the rest of make -C dm/gfx/tests run as on main. Ocarina of Time played to Kokiri Forest with a correct picture, with this and the resolution family change together on main at 2eb7182 (not rerun on this base, b2d46b8). No other game was run. frame-rate.md entry, symptom index, env_switches and the game's note updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…rs carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each (PR #1556) - Symptom: CPU-visible Metal memory (footprint's IOAccelerator (graphics)) of several hundred MB in thousands of 32-48 KB regions, most of it the 8 command allocators of the main submission. On Ocarina of Time (0400486024DEC000) before main's Ocarina fixes, scenes of about 150 passes a frame were slow while the GPU was not the limit. - Cause: while overlapping buffers or aliasing textures exist (large-pool chunks, image family heaps), resource_vis() adds MTL4VisibilityOptionResourceAlias, and m4_end put it on every encoder's end barrier (barrierAfterStages:beforeQueueStages:) as well as on its start barrier. On the end barrier it costs the command allocator about 100 KB per encoder; on the start barrier nothing measurable. An allocator keeps the size of the largest command buffer it held after a reset, so submissions of 600-1,400 encoders left all 8 at that size. - Fix: m4_end uses MTL4VisibilityOptionDevice. Every encoder the host opens on the main command buffer starts with a barrier carrying resource_vis() after all earlier work (draw passes, enc_r_begin, enc_c_begin, blit_open, the scaler's before MetalFX), so an encoder's writes are flushed for aliases before a later encoder reads them. A copy encoder that wrote through an aliased buffer address (g_blit_alias) keeps the flush on its end barrier. DM_NVN_END_ALIAS=1 brings the end flush back (ab alternates it with DM_AB). - DM_NVN_ALLOCPROF=2 gives each main submission a fresh allocator and reports command memory by operation, by encoder, by submission size, and the indirect-count multi-draws encoded. - Measured on an M3 Pro (18 GB): a test program's 600 render passes of 4 draws took 27.7 MB of command memory with the flush on start barriers only, 88.5 MB with it on end barriers too; device-only barriers on both sides read stale data through a second buffer over the same memory. Ocarina of Time in Kokiri Forest before main's Ocarina fixes: CPU-visible Metal memory about 800 -> 548 MB, frames 18-30 ms at p50 without the end flush against 31-45 ms with it (DM_NVN_END_ALIAS=ab). On main at 2eb7182: 392 -> 298 MB, the 8 allocators 181 -> 116 MB, no frame-time difference (A/B phase pairs within 2 ms). - Checked: check_t_copy_alias (126 checks, both encoding modes) with a new case, a shader writing through one no-copy buffer and the next encoder's shader reading through another over the same memory, which fails with device-only barriers; the rest of make -C dm/gfx/tests run as on main. Ocarina of Time played to Kokiri Forest with a correct picture, with this and the resolution family change together on main at 2eb7182 (not rerun on this base, b2d46b8). No other game was run. frame-rate.md entry, symptom index, env_switches and the game's note updated. - Landed on 7fe9fc4 with #1556 and #1557 together: the host builds; t_copy_alias, t_image_alias, t_pool_chunks, t_alias_takeover and t_rt_range pass. No game was run and scripts/regress.py was skipped. Co-authored-by: James <12531452+capibawa@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… only COMPRESSIBLE|IMAGE: Ocarina of Time's dynamic-resolution depth buffer (COMPRESSIBLE alone, in a virtual pool) kept 15-16 live sizes at one offset as separate 14.5 MB Metal textures (PR #1557) - Symptom: in Kokiri Forest on an M3 Pro (18 GB), 15-16 Depth32Float textures of 1,882-1,920 pixels described one pool offset, each with an allocation of its own. - Cause: image_shape required the flags to be exactly COMPRESSIBLE|IMAGE. A depth target carries COMPRESSIBLE alone, so its resolution family never formed. - Fix: image_shape takes any COMPRESSIBLE image without DISPLAY, VIDEO_DECODE, LINEAR, SPARSE or LINEAR_RENDER_TARGET; the other conditions (GPU-only mapped storage, 2D, one level, one sample, same pool, offset and format) are unchanged. - DM_CTL restop also prints a memory census (mem_census): textures by Metal format; which ones live descriptors, only retired ones, only the deferred-release queue or nothing names; storage addresses held as several Metal textures, with their texture and pool flags; memory pools by whether texture uploads or other uses looked them up. It is what found these textures. - Measured, before main's Ocarina fixes: Depth32Float 95 textures (458 MB) to 60 (191 MB) plus 21 family heaps (82 MB); GPU allocation 7,710 to 7,520 MB. On main at 2eb7182 the 17 sizes still form and share one heap, all Depth32Float textures 225 MB. - PR #1453's per-pool placement heaps leave virtual pools out, so they do not take this buffer. - Checked: check_t_image_alias (332 checks, direct and deferred encoding) with a new depth family case (Depth32F, COMPRESSIBLE alone) and a linear-exclusion case; the rest of make -C dm/gfx/tests run as on main; Ocarina of Time played to Kokiri Forest with a correct picture, with this and the end-barrier change together on main at 2eb7182 (not rerun on this base, b2d46b8). rendering.md entry and the game's note updated. - Landed on 7fe9fc4 with #1556 and #1557 together: the host builds; t_copy_alias, t_image_alias, t_pool_chunks, t_alias_takeover and t_rt_range pass. No game was run and scripts/regress.py was skipped. Co-authored-by: James <12531452+capibawa@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

seedds · 2026-11-07T07:28:06Z

Landed as 5b84d37, unchanged apart from a one-line conflict in Ocarina of Time's note (you are credited as co-author). I checked the premise on current main: every encoder the host opens (draw passes, compute, copies, uploads on the same queue, the scaler, present, snapshots) begins with a barrier that carries resource_vis(). On 7fe9fc4, with #1557 also applied, the host builds and t_copy_alias (126), t_pool_chunks, t_alias_takeover, t_rt_range and t_image_alias pass. No game was run on this side and regress.py was skipped, so a report of stale or aliased reads in any game would be welcome. Thank you.

…merges clang-tidy found one more core.NullDereference in nvn_mtl.m (tex_cpu_content's first-use log line read t->pool->flags without a pool, seedds#1604), and one more nullability.NullablePassedToNonnull each in t_copy_alias.m (a kernel function looked up by name, seedds#1556) and t_pso_archive.m (the archive path from a boxed C string, seedds#1307). The log line prints 0 for a texture without a pool; the tests stop when the lookup fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016qteK6QbhFrzr2Vop9t34k

…s first write stays resident while any of them holds it: releasing one took it out of the residency set (GPU page faults, issue seedds#1326) The serials of one storage share one packaged texture (g_pkg_share). At the GPU's first write over that storage, cpu_unc_drop gives each of them the package's texture itself (tex_put), so they all hold one texture that was in no shared set. The first holder released (tex_put's ghost-ring eviction, serial_put or serial_del, through obj_drop or defer_obj_release), or the package's own release (NvPkg dealloc's res_remove), took it out of the residency set while the other serials still drew with it, and a command buffer reading it faulted. pkg_hand_over enters the handed-over texture in g_alias_set and in g_handed, a list by the package's sharing key, so releasing one holder only drops a reference (alias_has). The package's release and its re-upload's spare leave a handed-over texture resident, mtl_alias_drop_pool retires it with its pool, and tex_upgrade moves its entry. It does not enter g_alias, so a new texture object over the same storage still gets a texture of its own. DM_UNC_SHARE=0 gives back the old hand-over. Ocarina of Time v131072, Mac mini M4 16 GB, macOS 27.2, on main b2d46b8 with seedds#1590, seedds#1557, seedds#1556, seedds#1575, seedds#1522, seedds#1543 and seedds#1390: page faults at 44, 29, 24 and 15.5 minutes, with the missing textures named by DM_RES_CHECK and a diagnostic of where each left the residency set, before two of them, as 260x260 terrain tiles handed over this way and released by the ghost-ring eviction. One binary: with the change, 107 minutes and no fault (86 hand-overs); with DM_UNC_SHARE=0, a fault at 37 minutes with the same kind of tiles named. t_cpu_tex --metal handover_shared: 97 checks, 0 failed; with DM_UNC_SHARE=0, 2 failed. The other GPU suites are unchanged. (cherry picked from commit 8cbc330)

…rs carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each (PR seedds#1556) - Symptom: CPU-visible Metal memory (footprint's IOAccelerator (graphics)) of several hundred MB in thousands of 32-48 KB regions, most of it the 8 command allocators of the main submission. On Ocarina of Time (0400486024DEC000) before main's Ocarina fixes, scenes of about 150 passes a frame were slow while the GPU was not the limit. - Cause: while overlapping buffers or aliasing textures exist (large-pool chunks, image family heaps), resource_vis() adds MTL4VisibilityOptionResourceAlias, and m4_end put it on every encoder's end barrier (barrierAfterStages:beforeQueueStages:) as well as on its start barrier. On the end barrier it costs the command allocator about 100 KB per encoder; on the start barrier nothing measurable. An allocator keeps the size of the largest command buffer it held after a reset, so submissions of 600-1,400 encoders left all 8 at that size. - Fix: m4_end uses MTL4VisibilityOptionDevice. Every encoder the host opens on the main command buffer starts with a barrier carrying resource_vis() after all earlier work (draw passes, enc_r_begin, enc_c_begin, blit_open, the scaler's before MetalFX), so an encoder's writes are flushed for aliases before a later encoder reads them. A copy encoder that wrote through an aliased buffer address (g_blit_alias) keeps the flush on its end barrier. DM_NVN_END_ALIAS=1 brings the end flush back (ab alternates it with DM_AB). - DM_NVN_ALLOCPROF=2 gives each main submission a fresh allocator and reports command memory by operation, by encoder, by submission size, and the indirect-count multi-draws encoded. - Measured on an M3 Pro (18 GB): a test program's 600 render passes of 4 draws took 27.7 MB of command memory with the flush on start barriers only, 88.5 MB with it on end barriers too; device-only barriers on both sides read stale data through a second buffer over the same memory. Ocarina of Time in Kokiri Forest before main's Ocarina fixes: CPU-visible Metal memory about 800 -> 548 MB, frames 18-30 ms at p50 without the end flush against 31-45 ms with it (DM_NVN_END_ALIAS=ab). On main at 077fb5c: 392 -> 298 MB, the 8 allocators 181 -> 116 MB, no frame-time difference (A/B phase pairs within 2 ms). - Checked: check_t_copy_alias (126 checks, both encoding modes) with a new case, a shader writing through one no-copy buffer and the next encoder's shader reading through another over the same memory, which fails with device-only barriers; the rest of make -C dm/gfx/tests run as on main. Ocarina of Time played to Kokiri Forest with a correct picture, with this and the resolution family change together on main at 077fb5c (not rerun on this base, 99edc18). No other game was run. frame-rate.md entry, symptom index, env_switches and the game's note updated. - Landed on 622391d with seedds#1556 and seedds#1557 together: the host builds; t_copy_alias, t_image_alias, t_pool_chunks, t_alias_takeover and t_rt_range pass. No game was run and scripts/regress.py was skipped. Co-authored-by: James <12531452+capibawa@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… only COMPRESSIBLE|IMAGE: Ocarina of Time's dynamic-resolution depth buffer (COMPRESSIBLE alone, in a virtual pool) kept 15-16 live sizes at one offset as separate 14.5 MB Metal textures (PR seedds#1557) - Symptom: in Kokiri Forest on an M3 Pro (18 GB), 15-16 Depth32Float textures of 1,882-1,920 pixels described one pool offset, each with an allocation of its own. - Cause: image_shape required the flags to be exactly COMPRESSIBLE|IMAGE. A depth target carries COMPRESSIBLE alone, so its resolution family never formed. - Fix: image_shape takes any COMPRESSIBLE image without DISPLAY, VIDEO_DECODE, LINEAR, SPARSE or LINEAR_RENDER_TARGET; the other conditions (GPU-only mapped storage, 2D, one level, one sample, same pool, offset and format) are unchanged. - DM_CTL restop also prints a memory census (mem_census): textures by Metal format; which ones live descriptors, only retired ones, only the deferred-release queue or nothing names; storage addresses held as several Metal textures, with their texture and pool flags; memory pools by whether texture uploads or other uses looked them up. It is what found these textures. - Measured, before main's Ocarina fixes: Depth32Float 95 textures (458 MB) to 60 (191 MB) plus 21 family heaps (82 MB); GPU allocation 7,710 to 7,520 MB. On main at 077fb5c the 17 sizes still form and share one heap, all Depth32Float textures 225 MB. - PR seedds#1453's per-pool placement heaps leave virtual pools out, so they do not take this buffer. - Checked: check_t_image_alias (332 checks, direct and deferred encoding) with a new depth family case (Depth32F, COMPRESSIBLE alone) and a linear-exclusion case; the rest of make -C dm/gfx/tests run as on main; Ocarina of Time played to Kokiri Forest with a correct picture, with this and the end-barrier change together on main at 077fb5c (not rerun on this base, 99edc18). rendering.md entry and the game's note updated. - Landed on 622391d with seedds#1556 and seedds#1557 together: the host builds; t_copy_alias, t_image_alias, t_pool_chunks, t_alias_takeover and t_rt_range pass. No game was run and scripts/regress.py was skipped. Co-authored-by: James <12531452+capibawa@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…rs carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each From nxnative main 5b84d37 (seedds, PR seedds#1556): scenes of about 150 passes a frame were slow while the GPU was not the limit, and CPU-visible Metal memory held several hundred MB of command allocators. Checked: check_t_copy_alias (126), check_t_res_guard, check_t_tex_recycle. Co-authored-by: seedds <seedds@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 6, 2026

oliverames mentioned this pull request Nov 6, 2026

WIP: Mario Kart World 1.8.0 on iOS - rendering, memory and presentation #1383

Draft

bettterlabs mentioned this pull request Nov 7, 2026

The Legend of Zelda: Ocarina of Time (0400486024DEC000): renderer, shader and pipeline archive fixes through the Forest Temple #1590

Closed

seedds closed this Nov 7, 2026

bettterlabs mentioned this pull request Nov 7, 2026

Keep a texture cpu_unc_drop hands to a package's serials resident while any of them holds it: Ocarina of Time (GPU page faults, #1326) #1644

Closed

2 tasks done

LeonardoBai12 mentioned this pull request Nov 7, 2026

Ocarina of Time on an M1 (16 GB): GPU hang (kIOGPUCommandBufferCallbackErrorHang) under memory pressure, about 24 fps at 720p / 30 fps #1605

Open

Navigation Menu

Repository navigation

An encoder's end barrier flushes to the device only; the start barriers carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each - #1556

An encoder's end barrier flushes to the device only; the start barriers carry the resource-alias flush: Ocarina of Time's main submission's command allocators no longer grow to 36-80 MB each#1556
capibawa wants to merge 1 commit into
seedds:mainfrom
capibawa:pr-end-alias

capibawa commented Nov 6, 2026 •

edited

Loading

Uh oh!

seedds commented Nov 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

capibawa commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why

Checked

Docs

Uh oh!

seedds commented Nov 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

capibawa commented Nov 6, 2026 •

edited

Loading