Skip to content

Prepare compute texture bindings before opening the dispatch encoder - #1755

Closed
thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/dispatch-own-encoder
Closed

thehobbit85 wants to merge 1 commit into
seedds:mainfrom
thehobbit85:codex/dispatch-own-encoder

Conversation

@thehobbit85

@thehobbit85 thehobbit85 commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Related to #1743. This fixes a reproducible host defect found while investigating that report; the issue stays open because the exact Ocarina of Time crash has not been confirmed against the game.

Resolving a compute shader's sampled texture can upgrade its allocation for a different-format view and copy the old pixels. Previously dispatch opened its compute encoder before resolving arguments. With DM_NVN_DISPATCH_OWN=1, the upgrade opened a second encoder on the same command buffer and crashed the Metal driver. With shared encoders, the upgrade copy came after the dispatch barrier and the shader could read stale pixels.

Prepare the compute argument table before selecting/opening the encoder and placing its dependency barrier. Compute bindings only populate the argument table, so they need no open encoder. The change preserves the existing barriers, direct/indirect group counts, and encoder selection.

The regression builds an 8×8 texture and a tiny compute shader entirely from synthetic data. It checks preserved pixels and driver group counts for direct/indirect dispatches, first-use/cached views, shared/separate encoders, and immediate/threaded encoding.

Validation after conflict resolution — 2026-11-08 07:50 UTC

Tested head ab2bfe2b6a5a324a8c8b6ec39feee9b219a1ed20 rebases a33b39c553d60af81b422c1e1b63228c9af446a2 onto main 62f6a52c4fa9b42b73e5c37ef9a88794c8d15ae5. Ocarina's findings list keeps both main's Pro Controller duplicate-device fix and this independent compute encoder finding. The seven feature paths have identical patch lines; all 18 upstream-only changed paths match main. The generated symptom index was refreshed.

Fresh checks on this exact head, Apple M2 Max, macOS 26.6 / Xcode27.2 (27B52):

  • python3 -m unittest discover -s dm/tools/tests -p test_docs.py: all 11 tests pass.
  • make -C dm/gfx/tests GAME=010015100B514000 check_t_dispatch_own: 21 checks in each of four shared/separate and immediate/threaded configurations pass with Metal validation.
  • make -C dm/host GAME=010015100B514000 -j3 and git diff --check 62f6a52c..HEAD pass.
  • LLVM23.1.4 is now available. python3 dm/tools/lint_c.py compdb succeeds; LLVM=<LLVM23.1.4> python3 dm/tools/lint_c.py tidy -j2 dm/gfx/nvn/nvn_mtl.m dm/gfx/tests/t_dispatch_own.m finds zero fixture diagnostics and 62 renderer/header diagnostics. The ratchet remains failing with two file/check pairs above the baseline (BitwiseShift and ArrayBound). The exact main source gives the same diagnostic messages and counts, ignoring shifted line numbers. No suppression or baseline edit was made.

The broader suite and baseline reproductions below remain historical at their named revision. No fresh gameplay, device test, full graphics run or remote-CI success is claimed; the exact Ocarina report and deferred-uniform/GPU-produced-count coverage remain open.

Earlier validation on a33b39c5

Apple M2 Max, arm64, macOS 26.6, Xcode 27.2 (27B52), base 555122a9069235441d5a88955105397ec6d53686.

  • Failing before: the final fixture against unchanged base aborts under Metal validation with Command buffer already has an open command encoder in separate-encoder mode. Shared mode fails its preserved-pixel assertion. Without validation, the original synthetic reproduction faults in AGX::ComputeContext::endComputePass at address 0x84; the report's M1 Max stack is in prepareForEnqueue at 0x98, so these are not claimed to be an identical game reproduction.
  • Passing after: make -C dm/gfx/tests GAME=010015100B514000 check_t_dispatch_own: 21 checks × four configurations, zero failures, with Metal API validation. This was rerun after the base comparison restored the fix.
  • Mac host: make -C dm/host GAME=010015100B514000 -j3 succeeds.
  • Graphics: all 122 test targets were attempted, continuing the remaining targets after the suite stopped. 119 pass; three fail identically with unchanged base: t_present_crop crashes; t_render_counter_overlap fails CopyBuffer0 (1/42); t_render_enable_queue fails 8/38. Both additional Python graphics checks pass. These same baseline failures are recorded in Fix CI fixtures and lint ratchets #1746. No test was weakened or skipped to obtain a green result.
  • Python: 165 modules / 1,852 tests / 13 skips; 163 modules pass. test_xcode_workflow has six subcase errors for a missing synthetic RomFS.img, and test_env_switches has stale table rows. Both reproduce in a separate source snapshot of the unchanged base and are already addressed by Fix CI fixtures and lint ratchets #1746.
  • Documentation: all 11 docs tests pass; symptom index regenerated; git diff --check passes.
  • Pinned ruff 0.17.2 / pyright 1.1.414: the unchanged test_patches.py has existing baseline drift (376 findings against 371 listed, five new counts), also covered by Fix CI fixtures and lint ratchets #1746. No Python source is changed here.
  • C lint: the compilation database generated successfully (260 sources); clang-tidy was unavailable during this original author run. The fresh focused LLVM23.1.4 comparison above supersedes that tool-availability limitation, but is not a full-source scan.

Game coverage and limits

No game files, extracted shaders, saves, keys, or device inputs were used. No gameplay, iPhone, or exact reported M1 Max crash validation is claimed. The fixture uses CPU-supplied indirect counts; it does not independently exercise GPU-generated counts or deferred uniform updates.

regress.py impact 555122a9..HEAD reports three affected games with manifests (Breath of the Wild, Pokémon Scarlet, Tears of the Kingdom), one unaffected, and 149 unknown because they have no manifest. The five selected gameplay checks would be:

Game Why
Ocarina of Time (0400486024DEC000) Reported separate-encoder crash and NVN2 compute bindings
Hyrule Warriors: Age of Calamity (01002B00111A2000) Dense compute dispatch batches on the shared encoder
Link's Awakening (01006BB00C6F0000) Compute light culling reads driver group counts
Breath of the Wild (01007EF00011E000) Impact manifest and compute uniform/texture bindings
Tears of the Kingdom (0100F2C0115B6000) Impact manifest and broader compute/resource paths

scripts/regress.py check --games 0400486024DEC000,01002B00111A2000,01006BB00C6F0000,01007EF00011E000,0100F2C0115B6000 could not run: no game-build/harness paths are available. Its configuration check exits 2 before any game run. All five games and other titles remain unchecked; there is no passing gameplay report.

Docs

  • Durable mechanism in the rendering reference, linked from Ocarina's “Found on this game” list with the remaining attribution limit.
  • Diagnostic-switch guide and generated symptom index updated.
  • No game data or agent planning/progress artifacts in the diff. The shader chain and front-end interfaces do not change.

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) docs Documentation game: 0400486024DEC000 A game labels Nov 8, 2026
Related to seedds#1743. Resolve format-view upgrade copies before encoder selection and dependency barriers; cover direct and indirect cold/warm dispatches with synthetic Metal fixtures.
@thehobbit85
thehobbit85 force-pushed the codex/dispatch-own-encoder branch from a33b39c to ab2bfe2 Compare November 8, 2026 08:03
@Amost-

Amost- commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Tested against Ocarina of Time on an M2 Pro (19-core GPU, 32 GB, macOS 27.2, the G14X driver). Here DM_NVN_DISPATCH_OWN=1 crashed the host as in #1743: CRASH sig=11 addr=0x98 in AGX::ComputeContext<AGX::G14X::Encoders, …>::prepareForEnqueue(bool)+0x2b0, in 5 of 5 runs with the Mac app's host, near frame 5,700 during the Kokiri Forest load.

The tree is main 863bc6b7 + #1585, #1331, #1368, #1176, #1377, #1379 and a few host picks of ours (chain 3f146103c47b). I ported this PR's dispatch() hunk by hand: the code around it is the same there, and enc_c_args ignores the encoder in that tree too. Then I built two hosts with make -C dm/host GAME=0400486024DEC000 dmhost, the same tree with and without the hunk. The runs were headless, with the app's data, DM_NVN_DISPATCH_OWN=1 (and DM_NVN_DISPATCH_OVERLAP=0, which turns off a local pick of ours) and a fresh copy of a Kokiri Forest save, loaded at about 99 s:

Host Runs Result
without the hunk 1 the crash above, at frame 5,700 (106 s)
with the hunk 3 no crash: the forest loads and draws correctly with an encoder per dispatch, until the GPU hang of #1326 at 114–155 s and the revoke

So in game, the change fixes this crash. The hang after it is the one tracked in #1326; this tree hangs there with shared encoders too (21 of 21 runs).

seedds added a commit that referenced this pull request Nov 8, 2026
…inding that upgrades its texture for a view of another format copies the pixels through the copy encoder, which opened a second encoder beside DM_NVN_DISPATCH_OWN's (Ocarina of Time crashed in AGX prepareForEnqueue at the Kokiri Forest load) and by default landed after the dispatch's barrier, so the shader could read the old pixels (PR #1755)

Compute bindings only write the argument table, so encode_args needs no open
encoder; any copy a binding issues now comes before the shared encoder's
barrier, as image_prepare already does for image writes. t_dispatch_own checks
pixels and driver group counts for direct and indirect dispatches, first and
cached views, both encoder modes and both encoding threads; it fails without the
change on an M6 (stale pixels shared, open-encoder abort separate).

Ocarina of Time on an M2 Pro (Amost-): 5 of 5 runs crashed before, 3 of 3
loaded the forest after.

Fixes #1743

Co-authored-by: Eliran Zach <4012041+thehobbit85@users.noreply.github.com>
@seedds

seedds commented Nov 8, 2026

Copy link
Copy Markdown
Owner

Landed as 7b6506d, with you credited as co-author. Thank you, and thanks to @Amost- for confirming it in Ocarina of Time (5 of 5 crashes before, 3 of 3 clean after).

The fix and the test are kept as written. Doc changes:

  • Ocarina's note and the rendering reference now mark the crash resolved, with the M2 Pro result, and name the stale read on the default shared-encoder path as a bug of its own.
  • The extra sentence in the DM_NVN_DISPATCH_OWN row of 04-graphics.md is dropped; the reference entry covers it.

Checked on an M6: check_t_dispatch_own passes 21 of 21 in all four configurations, and without the fix it fails both ways (stale pixels shared, open-encoder abort separate). Quick regression on Kirby and the Forgotten Land: the candidate built compute pipelines and ran to frame 1,898 at 30 fps with no new red lines; the baseline hit Kirby's known silent-sink crash.

@seedds seedds closed this Nov 8, 2026
Know-It-All-Brother pushed a commit to Know-It-All-Brother/nxnative that referenced this pull request Nov 9, 2026
…inding that upgrades its texture for a view of another format copies the pixels through the copy encoder, which opened a second encoder beside DM_NVN_DISPATCH_OWN's (Ocarina of Time crashed in AGX prepareForEnqueue at the Kokiri Forest load) and by default landed after the dispatch's barrier, so the shader could read the old pixels (PR seedds#1755)

Compute bindings only write the argument table, so encode_args needs no open
encoder; any copy a binding issues now comes before the shared encoder's
barrier, as image_prepare already does for image writes. t_dispatch_own checks
pixels and driver group counts for direct and indirect dispatches, first and
cached views, both encoder modes and both encoding threads; it fails without the
change on an M6 (stale pixels shared, open-encoder abort separate).

Ocarina of Time on an M2 Pro (Amost-): 5 of 5 runs crashed before, 3 of 3
loaded the forest after.

Fixes seedds#1743

Co-authored-by: Eliran Zach <4012041+thehobbit85@users.noreply.github.com>
thomasruegsegger pushed a commit to thomasruegsegger/nxnative that referenced this pull request Nov 9, 2026
… and archived, the dynamic resolution that climbs on measured load and recycled render-target heaps, pipeline archive indexes, the -fwrapv, raw vector miss and mesh object fixes, the mist fix through frag_z_snap

- Our frag_z (d78e724, PR seedds#1742) gives way to upstream's frag_z_snap for the same mist specks: dmsh_build.py, its tests and notes are upstream's
- tex_mtl_: the GPU-only pool's heap (pool_heap_texture) first, then upstream's recycled heap of its own (rh_texture), then DM_NVN_RT_RECLAIM's free list, then a new texture
- dispatch keeps our (dp, next) signature; upstream's seedds#1755 body is the same
- The translator dlls, lock and chain header are upstream's here; they must be rebuilt from these sources (our ResolveRawRuns beside upstream's raw vector miss and follower-group fixes in GlobalToStorage.cs) before a build

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Aax1HnchwCweUDRNWjhHx9
@thehobbit85
thehobbit85 deleted the codex/dispatch-own-encoder branch November 10, 2026 03:36
Fasen24-AI added a commit to Fasen24-AI/swordboiMacNative that referenced this pull request Nov 10, 2026
…der)

Upstream pull requests applied in this commit (their authors' work, brought onto this tree with a three-way merge; the description is in each PR):
- seedds#1755 by thehobbit85 (closed): Prepare compute texture bindings before opening the dispatch encoder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400486024DEC000 A game nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants