Skip to content

vk: inline the global-load helper in task and mesh stages (the floating trees, #994) - #1334

Merged
shayderrr merged 1 commit into
NotRexed:mainfrom
blushOFF:fix/task-mesh-load-inline
Nov 9, 2026
Merged

shayderrr merged 1 commit into
NotRexed:mainfrom
blushOFF:fix/task-mesh-load-inline

Conversation

@blushOFF

@blushOFF blushOFF commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the wall of floating trees from #994 at the spots I could test (Kakariko's entrance, the Hyrule Castle cliff), on an RTX 3070 / Windows 11.

Cause. The tree task stage 3cb7682790143836 computes each visible cell's tree count as min(offs[e+1] - offs[e], 256) from the placement file. Traced per load (details in #994): the offs[e+1] load went to file + (cells[e] + 9 + e) * 4 instead of file + (n + 9 + e) * 4, the result of the previous call to the global-load helper where the one before it belongs. The address is outside every buffer, the load returns 0, and every cell is drawn with 256 trees, the following cells' trees placed in it. The IR and the emitted SPIR-V are both right; the compiled shader is not, while the helper is DontInline.

Change. The global-load helper is emitted Inline for the task and mesh stages. The compute stages keep DontInline: their culling programs have hundreds of global loads, and inlining was what made NVIDIA's compiler take 41 s (the comment above define_load). EDEN_NX2_GM_INLINE=0 brings the old behaviour back; as an EDEN_ switch it is part of the program cache's stamp.

Tested (RTX 3070, Windows 11, NVIDIA driver 610.88 as Vulkan reports it):

  • Kakariko's entrance, same scripted camera: the pair's mesh stage went from 1184 threads / 2082 primitives to 160 / 78 (the file holds 81 trees for the frame's 10 cells), 0 global-memory misses instead of 3200 a frame, no tree left in the sky. Also with an empty program cache.
  • Hyrule Castle cliff view, with and without the switch: the trees above the cliff are gone with it.
  • A teleport tour of 15 areas (Kokiri Forest to Zora's Fountain): no device lost, no pipeline refused.

Not tested: AMD, Intel, Linux; the open-field view toward Death Mountain; frame rate in a proper A/B (a first single run at Kakariko gave about 7 fps more, which I would not rely on). @Nitnek offered to run it at the castle cliff on another driver version.

No game data in this change.

NVIDIA's compiler gave the tree task 3cb76827 a wrong address for its
offs[e+1] load while the helper was DontInline: every visible cell drew
256 trees, the floating tree wall. Compute stages keep DontInline
(compile time). EDEN_NX2_GM_INLINE=0: off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMVHfecF8UiqmMfKTDgxgH
@Nitnek

Nitnek commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Tested at the Hyrule Castle cliff view, as offered: RTX 3070 8 GB, Windows 11, main 7dbdcdc + this PR. The setup:

  • a scripted run loads the Hyrule Castle autosave and turns the camera to the cliff;
  • both runs reach the same view;
  • EDEN_NX2_GM_INLINE=0 is the "before".
3cb7682790143836 mesh threads its fragments on screen
EDEN_NX2_GM_INLINE=0 39392 2.1M the wall of spruces in the sky over the cliff
this PR 6944 267k a few spruces standing on the hill, no wall

The pair's task groups barely change (164 vs 208 from b612216258832b9c's list); what each group emits drops ~6x. That matches the per-cell count no longer coming out at the 256 clamp.

  • No device lost, no failed programs. Frame rate is the same in both runs (~50 fps over the load and the turn).
  • The floating-tree wall is gone at this spot too, a second spot on a different driver than yours.

Edit: also checked by hand in Kakariko with this PR and #1338 on the same build (the latter for the foliage that went bare at mid distance, #1151). Looking from the village entrance toward Hyrule Field, the render is as expected: no floating trees, and the spruces keep their needles.

@shayderrr

Copy link
Copy Markdown
Collaborator

Thank you!

@shayderrr
shayderrr merged commit 75a676e into NotRexed:main Nov 9, 2026
10 checks passed
@msxdan

msxdan commented Nov 9, 2026 •

Copy link
Copy Markdown

Tested on Linux: the wall of trees is gone, same numbers with and without the switch as on Windows

The tree pair 3cb7682790143836 / 4f0c0b40ab3a32cd, canyon save:

mesh threads primitives fragments (3cb76827)
EDEN_NX2_GM_INLINE=0 32608 64878 5.51M
this PR (default) 9760 3712 0.31M
  • With the PR the conifer wall on the plateau is gone and the far mountains show. From the Too many Trees #1306 spot the view now matches the reporter's reference picture: the castle, the hills, single trees.
  • No device lost in 5 cold starts. Preparing 38809 programs with an empty program cache: 33.7 s with the PR, 32.9 s with the switch at 0.
  • 60 fps in both (frame-limited, so no frame rate comparison).

Not tested: AMD, a long play session, Kakariko and the castle cliff.

No game data attached.

@NotRexed NotRexed mentioned this pull request Nov 10, 2026
3 tasks done
NotRexed pushed a commit that referenced this pull request Nov 10, 2026
On the tested NVIDIA driver, the compiled Deku Tree web vertex shader
uses the grid origin where the SPIR-V calls for the grid dimensions.
The resulting coarse cell index is out of range and the web is dark.

Request inlining for the 128-bit load helper in that vertex shader.
Use the existing EDEN_NX2_GM_INLINE switch and cover the helper's
function controls in the game-free SPIR-V fixture.

Refs #455 and #1002. Related workaround: #1334.

Assisted-by: OpenAI Codex

Co-authored-by: Codex <codex@openai.com>
@renannpaiva

renannpaiva commented Nov 10, 2026 •

Copy link
Copy Markdown
image

Ryzen 7 9800x3d RTX 5070 32gb

Everything is perfect where there used to be issues; however, it gives the impression that it’s on the verge of freezing whenever VRAM usage hits a high percentage. The game feels like it’s running twice as fast, haha! I’ll play a bit more so I can provide some precise feedback!

@NotRexed

Joey-Magnus pushed a commit to Joey-Magnus/Hookshot that referenced this pull request Nov 10, 2026
…otRexed#1338): row removed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Joey-Magnus pushed a commit to Joey-Magnus/Hookshot that referenced this pull request Nov 10, 2026
…otRexed#1338): row removed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nowhere02 added a commit to Eglantine-Community/Eglantine that referenced this pull request Nov 13, 2026
Inline global loads in emulated task and mesh stages on Pascal GPUs. This fixes floating trees in Ocarina of Time on the tested GTX 1060.

Based on NotRexed/Hookshot#1334.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants