Repository navigation
vk: RX_VK_MESH_NOCULL, the Steam Deck's task shader hang without leaving models out (#817) - #1300
Merged
Merged
Conversation
…ing models out (issue 817) OoT's meshlet culling task shader 2c56dc1006942f74 walks a coarse depth grid cell by cell for every meshlet; on RDNA 2 (Steam Deck) one draw runs past the kernel's 10 s and the GPU is reset. Its answer is visible = (result & 1) | (c3[2320] & 4), bit 4 being the game's own "draw everything". With the switch set, that comparison becomes true and dead code elimination removes the grid walk: every meshlet is drawn, none is left out (RX_VK_SKIP_CODE leaves the programs out, and their models with them). Off unless set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new renderer switch,
RX_VK_MESH_NOCULL=<task shader code key prefix,...>, turns off the culling inside the task shaders it names. On a Steam Deck,RX_VK_MESH_NOCULL=2c56dc1006942f74avoids the GPU hang of #817 with every model drawn, whereRX_VK_SKIP_CODE=2c56dc1006942f74(#1165) avoids it by leaving out the ~30 programs that use that task shader, and the houses, platforms and rupees they draw.gpu/vk/vk_shader.cpp:mesh_nocull()runs on the translated task program, beforedump_ir. It looks forINotEqual(BitwiseOr32(x, BitwiseAnd32(4, GetCbufU32(3, 2320))), 0), replaces its uses withtrueand runsDeadCodeEliminationPass. If the pattern is not found, the program is left as it is and the log says so. It logs the first 8 task shaders it met.readme.md: the Deck troubleshooting row now points to this switch first, and the switch is described under Switches for testing, next toRX_VK_SKIP_CODE.Why it works
Task shader
2c56dc1006942f74projects each meshlet's bounding sphere onto a grid ofc3[3088]xc3[3092]cells and, for every covered cell, reads four words fromc3[3120 + cell * 16]by a dynamic index (two nested loops). On RDNA 2 some draws run for seconds, and past 10 s the kernel resets the ring. The shader's answer is:Bit 4 of
c3[2320]is the game's own "draw everything" switch. Making that comparison true lets dead code elimination take the grid walk with it: no loops are left, every meshlet goes to the mesh stage, and the rasterizer clips what is off screen. It costs some GPU time on cards that cull quickly, so it is off unless set. The name is underRX_VK_MESH, so the program cache starts again when it changes.Tested
Steam Deck (RADV VANGOGH), SteamOS, Game Mode,
HOS_HANDHELD=1,RX_VK_MESH_OPT=1, the memory caps from #817, with the same change on by default in a local build:RX_VK_SKIP_CODEhides are all there.tests/run_all.pygives the same results as main without the change. I did not run this exact build on the Deck: the runs above are the same function, on by default.The flashing on integrated GPUs (#1186, #1126) is a separate problem and is not touched here.
Generated with Claude Code