Repository navigation
vk: a mesh program's stage buffer starts at zero (foliage shadow lines on AMD) - #475
Merged
Merged
Conversation
…s on AMD) The mesh epilogue hands Vulkan every vertex up to the program's maximum and the index bytes of every primitive below its count, from the map and attribute regions in workgroup memory. Workgroup memory holds whatever the last workgroup on that unit left, so a vertex or an index byte the game's program leaves unwritten was a leftover. NVIDIA's comes out clean in practice; AMD's does not. On an RX 7700S (Adrenalin 26.10.1, Hyrule Field) the 2400 x 2400 D32F shadow map around Link, drawn by the foliage's mesh programs 02ee95fbf9e0050d and b0b3b40244c71ec7, had hundreds of long thin triangles across the whole map (an RX_VK_DUMP_SAMPLED dump of 3427a1dbefd5e02c's shadow map), and the ground, the water and Link showed dark lines that changed every frame. With both regions zeroed in the prologue those triangles are gone: an unwritten primitive is (0,0,0), no area. Straight-line stores, each clamped into its array: the prologue's entry block is a phi predecessor, so no branch. RX_VK_MESH_NO_ZERO=1 restores the old behaviour. This changes the shader stamp, so the program cache starts again once. tests/test_mesh_stage_buffer_zero.py checks the prologue. Not fixed by this: inside the foliage patch the same shadow map still has long streaks across the patch on this GPU, so the striped pattern around Link remains (gone with RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d, b0b3b40244c71ec7). That is a separate problem in those programs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On an AMD laptop (Radeon RX 7700S 8 GB, Windows 11, Adrenalin 26.10.1) the ground, the water and Link showed dark lines in a square around Link in Hyrule Field, changing every frame. They were in the shadow mask (
3427a1dbefd5e02c), and anRX_VK_DUMP_SAMPLEDdump of the shadow map it samples (2400 x 2400 D32F, drawn only by the foliage's mesh programs02ee95fbf9e0050dandb0b3b40244c71ec7) showed hundreds of long thin triangles across the whole map.Why
MeshEpiloguehands Vulkan every vertex up to the program's maximum and the index bytes of every primitive below its count, from the map and attribute regions in workgroup memory. Workgroup memory holds whatever the last workgroup on that unit left, so a vertex or an index byte the game's program leaves unwritten is a leftover from another meshlet. On NVIDIA that memory comes out clean in practice; on AMD it does not.The change
EmitMeshProloguezeroes both regions in mesh programs, before the primitive count and its barrier. An unwritten primitive is then (0,0,0), with no area. The stores are straight-line and clamped into their arrays, because the prologue's entry block is a phi predecessor and must not branch.RX_VK_MESH_NO_ZERO=1restores the old behaviour. It changes the shader stamp, so the program cache starts again once.Result
The triangles across the whole shadow map are gone (same spot, before and after dumps). Not fixed by this: inside the foliage patch the same map still has long streaks on this GPU, so a striped pattern around Link remains. It goes away with
RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d,b0b3b40244c71ec7, which points at those two programs; that is a separate problem, reported in #302.Tested
device-lost-fixwith this change: the shadow map dumps above; all programs translate as before (same 57 failures, none of them mesh programs).tests/test_mesh_stage_buffer_zero.py(new, wired intosource-regressions.yml) checks the prologue: both regions zeroed before the count and the barrier, no branch, clamped stores, the switch. It fails onmainwithout this change.🤖 Generated with Claude Code