Repository navigation
fix(vk): tree-shadow flicker without the TAAU network: the shadow mask evened out over frames (rebase of #477) - #1581
Merged
hckrman101 merged 3 commits intoNov 11, 2026
Conversation
This was referenced Nov 10, 2026
rakefire-impala
force-pushed
the
fix/tree-shadow-flicker-history
branch
from
November 10, 2026 11:31
e37b328 to
936eeaa
Compare
rakefire-impala
force-pushed
the
fix/tree-shadow-flicker-history
branch
from
November 10, 2026 23:00
936eeaa to
c10a3b4
Compare
…e after our own pushes Tree shadows (OoT's 3427a1dbefd5e02c) flickered in fast thin streaks over sunlit ground and shadow edges while the camera stood still. The pass offsets each pixel's shadow-map lookup by a normal it reconstructs from a gather of the neighbouring depths; on grass blades and leaf edges those neighbours flip between the blade and the ground behind it with the sub-pixel camera shift, so pixels flip between lit and shadowed every frame (the console's TAAU averages that away). Right after that draw, its mask (G and B of an RGBA8 target, MIN blended) is blended with its own history (shaders/shadow_stab.comp). RX_VK_MASK_BLEND=<w> (default 0.85, 0: off), RX_VK_MASK_RADIUS=<r> (history bound, default 0: none; a 3x3 bound kept nothing, the streaks move as blocks). The TAAU and movie stand-ins push descriptors of their own layout without telling bind_descriptors' cache, so the next dispatch of the cached layout could skip its push and read the stand-in's bindings (found as a device lost after a save load once another compute pass was inserted after the network). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> [rebased on main 9447e1d: main now marks the cache stale after the stand-ins (descriptors_disturbed), so only the new pass does it here, through that helper] (cherry picked from commit 29ab07f)
…our own pushes Checks that the blend follows the tree-shadow pass (3427a1dbefd5e02c) only while RX_VK_MASK_BLEND is above 0, needs an RGBA8 storage mask and push descriptors and runs outside the render pass, and that every push of the stand-ins' or the mask's own layout marks bind_descriptors' cache stale (fails with one of the three marks removed).
…story The header still described the history as bounded by the 3x3 neighbourhood; the default is radius 0 (unbounded), as the comment at the bound and RX_VK_MASK_RADIUS's description say. Comment only: the SPIR-V is unchanged.
rakefire-impala
force-pushed
the
fix/tree-shadow-flicker-history
branch
from
November 10, 2026 23:00
c10a3b4 to
52ea5bf
Compare
Contributor
Author
|
Rebased on main One more change came with the rebase. Main now calls I also fixed your first nit: the header comment of |
This was referenced Nov 11, 2026
Merged
TechHammerBR
pushed a commit
to TechHammerBR/RecompZel
that referenced
this pull request
Nov 11, 2026
…ce lost near Hyrule Castle) (NotRexed#2374) **Release blocker for 2.1.1.** The owner's test of main crashed near Hyrule Castle: device lost after a ~2.5 s GPU hang on the mesh draw `b0b3b40244c71ec7`. In the checkpoints, the draw that follows it is OoT's tree-shadow pass `3427a1dbefd5e02c`, the one NotRexed#1581's mask blend (`mask_stabilize`) runs after. | build, the owner's castle save, native mesh | result | |---|---| | v2.1.0 (before NotRexed#1581) | 48,960 presents, no crash | | main `49f75dc8` | device lost at ~30,000, **twice** | | main, mesh through the compute path | 49,560, no crash (flashes) | | main, `RX_VK_MASK_BLEND=0` | **51,600, no crash** | Two of last night's route runs (`sun_nodefer`, `c8_on`) died the same way, on the same program. This makes the blend opt-in (`RX_VK_MASK_BLEND=<weight>`; the default was 0.85). Without the TAAU network, the tree-shadow streaks it evened out come back until the hang is found. `tests/test_tree_shadow_mask.py` is updated (passes). cc the author of NotRexed#1581: the hang is on the draw before your pass, so it may be the mask image's history or barrier rather than the blend itself. Logs are available. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
NotRexed
pushed a commit
that referenced
this pull request
Nov 12, 2026
…ols the GPU writes (#2154) > **Update (2026-11-11):** a second commit widens the fix to save reloads and logs the mirrors still given back. The pause fix below is tested in game; the second commit is not yet (see "Save reloads"). ## Problem Pause the game, leave it in the background for 1–2 minutes (another window or the Start menu in front), then come back and resume: for a few seconds the picture has cyan patches. On an RX 5500M (RDNA1, 4 GB, Windows 11, AMD driver) this happened on every resume. While the game is paused, most of its memory pools are not bound. After `kIdleEpochs` (600) submissions an unbound pool's VRAM mirror is idle, and `mirrors_give_back` gives it back once it has been idle past `kLongIdle` (6000 submissions) or when a wanted mirror does not fit. In one such pause the running count in the log went from 35 to 51 (`51 mirrors given back (1136 MB in all)`). At the pool's next binding, `mirror_make` makes a new mirror from guest memory (plus the CPU's newer writes from the captures). Most of what the GPU writes into a mirror is copied back to guest memory: a written binding queues a write-back. But some of it is never copied back, by design, and is noted only as a hazard by `memory_note_gpu_write`: what a shader writes through a bare entry's guessed range, the pointer table's range or a wide binding. (Copying the whole guessed range back overwrote the CPU's own data next to it.) That data was in the old mirror alone. The new mirror starts without it, and the GPU goes on from stale data until it has rebuilt it, which is the few seconds of cyan. ## Change - `Pool::gpu_only`: set by `memory_note_gpu_write` on every imported pool its range reaches. - `mirrors_give_back` leaves an idle `gpu_only` pool's mirror alone. Its other idle mirrors go as before. - A pool that takes others in (`pool_lookup`'s merge) keeps their mark: their mirrors' contents come with them. - The give-back's log line also says how many MB of idle mirrors are kept this way. - Second commit: `Pool::gpu_written`, set by `gpu_buffer_bind` for a written binding of the mirror, carried over a merge like `gpu_only`, and the idle give-back keeps those mirrors too. - Second commit: every mirror still given back is logged (the first 200), with why (retired, retired and not inherited in time, idle, or idle for room), how many submissions since its sync, and the pool's marks. A pool that is refused a mirror for the budget meanwhile is read in system memory, which is slower but not wrong. Retired pools are unchanged: their GPU contents already go to the pool that took them in (`inherit_pending`). ## Tested RX 5500M, Windows 11, AMD driver, main `92907f7` with #1581 and #2125, same settings throughout. Each pause was 1–2 minutes with the game in the background: | | cyan after resume | mirrors given back during the run | |---|---|---| | main (`92907f7` + #1581 + #2125) | every resume (tried at Auto), for a few seconds | 16 in one pause | | `RX_VK_MIRROR_KEEP=1` (no give-back at all) | none, one pause at Auto and one at 33% Render resolution | 0 | | this PR | none, one pause at Auto and one at 33% | 48 in the session; 112 MB of idle mirrors kept at the last count | A very short black flash right on resume is still there at both resolutions. At Auto it has been there since the first builds on this card, including the `RX_VK_MIRROR_KEEP=1` run, so it looks separate. ## Save reloads With the first commit, cyan came back after reloading a save within a run (Hyrule Castle, Hyrule Field), in play and over the Start menu, with no pause and no `not imported` pools in the log. (That run also had `RX_VK_FIFO=1`, which turned out not to matter.) Against `RX_VK_MIRROR_KEEP=1`, which never gives a mirror back: | | run 1 | run 2 | |---|---|---| | first commit only | cyan after each reload (castle, field) | — | | `RX_VK_MIRROR_KEEP=1` | three reloads, no cyan over the Start menu | fewer flashes, some on the third reload | So mirrors given back explain most of the cyan after a reload, but not all of it. The first commit keeps only pools the GPU wrote past the write-back. The pools whose mirrors still went are the ones the GPU writes through a normal binding, which the second commit keeps too. **Not tested in game yet:** the second commit. I haven't played a build with it, so I don't know yet whether it matches `RX_VK_MIRROR_KEEP=1` or what it costs in VRAM. Its log line is there to answer both, and to show which mirrors are still given back before any flash that's left. ## Tests `tests/test_mirror_gpu_only.py` (new) reads the source and checks four things: `memory_note_gpu_write` marks the pools it reaches, `gpu_buffer_bind` marks a pool written through its mirror, the idle give-back leaves marked pools, and a merge keeps both marks. It fails on current main and passes with this change. `nx2/test_import_windows.py` builds `pool_lookup` against a `Pool` of its own, which now has both fields. `tests/run_all.py` on main `4d68bc83` with this branch merged: 311 passed. The 6 that fail are the same 6 that fail on plain main on this PC. Related: #1005 (cyan and UI flashes after focus loss), #2125.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebase of #477 by @imbahzor onto current main, as asked when it was closed ("please rebase on current main and reopen"). The commit is theirs, cherry-picked with
-x; the only changes are the conflict resolution and the push-descriptor mark, which now goes through main'sdescriptors_disturbed()(see Change).Problem
Over sunlit ground and along shadow edges near trees, shadows flicker constantly in thin streaks, even with the camera still. This happens on GPUs where the gameplay TAAU network is left out ("a network left out on this card" in the log, for example an RX 5500M), so the bilinear stand-in runs instead.
The tree-shadow pass
3427a1dbefd5e02cis a full-screen draw. It rebuilds a normal from anImageGatherof the neighbouring depths and offsets its shadow-map lookup by that normal. On grass blades and leaf edges, those neighbours flip between the blade and the ground behind it as the camera shifts by a sub-pixel each frame, so the pixels flip between lit and shadowed. On the console, the TAAU network averages that away.Change
shaders/shadow_stab.comp: two history images used in turn, then G and B copied back.RX_VK_MASK_BLEND=<w>: the weight, default 0.85;0turns it off.RX_VK_MASK_RADIUS=<r>: limits the history to a (2r+1)² neighbourhood. Default 0 (no limit), because a 3×3 limit kept nothing.descriptors_disturbed()right after the push. Without that, the next dispatch with the cached layout skipped its push and read the pass's bindings, which caused a device lost after a save load in vk: tree-shadow flicker evened out; push-descriptor cache stale after the stand-ins' pushes #477's testing. vk: tree-shadow flicker evened out; push-descriptor cache stale after the stand-ins' pushes #477 also markedmovie_upscaleandtaau_stand_in; main now does that itself.Tests
tests/test_tree_shadow_mask.py(new) checks that the blend follows the tree-shadow pass only whileRX_VK_MASK_BLENDis above 0, needs an RGBA8 storage mask and push descriptors, runs outside the render pass, and that every push of the stand-ins' or the mask's own layout reachesdescriptors_disturbed()before its function returns. It fails with any one of those three calls removed.Tested
779bdcf): played on an RX 5500M (RDNA1, 4 GB, Windows 11, AMD driver; main779bdcfwith fix(vk): rainbow block-noise textures on low-VRAM cards: a failed BC3 upload is tried again, not drawn as noise #1505–fix(vk): texture noise when system RAM runs low: a staging arena of the size needed when 64 MB can't be pinned #1514) in Hyrule Field. The constant shadow flicker is gone. The log showstree-shadow mask blend on #266 768x432, weight 0.85; the pass runs once a frame there.88b554ewith fix(vk): mesh fallback flicker: wait for the last draw before clearing the shared buffers (#1383 review) #1509–fix(vk): coloured specks on fallback foliage (AMD): the generated vertex stage's outputs start at (0,0,0,1) #1513 and this PR): no flicker, specks or tree-shadow problems noticed in a 23-minute session; see their comment on fix(vk): missing floor grass without mesh shaders: indirect task draws ran no mesh workgroups #1510.9447e1d: one conflict, the renamedskin_checkcall next to the new guard. Compile check clean.tests/run_all.py: 291 passed; the 6 that fail are the same 6 that fail on plain main on this PC, so no new failures. Not played again: the only code change from the played version is the push-descriptor mark going throughdescriptors_disturbed().