Skip to content

fix(vk): build task and mesh programs with the optimizer on RADV - #1965

Merged
hckrman101 merged 2 commits into
NotRexed:mainfrom
addisonbair:fix/radv-mesh-optimizer
Nov 11, 2026
Merged

hckrman101 merged 2 commits into
NotRexed:mainfrom
addisonbair:fix/radv-mesh-optimizer

Conversation

@addisonbair

@addisonbair addisonbair commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

On Linux with Mesa's RADV driver, the GPU resets while the save loads at the Great Deku Tree on the Steam Deck (and RDNA2 desktop cards) unless players find RX_VK_MESH_OPT=1 in an issue thread. The cause is the PR 430 rule that builds task and mesh programs without the driver's optimizer on every AMD GPU. That rule was made and tested on AMD's Windows driver; on RADV, the unoptimized foliage task program runs long enough to trip the GPU timeout.

What changed

build_graphics (gpu/vk/vk_draw.cpp) now applies the no-optimizer rule to AMD's own drivers only, not to VK_DRIVER_ID_MESA_RADV. On RADV, task and mesh programs are built with the optimizer, like every other pipeline. AMD on Windows is unchanged, and RX_VK_MESH_OPT=1 still forces the optimizer on any driver. The check reads dev.driver_id, which create_device already stores. tests/test_radv_mesh_optimizer.py checks that RADV stays out of the rule and the override stays.

What it doesn't fix

This fixes the reset while the save loads. Other RADV hangs remain, and the optimizer doesn't cure them: rere225 (RX 6900 XT) still hangs entering the Great Deku Tree with RX_VK_MESH_OPT=1, and KewinBR (Deck LCD) still hangs in Lon Lon Ranch's open view with it (RX_VK_FORCE_MESH_FALLBACK=1 played). Both runs had the optimizer on already, so neither is a regression from this change.

Testing

Steam Deck OLED: RADV VANGOGH, Mesa 26.2.0-devel, SteamOS 3.9.5. v2.0.14 with only this change, Kokiri Forest, loading the same save each run, RX_VK_MESH_OPT not set:

Renderer Result
v2.0.14 as released ring gfx_0.0.0 timeout and MODE2 reset while the save loads. The log says "task and mesh programs built without the driver's optimizer"
v2.0.14 + this change Plays with no reset. That line is not logged. 15.4 fps (the same as RX_VK_MESH_OPT=1 on the release)

RX_VK_MESH_OPT=1 on the release has also run about 30 minutes across five launches with no reset on this Deck. @manianac reports the same on an RX 6800 XT, RADV 26.2.6 (#1000).

tests/run_all.py (Ubuntu 22.04 with g++-13, as in CI): 321 passed, 13 failed, 11 skipped, both on main and with this change: the same 13 fail without it (tools missing in my container), none new.

Related: #1000, #817, #430.

🤖 Generated with Claude Code

@omgitsgela

Copy link
Copy Markdown

Additional Steam Deck LCD measurements: 3601726 and pinned main

Hardware in the test logs: Steam Deck LCD, RADV VANGOGH, driver 0x6801063.

Legacy 3601726 baseline: enabling RX_VK_MESH_OPT=1 on the original renderer let the opening sequence reach player control, where earlier trials had failed. It did not resolve the later color/particle defects or camera hitches. Our subsequently tested legacy branch combines the RADV optimizer policy with upstream-derived DMA-buf imports, allocator-aware texture squeeze, and smaller/growing recording streams. It is not an optimizer-only performance result.

In a matched legacy host/renderer/save/settings DMA A/B, settled room samples rose from roughly 20 to 25–26 fps and queue-submit CPU timing fell from roughly 21 to 2 ms. That gain belongs to the DMA change, not this PR. The combined legacy build was subjectively playable during Kokiri traversal. A separate 30-minute mixed play/menu/view observation had median 20.0 fps and range 10.2–27.7 in five-second VI reports, with no captured kernel timeout. This is not locked-25/30-fps or continuous-route acceptance.

Pinned main ebbba2ea868458cde654a008e481f474841f0dff: official nightly toolchain and matching packaged renderer, FSR 3.1 Performance, frame generation off, isolated copies of the same Kokiri save:

  • Default optimizer policy: the log says task/mesh optimization is disabled on AMD; loading Kokiri caused gfx_0.0.0 timeout, a GPU reset, and VK_ERROR_DEVICE_LOST.
  • Same binaries/options with only RX_VK_MESH_OPT=1 added: Kokiri loaded and rendered terrain/characters; no kernel timeout during the captured gameplay window. The sampled docked Saria view was about 16 fps.
  • A subsequent Handheld-profile sample of the comparable view was about 18–19 fps. Red particles remain; full intro/audio-sync and long-route stability are not cleared.

The reset and cache-stamped option change prevented a cache-matched performance A/B, so I am treating this as additional hardware corroboration for the RADV policy, not proof of a universal speedup or a complete correctness fix.

Public legacy source milestone (not current-main tested source patches): omgitsgela@8d1347f . No game files, saves, cache files, or raw game-data logs attached.

@NotRexed

Copy link
Copy Markdown
Owner

Note from the duplicate #2311 (closed): on the Steam Deck (Van Gogh), the foliage culling mesh program 89ca750f9852b7ca built without the optimizer wedged the GPU about 30 s after a save loaded in Kokiri Forest, every run (ring gfx timeout, then a reset); built with it, the forest plays.

@addisonbair

Copy link
Copy Markdown
Contributor Author

Thanks. All three are in c81cf97:

  • The rule reads dev.driver_id. The second properties query is gone.
  • The comment and description now say this fixes the reset while the save loads, and that other RADV hangs remain. rere225's and KewinBR's reports are listed as not covered.
  • New tests/test_radv_mesh_optimizer.py checks that RADV stays excluded and RX_VK_MESH_OPT stays. I checked that it fails if the RADV exclusion is removed.

The renderer builds clean. I don't have before/after shader-prep times on the Deck yet.

🤖 Generated with Claude Code

@Joey-Magnus Joey-Magnus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still needed: #2311 was closed as a duplicate of this PR, not merged, and main still builds task/mesh programs without the optimizer on every AMD driver (gpu/vk/vk_draw.cpp:2187). Your original commit merges cleanly and run_all passes on it (376 passed, Linux).

The new commit c81cf97 conflicts with main, though. Since #2042, main reads the override as env_on("RX_VK_MESH_OPT") (vk_draw.cpp:2192), and the branch still has getenv(...) != nullptr. Either way of resolving it fails a test. If you keep getenv, tests/test_env_on.py flags RX_VK_MESH_OPT (gpu/vk/vk_draw.cpp:2191). If you keep env_on, your new tests/test_radv_mesh_optimizer.py fails with "RX_VK_MESH_OPT=1 no longer builds them with the optimizer", because it looks for the getenv text. I tried both. The fix is to rebase, keep your amd line with dev.driver_id (create_device always sets it), keep main's env_on, and change the test's needle to env_on("RX_VK_MESH_OPT"). Using dev.driver_id and the softer comment both look right to me. Since the RC is today, this is worth doing quickly.

addisonbair and others added 2 commits November 11, 2026 22:43
The PR 430 rule builds task and mesh programs without the driver's
optimizer on every AMD GPU. It was made and tested on AMD's Windows
driver. On Mesa's RADV the unoptimized foliage task program runs long
enough to reset the GPU at the Great Deku Tree (Steam Deck OLED,
RX 6800 XT; NotRexed#1000), and the optimized one plays. Keep the rule for
AMD's own drivers and leave RADV to build them as everything else.
RX_VK_MESH_OPT=1 still forces the optimizer on any driver.

Assisted-by: Claude Opus 5.5 (Anthropic)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ich hang it fixes, and has a source check

The rule reads the driver ID create_device already stored instead of a second properties query. The comment now says it
fixes the GPU reset while the save loads at the Great Deku Tree, and that other RADV hangs remain (issue 1000).
tests/test_radv_mesh_optimizer.py checks that RADV is left out of the rule and that RX_VK_MESH_OPT=1 still overrides it.

Assisted-by: Claude Opus 5.5 (Anthropic)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@addisonbair
addisonbair force-pushed the fix/radv-mesh-optimizer branch from c81cf97 to 55c48f7 Compare November 11, 2026 22:47
@hckrman101
hckrman101 merged commit eed4d91 into NotRexed:main Nov 11, 2026
1 check passed
@mooki000 mooki000 mentioned this pull request Nov 12, 2026
3 tasks done
dazemc added a commit to dazemc/Hookshot-rs that referenced this pull request Nov 12, 2026
Ports of upstream NotRexed#1965 and NotRexed#2215. On RADV the task and mesh programs
built without the driver's optimizer ran long enough to reset the GPU
while the Great Deku Tree save loads; the AMD no-optimizer rule now
excludes Mesa's RADV, and RX_VK_MESH_OPT still forces it on. The default
task+mesh fallback also covers 0x73af (RX 6900 XT XTXH) and 0x73a5
(RX 6950 XT), the same Navi 21 chip.
dazemc added a commit to dazemc/Hookshot-rs that referenced this pull request Nov 12, 2026
Record NotRexed#1965 and NotRexed#2215 and the green b961fc4; rs.7 is built from it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants