Skip to content

fix(vk): texture noise when system RAM runs low: a staging arena of the size needed when 64 MB can't be pinned - #1514

Merged
NotRexed merged 1 commit into
NotRexed:mainfrom
rakefire-impala:fix/texture-noise-low-ram-staging-fallback
Nov 10, 2026
Merged

NotRexed merged 1 commit into
NotRexed:mainfrom
rakefire-impala:fix/texture-noise-low-ram-staging-fallback

Conversation

@rakefire-impala

@rakefire-impala rakefire-impala commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Without resizable BAR, a texture staging arena is 64 MB of system memory the driver must pin. On a 16 GB laptop with RAM nearly gone, no 64 MB arena could be made, even for a 36 KB upload, so texture uploads failed and drew as noise. This falls back to an arena of the size the upload needs.

Related: #993. Part of splitting up #1023.

Problem

On an RX 5500M laptop (4 GB VRAM, 16 GB RAM), the game's pools were imported (about 2 GB pinned), only 2.9 GB of RAM was available, and Windows was paging. texture_frame had given the idle arenas back, as it does after 30 s on a card of 6 GB or less, and making a new 64 MB one failed:

vk: staging: an arena of 64 MB for 36 KB could not be made (1 arenas)
vk: memory heaps: heap 0 (device): ... used 2972 of 2972 MB; heap 1 (system): ... used 3661 of 7206 MB; heap 2 (device): ... used 198 of 198 MB;
vk: BC3 texture #... its upload failed (no staging memory for its ASTC blocks): tried again in 30 frames

The system heap's budget had room, but pinning a new 64 MB block failed under RAM pressure. Every upload failed until an arena came free.

Change

  • When a 64 MB arena can't be made, staging() tries one of the size the upload needs, rounded up to 1 MB. Both outcomes are logged once in a while: vk: staging: no 64 MB arena could be made (N kept); one of the size asked for instead, or ...; the upload waits.
  • A buffer that is made but not mapped is now destroyed instead of leaked.
  • texture_frame gives the small arenas back as it does the others. Upstream's arena timing log is kept.

Who it affects

Only a machine where a 64 MB staging arena can't be pinned: low free RAM, no resizable BAR. Everywhere else staging() makes the same 64 MB arenas as before, and the new path never runs.

Tests

tests/test_staging_arena_fallback.py (new) checks the order of the attempts, the cleanup and the log line. It fails on main without this change.

Tested on

Tested on RX 5500M (RDNA1, 4 GB VRAM, 16 GB RAM), Windows 11, AMD driver; Hookshot main 0c412ad (v2.0.11-383) with these PRs, 2026-11-10
Result In one session, no 64 MB arena could be made ...; one of the size asked for instead 8 times, 0 failed texture uploads, no noise

…asked for

Without resizable BAR a staging arena is new system memory the driver
pins. On a 16 GB laptop (RX 5500M, 4 GB) with the game's pools imported
and 2.9 GB of RAM available, Windows paging, no 64 MB arena could be made
even for a 36 KB upload: texture_frame had given the idle arenas back,
and every texture upload failed until one was free again. BC3 textures
were drawn as noise meanwhile (a failed upload marks the texture
uploaded).

staging() now tries an arena of the size the upload asks for (rounded to
1 MB) when a 64 MB one cannot be made, and logs it ("vk: staging: no 64 MB
arena could be made"). A buffer made but not mapped is destroyed, not
leaked. texture_frame gives the small arenas back as it does the others.

tests/test_staging_arena_fallback.py checks the order, the cleanup and the
log line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants