[corentinbonjour]

The frame is bound by the Vulkan replay's CPU time on the game's render thread (submit 23-28 ms for 2300-3000 draws against ~12 ms of GPU on an RTX 5070 Laptop / 275HX), and every draw searches three thread-local std::unordered_maps: a modulo by a prime and a node pointer chase each time, often a cache miss once the draw's other work has gone through the cache.

What changed

• gpu/vk/flat_map.h: FlatMap<K, V>, an integer-keyed table in one array (open addressing, linear probing, at most 3/4 full, slot from a multiplicative hash). It has only what these caches use: find/end, operator[], size, clear. No single-key erase; clear keeps a small array and gives back one over 4096 slots.
• Three maps now use it: handle_hits (texture_view_of), graphics_pipeline's mine (vk_draw.cpp), and picks (program_for, vk_shader.cpp). texture_answer's cache was already a flat array by slot on main, so it is left as it is.
• HandleHit is reordered, and tex_epoch is now uint32_t (the type texture_epoch() returns). The struct drops from 48 to 40 bytes and a table slot from 64 to 56.
• The handle_hits clear threshold (more than 65536 entries, checked once a submission) and the picks one are unchanged, so entries are kept or dropped exactly as before.

Measurements

Micro-benchmark, llvm-mingw clang 23, -O2 -march=x86-64-v3, on the 275HX. One draw does what the replay does: a picks lookup, a mine lookup and 6 handle_hits lookups with the version/generation compare. Draws pick from N materials on a skewed distribution. std::unordered_map (with the old HandleHit) and FlatMap alternate within each process, 21 repetitions, two processes per configuration. ns per draw, median (p25-p75), at the 3/4 load limit:

materials, evicted between draws	std	flat	flat/std
1000, none	146 (143-149), 173 (159-189)	104 (103-105), 120 (113-129)	0.69-0.71
1000, 256 KB	218 (186-387), 170 (165-194)	156 (116-220), 132 (117-155)	0.71-0.78
4000, 256 KB	242 (202-296), 371 (303-742)	162 (143-268), 235 (189-449)	0.63-0.67

The first version, at most half full, measured 0.53-0.70; 3/4 trades a little of that for half the memory.

In the game

Two A/B runs by reviewers, each swapping only hookshot_vk.dll:

• Ryzen 5 5600X, RTX 3060 Ti (klikso), Hyrule Field, ~1,200 draws a frame, three launches per arm: replay submit 10.03 -> 9.62 ms a frame (-4%), every run with the PR under every run without; process CPU 54.7 -> 52.7 ms a frame. Castle Town Market was too noisy on that card to read.
• i9-13900K, RTX 4090 (Jiro131), Kokiri Forest, 1,000-4,000 draws, uncapped: the graphics_pipeline lookup is 25-30% cheaper in every run and every draw-count bin (about 0.08 ms a frame at 2,500 draws), picks about 0.07 ms cheaper. The total submit change is within run-to-run noise on that CPU.

No crash, device loss or picture difference in either set of runs. The gain is a small share of the submit on a fast CPU and shows more on a mid-range one.

Memory of handle_hits per recording thread (bytes allocated):

entries	std	flat
4000	0.32 MB	0.44 MB
32768	2.64 MB	3.50 MB
65537 (the clear threshold)	5.28 MB	7.00 MB

A clear at that threshold now gives the array back, so the peak is not kept.

Testing

• New tests/test_flat_map.py, stdlib only, run by run_all with --cxx:
  • Map level: a random fuzz of FlatMap against std::unordered_map, 1.1M operations. It covers the replay's key shapes (handle keys, hashes, dense keys that share low bits), the replay's value types, growth and whole clears.
  • Code level: HandleHit, handle_hits and texture_view_of are cut out of vk_draw.cpp and built twice, once over each map. Both copies get the same 300k calls while slots are registered anew, textures destroyed and epochs pass. The views, levels, textures, and the stub calls and their order must match.
  • It also asserts that the three declarations use FlatMap. It fails on an untouched main, and it catches a FlatMap that drops an entry while growing.
• vk_draw.cpp and vk_shader.cpp compile with the build's renderer flags (llvm-mingw, -O2).
• python -I -B tests/run_all.py, with llvm-mingw on PATH and RX_CPM set: 434 passed, 24 failed, 38 skipped. That is the same set of failing checks as an untouched origin/main worktree (missing tools on this PC). The only differences were test_gpu_hang_watch and test_setup_tar_diagnostics, one in each run under 24-way load; both pass alone in both trees.

Notes

• Memory: at handle_hits' 65536-entry clear threshold the table is 7 MB against about 5 MB for the node map, per recording thread, and the clear gives it back. The threshold is unchanged, so entries are kept or dropped as before.
• The change touches vk_shader.cpp, which is part of HOOKSHOT_SHADER_STAMP, so players' program caches are translated again once after updating, as with any vk_shader.cpp change.
• perf(vk): cheaper replay: uniform-only descriptor rebuilds and flat program picks #2644 also edits picks in program_for (what a pick keeps) and lists "a flat table for the picks" as later work: this is that table, for picks and the two other per-draw maps. The two touch neighbouring lines, so whichever lands second needs a small rebase.
• An insert can move every entry. None of the three call sites holds a pointer across an insert: picks[key] is the only insert in program_for, and mine/handle_hits insert after their last use of the lookup result.

[Joey-Magnus]

I merged this into current main and ran tests/test_flat_map.py with clang++ and g++, and the vk/shader subset of run_all. All pass, and it still passes merged together with #2644. On reference stability: in program_for, Pick& pk = picks[key] is held across program_pick(), but program_pick never reaches picks or program_for, so no insert happens in between. In graphics_pipeline and texture_view_of, the insert comes after the last use. tex_epoch as uint32_t matches texture_epoch(). Two things before merging:

• Performance: the 0.3-0.6 ms a frame is extrapolated from a micro-benchmark, and the description says it hasn't been measured in the game. Please add an A/B in Ocarina (replay submit ms or frame time, same scene) before this goes in as a perf change.
• Memory: at the 65536 clear threshold (vk_draw.cpp:4721), handle_hits goes from about 5 MB to 14 MB per recording thread, and FlatMap::clear (flat_map.h:38) keeps the array for good. That is per thread, so it multiplies with RX_VK_PARALLEL. A 3/4 load factor, or dropping the array on clear when it is far above the size, would keep it closer to what main has.

[corentinbonjour]

Thanks for checking the reference lifetimes. On memory: FlatMap now fills up to 3/4 and its clear gives back an array over 4096 slots, so handle_hits at the 65536 threshold is 7 MB per recording thread instead of 14 (main's node map: about 5.3 MB), and it does not keep that after the clear. Re-measured at 3/4: flat/std 0.63-0.78 across the same configurations (it was 0.53-0.70 half full). tests/test_flat_map.py still passes, the description has the new numbers.

[klikso]

An in-game measurement on a slower desktop CPU: the replay's submit is ~0.4 ms a frame shorter in Hyrule Field in all three pairs; the Market was too noisy on this card to say.

Windows 11, Ryzen 5 5600X (6 cores, 12 threads), RTX 3060 Ti 8 GB (driver 32.0.16.2114), output 2560x1440, DLSS model K. main ba005ca with #2711, #2731, #2685, #2709, #2825 and #2826 merged locally, with and without this PR on top. Two renderer DLLs built from the two trees and swapped between launches (each launch prepares its shaders again first), arms alternated, three launches each. RX_VK_FRAMEGEN=0, HOS_FPS_MAX=240. CPU is the process's user + kernel time over the window divided by the frames shown.

Hyrule Field child, 30 s of the camera turning (~1,200 draws a frame):

	fps mean	replay submit a frame	CPU a frame	GPU a frame
base	66.3 / 67.6 / 67.4	10.23 / 10.06 / 9.80 ms	55.1 / 54.3 / 54.8 ms	7.9-8.1 ms
this PR	68.3 / 66.7 / 68.5	9.66 / 9.72 / 9.49 ms	53.0 / 52.3 / 53.0 ms	7.9 ms

Submit is 10.03 -> 9.62 ms on the means (-4%) and every run with the PR is under every run without; CPU a frame 54.7 -> 52.7 ms. The fps difference (+0.8) is inside the spread.

Castle Town Market entered from the field, 40 s (1,500-3,000 draws): base 45.9 / 49.4 / 34.2 fps, this PR 51.9 / 46.4 / 47.7. One base launch landed in this card's slow state (GPU 14.7 ms a frame against 7.6-7.8 in the other five), so I would not read a gain into it; without that launch the two arms overlap (submit 13.9-14.2 against 12.8-14.2 ms).

No crash, device loss or picture difference seen in the six launches with it (start-of-window screenshots only; I did not watch them live).

[Jiro131]

In-game A/B on Windows: the per-thread pipeline lookup gets clearly cheaper, but the total replay submit shows no change above run-to-run noise.

Setup

• Hardware: i9-13900K, RTX 4090 (driver 32.0.16.2101), Windows 11.
• Builds: main 51da046 (A) against this PR at aff9633 (B), both built release with RX_VK_ONLY=1. Only hookshot_vk.dll was swapped, in one game folder.
• Each arm kept its own program and pipeline caches, since vk_shader.cpp is in the translator stamp. Each arm had one warm-up run first.
• Settings: HOS_FPS_MAX=1000 (uncapped). The log confirms "frame rate uncapped", and the GPU-bound wait was about 0 throughout.
• Route: about 2 minutes in Kokiri Forest, the same route each time, with 1,000 to 4,000 draws per frame.
• Runs, in order: A B A B B A.
• Figures come from the replay's 120-present windows with at least 1,000 draws, leaving out the first window after the scene load.

Replay time on the Presentation Thread, µs per draw (median of each run's windows)

	A run 1	A run 3	A run 6	B run 2	B run 4	B run 5
pipelines	0.158	0.122	0.129	0.092	0.086	0.087
programs	0.722	0.571	0.570	0.562	0.527	0.516
submit (all)	6.06	4.46	4.65	4.65	4.32	4.25
memory sync (untouched)	0.345	0.242	0.258	0.266	0.244	0.270
fps (median)	56.9	70.6	67.8	73.2	76.7	80.8

Run 1 (A) was about 25% slower on every timer, including memory sync and queue submit, which this PR does not touch. That points to the machine's state, so the comparison below leaves it out. Leaving out an A run favours neither arm here: the slow run would only make B look better.

At equal draw counts (A = runs 3 and 6, B = runs 2, 4 and 5, ms per frame)

draws	A pipelines	B pipelines	A submit	B submit	A fps	B fps
1000-1500	0.205	0.142	7.25	7.02	82.3	86.5
1500-2000	0.241	0.165	8.57	8.10	76.2	80.3
2000-2500	0.273	0.198	10.14	9.98	71.2	72.9
2500-3000	0.310	0.218	11.80	11.04	63.9	69.1
3000-4000	0.343	0.255	12.72	12.74	62.0	63.1

Reading

• Pipelines: the graphics_pipeline thread cache is the clear effect.
  • It is 25-30% cheaper in every B run than in every A run, and in every draw-count bin, about 0.08 ms a frame at 2,500 draws.
  • Both the fixed part and the per-draw part of a linear fit go down.
• Programs: the picks cache looks about 0.07 ms a frame cheaper, but run 2 (B) overlaps the A runs.
• Submit: I can't attribute a change in the total to this PR.
  • The two A runs differ from each other by about as much as A differs from B.
  • The fit puts B's gain in the fixed part, not in the per-draw part: B's per-draw slope is even slightly steeper (2.62 against 2.47 µs/draw).
  • The 3000-4000 bin shows no gain at all.
• FPS: B is ahead in every bin, by 1 to 5 fps. That gap is far larger than the ~0.15 ms the two caches save, so most of it is noise or something else, not this change.
• Correctness:
  • No visual difference was seen between the arms.
  • Both arms log the same warnings, with 0 failed programs and no driver-fault file.

In short, the change does what it says for the pipeline lookup. On this CPU at Kokiri's draw counts, that lookup was a small share of the submit to begin with.

[corentinbonjour]

Thanks @klikso and @Jiro131 for the in-game runs. I added both to the description under "In the game" and dropped the frame-time figure extrapolated from the micro-benchmark. In short: the pipeline lookup is clearly cheaper on both machines, and the total submit gain shows on the 5600X (-4%) but stays within noise on the 13900K.