Repository navigation
fix(vk): report a driver fault in a pipeline compile on Linux - #975
tickernelz wants to merge 5 commits into
Conversation
|
Real-GPU evidence for the second and fourth commits, on an RX 6800 XT (RADV NAVI21, Mesa 26.2.6, Fedora 44, kernel 7.2.15) with the v2.0.5 game and this branch's The
Two things this pinned down for me:
Caveat: the run renders offscreen (no display in the session I ran it from), so I can confirm warm-up, precompile and frame rate, not the window. |
|
Re-verified at Build:
RADV NAVI21, Mesa 26.2.6, Fedora 44, kernel 7.2.15. The run is offscreen 1920×1080 because this shell has no display: a virtual X server here has no DRI3, and Mesa refuses Vulkan presentation without it ( Tests: CI at |
Windows wraps vkCreateComputePipelines and vkCreateGraphicsPipelines in __try so a driver that faults on a shader costs one pipeline, not the game. Linux had no equivalent: the call went to the driver as it was, and an access violation inside RADV's compiler ended the process during the first start's warm-up (issues 915 and 920, and the RX 6800 XT report). Arm a thread-local window around the driver call and catch SIGSEGV and SIGBUS inside it, reporting the call's failure with the same values the Windows branch returns. A fault outside the window is passed to whoever held the signal, so the game's own crash report and its slow-path handlers are unchanged. tests/test_linux_driver_fault_guard.py compiles the actual guarded calls from vk_draw.cpp against a fake Vulkan that faults on demand: a fault in the compute or graphics call returns -77 or VK_ERROR_UNKNOWN and the next compile still reaches the driver, a fault outside the window still ends the process, and the unfixed source dies on the same fault.
A silent missing pipeline is harder to place in a report than one line naming which kind went.
The guarded calls now log the pipeline they drop, so the compiled harness needs that symbol.
The Windows branch wraps the pipeline calls in __try and catches every violation. The signal guard took only SIGSEGV and SIGBUS, so a driver that faults with an illegal instruction or an arithmetic trap still ended the game. The host's own handler (host/posix/win32_posix.c) already translates all four, so guard the same four and keep the previous handler per signal. A sigaction that fails used to leave the process unguarded for that signal with nothing in the log.
A fault outside the window has to reach whatever handler held the signal before the guard. A guarded compile now runs first, so that handler is the guard's predecessor rather than the default, and the case fails unless it runs. The fake driver can also raise SIGILL and SIGFPE, so the wider signal set is covered instead of assumed.
ca9dc50 to
bad04d0
Compare
|
Rebased onto upstream
The three runs on this pull request itself are still |
|
Superseded by #834, which is in v2.0.6 (fa43939): Closing to keep the queue clear rather than carrying a band-aid on top of a fixed root. The guard still has a residual use (any RADV/AMDVLK fault inside a pipeline compile, e.g. issue 26 on Windows), so say the word and I will rebase and reopen it as a standalone hardening PR instead of a fix for 915/920. |
What this fixes
On Linux a driver that faults while compiling a pipeline ends the game. Windows has had a guard for this since the amdvlk mesh-program crash (issue 26):
__tryaroundvkCreateComputePipelinesandvkCreateGraphicsPipelines, one lost pipeline. The#elsebranch called the driver as it was.That is the difference between a report and a dead process in issues 915 and 920 (RX 6700 XT and RX 7900 XTX, RADV, crash right after shader preparation), and in a report on an RX 6800 XT with Mesa 26.2.6:
The SPIR-V there is invalid, and RADV dereferences what
spirv_to_nirreturns for it. The invalid variants themselves are the subject of #894, #834 and #848; this PR is the container around the driver call, so the next invalid shader costs one pipeline instead of the start.Change
gpu/vk/vk_draw.cpp: the Linuxguarded_computeandguarded_graphicsarm a thread-local window around the driver call and catchSIGSEGV,SIGBUS,SIGILLandSIGFPEinside it, returning the same values the Windows branch returns (-77for compute,VK_ERROR_UNKNOWNfor graphics). Those are the four the host's own handler translates (host/posix/win32_posix.c), so the guard sees the same set the crash report does.A fault outside the window is handed to whoever held the signal before, per signal, so the game's existing crash report and its other signal handlers are unchanged.
A
sigactionthat fails leaves that one signal unguarded; the log now says which one, instead of the guard looking installed.No behaviour change without a fault: the call is made where it was made.
gpu/vk/vk_draw.cpp(precompile_run): the compute worker calledvkCreateComputePipelinesitself, so the guard never covered the phase that builds the cached compute pipelines. It now goes throughguarded_compute. This is the call Guard cached compute precompilation against driver faults #758 changes for Windows; on Linux it is what makes that change reach the driver at all.Testing
tests/test_linux_driver_fault_guard.py(game-free, registered intests/run_all.py, skipped on Windows) compiles the actual guarded calls fromvk_draw.cppagainst a fake Vulkan that faults on demand:VK_SUCCESS-77, no pipeline published, the next compile reaches the driverVK_ERROR_UNKNOWN, no pipeline publishedSIGILLorSIGFPE-77, no pipeline publishedvk_draw.cpppython -I -B tests/run_all.py --jobs 1 linux_driver_fault_guardpasses with clang++ and g++.hookshot_vk.so: warm-up completes, 10,083 compute pipelines built, then 59-60 fps steady for 15 minutes with no crash. Without the fix the same folder crashes in the first minute atlibvulkan_radeon.so+0x14a7d2.Not covered
SIGABRT,SIGTRAPand the like) are still handed to the game's own handler, so a driver that ends the process that way is not covered.