Exploring rust-gpu internals: MIR to SPIR-V + debugging questions #260
Replies: 3 comments 3 replies
-
|
Hi there, just checking again, if someone has any pointers on this. TL;DRThere doesn’t seem to be a supported or documented way to directly step through Any pointers on this kind of low-level integration with Question:Is there a way to directly debug From what I’ve explored so far, I haven’t found a clear path to doing this. The only debugging avenues currently available seem to be:
My Approach: Programmatic MIR Generation and Backend Invocation
Here’s a minimal example I tried: use rustc_driver::{Callbacks, Compilation};
use rustc_interface::{interface, Queries};
struct SpirvCaller;
impl Callbacks for SpirvCaller {
fn after_analysis<'tcx>(
&mut self,
_compiler: &interface::Compiler,
queries: &'tcx Queries<'tcx>,
) -> Compilation {
queries.global_ctxt().unwrap().enter(|tcx| {
let metadata = rustc_metadata::EncodedMetadata::from_path(
PathBuf::from("../../examples/shaders/simplest-shader/src/lib.rs"),
None,
).unwrap();
let backend = SpirvCodegenBackend;
let result = backend.codegen_crate(tcx, metadata, false);
println!("SPIR-V codegen result: {:?}", result.type_id());
});
Compilation::Stop
}
}To drive this setup, I used rustc directly with the required arguments: rustc_driver::RunCompiler::new(&args, &mut SpirvCaller)
.run()
.expect("Failed to run rustc driver");The arguments were extracted from a working cargo build-plan and adjusted for use with These errors suggest that
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @eddyb — folks on the t-compiler Zulip suggested you might be the best person to talk to about this. Would you happen to have any pointers or suggestions? Or perhaps a better way to approach the problem? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @LegNeato — do you happen to have any thoughts on this? The only other option I can think of is to start instrumenting |
Beta Was this translation helpful? Give feedback.
-
|
@eddyb would definitely have more info...we have a recipe around here somewhere for how to debug the compiler backend. Not sure exactly what you are looking for. There are various options to dump different stages of IRs: https://github.com/Rust-GPU/rust-gpu/blob/main/docs/src/codegen-args.md But that of course won't be the same as tracing through with a debugger (which may be a good thing?) |
Beta Was this translation helpful? Give feedback.
-
this would be great if there is one.
To start with, I’m interested in understanding how the backend maps something simple—like a type represented in MIR—to its corresponding SPIR-V type. My goal is to build a basic mental model of how the backend works internally. I’ve been able to glean some insights through static analysis, but that only gets me so far. Here's a quick example: #![cfg_attr(target_arch = "spirv", no_std)]
// HACK(eddyb) can't easily see warnings otherwise from `spirv-builder` builds.
#![deny(warnings)]
use shared::glam::{Vec4, vec4};
use spirv_std::spirv;
#[spirv(fragment)]
pub fn main_fs(output: &mut Vec4) {
*output = vec4(1.0, 0.0, 0.0, 1.0); /******************how do we get this vec4 type to map to Spirv's OpTypeVector*************/
}
#[spirv(vertex)]
pub fn main_vs(
#[spirv(vertex_index)] vert_id: i32,
#[spirv(position, invariant)] out_pos: &mut Vec4,
) {
*out_pos = vec4(
(vert_id - 1) as f32,
((vert_id & 1) * 2 - 1) as f32,
0.0,
1.0,
);
}the corresponding SPIRV code for this is ; SPIR-V
; Version: 1.3
; Generator: Google rspirv; 0
; Bound: 31
; Schema: 0
OpCapability Shader
OpMemoryModel Logical Simple
OpEntryPoint Fragment %1 "main_fs" %2
OpEntryPoint Vertex %3 "main_vs" %gl_VertexIndex %gl_Position
OpExecutionMode %1 OriginUpperLeft
OpDecorate %2 Location 0
OpDecorate %gl_VertexIndex BuiltIn VertexIndex
OpDecorate %gl_Position BuiltIn Position
OpDecorate %gl_Position Invariant
%float = OpTypeFloat 32
%v4float = OpTypeVector %float 4 /***************** how do we get to this ******************/
%_ptr_Output_v4float = OpTypePointer Output %v4float
%void = OpTypeVoid
%12 = OpTypeFunction %void
%float_1 = OpConstant %float 1
%float_0 = OpConstant %float 0
%2 = OpVariable %_ptr_Output_v4float Output
%int = OpTypeInt 32 1
%_ptr_Input_int = OpTypePointer Input %int
%gl_VertexIndex = OpVariable %_ptr_Input_int Input
%int_1 = OpConstant %int 1
%int_2 = OpConstant %int 2
%gl_Position = OpVariable %_ptr_Output_v4float Output
%30 = OpConstantComposite %v4float %float_1 %float_0 %float_0 %float_1
%1 = OpFunction %void None %12
%19 = OpLabel
OpStore %2 %30
OpReturn
OpFunctionEnd
%3 = OpFunction %void None %12
%21 = OpLabel
%22 = OpLoad %int %gl_VertexIndex
%23 = OpISub %int %22 %int_1
%24 = OpConvertSToF %float %23
%25 = OpBitwiseAnd %int %22 %int_1
%26 = OpIMul %int %25 %int_2
%27 = OpISub %int %26 %int_1
%28 = OpConvertSToF %float %27
%29 = OpCompositeConstruct %v4float %24 %28 %float_0 %float_1
OpStore %gl_Position %29
OpReturn
OpFunctionEnd
|
Beta Was this translation helpful? Give feedback.
-
|
Follow up: I managed to get
use rustc_driver::{Callbacks, Compilation};
use rustc_interface::{interface, Queries};
struct SpirvCaller;
impl Callbacks for SpirvCaller {
fn after_analysis<'tcx>(
&mut self,
_compiler: &interface::Compiler,
queries: &'tcx Queries<'tcx>,
) -> Compilation {
queries.global_ctxt().unwrap().enter(|tcx| {
// Collect metadata manually
let metadata = rustc_metadata::EncodedMetadata::from_path(
<PathBuf as From<&str>>::from("/Users/nihalpasham/devspace/rust/gpu/rust-gpu/tests/ui/hello_world.rs"),
None,
)
.unwrap();
let backend = SpirvCodegenBackend;
let result = backend.codegen_crate(tcx, metadata, false);
println!("SPIR-V codegen result: {:?}", result.type_id());
println!("SPIR-V codegen result: {:?}", tcx.crate_name(CrateNum::from_u32(0)));
});
Compilation::Stop
}
}
pub fn quick_backend_test() {
let args = vec![
"rustc",
"/Users/nihalpasham/devspace/rust/gpu/rust-gpu/tests/ui/hello_world.rs",
"-L",
"/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-results",
"--target=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/tests/../crates/spirv-builder/target-specs/spirv-unknown-spv1.3.json",
"--error-format", "json",
"-C", "prefer-dynamic",
"-o", "/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-results/hello_world.spv1.3.spv",
"-A", "unused",
"-Zcodegen-backend=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/release/deps/librustc_codegen_spirv.dylib",
"-Zbinary-dep-depinfo",
"-Csymbol-mangling-version=v0",
"-Zcrate-attr=feature(register_tool)",
"-Zcrate-attr=register_tool(rust_gpu)",
"-Coverflow-checks=off",
"-Cdebug-assertions=off",
"-Zinline-mir=off",
"-Zmir-enable-passes=-GVN",
"-Cdebuginfo=2",
"-Cembed-bitcode=no",
"-Ctarget-feature=+Int8,+Int16,+Int64,+Float64,+ShaderClockKHR,+ext:SPV_KHR_shader_clock",
"-L", "dependency=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/debug/deps",
"-L", "dependency=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/spirv-unknown-spv1.3/debug/deps",
"--edition", "2021",
"--extern", "noprelude:core=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/spirv-unknown-spv1.3/debug/deps/libcore-97d3787f8d654b18.rlib",
"--extern", "noprelude:compiler_builtins=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/spirv-unknown-spv1.3/debug/deps/libcompiler_builtins-532e4ba355802893.rlib",
"--extern", "spirv_std_macros=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/debug/deps/libspirv_std_macros-8e1d2a17232f3f1b.dylib",
"--extern", "spirv_std=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/spirv-unknown-spv1.3/debug/deps/libspirv_std-372003debcd81735.rlib",
"--extern", "glam=/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-deps/spirv-unknown-spv1.3/debug/deps/libglam-7076740f3e08598a.rlib",
"--crate-type", "dylib",
"-Zunstable-options",
"-Zcrate-attr=no_std",
"-Zcrate-attr=feature(asm_experimental_arch)",
"-L", "/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/compiletest-results/hello_world.spv1.3.aux",
];
let args_r: Vec<String> = args.into_iter().map(|s| s.to_string()).collect();
// Debug output
println!("Running rustc with args:");
for arg in &args_r {
println!(" {}", arg);
}
rustc_driver::RunCompiler::new(&args_r, &mut SpirvCaller)
.run()
.expect("Failed to run rustc driver");
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn check_quick_backend_test() {
quick_backend_test();
}
}Running the above test using: cargo test --package rustc_codegen_spirv --lib -- test::check_quick_backend_test --exact --show-output…produces: SPIR-V codegen result: TypeId(0x3251abd8734b52e1279a063b41f0e7bc)
SPIR-V codegen result: "hello_world"
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 144 filtered out; finished in 0.18s⸻ However, I still can’t breakpoint-debug this, which is what I need to observe the MIR → SPIR-V conversion flow. The debug output is: nihalpasham@Nihals-Mac-mini rust-gpu % /Users/nihalpasham/.vscode/extensions/vadimcn.vscode-lldb-1.11.4/adapter/codelldb terminal-agent --connect=58809
dyld[15875]: Library not loaded: @rpath/librustc_driver-0c1b53b2a16b2162.dylib
Referenced from: <62A904B6-FB6E-31B4-9C84-96911FCD9FE9> /Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/debug/deps/rustc_codegen_spirv-64cd2f91fb90afc5
Reason: no LC_RPATH's foundThis seemed like a dynamic linker issue (on a Mac) to ensure {
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "Debug quick_backend_test",
"program": "cargo",
"args": [
"test",
"--package",
"rustc_codegen_spirv",
"--lib",
"--",
"test::check_quick_backend_test",
"--exact",
"--show-output"
],
"env": {
"DYLD_FALLBACK_LIBRARY_PATH": "/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/debug/build/spirv-tools-sys-5247ebcd26373c70/out:/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/debug/deps:/Users/nihalpasham/devspace/rust/gpu/rust-gpu/target/debug:/Users/nihalpasham/.rustup/toolchains/nightly-2024-11-22-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib:/Users/nihalpasham/.rustup/toolchains/nightly-2024-11-22-aarch64-apple-darwin/lib:/Users/nihalpasham/lib:/usr/local/lib:/usr/lib"
},
"cwd": "${workspaceFolder}"
}
]
}Hoping someone here has ideas on how to fix this debugger problem. Please let me know if you need additional context. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone,
I finally found some time to explore this project —
rust-gpuhas been on my to-do list for a long while! I recorded a (relatively) short stream where I dig into its internals, primarily to build a mental model of how the MIR → SPIR-V mapping works. I plan to make more of these as I continue exploring.I’d really appreciate any corrections or feedback if I got anything wrong in the video.
Also, I have a question:
rustc_codegen_spirvand breakpoint-debug the entire backend flow? The current examples use a build script that precompiles the backend and stores it in a known location, which makes breakpoint debugging the actual backend code quite difficult.SpirvCodegenBackend::codegen_crateprogrammatically to trace the flow end-to-end during debugging. This should work -right?YouTube link
Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions