Description
I'm trying to rewrite kmscube in rust, targetting an environment with musl as libc implementation.
kmscube's goal is to demonstrate the use of opengl without any compositor. It relies on mesa3d's libgbm, which is present as a shared library in my musl environment, at /usr/lib/libgbm.so
. There is a rust crate wrapping libgbm : gbm.rs. I'm using the crate as follows:
extern crate gbm;
use gbm::Device as GBMDevice;
use std::fs::OpenOptions;
fn main() {
let card = {
let mut options = OpenOptions::new();
options.read(true);
options.write(true);
options.open("/dev/dri/card0").unwrap()
};
println!("pre-gbm");
let _card = GBMDevice::new(card).unwrap();
println!("post-gbm");
}
Using the gnu (non-musl) target, this happens:
cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.01s
Running `target/debug/azimuth`
pre-gbm
post-gbm
Using the musl target, this happens:
cargo run --target=x86_64-unknown-linux-musl
Finished dev [unoptimized + debuginfo] target(s) in 0.01s
Running `target/debug/azimuth`
pre-gbm
Segmentation fault (core dumped)
And this SIGSEGV is my problem.
Analysis
readelf
reveals the dynamic section of each file:
GNU
0x0000000000000001 (NEEDED) Shared library: [libEGL.so.1]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
[numeric entries]
0x000000006ffffffb (FLAGS_1) Flags: NOW PIE
[numeric entries]
MUSL
0x0000000000000001 (NEEDED) Shared library: [libgbm.so.1]
[numeric entries]
0x000000006ffffffb (FLAGS_1) Flags: NOW PIE
[numeric entries]
As well as the program headers:
GNU
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000000040 0x0000000000000040
0x0000000000000310 0x0000000000000310 R 0x8
INTERP 0x0000000000000350 0x0000000000000350 0x0000000000000350
0x000000000000001c 0x000000000000001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x00000000000040e0 0x00000000000040e0 R 0x1000
LOAD 0x0000000000005000 0x0000000000005000 0x0000000000005000
0x00000000000265a5 0x00000000000265a5 R E 0x1000
LOAD 0x000000000002c000 0x000000000002c000 0x000000000002c000
0x0000000000009d58 0x0000000000009d58 R 0x1000
LOAD 0x0000000000036320 0x0000000000037320 0x0000000000037320
0x0000000000001d18 0x0000000000001e98 RW 0x1000
DYNAMIC 0x00000000000377a8 0x00000000000387a8 0x00000000000387a8
0x0000000000000240 0x0000000000000240 RW 0x8
NOTE 0x0000000000000370 0x0000000000000370 0x0000000000000370
0x0000000000000020 0x0000000000000020 R 0x8
NOTE 0x0000000000000390 0x0000000000000390 0x0000000000000390
0x0000000000000044 0x0000000000000044 R 0x4
TLS 0x0000000000036320 0x0000000000037320 0x0000000000037320
0x0000000000000000 0x0000000000000098 R 0x20
GNU_PROPERTY 0x0000000000000370 0x0000000000000370 0x0000000000000370
0x0000000000000020 0x0000000000000020 R 0x8
GNU_EH_FRAME 0x0000000000030158 0x0000000000030158 0x0000000000030158
0x0000000000000b9c 0x0000000000000b9c R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x0000000000036320 0x0000000000037320 0x0000000000037320
0x0000000000001ce0 0x0000000000001ce0 R 0x1
MUSL
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000006e10 0x0000000000006e10 R 0x1000
LOAD 0x0000000000007000 0x0000000000007000 0x0000000000007000
0x00000000000420eb 0x00000000000420eb R E 0x1000
LOAD 0x000000000004a000 0x000000000004a000 0x000000000004a000
0x000000000000fda4 0x000000000000fda4 R 0x1000
LOAD 0x000000000005a060 0x000000000005b060 0x000000000005b060
0x0000000000002100 0x0000000000003d08 RW 0x1000
DYNAMIC 0x000000000005b788 0x000000000005c788 0x000000000005c788
0x0000000000000140 0x0000000000000140 RW 0x8
NOTE 0x0000000000000270 0x0000000000000270 0x0000000000000270
0x0000000000000024 0x0000000000000024 R 0x4
TLS 0x000000000005a060 0x000000000005b060 0x000000000005b060
0x0000000000000000 0x00000000000000d8 R 0x20
GNU_EH_FRAME 0x0000000000051e24 0x0000000000051e24 0x0000000000051e24
0x000000000000125c 0x000000000000125c R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x000000000005a060 0x000000000005b060 0x000000000005b060
0x0000000000001fa0 0x0000000000001fa0 R 0x1
Using strace
, I was able to see that the musl binary never tries to reach for libgbm.
The function call at line 15 results in the use of the symbol gbm_create_device
, which was never resolved and points to NULL.
→ segmentation fault.
Interpretation
Upon inspection of the musl target specification in rust-lang/rust, I understand that the musl target systematically produces static binaries. Further in the targets code, the static_position_independent_executables
target option is converted to a command line argument for gcc: -static
.
From the GCC manual:
-static: On systems that support dynamic linking, this overrides -pie and prevents linking with the shared libraries.
I think GCC achieves this by simply removing the code relocating imported symbols, leaving them unresolved. If we try to use them, we get a segfault.
Does this mean that the musl target is incompatible with any crate that links with shared library at runtime ?
My motivation for using musl is not getting fully static binaries, I just like that it's lightweight and simple. If this incompatibility exists, I'd be happier if I could disable the static_position_independent_executables
as a feature, so I could use all crates with musl.
I'm reporting this as a bug because rustc does not warn the developer that using crates linking to a shlib will lead to segfaults.
Meta
rustc --version --verbose
:
rustc 1.50.0 (cb75ad5db 2021-02-10)
binary: rustc
commit-hash: cb75ad5db02783e8b0222fee363c5f63f7e2cf5b
commit-date: 2021-02-10
host: x86_64-unknown-linux-gnu
release: 1.50.0
Activity
ath-inactive-account commentedon Feb 16, 2021
I will try to build the musl target with
static_position_independent_executables
set tofalse
to see if the dynamic linking takes place.ath-inactive-account commentedon Feb 18, 2021
This stackoverflow question has good information on the subject of static pie and relocation.
ath-inactive-account commentedon Feb 18, 2021
Although this is not solved, I solved it for my case. In config.toml, I put this rustc flag:
It looks like this overrides the
static_position_independent_executables
target configuration.And while it didn't work on my mixed-libc environment, it worked on my musl-only environment.
Now the binary links to musl. I'm not sure exactly which librariy is dynamically linked and which is statically linked, but at least libgbm symbols are now resolved at runtime.
I don't think I should close the issue, as I don't fully understand how the problem was solved. I welcome any explanation.
mati865 commentedon Feb 19, 2021
Were you building the executable on musl based system (like Alpine or musl variant of Void Linux)?
From the description it seems you are trying to make musl binary linked to shared library that is linked to glibc.
ath-inactive-account commentedon Feb 19, 2021
I was trying to build my crate on two different systems. Initially both would segfault.
First one has arch-linux with glibc but also musl in /usr/lib/musl/lib.
Second one is a buildroot embedded linux distribution with musl in /usr/lib.
The config.toml trick worked on the buildroot system immediately (to my surprise), but it was generating a lot of problems on the arch system which I was first testing on.
The mess between musl-libc and glibc on the arch system must have been the reason for all the subsequent problems.
mati865 commentedon Feb 19, 2021
When cross compiling musl target it defaults to static linking yet
gbm.rs
somehow got linked dynamically (not sure ifgbm.rs
bug or what). This didn't work because statically linked musl will not load any dynamic libraries.target-feature=-crt-static
made musl to be linked dynamically (it's default when not cross compiling, i.e. when using musl host) so it could load dynamic libraries.You were trying to mix glibc and musl libraries which is not supposed to work.
ath-inactive-account commentedon Feb 19, 2021
Thanks for the explanation.
I guess that's the real bug then. The crate which links to libgbm is, to be precise, the gbm-sys subcrate. It uses bindgen and its code is there. The link attribute is located here.
mati865 commentedon Feb 19, 2021
So it's
gbm-sys
bug since it tells Rust to always link shared libgbm: https://doc.rust-lang.org/reference/items/external-blocks.html#the-link-attributeath-inactive-account commentedon Feb 19, 2021
I don't understand, the link attribute should select the appropriate link type if the
kind
key is not specified. An error should be reported if the linking cannot happen, way before it segfault, don't you think ? In which case it's a bug from rustc.mati865 commentedon Feb 19, 2021
AFAIK it cannot do it because of backwards compatibility. If you look at this text:
Empty
kind
means dynamic linking.ath-inactive-account commentedon Feb 19, 2021
Oh, I see.
Should I leave the issue open for this part ?
mati865 commentedon Feb 19, 2021
I'll leave it to somebody more experienced like @petrochenkov but IMO this would be hard to take any action here because some libc implementations allow to mix static libc with shared libraries.
12101111 commentedon May 18, 2021
I think this is another duplicate of #71647, #81923, #82912 and #39998
20 remaining items