rust-lld since 1.38 overlaps .text with .rodata for embedded arm target

[lightblu]

Minimal blinky example for a cortex-m4 using cortex-m-rt crate (and a svd2rust generated device crate) since 1.38 (also present with 1.40 nightly) fails linking:

error: linking with `rust-lld` failed: exit code: 1
  |
  = note: "rust-lld" "-flavor" "gnu" "-L" "/home/.rustup/toolchains/    nightly-x86_64-unknown-linux-gnu/lib/rustlib/thumbv7em-none-eabihf/lib" "/home/work/remb    /target/thumbv7em-none-eabihf/debug/deps/remb-0e89ac955194710a.1u2vah3l27soigvo.rcgu.o" "/    home/lb/work/remb/target/thumbv7em-none-eabihf/debug/deps/    remb-0e89ac955194710a.20vhaom7k5jk5m2z.rcgu.o" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/remb-0e89ac955194710a.21jk4vxquq5fsklp.rcgu.o" "/home/    work/remb/target/thumbv7em-none-eabihf/debug/deps/    remb-0e89ac955194710a.2v63og99w4q7umup.rcgu.o" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/remb-0e89ac955194710a.40ihsdzg5hvegvya.rcgu.o" "/home/    work/remb/target/thumbv7em-none-eabihf/debug/deps/    remb-0e89ac955194710a.4edrqqhvp54adqvm.rcgu.o" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/remb-0e89ac955194710a.ela5gwo3oi0mbal.rcgu.o" "-o" "/home/    lb/work/remb/target/thumbv7em-none-eabihf/debug/deps/remb-0e89ac955194710a"     "--gc-sections" "-L" "/home/work/remb/target/thumbv7em-none-eabihf/debug/deps" "-L" "/    home/lb/work/remb/target/debug/deps" "-L" "/home/work/remb/target/thumbv7em-none-eabihf/    debug/build/remb-4f277a380d027b54/out" "-L" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/build/cortex-m-cc9ea6fae48cdb10/out" "-L" "/home/work/remb/    target/thumbv7em-none-eabihf/debug/build/cortex-m-rt-e72a5142fcf97ff7/out" "-L" "/home/    work/remb/target/thumbv7em-none-eabihf/debug/build/cortex-m-semihosting-62a07b6b9f966603/    out" "-L" "/home/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/    thumbv7em-none-eabihf/lib" "-Bstatic" "/home/work/remb/target/thumbv7em-none-eabihf/    debug/deps/libcortex_m_rt-d442f13f5d2d3c17.rlib" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/libr0-44368e36351c6aca.rlib" "--start-group" "/home/    work/remb/target/thumbv7em-none-eabihf/debug/deps/libpanic_itm-54548bdc2864396e.rlib" "/    home/lb/work/remb/target/thumbv7em-none-eabihf/debug/deps/    libcortex_m-7bfd5c3ea6d14260.rlib" "/home/work/remb/target/thumbv7em-none-eabihf/debug/    deps/libvolatile_register-5316b243390db931.rlib" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/libvcell-8547e68540e6b860.rlib" "/home/work/remb/target    /thumbv7em-none-eabihf/debug/deps/libbare_metal-5adef261ba9f8cbc.rlib" "/home/work/remb/    target/thumbv7em-none-eabihf/debug/deps/libaligned-074dd0e0ef261e42.rlib" "/home/work/    remb/target/thumbv7em-none-eabihf/debug/deps/libas_slice-e5decefefa2360b9.rlib" "/home/    work/remb/target/thumbv7em-none-eabihf/debug/deps/    libstable_deref_trait-8119af54d84c3dce.rlib" "/home/work/remb/target/    thumbv7em-none-eabihf/debug/deps/libgeneric_array-fd3cfec1f6f858d0.rlib" "/home/work/    remb/target/thumbv7em-none-eabihf/debug/deps/libtypenum-4c848b432b16c46c.rlib" "/home/    lb/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/thumbv7em-none-eabihf/    lib/librustc_std_workspace_core-bd695e8dc7ecd4c4.rlib" "/home/.rustup/toolchains/    nightly-x86_64-unknown-linux-gnu/lib/rustlib/thumbv7em-none-eabihf/lib/    libcore-d0e985a669afa02b.rlib" "--end-group" "/home/.rustup/toolchains/    nightly-x86_64-unknown-linux-gnu/lib/rustlib/thumbv7em-none-eabihf/lib/    libcompiler_builtins-58bdfb0b01f66c08.rlib" "-Tlink.x" "-Map=./Application.map" "-Bdynamic"
  = note: rust-lld: error: section .text file range overlaps with .rodata
          >>> .text range is [0x1410, 0x4471]
          >>> .rodata range is [0x41D0, 0x4B5F]
          
          rust-lld: error: section .text virtual address range overlaps with .rodata
          >>> .text range is [0x410, 0x3471]
          >>> .rodata range is [0x31D0, 0x3B5F]
          
          rust-lld: error: section .text load address range overlaps with .rodata
          >>> .text range is [0x410, 0x3471]
          >>> .rodata range is [0x31D0, 0x3B5F] 

My device needs .text to start at 0x410, for what I use the commented option in memory.x by cortex-m-quickstart generated example:

/* You can use this symbol to customize the location of the .text section */
/* If omitted the .text section will be placed right after the .vector_tablesection */
/* This is required only on microcontrollers that store some configuration right after the vector table */
_stext = ORIGIN(FLASH) + 0x410;

Without this offset, it links fine but places .text already at 016c (my device crate replaces __INTERRUPTS with 75 vectors, which makes the hole vector section fill til 0x16c only), but for the device it needs to start at 0x410.

This builds fine with rust-lld 1.37 and also with binutils ld ("-C", "linker=arm-none-eabi-ld" in .cargo/config) under 1.38 and nightly, just with rust-lld it is failing since 1.38.

Interesting is, that when I don't specify the offset in memory.x, and .text starts at 0x16c, .rodata starts at 0x31D0, too. So it seems that this offset via _stext is taken into account for .text but not for .rodata? Though I cannot see why this should happen via cortex-m-rt link script:

https://github.com/rust-embedded/cortex-m-rt/blob/master/link.x.in

I searched for relates issues but found only some which were related to alignment issues and small overlaps of some bytes; this overlap of 0x3471 - 0x31D0 = 673 bytes seems to happen because the offset of 0x410 - 0x16c = 676 bytes which puts .text fine at its correct starting position 0x410 is not obeyed for following .rodata section?

Attaching minimal example files; device.x and the vector table definition in main.rs are coming from the device crate and just have been incorporated to provide a smaller example.

minimal-rust-lld-issue.zip

[mati865]

cc @nikic

[nikomatsakis]

cc @japaric @therealprof @jamesmunns -- maybe somebody from the embedded wg has some insight into what is happening here?

[jonas-schievink]

I know that overriding _stext was somewhat broken and was fixed for the RISC-V runtime here: rust-embedded/riscv-rt#36. I don't know if those issues were also in the original cortex-m-rt, or if they were ever fixed there.

Perhaps the linker script in cortex-m-rt had other issues that have now surfaced with the newer LLD?

[jonas-schievink]

(related: would it be helpful to add a WG-embedded label for issues impacting embedded work?)

[jamesmunns]

Hey @lightblu, thanks for reporting. It seems that the behavior of LLD changed between 1.37 and 1.38 in a way that breaks the ABI of the cortex-m-rt crate (likely due to an LLVM version upgrade).

This is likely either:

• An intended change to LLVM/LLD behavior
• A regression in LLD
  • Possibly related to _stext fails to move .text with lld rust-embedded/cortex-m-rt#188 and https://reviews.llvm.org/D60744 ?

If it is the former, unfortunately as far as I understand, we have not gotten official guidance from the upstream rust project that use of custom linker scripts should be considered part of Rust's stability guarantees, so this breakage will likely need to be followed up with in the cortex-m-rt repo. For now, I will assume this is the same root cause as rust-embedded/cortex-m-rt#188, but if not we can open a separate issue.

@yodaldevoid did you ever follow up with this re: the LLVM regression in the ELF headers? Would you know how to check whether the minimal example posted above matches the same regression?

[jamesmunns]

@nikomatsakis were there any LLVM changes that landed in nightly before 2019-04-13 or so, that stabilized in 1.38 (2019-09-26 or so)? If so, that might help us diagnose the specific changes that cause this regression for stable embedded targets.