Skip to content

Memory unsafety problem in safe Rust #69225

Closed
@dfyz

Description

@dfyz
Contributor

I have a small program (a simplification of a test function from a larger project) that slices a small array and tries to access an out-of-bounds element of the slice. Running it with cargo run --release using the stable 1.41.0 release prints something like this (tested on macOS 10.15 and Ubuntu 19.10):

0 0 3 18446744073709551615
[1]    21065 segmentation fault  cargo run --release

It looks like the resulting slice somehow has length 2**64 - 1, so the bounds checking is omitted, which predictably results in a segfault. On 1.39.0 and 1.40.0 the very same program prints what I would expect:

0 0 3 0
thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 16777216', src/main.rs:13:35
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

The problem goes away if I do any of the following:

  • remove either of the two do_test(...); calls in main();
  • remove the for _ in 0..1 { loop;
  • replace the for y in 0..x { loop with for y in 0..1 {;
  • remove the z.extend(std::iter::repeat(0).take(x)); line or replace it with z.extend(std::iter::repeat(0).take(1));;
  • replace the for arr_ref in arr { loop with let arr_ref = &arr[0];;
  • specify RUSTFLAGS="-C opt-level=2";
  • specify RUSTFLAGS="-C codegen-units=1".

My best guess is -C opt-level=3 enables a problematic optimization pass in LLVM, which results in miscompilation. This is corroborated by the fact that MIR (--emit mir) and LLVM IR before optimizations (--emit llvm-ir -C no-prepopulate-passes) is the same for both -C opt-level=2 and -C opt-level=3.

Some additional info that might be helpful:

  • I can't reproduce the problem in the Rust playground (presumably because it uses codegen-units = 1);
  • I can't reproduce the problem on Windows 10 with the same 1.41.0 release (no idea what makes it different);
  • cargo-bisect-rustc says the regression first happened in the 2019-12-12 nightly, specifically in this commit. This seems suspicious to me, given that 1.40.0, which does not exhibit the problem, was released after this date.

I'm attaching the program inline in case the GitHub repo doesn't work (if you want to compile it without Cargo, use rustc -C opt-level=3 main.rs):

fn do_test(x: usize) {
    let arr = vec![vec![0u8; 3]];

    let mut z = Vec::new();
    for arr_ref in arr {
        for y in 0..x {
            for _ in 0..1 {
                z.extend(std::iter::repeat(0).take(x));
                let a = y * x;
                let b = (y + 1) * x - 1;
                let slice = &arr_ref[a..b];
                eprintln!("{} {} {} {}", a, b, arr_ref.len(), slice.len());
                eprintln!("{:?}", slice[1 << 24]);
            }
        }
    }
}

fn main() {
    do_test(1);
    do_test(2);
}

Activity

added
A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.
on Feb 17, 2020
pietroalbini

pietroalbini commented on Feb 17, 2020

@pietroalbini
Member

cc @rust-lang/compiler
@rustbot ping icebreakers-llvm

The release team is considering making a point release for Rust 1.41 (we briefly discussed it in last week's meeting), and I'd love for this to be included in it if we can get a PR up soon.

rustbot

rustbot commented on Feb 17, 2020

@rustbot
Collaborator

Hey LLVM ICE-breakers! This bug has been identified as a good
"LLVM ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @comex @DutchGhost @hanna-kruppe @hdhoang @heyrutvik @JOE1994 @jryans @mmilenko @nagisa @nikic @Noah-Kennedy @SiavoshZarrasvand @spastorino @vertexclique @vgxbj

RalfJung

RalfJung commented on Feb 17, 2020

@RalfJung
Member

Running it with cargo run --release using the stable 1.41.0 release prints something like this (tested on macOS 10.15 and Ubuntu 19.10):

I cannot reproduce this on playground. The program works fine there on 1.41.0 in release mode.

EDIT: Ah, you already said that.
Also the program is fine in Miri, so this is likely not UB but a miscompilation.

BurntSushi

BurntSushi commented on Feb 17, 2020

@BurntSushi
Member

Just to add a data point, I can reproduce this on Linux with the latest nightly:

[andrew@krusty rust-69225]$ rustc --version
rustc 1.43.0-nightly (5e7af4669 2020-02-16)

[andrew@krusty rust-69225]$ cat main.rs
fn do_test(x: usize) {
    let arr = vec![vec![0u8; 3]];

    let mut z = Vec::new();
    for arr_ref in arr {
        for y in 0..x {
            for _ in 0..1 {
                z.extend(std::iter::repeat(0).take(x));
                let a = y * x;
                let b = (y + 1) * x - 1;
                let slice = &arr_ref[a..b];
                eprintln!("{} {} {} {}", a, b, arr_ref.len(), slice.len());
                eprintln!("{:?}", slice[1 << 24]);
            }
        }
    }
}

fn main() {
    do_test(1);
    do_test(2);
}

[andrew@krusty rust-69225]$ rustc -C opt-level=3 main.rs

[andrew@krusty rust-69225]$ ./main
0 0 3 18446744073709551615
zsh: segmentation fault (core dumped)  ./main

I was able to reproduce the above with the exact same output with Rust 1.41 stable. Rust 1.40 stable does not exhibit the problem:

$ ./main
0 0 3 0
thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 16777216', main.rs:13:35
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

I think this is all consistent with @dfyz's report, except this at least confirms that it isn't macOS specific.

SimonSapin

SimonSapin commented on Feb 17, 2020

@SimonSapin
Contributor
  • cargo-bisect-rustc says the regression first happened in the 2019-12-12 nightly, specifically in this commit. This seems suspicious to me, given that 1.40.0, which does not exhibit the problem, was released after this date.

This is expected. 1.40.0 was released on 2019-12-19 based on what was then the beta branch, which was branched from master six weeks earlier (around the time of the 1.39.0 release). See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more about release channels.

RalfJung

RalfJung commented on Feb 17, 2020

@RalfJung
Member

If I had to guess, I'd say #67015 is the likely culprit. This fixed 3 codegen issues, so it touches codegen-critical code.
Cc @osa1 @oli-obk @wesleywiser

added
E-needs-bisectionCall for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc
on Feb 17, 2020

63 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.C-bugCategory: This is a bug.I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessICEBreaker-LLVMBugs identified for the LLVM ICE-breaker groupP-mediumMedium priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @cuviper@dfyz@comex@shahn@pnkfelix

        Issue actions

          Memory unsafety problem in safe Rust · Issue #69225 · rust-lang/rust