Skip to content

38x performance regression in lewton since rust 1.26 #55446

Closed
@est31

Description

@est31
Member

Decoding the test assets in the benchmark suite of lewton takes 2.5449s if compile with rustc 1.25.0 and 96.5645s with rustc 1.26.0. This is a 38 fold regression.

To reproduce:

$ git clone https://github.com/RustAudio/lewton/
$ cd lewton/dev/cmp
$ RUSTFLAGS="-C codegen-units=1" cargo +1.25.0 run bench --release
    Finished release [optimized] target(s) in 0.0 secs
     Running `target/release/cmp bench`

File bwv_1043_vivace.ogg has matching hash inside hash list, skipping download
File bwv_543_fuge.ogg has matching hash inside hash list, skipping download
File maple_leaf_rag.ogg has matching hash inside hash list, skipping download
File hoelle_rache.ogg has matching hash inside hash list, skipping download
File thingy-floor0.ogg has matching hash inside hash list, skipping download
File audio_simple_err.ogg has matching hash inside hash list, skipping download

Comparing speed for bwv_1043_vivace.ogg : libvorbis=0.4688s we=0.6376s difference=1.36x
Comparing speed for bwv_543_fuge.ogg    : libvorbis=0.6750s we=0.9988s difference=1.48x
Comparing speed for maple_leaf_rag.ogg  : libvorbis=0.1916s we=0.2751s difference=1.44x
Comparing speed for hoelle_rache.ogg    : libvorbis=0.3393s we=0.4721s difference=1.39x
Comparing speed for thingy-floor0.ogg   : libvorbis=0.1525s we=0.1699s difference=1.11x

Overall time spent for decoding by libvorbis: 1.8272s
Overall time spent for decoding by us: 2.5536s
Overall ratio of difference: 1.40x
$ RUSTFLAGS="-C codegen-units=1" cargo +1.26.0 run bench --release
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/cmp bench --release`

File bwv_1043_vivace.ogg has matching hash inside hash list, skipping download
File bwv_543_fuge.ogg has matching hash inside hash list, skipping download
File maple_leaf_rag.ogg has matching hash inside hash list, skipping download
File hoelle_rache.ogg has matching hash inside hash list, skipping download
File thingy-floor0.ogg has matching hash inside hash list, skipping download
File audio_simple_err.ogg has matching hash inside hash list, skipping download

Comparing speed for bwv_1043_vivace.ogg : libvorbis=1.7041s we=22.9206s difference=13.45x
Comparing speed for bwv_543_fuge.ogg    : libvorbis=2.7188s we=38.3622s difference=14.11x
Comparing speed for maple_leaf_rag.ogg  : libvorbis=0.8191s we=11.1756s difference=13.64x
Comparing speed for hoelle_rache.ogg    : libvorbis=1.3928s we=17.8183s difference=12.79x
Comparing speed for thingy-floor0.ogg   : libvorbis=0.9074s we=6.2878s difference=6.93x

Overall time spent for decoding by libvorbis: 7.5422s
Overall time spent for decoding by us: 96.5645s
Overall ratio of difference: 12.80x

The same behaviour also occurs without the RUSTFLAGS env var but I've added it here to indicate that this bug is a different issue from #53833 or #47745.

Note that the bug is not fixed in stable 1.30.0 (da5f414c2 2018-10-24) or 1.31.0-nightly (cae6efc37 2018-10-27) (latest nightly as of filing this bug): On stable, the benchmark takes 80.0889s, on nightly, it takes 77.7243s.

Activity

added
I-slowIssue: Problems and improvements with respect to performance of generated code.
regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.
C-bugCategory: This is a bug.
WG-llvmWorking group: LLVM backend code generation
on Oct 28, 2018
added
T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
on Oct 28, 2018
est31

est31 commented on Oct 28, 2018

@est31
MemberAuthor

Ohhhh I see it. Of course.

Running target/debug/cmp bench --release

Running target/release/cmp bench

This seems to be a weird regression in cargo that made cargo run bench --release being treated differently between the two releases, somehow ignoring the --release flag after 1.26.0.

Closing lol I'm stupid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: This is a bug.I-slowIssue: Problems and improvements with respect to performance of generated code.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.WG-llvmWorking group: LLVM backend code generationregression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @kennytm@Mark-Simulacrum@est31

        Issue actions

          38x performance regression in lewton since rust 1.26 · Issue #55446 · rust-lang/rust