major performance regression

[thestinger]

The benchmarks in #8981 had a significant drop in performance on my platform (Linux 3.11.5, glibc 2.18).

Last good commit: 0.8-331-g6db8899
First known bad commit: 0.8-369-g62812f1

@cmr bisected this and identified d858360 as the cause.

cc @alexcrichton

[pnkfelix]

@cmr bisected this and identified d858360 as the cause.

that ... is bizarre. Looking more deeply now.

[alexcrichton]

I built before and after that commit on OSX, and I didn't detect a regression either way, although I didn't detect a regression on OSX in #8981 on master either, so I was either measuring incorrectly or this is only a regression on linux.

[pnkfelix]

I took some samples of builds from four bors-merge points over the last 20 days: sep 24, oct 01, oct 05, and oct 10.

All of the builds were configured with configure --disable-debug --enable-optimize --enable-ccache --enable-clang --prefix=~/opt/rust. My platform is a 2.7 Ghz Intel Core i7 MacBook Pro running OS X 10.8.5.

I do not see a performance regression, at least not on Mac OS X. If anything it seems like things have gotten better for the change proposeed in #8981.

Maybe someone with a Linux machine handy can double-check thestinger's findings.

% for d in sep24 oct01 oct05 oct10 ; do ls -Fd rust-$d-* ; ./rust-$d-*/objdir-opt/x86_64-apple-darwin/stage2/bin/rustc -Aunused-imports --opt-level=3 --test /tmp/b.rs && /tmp/b --bench ; done
rust-sep24-a7d68adbdd75300a4a10b2dcc59e1048f613db48/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 58 ns/iter (+/- 7)
test bench::push ... bench: 37438 ns/iter (+/- 3493)
test bench::push_preallocated ... bench: 34575 ns/iter (+/- 10314)
test bench::push_preallocated_std ... bench: 34751 ns/iter (+/- 5529)
test bench::push_std ... bench: 43656 ns/iter (+/- 4786)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct01-88b0b511beed1599c5bddaf05b9cd0f98bd714ca/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 59 ns/iter (+/- 6)
test bench::push ... bench: 11623 ns/iter (+/- 2518)
test bench::push_preallocated ... bench: 5882 ns/iter (+/- 744)
test bench::push_preallocated_std ... bench: 37822 ns/iter (+/- 5728)
test bench::push_std ... bench: 42381 ns/iter (+/- 20040)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct05-549f70989e3423642bacc41bb6c58ed81d591c03/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 58 ns/iter (+/- 9)
test bench::push ... bench: 11789 ns/iter (+/- 3440)
test bench::push_preallocated ... bench: 5839 ns/iter (+/- 2173)
test bench::push_preallocated_std ... bench: 37940 ns/iter (+/- 3766)
test bench::push_std ... bench: 42484 ns/iter (+/- 5002)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct10-2e1df8e35bb0185dc35f5e7976c0b46608c32404/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 59 ns/iter (+/- 14)
test bench::push ... bench: 11881 ns/iter (+/- 2696)
test bench::push_preallocated ... bench: 5827 ns/iter (+/- 457)
test bench::push_preallocated_std ... bench: 37639 ns/iter (+/- 4246)
test bench::push_std ... bench: 42216 ns/iter (+/- 7903)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

% 

[pnkfelix]

(of course I forget to include a build from today in the data above. will add that shortly.) (done)

[emberian]

I had double-checked his findings. It could be a linux-specific regression.

On Mon, Oct 14, 2013 at 10:06 PM, Felix S Klock II <notifications@github.com

wrote:

I took some samples of builds from four bors-merge points over the last 20
days: sep 24, oct 01, oct 05, and oct 10.

All of the builds were configured with configure --disable-debug
--enable-optimize --enable-ccache --enable-clang --prefix=~/opt/rust

I do not see a performance regression, at least not on Mac OS X. If
anything it seems like things have gotten better for the change proposeed
in #8981 #8981.

Maybe someone with a Linux machine handy can double-check thestinger's
findings.

% for d in sep24 oct01 oct05 oct10 ; do ls -Fd rust-$d-* ; ./rust-$d-*/objdir-opt/x86_64-apple-darwin/stage2/bin/rustc -Aunused-imports --opt-level=3 --test /tmp/b.rs && /tmp/b --bench ; done
rust-sep24-a7d68adbdd75300a4a10b2dcc59e1048f613db48/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 58 ns/iter (+/- 7)
test bench::push ... bench: 37438 ns/iter (+/- 3493)
test bench::push_preallocated ... bench: 34575 ns/iter (+/- 10314)
test bench::push_preallocated_std ... bench: 34751 ns/iter (+/- 5529)
test bench::push_std ... bench: 43656 ns/iter (+/- 4786)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct01-88b0b511beed1599c5bddaf05b9cd0f98bd714ca/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 59 ns/iter (+/- 6)
test bench::push ... bench: 11623 ns/iter (+/- 2518)
test bench::push_preallocated ... bench: 5882 ns/iter (+/- 744)
test bench::push_preallocated_std ... bench: 37822 ns/iter (+/- 5728)
test bench::push_std ... bench: 42381 ns/iter (+/- 20040)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct05-549f70989e3423642bacc41bb6c58ed81d591c03/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 58 ns/iter (+/- 9)
test bench::push ... bench: 11789 ns/iter (+/- 3440)
test bench::push_preallocated ... bench: 5839 ns/iter (+/- 2173)
test bench::push_preallocated_std ... bench: 37940 ns/iter (+/- 3766)
test bench::push_std ... bench: 42484 ns/iter (+/- 5002)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

rust-oct10-2e1df8e35bb0185dc35f5e7976c0b46608c32404/
warning: no debug symbols in executable (-arch x86_64)

running 7 tests
test test::push ... ignored
test bench::empty ... bench: 5 ns/iter (+/- 0)
test bench::empty_std ... bench: 59 ns/iter (+/- 14)
test bench::push ... bench: 11881 ns/iter (+/- 2696)
test bench::push_preallocated ... bench: 5827 ns/iter (+/- 457)
test bench::push_preallocated_std ... bench: 37639 ns/iter (+/- 4246)
test bench::push_std ... bench: 42216 ns/iter (+/- 7903)

test result: ok. 0 passed; 0 failed; 1 ignored; 6 measured

%

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/9858#issuecomment-26304375
.

[thestinger]

@pnkfelix: the numbers here are basically the reverse of your sept24 -> oct10

[thestinger]

This is no longer an issue, since whatever went wrong with jemalloc has been "fixed" by the removal.