Rust tests fail sometimes with "Resource temporarily unavailable"

[dankamongmen]

More often than not, when I run the libnotcurses-sys unit tests via cargo test, they fail with the following:

test bindgen_test_layout_ncmenu_item ... ok
test bindgen_test_layout_ncmenu_options ... ok
test bindgen_test_layout_ncmenu_section ... ok
test bindgen_test_layout_ncmselector_item ... ok
error: io error when listing tests: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
test bindgen_test_layout_ncmultiselector_options ... error: test failed, to rerun pass '--lib'

[dankamongmen]

The notcurses unit tests see this as well:

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

[schwarzgerat](0) $ cargo test --lib
    Finished test [unoptimized + debuginfo] target(s) in 0.02s
     Running target/debug/deps/notcurses-baa7c69b18166c96

running 2 tests
Term: 78x104 alacritty (alacritty terminal emulator)

 notcurses 1.5.1 by nick black et al
  78 rows, 104 columns (126.75KiB), 256 colors (direct)
  compiled with gcc-9.3.0
  terminfo from ncurses 6.2.20200212
  avformat 58.29.100 avutil 56.31.100 swscale 5.5.100
test tests::create_context ... okerror: io error when listing tests: Os { code: 11, kind: WouldBlockTerm: 78x104 alacritty (alacritty terminal emulator)
, message: "Resource temporarily unavailable" }


 notcurses 1.5.1 by nick black et al
  78 rows, 104 columns (126.75KiB), 256 colors (direct)
  compiled with gcc-9.3.0
  terminfo from ncurses 6.2.20200212
  avformat 58.29.100 avutil 56.31.100 swscale 5.5.100
error: test failed, to rerun pass '--lib'
    Finished test [unoptimized + debuginfo] target(s) in 0.02s
     Running target/debug/deps/notcurses-baa7c69b18166c96

running 2 tests
Term: 78x104 alacritty (alacritty terminal emulator)

 notcurses 1.5.1 by nick black et al
  78 rows, 104 columns (126.75KiB), 256 colors (direct)
  compiled with gcc-9.3.0
  terminfo from ncurses 6.2.20200212
  avformat 58.29.100 avutil 56.31.100 swscale 5.5.100
test tests::stdplane_dims ... ok
Term: 78x104 alacritty (alacritty terminal emulator)

 notcurses 1.5.1 by nick black et al
  78 rows, 104 columns (126.75KiB), 256 colors (direct)
  compiled with gcc-9.3.0
  terminfo from ncurses 6.2.20200212
  avformat 58.29.100 avutil 56.31.100 swscale 5.5.100
test tests::create_context ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

[schwarzgerat](0) $

[dankamongmen]

With the recent work involving redirection support, can we maybe just send the rust unit tests to /dev/null, and still get a meaningful result?

[dankamongmen]

Here's a new one:

test bindgen_test_layout_ncreader_options ... ok
test bindgen_test_layout_ncreel_options ...
error: io error when listing tests:   avformat 58.29.100 avutil 56.31.100 swscale 5.5.100
thread 'tests::create_notcurses_context' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libtest/lib.rsnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'failed printing to stderr: Resource temporarily unavailable (os error 11)', src/libstd/io/stdio.rs:805:9
okerror: test failed, to rerun pass '--lib'
[schwarzgerat](101) $

[dankamongmen]

If I run with cargo test -- --test-threads 1, it always succeeds. So this definitely looks to be a concurrency thing. Unfortunately, it seems that serial_test can only apply to our tests. I don't see any way to apply it to all other tests :/.

[dankamongmen]

rust-lang/rust#43155

[dankamongmen]

https://users.rust-lang.org/t/serializing-against-tests-outside-my-crate/46202

[joseluis]

I personally use a convenience alias for testing, in .cargo/config.toml:

[alias]
t = "test -- --test-threads 1"

[dankamongmen]

@joseluis yeah, i can do that, but that doesn't help with regards to general CI or other people trying the tests. The idiom is just to run cargo test, and that's expected to work, right? Any solution which requires running something else smells to me like it will be incompatible with the rest of the ecosystem, though I'm sure you know better...?

[joseluis]

@dankamongmen there doesn't seem to be a way of configuring the default way of running tests so that it automatically uses n number of threads. The easiest solution sems to be to use an alias (or a script), and tell to use that in the docs.

The only other alternative appears to be not to use the default test-harness and substitute it for custom binaries, but I don't think that's worth it: https://stackoverflow.com/a/43458688/940200