QuickCheck port for Rust

[toddaaro]

One of the things Haskell is known for is Quick Check, a library for easily generating test cases that test invariants specified by the user. This testing approach has been ported to many languages, and it would be great if Rust ended up one of those languages.

http://en.wikipedia.org/wiki/QuickCheck
http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck

[kud1ing]

See also http://static.rust-lang.org/doc/std/rand.html which provides a Trait for generating random values.

[dbp]

I've started playing around with this. I have a very early implementation here: https://github.com/dbp/rust-quickcheck

I opted not to use the Rand trait for random values, but rather to introduce a different trait, Arbitrary. The reason for this is that I'm not sure if a uniform distribution is actually what is going to be the most useful for testing. For example, when testing vectors, I'd want to bias the empty one relatively high, because I think that's an edge case that is often forgotten. I'm not sure if changing the distribution is the right way to aim for common edge cases, but in case it is... not tying this to Rand seems useful.

The other main trait is Testable - which is a way to mimic lisp apply-like behavior for functions of different arity (this seems to be the approach in the Haskell library - though with a lot more layers of abstraction!)

Thoughts welcome!

[graydon]

I have many thoughts about this! Will try to find time to summarize. Very excited.

[glaebhoerl]

It might also be worth stealing ideas from SmallCheck: http://ro-che.info/articles/2013-02-19-smallcheck.html

[dbp]

Oh, this is relevant :) https://github.com/blake2-ppc/qc.rs

Should have searched more!

[metajack]

You'll want to be able to control the distribution as QuickCheck allows.

Also, we're going to want a Shrink trait, so that we can shrink failing cases to something reasonable.

And will probably want something like eqc_statem for doing model based testing with state.

We're extremely interested in this stuff for Servo.

[bluss]

The qc.rs Arbitrary trait passes down a size parameter. It's quite rudimentary, but it starts with small sizes and gradually increases the mean size. The Shrink thing is using a crazy lazy list thing (because it's interesting), maybe you can come up with something better there.

[killerswan]

So, how's Rust QuickCheck looking lately, @dbp, @mcandre, @blake2-ppc, and @jruderman? What's the state of the art?

[mcandre]

I'm not actively working on it, but I'm happy to accept any pull requests.

[pnkfelix]

Visiting for long delayed triage.

This strikes me as something that's nice to have, but need not be integrated into the central rust distribution.

[BurntSushi]

Just though I'd chime in and say that I've started fresh here: https://github.com/BurntSushi/quickcheck --- it's a pretty faithful port of Haskell's QuickCheck (shrinking, sized arbitrary, properties are polymorphic). Most of the shrinking is lazy with iterators, but not all of it.

[mcandre]

Excellent! I haven't kept up with rust, so I expect BurntSushi/quickcheck would be a lot better than my rust 0.6-based port. And I never added test case simplification, so that's amazing, too.

I'd be happy to deprecate mcandre/rustcheck in favor of this if the rust community likes it.

[sinistersnare]

If #14100 was closed because the Rust team was unsure of whether we actually want QuickCheck in-tree, should this be closed too?

In the same vein as #6167, I'm going to close this. @BurntSushi's library and all the alternatives are amazing but I don't think we need to track their progress in this issue. Please feel free to reopen if you strongly disagree.