Closed
Description
Filing a proposal on behalf of @kcc and @dvyukov:
They request that cmd/go support fuzzing natively, just like it does tests and benchmarks and race detection today.
https://github.com/dvyukov/go-fuzz exists but it's not as easy as writing tests and benchmarks and running "go test -race" today.
Should we make this easier?
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
ianlancetaylor commentedon Feb 15, 2017
I think it would be easier to evaluate the idea if it were slightly less abstract.
For example:
FuzzXxx(f *testing.F, data []byte)
data
testing.F
argument in the usual wayf.Useful()
may be called to indicate useful data, i.e., data that parses correctlyf.Discard()
may be called to indicate that the data should be discardedgo test -fuzz=.
runs the fuzz functions using a regexp like-test
and-bench
go test -fuzz
must also rebuild the package in fuzz mode$GOROOT/pkg
, but where?bradfitz commentedon Feb 15, 2017
@ianlancetaylor, yes,
FuzzXxx(f *testing.F, ...)
is what this is about. The exact API is probably TBD.I think the first step before it's designed completely is to determine whether there's interest.
ianlancetaylor commentedon Feb 15, 2017
As a general concept, I'm in favor.
dsnet commentedon Feb 15, 2017
I would expect that there would be an additional required flag (when fuzzing) where you specify the corpus directory.
ianlancetaylor commentedon Feb 15, 2017
Can we just cache the corpus somewhere under
$GOROOT/pkg
? Are there cases where a typical user would be expected to modify the corpus themselves?dsnet commentedon Feb 15, 2017
I think it's wrong to think of the corpus as strictly a cache. The corpus is the save state of the fuzzer and the documentation for go-fuzz even recommends committing them into a version control system. The
pkg
directory is treated strictly as cache and it is not uncommon for people to recommend clearing out the directory, which will unfortunately delete the fuzzer state.A specified corpus is not so much for the user modify the corpus themselves, but for them to specify how to persist the corpus data.
jimmyfrasche commentedon Feb 15, 2017
Could there be some default convention say a _fuzz/xxx directory (where xxx corresponds with FuzzXxx) and a method on the *testing.F object to load a different corpus from the _fuzz/ directory if necessary? It seems like it should just know where the corpus is.
minux commentedon Feb 15, 2017
cznic commentedon Feb 16, 2017
Quoting @dvyukov
It was very useful for me - found bugs in several lexers.
mvdan commentedon Feb 16, 2017
I use it regularly on a lexer/parser/formatter for Bash (https://github.com/mvdan/sh).
Having it be a first-class citizen would simplify things for me and for contributors.
dsnet commentedon Feb 16, 2017
Found a bug in the C decoder for google/brotli by fuzzing a Go implementation of a Brotli decoder.
Also found some divergences in Go bzip2 decoders from the canonical C decoder (this and #18516). All by fuzzing.
fatih commentedon Feb 16, 2017
My coworker at DigitalOcean was working on a side project to make fuzzing easier. Check his repo out here: https://github.com/tam7t/cautious-pancake Adding it here as I think it would be a valuable piece of information for this discussion.
168 remaining items