A software QA engineer walks into a bar. He orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers.
First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.
Using symbolic execution to simultaneously test all edge cases is a really powerful idea. DeepState is especially powerful since it'll generate test cases you can use to fix bugs and then use them in your traditional CI tests.
>it'll generate test cases you can use to fix bugs and then use them in your traditional CI tests
That's what I like about hypothesis, it will try to generate minimal testing cases and save them for later. If you're a Python dev and you like what DeepState does, consider hypothesis too.
DeepState provides a Google Test-compatible interface to writing C++ unit tests; however, underneath it all, it is really a C unit testing framework. That is the reason for some of the strange naming of functions like DeepState_Int: these are the underlying C interfaces. If you're using C++, you can choose to use Symbolic<int> or symbolic_int. However, if your codebase is pure C, then have no fear, DeepState can still help you!
I've been doing a similar thing in Rust with cargo-fuzz and the `arbitrary` crate. One thing that could have been made more clear is that a big advantage of something like DeepState is that the fuzzers it uses support instrumentation guided fuzzing, which is exponentially more powerful than the simple random fuzzing from regehr's original post.
A software QA engineer walks into a bar. He orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers.
First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.
Using symbolic execution to simultaneously test all edge cases is a really powerful idea. DeepState is especially powerful since it'll generate test cases you can use to fix bugs and then use them in your traditional CI tests.
>it'll generate test cases you can use to fix bugs and then use them in your traditional CI tests
That's what I like about hypothesis, it will try to generate minimal testing cases and save them for later. If you're a Python dev and you like what DeepState does, consider hypothesis too.
https://hypothesis.readthedocs.io/en/latest/
DeepState provides a Google Test-compatible interface to writing C++ unit tests; however, underneath it all, it is really a C unit testing framework. That is the reason for some of the strange naming of functions like DeepState_Int: these are the underlying C interfaces. If you're using C++, you can choose to use Symbolic<int> or symbolic_int. However, if your codebase is pure C, then have no fear, DeepState can still help you!
I've been doing a similar thing in Rust with cargo-fuzz and the `arbitrary` crate. One thing that could have been made more clear is that a big advantage of something like DeepState is that the fuzzers it uses support instrumentation guided fuzzing, which is exponentially more powerful than the simple random fuzzing from regehr's original post.
Good point! I added a little paragraph emphasizing the reasons you want libFuzzer etc. over a dumb fuzzer, in the long run.
The second part is already available: https://blog.trailofbits.com/2019/01/23/fuzzing-an-api-with-...