Hi,
I would note that, for my own simplistic random program generators, it
was worth having the generator create illegal (spec violating tests)
for various reasons: fuzzing, checking the front-end does reject
illegal constructs, exercising error handling code paths etc. This is
especially useful in relatively immature compilers.
Of course, its good to ensure that the generator knows the legality of
what it generates!
Cheers,
George
On 8 June 2011 21:09, John Regehr<regehr@cs.utah.edu> wrote:
"Mission creep" is a bit strong. Csmith is a random program generator and
memory unsafety is an almost trivial variation on the existing theme.
However I am back from PLDI with several ideas that will certainly count as
mission creep. Also Yang says he's asking his nVidia supervisors if he can
contribute back the CUDA-generation work he will do over the summer.
I'd be interested to have the discussion you mention, but the conclusion I
had privately come to is that Csmith is probably just going to become (even
more) a big crappy pile of features and that's OK. The fix will be to
redesign and reimplement it.
John
On 06/08/2011 07:17 PM, Eric Eide wrote:
John> The purpose of these features is to test static and dynamic
John> memory safety checking tools.
The words that come to my mind are "mission creep."
This sounds pejorative, but I don't really mean it to be so. Testing the
checkers is good. It's just not what Csmith is designed to do now.
For me, the more general question is how to best deal with the increasing
number of options that we and other people want. I think that this
requires
some thinking, before the options get completely out of hand.
Eric.