[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [csmith-dev] feature request: generate memory unsafe code

George, I agree that by generating incorrect code Csmith would probably find a lot of interesting quality-of-implementation issues in compilers.

We have not done so simply because it's often hard to convince people to fix these kinds of issues, and because we considered the wrong-code bugs to be the highest priority.


On 06/08/2011 10:17 PM, George Russell wrote:

I would note that, for my own simplistic random program generators, it
was worth having the generator create illegal (spec violating tests)
for various reasons: fuzzing, checking the front-end does reject
illegal constructs, exercising error handling code paths etc. This is
especially useful in relatively immature compilers.

Of course, its good to ensure that the generator knows the legality of
what it generates!


On 8 June 2011 21:09, John Regehr<regehr@cs.utah.edu>  wrote:
"Mission creep" is a bit strong.  Csmith is a random program generator and
memory unsafety is an almost trivial variation on the existing theme.

However I am back from PLDI with several ideas that will certainly count as
mission creep.  Also Yang says he's asking his nVidia supervisors if he can
contribute back the CUDA-generation work he will do over the summer.

I'd be interested to have the discussion you mention, but the conclusion I
had privately come to is that Csmith is probably just going to become (even
more) a big crappy pile of features and that's OK.  The fix will be to
redesign and reimplement it.


On 06/08/2011 07:17 PM, Eric Eide wrote:

        John>    The purpose of these features is to test static and dynamic
        John>    memory safety checking tools.

The words that come to my mind are "mission creep."

This sounds pejorative, but I don't really mean it to be so.  Testing the
checkers is good.  It's just not what Csmith is designed to do now.

For me, the more general question is how to best deal with the increasing
number of options that we and other people want.  I think that this
some thinking, before the options get completely out of hand.