[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [csmith-dev] arthur's bugs

On Fri, Apr 29, 2011 at 12:10 PM, John Regehr <regehr@cs.utah.edu> wrote:
> Thanks for the details, Arthur!
> It sounds like your process is working well.  If you are going to test GCC
> in this fashion, I may focus our CPU cycles in a LLVM for the time being.
>  I'm just not finding a lot wrong with GCC these days, for the first time in
> several years (I wonder if others besides you are running Csmith?).
> Your reduction process sounds good-- perhaps we can use these tricks in an
> automated reducer.

Oh, definitely.

By the way, I've been wondering about your original motivation for
printing only a final checksum, rather than inserting calls to
printf() throughout the program. Was it just the most obvious way to
avoid winding up with a printf() inside an infinite loop? One of the
first code changes to Csmith that I would investigate would be to
insert printf("%appropriate-specifier", arbitrary-complex-expression)
at random points in the generated code. If an infinite amount of
output worries you, then you could always do something like
    #define printf if (global_counter < 1000) global_counter++, printf
at the top of the file.  (Untested. I bet the above wouldn't work in
gcc's C99 mode because printf is already a macro. But you get the
general idea.)

> Do you know about this delta debugger?  It tends to work very well for crash
> bugs (less so for wrong-code bugs):
>  http://delta.tigris.org/

I've seen you mention it on your blog before, but I've never looked at
it.  I would assume that your long-term goal is to incorporate a
reducer directly into Csmith, though, rather than improving that
standalone reducer.  Csmith already knows the syntax tree, and maybe
has (or will have) some interesting static-analysis factoids that
could be exploited during reduction.  (For example, "g_42 is never
NULL", or, as you mentioned "func_42() is never called from outside
this module, therefore it's okay to change its prototype".)

>> I find that most bugs aren't affected by search-and-replacing all type
>> names with "int". (Actually, I'm pretty sure I'd prefer it if Csmith
>> didn't use typedefs for the integer types in the first place.)
> Can you try compiling Csmith output with the CSMITH_MINIMAL CPP symbol
> defined?  This includes an alternate set of files where (among other things)
> types like int32_t is a macro instead of a typedef.  The result is easier to
> reduce.

If the C code generated by Csmith is identical (and it must be, since
you're talking only about "compiling Csmith output" differently), then
it can't possibly be any easier or harder to reduce by my *current*
method.  I deal entirely with the unpreprocessed output, unless I
discover that the test case actually depends on a safe math function,
at which point I run it through "gcc -E | grep" to figure out the
exact text of the safe math function, then cut-and-paste that text
back onto the top of the unpreprocessed output.