[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [csmith-dev] arthur's bugs



On Thu, Apr 28, 2011 at 9:48 AM, John Regehr <regehr@cs.utah.edu> wrote:
> I wanted to send pointers to some bugs that Arthur O'Dwyer has found and
> reported in GCC:
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48784
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48766
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48739
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48723
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48722
>
> This is great work, and very impressive without an automated reducer (or
> maybe you have your own tools for this?).  I am actually hacking on one of
> our reducers today!
>
> Arthur, I wonder if you could tell us a bit about your methodology?  For
> example, are you randomly picking compiler flags or is there a set you're
> particularly interested in?  Testing flags other than -On is something we've
> always thought would be interesting, but have never attempted.

I don't have an automated reducer; I'm just doing that part by hand.
I'll write a little bit about manual reducing below, but I'm sure it
won't be news to you guys.

For the compile line, I wrote a Perl script [attached go.pl] that does
the following in a loop over all possible seeds starting from a random
point:
  Generate test${seed}.c using "csmith --bitfields --packed-struct -s ${seed}".
  Now do three random compiles of that source file, as follows:
    Pick one of (-O0, -O1, -O2, -O3, -Os, -Ofast)
    Pick one of (-march=native, "-m32 -march=i386", "-march=corei7-avx", ...)
    Pick between two and six of (-fdefault-inline, -fdefer-pop,
-fforward-propagate, ...), possibly negated by inserting "no-" after
the first two characters
    Pick between one and four of (-g, -ggdb, ..., -fpcc-struct-return,
-fshort-enums, ...)
    Concatenate all those options together on a gcc-4.7 command line.
    If the command line matches any regex on a hard-coded blacklist,
throw it out; this reflects the fact that e.g. -fomit-frame-pointer
and -pg cannot be used on the same command line, and it also helps
avoid duplicates of known bugs with certain option combinations.
    Run the command line with "ulimit -t 8"; if it fails, tell how it
failed and save the test case.
  If the test case makes it through all three random compiles without
failing, delete it from the working directory.

The attached "run.pl" is the same thing, slightly expanded to do run
testing as well as compile-only testing. Surprisingly, my run testing
hasn't found any actual bugs yet --- only a couple of obscure things
that don't play well with -fpack-struct= for reasons that are obvious
in retrospect.

I got the list of options by manual extraction from
http://gcc.gnu.org/onlinedocs/gcc/Invoking-GCC.html under "Debugging
Options", "Optimize Options", and "Code Gen Options".  I had to delete
a few options like -fschedule-insns which were just hopelessly broken
on x86.

I started working on a way to reduce the command-line options, but
didn't get any farther than simply replacing -O1, -O2, etc. with their
documented equivalent -f flags. It turned out that it was easier for
me to just reduce the command line by hand at the same time that I was
reducing the test case by hand.  (Besides, I'd started wrong. The
first step in reducing the command line "gcc -O2 ..." should be to see
if the same symptom reproduces with "gcc -O1 ...".)

I've made an SVN checkout of gcc; I build it with the
--program-prefix=ajo- configure option so as not to confuse it with my
system's packaged gcc (which is 4.4.5).  I've also installed the
llvm-gcc package, which claims to be gcc 4.5.1, but I have not been
testing LLVM yet.

To reduce a test case by hand, start at the bottom of the file. Delete
the last function; re-compile; if it still fails, delete the new last
function. Repeat until you've figured out that the failure depends on
func_42() being present.  Try deleting everything except func_42().
If that makes the symptom go away, then try deleting the other
functions one at a time.  Now start back at the bottom of the program
again, working at the statement level. Delete big chunks of func_42().
 The indentation style of Csmith's output actually makes it nice and
easy to delete "else" blocks.  It's also easy to delete a "for (...)"
header, thus simplifying the control flow, without deleting the whole
loop body.

Once you get func_42() down to the bare minimum of statements, do the
same kind of simplification on expressions.  Start on the most deeply
nested subexpression and replace it with (0); then recurse outward,
replacing larger and larger chunks with (0).  Of course humans can
take shortcuts; if it looks like the primary feature of "*g_37 =
func1(func2(0, 1u + g_42[1][7][0]), 23u, *g_37 + 1u)" is that it
dereferences g_37, then I'd typically test that hypothesis by
replacing it with "*g_37 = 0" before doing the long slow reduction.

For the "safe math" functions, I try just replacing them with their
"unsafe" equivalents.  If that fails to reproduce the symptom, adding
(unsigned) casts sometimes helps.  If all else fails, I just
cut-and-paste the preprocessed static inline function into the reduced
test case and then reduce it using the above methods.  Function foo()
in bug 48768 was originally one of the safe math functions; I forget
which.

I find that most bugs aren't affected by search-and-replacing all type
names with "int". (Actually, I'm pretty sure I'd prefer it if Csmith
didn't use typedefs for the integer types in the first place.)
Getting rid of unnecessary array types, "volatile" qualifiers, and
bitfield macros comes next.  By that point, especially after removing
the typedefs, there's nothing left in the test case that depends on
"csmith.h", so I can remove that #include line from the test case.
And then that's pretty much it!

I've used my first bug report as a template when making all my other
reports; that way I don't forget to include the output of "gcc -v" or
the original Csmith command line.


> Also, I wonder if you would mind CC'ing either me or my student Yang
> (chenyang@cs.utah.edu) on these kinds of bugs?  We are very interested not
> just in the bugs, but also in what gcc targets and compiler flags you are
> looking at.

Adding your email to the CC list on the Bugzilla page, you mean?  I can do that.
As for the targets and compiler flags, hopefully the attached Perl
scripts should answer those questions. :)

-Arthur

Attachment: go.pl
Description: Perl program

Attachment: run.pl
Description: Perl program