On Thu, Apr 28, 2011 at 9:48 AM, John Regehr <regehr@cs.utah.edu> wrote: > I wanted to send pointers to some bugs that Arthur O'Dwyer has found and > reported in GCC: > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48784 > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48766 > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48739 > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48723 > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48722 > > This is great work, and very impressive without an automated reducer (or > maybe you have your own tools for this?). I am actually hacking on one of > our reducers today! > > Arthur, I wonder if you could tell us a bit about your methodology? For > example, are you randomly picking compiler flags or is there a set you're > particularly interested in? Testing flags other than -On is something we've > always thought would be interesting, but have never attempted. I don't have an automated reducer; I'm just doing that part by hand. I'll write a little bit about manual reducing below, but I'm sure it won't be news to you guys. For the compile line, I wrote a Perl script [attached go.pl] that does the following in a loop over all possible seeds starting from a random point: Generate test${seed}.c using "csmith --bitfields --packed-struct -s ${seed}". Now do three random compiles of that source file, as follows: Pick one of (-O0, -O1, -O2, -O3, -Os, -Ofast) Pick one of (-march=native, "-m32 -march=i386", "-march=corei7-avx", ...) Pick between two and six of (-fdefault-inline, -fdefer-pop, -fforward-propagate, ...), possibly negated by inserting "no-" after the first two characters Pick between one and four of (-g, -ggdb, ..., -fpcc-struct-return, -fshort-enums, ...) Concatenate all those options together on a gcc-4.7 command line. If the command line matches any regex on a hard-coded blacklist, throw it out; this reflects the fact that e.g. -fomit-frame-pointer and -pg cannot be used on the same command line, and it also helps avoid duplicates of known bugs with certain option combinations. Run the command line with "ulimit -t 8"; if it fails, tell how it failed and save the test case. If the test case makes it through all three random compiles without failing, delete it from the working directory. The attached "run.pl" is the same thing, slightly expanded to do run testing as well as compile-only testing. Surprisingly, my run testing hasn't found any actual bugs yet --- only a couple of obscure things that don't play well with -fpack-struct= for reasons that are obvious in retrospect. I got the list of options by manual extraction from http://gcc.gnu.org/onlinedocs/gcc/Invoking-GCC.html under "Debugging Options", "Optimize Options", and "Code Gen Options". I had to delete a few options like -fschedule-insns which were just hopelessly broken on x86. I started working on a way to reduce the command-line options, but didn't get any farther than simply replacing -O1, -O2, etc. with their documented equivalent -f flags. It turned out that it was easier for me to just reduce the command line by hand at the same time that I was reducing the test case by hand. (Besides, I'd started wrong. The first step in reducing the command line "gcc -O2 ..." should be to see if the same symptom reproduces with "gcc -O1 ...".) I've made an SVN checkout of gcc; I build it with the --program-prefix=ajo- configure option so as not to confuse it with my system's packaged gcc (which is 4.4.5). I've also installed the llvm-gcc package, which claims to be gcc 4.5.1, but I have not been testing LLVM yet. To reduce a test case by hand, start at the bottom of the file. Delete the last function; re-compile; if it still fails, delete the new last function. Repeat until you've figured out that the failure depends on func_42() being present. Try deleting everything except func_42(). If that makes the symptom go away, then try deleting the other functions one at a time. Now start back at the bottom of the program again, working at the statement level. Delete big chunks of func_42(). The indentation style of Csmith's output actually makes it nice and easy to delete "else" blocks. It's also easy to delete a "for (...)" header, thus simplifying the control flow, without deleting the whole loop body. Once you get func_42() down to the bare minimum of statements, do the same kind of simplification on expressions. Start on the most deeply nested subexpression and replace it with (0); then recurse outward, replacing larger and larger chunks with (0). Of course humans can take shortcuts; if it looks like the primary feature of "*g_37 = func1(func2(0, 1u + g_42[1][7][0]), 23u, *g_37 + 1u)" is that it dereferences g_37, then I'd typically test that hypothesis by replacing it with "*g_37 = 0" before doing the long slow reduction. For the "safe math" functions, I try just replacing them with their "unsafe" equivalents. If that fails to reproduce the symptom, adding (unsigned) casts sometimes helps. If all else fails, I just cut-and-paste the preprocessed static inline function into the reduced test case and then reduce it using the above methods. Function foo() in bug 48768 was originally one of the safe math functions; I forget which. I find that most bugs aren't affected by search-and-replacing all type names with "int". (Actually, I'm pretty sure I'd prefer it if Csmith didn't use typedefs for the integer types in the first place.) Getting rid of unnecessary array types, "volatile" qualifiers, and bitfield macros comes next. By that point, especially after removing the typedefs, there's nothing left in the test case that depends on "csmith.h", so I can remove that #include line from the test case. And then that's pretty much it! I've used my first bug report as a template when making all my other reports; that way I don't forget to include the output of "gcc -v" or the original Csmith command line. > Also, I wonder if you would mind CC'ing either me or my student Yang > (chenyang@cs.utah.edu) on these kinds of bugs? We are very interested not > just in the bugs, but also in what gcc targets and compiler flags you are > looking at. Adding your email to the CC list on the Bugzilla page, you mean? I can do that. As for the targets and compiler flags, hopefully the attached Perl scripts should answer those questions. :) -Arthur
Attachment:
go.pl
Description: Perl program
Attachment:
run.pl
Description: Perl program