[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[csmith-dev] manual test case reduction

The next step after finding a failing program with csmith is to minimize the program. Regehr just posted a blog entry about delta debugging. Here is my manual algorithm, which i ended up with after fixing some libfirm bugs.

"Try" means remove one by one and check, if the program still fails.

1. Instead of test.c use the preprocessed file to get rid of
   dependencies (runtime/csmith.h).
2. Identify the function (e.g. func_10) where compiler fails (via gdb)
3. Try to set all other func_XX and main to { return 0; }
   So far i only care about compilation failures. If the generated
   program does the wrong stuff, you probably want to keep the main
   function with all the checksum printing.
4. Try to remove calls from func_10
   I do not remove the arguments, so "func_1(x,y)" => "(x,y)".
5. Try to reduce the control flow graph
   1. Remove "else", "continue", "break", "goto XX"
   2. Remove "if (...)", "for (...)", "while (...)"
6. Try to remove calculations (assignments)
7. Try to simplify the control flow graph
   Shorten conditions in remaining if/for/while statements
8. Try to remove variables
   Start from the end of the function, since the removal of a
   declaration could enable the removal of another one.
   Also, try to reuse variables to enable removals.
9. Try to use basic data types. "uint64" => "long long" => "int".
   Ideally, only ints are left afterwards. No "volatile", no "const".
10.Now remove stuff, which is not used by func_10 anymore, which
   usually includes all stuff, which was included, and all func_XX

Now the program should only consist of a main, which does nothing, and func_10, which is hopefully small. Here is an example, which now lives in our testsuite:

Maybe this helps with your work on test case reduction? Maybe someone has a suggestion to improve my workflow?

Andreas Zwinkau

 Karlsruhe Institute of Technology (KIT)
 Institut für Programmstrukturen und Datenorganisation (IPD)
 Lehrstuhl Prof. Snelting
 Adenauerring 20a
 76131 Karlsruhe

 Phone:  +49 721 608 48351
 Fax:    +49 721 608 48457
 Email:  zwinkau@kit.edu
 Web:    http://pp.info.uni-karlsruhe.de/person.php?id=107

 KIT – University of the State of Baden-Wuerttemberg and
 National Research Center of the Helmholtz Association