[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [csmith-dev] Complicatedness limits might need a little tweaking

Hi Arthur, you're right.

The solution we've discussed is to put caps on the sizes of more constructs. For example I don't think there's any cap right now on total bytes of global variables. There are limits on function size, struct size, array size, and probably a bunch more that I'm forgetting about. Not very elegant, but probably it solves the problem.

Currently our driver scripts (the nasty internal ones, not the ones we released) put a timeout on Csmith as well as on everything else :(.


On 05/07/2011 01:38 AM, Arthur O'Dwyer wrote:
For the most part, Csmith generates pretty reasonable medium-sized
test cases, but it is still possible for it to get carried away. I
just ran across this unusual case where Csmith 2.0.0 was generating a
main() function of more than 6000 lines --- thanks to hundreds of
global variables, a surprising number of which were structs containing
structs containing structs.

csmith --bitfields --packed-struct -s 785395829>test.c
wc test.c ; md5sum test.c
    9043  36116 839634 test.c
0814cae40428306881450005f3d23a6c  test.c

I mention it just in case it inspires anyone to figure out a better
way to limit the worst-case complicatedness of tests.  (Or is limiting
worst-case complicatedness even a reasonable goal? The occasional
ridiculously huge test could be seen as a positive thing, by someone
who's not me.)