Re: [csmith-dev] endless loops?

[Eric Eide <eeide@cs.utah.edu>]

Detlef Riekenberg <wine.dev@web.de> writes:

> How long do you run the compiles programs?

I would run them for a "human-short" "computer-long" time limit, like maybe
four or five seconds or so.  If a Csmith-generated program doesn't terminate
after 5 seconds, there is a pretty good chance that it will never terminate.

> Does csmith tries to avoid endless loops?

Csmith-generated programs are not guaranteed to terminate.

We wrote this in our conference paper about Csmith: "It is not difficult to
generate random programs that always terminate.  However, we judged that this
would limit Csmith's expressiveness too much: for example, it would force loops
to be highly structured.  Additionally, always-terminating tests cannot find
compiler bugs that wrongfully terminate a non-terminating program.  (We have
found bugs of this kind.)  About 10% of the programs generated by Csmith are
(apparently) non-terminating.  In practice, during testing, they are easy to
deal with using timeouts."

You can get the full paper, if you're interested, at:
http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf

> Then this seems to be a bug, otherwise this is an idea for an enhancement.

Sure!  This would be an interesting enhancement for sure.

Eric.

-- 
-------------------------------------------------------------------------------
Eric Eide <eeide@cs.utah.edu>  . University of Utah Kahlert School of Computing
https://www.cs.utah.edu/~eeide/ . +1 801-585-5512  .  Salt Lake City, Utah, USA