Re: [csmith-dev] Floating-point support in csmith

To: Shafiul Azam <ishafiul@gmail.com>, John Regehr <regehr@cs.utah.edu>

Subject: Re: [csmith-dev] Floating-point support in csmith

From: Alastair Donaldson <alastair.donaldson@imperial.ac.uk>

Date: Fri, 13 Nov 2015 16:52:58 +0000

Cc: Csmith Dev Mailing List <csmith-dev@flux.utah.edu>

Dkim-signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=imperial.ac.uk; s=main01; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:Cc:References:To:Subject; bh=X74bV0gV49Eed3hsScOBWmvg9JUX6OCZZR/Zzq2SlMA=; b=03SCckhSMlPU047MGxhbRMo/mx PwpS4SxwK3gzqQIcKQkosOdDVcNsOqYYlyNjbK3OYKFS83XPeKkfgdHrgiKIkAeYDlMylUYj5acCP EF79D0a5y6z8YBbwKR5YqhCa3HpaXQtVc9XrHsL5AYsmpufsXe9zTFP7Szrr2aQbNc/E=;

In-reply-to: <CAGoTxu2=QK992wEBPN2An2ovuOgzYVAtZ1O2yDFR_31bBaYHkw@mail.gmail.com>

List-archive: </listarchives/csmith-dev>

List-help: <mailto:csmith-dev-request@flux.utah.edu?subject=help>

List-id: Csmith Development Mailing List <csmith-dev.flux.utah.edu>

List-post: <mailto:csmith-dev@flux.utah.edu>

List-subscribe: <http://www.flux.utah.edu/mailman/listinfo/csmith-dev>, <mailto:csmith-dev-request@flux.utah.edu?subject=subscribe>

List-unsubscribe: <http://www.flux.utah.edu/mailman/options/csmith-dev>, <mailto:csmith-dev-request@flux.utah.edu?subject=unsubscribe>

References: <CAGoTxu2xivya22bt+EtCZaNTTuitToRKp6nim-CzfZUAtjbf4Q@mail.gmail.com> <563A7765.6070203@cs.utah.edu> <CAGoTxu2=QK992wEBPN2An2ovuOgzYVAtZ1O2yDFR_31bBaYHkw@mail.gmail.com>

User-agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0

Hi all

Sorry for the delay in responding.

On 06/11/2015 22:03, Shafiul Azam wrote:

Thank you All very much for responses! To summarize, csmith with --float can be used to detect crash bugs for the moment, and should NOT be used for wrong code bugs. Please correct me if I got it wrong.

Sincerely,

Shafiul

On Wed, Nov 4, 2015 at 3:23 PM, John Regehr <regehr@cs.utah.edu> wrote:

- Floating-point support was introduced in version 2.2.0 according to
the release note [1]. Floating point support must be turned on passing
--float command-line option. This note acknowledges work of Dr. Alastair
Donaldson for the initial implementation.

Yes, please check with Ally, he and his people have been doing this. I think basically all of us are interested but it's been a matter of making time to work on it.

We submitted a pull request, and I believe it still needs to be reviewed. Yang, is this something you might be able to look at if you haven't already? Our pull request fixed some bugs with the way floats were handled, and also added the option of tracking floats via intervals.

It would be good to get this in and I can devote some time to helping with it in a couple of weeks.

A

- I conducted a run of csmith 2.2.0 with --float argument on Ubuntu
10.04 (gcc 4.4.3). Number of crash-programs is significantly more now.
Running csmith without --float found 3 crash-bugs, while running with
--float found more than 300 crash-bugs within the (almost) same runtime.

Nice. There are a few things going on here. As Yang observes, there's some number <300 of bugs that are getting triggered multiple times. To find the true set of bugs you can start to report them, one at a time.

Another thing that is going on is that GCC has built up a huge resistance to Csmith. We've beaten it up for years now. We have not beaten on its FP code so of course there's more low-hanging fruit.

I am interested to know current support-level for floating points, any
limitations and any future works you plan regarding float-point support.

I'll also appreciate any source which explains current support level of
floating-point in csmith.

Hopefully Ally will chime in. You should the source code too, there's not that much that deals with FP, as far as I know.

John