[csmith-dev] Alignment of struct members cause target dependent checksum

[Nicholas Mc Guire]

On Wed, 05 Jun 2013, John Regehr wrote:

>> would you have a quantifiation/estimate of "quite rare" ?
>
> I'd say we've seen this may 10-15 times out of ~450 total compiler bugs  
> we've reported.
>
> The thing that's sort of interesting about the "wrong in the same way at  
> all optimization level" bugs is that they're different.  Instead of  
> being logic errors in optimization passes (this is what most wrong-code  
> bugs are) they tend to be misinterpretations of the specification by the  
> compiler team.  I'd argue that this kind of bug is probably best found  
> by a standards conformance test suite (several are commercially  
> available).  This kind of thing isn't really Csmith's strength.
>
>> Would it make sense to have a two step process to minimize false positives
>> something like:
>>   1) same compiler -O -O2 -Os (or similar)
>>   2) those that did not trigger in 1) rerun agains compiler A/B/C ?
>
> Are you assuming that in step 2 the compiler might have different  
> implementation-defined characteristics?  If so, then things are a bit  
> tricky since manual work will be needed to sift through the outputs, and  
> that gets old fast.
>
thanks

just realized that my "rational" is totally botched - weeding out the "many"
cases of course does not significantly reduce the test-cases so the number of
test-cases in 2) is de-facto the full set. Just trying to locate methods how
to reduce efforts and find out when to stop a testing-campaign. Will look at
time-stamps next - the (again probably naive) model being that if the input
is truly random then the fault discoveries should actually exponentially
decrease (assuming that the probability of bugs impacting a larger input
space will trigger earlier than those with a small input space) and thus one
(maybe) can do a kind of residual fault prediction from the time-stamps of
the discovered bugs.

thx!
hofrat