[csmith-dev] Alignment of struct members cause target dependent checksum

[Nicholas Mc Guire]

On Wed, 05 Jun 2013, John Regehr wrote:

> Hi Kees,
>
> It is not generally expected that Csmith programs will have the same  
> results across compilers or platforms except when those  
> compilers/platforms make the same choices for implementation-defined  
> characteristics such as alignment, integer width and representation, etc.
>
> When creating Csmith we had a choice between generating more portable  
> code, which would permit differential testing across more platforms, and  
> less portable code, which (we think) finds more compiler bugs.  We  
> decided to do the second one.  Basically this means that you will need  
> to choose your compilers carefully when doing differential testing.
>
> Most compiler bugs can be found simply by comparing different  
> optimization levels of the same compiler.  We've seen a few bugs where a  
> compiler produces that same wrong result at all optimization levels, but  
> this is quite rare.
>

would you have a quantifiation/estimate of "quite rare" ?

Would it make sense to have a two step process to minimize false positives
something like:
 1) same compiler -O -O2 -Os (or similar)
 2) those that did not trigger in 1) rerun agains compiler A/B/C ?

thx!
hofrat