[creduce-dev] A reduction attempt that creduce handled poorly, where delta was able to make progress

[Tony Kelman <tony.kelman@juliacomputing.com>]

Hi! First I'll say creduce is amazing, it is almost magic when it
works well. In https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71505 it
reduced the Cholmod sparse linear algebra library down to a 10-line
ICE. Now I'm trying to get creduce to help with
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77333 where the test case
so far is a large chunk of LLVM.

It's not working well with what I've tried, so maybe this is a case of

> If you find C-Reduce to not work very well for reducing C++ programs, please send us a note including a test case.

or I could be going about it in a way that's doomed from the outset.
Appreciate any input!

The bug I'm looking at involves using a gcc cross-compiler to make a
good and bad build of LLVM, so I've made a Dockerfile based on
opensuse 42.1 (which has a subject-to-the-bug version of the gcc
cross-compiler available in its repos) to reproduce this in a
controlled environment. See
https://raw.githubusercontent.com/tkelman/docker-gcc-bisect/creduce/Dockerfile
for the full Dockerfile recipe.

My interesting-ness test script is currently:

#!/bin/bash -e
cd /opt/llvm/build-good && ninja opt
cd /opt/llvm/build-bad && ninja opt
cd /opt/llvm/llvm-3.7.1.src/lib/Transforms/Vectorize
rm -rf /tmp/.wine*
diff /opt/llvm/build-good/good-output.txt <(timeout 20 wine
/opt/llvm/build-good/bin/opt.exe -slp-vectorizer -S
/opt/llvm/llvm-3.7.1.src/test/Transforms/SLPVectorizer/X86/vector.ll)
! timeout 20 wine /opt/llvm/build-bad/bin/opt.exe -slp-vectorizer -S
/opt/llvm/llvm-3.7.1.src/test/Transforms/SLPVectorizer/X86/vector.ll


and I'm calling creduce on a single file in LLVM, SLPVectorizer.cpp.
This relies on ninja to re-link good (built with -fno-ipa-cp) and bad
(built with default CXXFLAGS) copies of opt.exe, comparing to
known-good output vs a segfault or other error exit code.

The pass statistics when creduce exits say that pass_blank::0 worked
twice, but all other passes failed, so the output file wasn't
meaningfully different than the input file. Using old-fashioned
line-by-line delta with the exact same interestingness script takes a
long time, but does start cutting out lines from the test case and
gives me something about a quarter the original size after its first
run.

Is this my fault or creduce's fault? Any ideas how to help it do
better with test script / setup modifications? I could maybe try some
modifications of the LLVM build system to try to get a single-file
input version of opt.exe, but I'm not sure of a good way to go about
that. Are there some standard helpers for that, maybe using the
intercept-build utility from scan-build?

Thanks,
Tony