Xuejun Yang, Yang Chen, Eric Eide, and John Regehr

Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) 2011.

DOI: 10.1145/1993498.1993532

© Copyright 2011 by ACM, Inc. Posted by permission of ACM; the copies posted here may not be redistributed. The definitive copy of this work is available from the ACM Digital Library.

Award paper.

View PDF, slides, or BibTeX.

areas
Languages, Software Testing

abstract

Compilers should be correct.  To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs.  During this period we reported more than 325 previously unknown bugs to compiler developers.  Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input.  In this paper we present our compiler-testing tool and the results of our bug-hunting study.  Our first contribution is to advance the state of the art in compiler testing.  Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the undefined and unspecified behaviors that would destroy its ability to automatically find wrong-code bugs.  Our second contribution is a collection of qualitative and quantitative results about the bugs we have found in open-source C compilers.