Apt: A Platform for Repeatable Research in Computer Science
ACM SIGOPS Operating Systems Review 49(1), January 2015.
DOI: 10.1145/2723872.2723885
areas
Networking,
Operating Systems,
Testbeds,
Software Testing
abstract
Repeating research in computer science requires more than just code and data: it requires an appropriate environment in which to run experiments. In some cases, this environment appears fairly straightforward: it consists of a particular operating system and set of required libraries. In many cases, however, it is considerably more complex: the execution environment may be an entire network, may involve complex and fragile configuration of the dependencies, or may require large amounts of resources in terms of computation cycles, network bandwidth, or storage. Even the "straightforward" case turns out to be surprisingly intricate: there may be explicit or hidden dependencies on compilers, kernel quirks, details of the ISA, etc. The result is that when one tries to repeat published results, creating an environment sufficiently similar to one in which the experiment was originally run can be troublesome; this problem only gets worse as time passes. What the computer science community needs, then, are environments that have the explicit goal of enabling repeatable research. This paper outlines the problem of repeatable research environments, presents a set of requirements for such environments, and describes one facility that attempts to address them.