I would suggest that you start out dealing only with one or two passes, perhaps the line remover and the token remover. These do some heavy lifting, hopefully never crash, and are always easy to understand.
I think if you do this and don't worry about things like command-line arguments (or working across a network) this should end up being easy to get off the ground.
One area where you'll get a bit of a free pass is process management: the current scheme wants to asynchronously kill obviously-useless work and that is a bit painful. If you have a merge strategy it doesn't sound like you'll ever want to kill off a test.
One place to be careful is in managing the pass state objects, which only make sense in the context of a variant. You have to keep these paired up. I could have included the variant in the pass state object, this would be much cleaner but would not be fast.
John