[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [creduce-dev] [RFC] Switching from Perl to Python

Hi Moritz, I need to give your version a careful read and run it against the default version-- will get back to you. Thanks!


On 6/19/16 9:08 PM, Moritz Pflanzer wrote:
Hi John,

It took me a good time longer than I expected but I now my Python version of C-Reduce is complete. I ported all passes to Python and except for the "skip key" feature all options are supported. The skip-key feature seems indeed to be quite a problem under Linux as there is no non-blocking read function which reads only a single character (not followed by ENTER). For now I stopped trying to implement the feature to see if there is further interest at all.

I pushed the current version to the "python" branch of my repository at: https://github.com/mpflanzer/creduce/tree/python
I added a second table to the spreadsheet for a speed comparison for the complete set of passes: https://docs.google.com/spreadsheets/d/1FIvuHr29X2T2H2wOrnGCU0BUM3NeQrvJY_GpKMVJRCA/edit?usp=sharing

Yes, absolutely, there's no requirement for a line-by-line rewrite. I'll have to look at the code to find things I'm unhappy with, but the short version is I've rewritten the C-Reduce core enough times that I'm pretty happy with it.

I think I found a few minor bugs in some passes which I fixed in the Python version. So if the Python version has no future on its own I will compare it against the Perl version and report the issues I found.

The only major change -- I think -- is that interestingness test are now Python modules, i.e. Python scripts. Currently each script has to define a "run" and a "check" function. Both function take a list of test cases as inout, "check" has to return True or False depending on the interestingness of the test cases and "run" has to exit with either 0 or 1; also depending on the interestingness.
Except for the fact that it have to be Python the "API" of the script could easily be changed. They have to be Python scripts because this allows to use the "multiprocessing" module which is platform independent and supports everything that is needed to run the interestingness tests as separate process and to wait on any process to finish.

One thing I've wanted is a way to better specify the pass ordering, which is kind of hacky.  It would also be nice to allow users to customize the passes using a little file.  Also we'd like to customize the passes based on the language that is being reduced.

At the top-level passes are now represented in groups. This allows to make different configurations based on languages etc. Each group consists of three subgroups (first, main, last) which itself contain a list with passes. The priority is now implied by the position in the list.
The downside is that the passes have to be repeated for each group but this layout makes it easy to take for instance a JSON file as input to define a custom group of passes (not implemented yet).

Regarding the passes, there's some pretty bad stuff in the regex-based passes.  These are basically the original C-Reduce code from the mid/late 2000s.  It would be great to find better ways to do these rewrites.

I hope the nested regular expressions are now a bit nicer and maybe even faster. I had to write a custom parser for nested expressions as there seems to be no build-in approach. Currently it is more like a prototype -- though fully functional -- and I am sure it could be further improved. Both in terms of ease of use and performance.

I am happy about any questions or comments. In the meantime I will try to keep my version up-to-date with the master branch of C-Reduce.