Arriving some weeks later than planned, the new version includes new functionality, several bug fixes and refactored code.
I've added two new commandine options: --phase and --scan, that allow the user to specify a particular range of tests to run. Several bugs have been fixed and the Group code (and concept) is now gone - this makes the code size more manageable and hasn't removed any currently used functionality.
I also had a chance to analyse the performance of FRET for the first time. As expected, the ScanGrind analysis (it looks for data that is common to two of more files) is the major contributor. Its analysis duration grows exponentially with file size. This can unfortunately lead to very long durations for files over 1 MB - I estimated that it takes 27 hours to analyse two 1.7 MB files. This is not unexpected although I am sure there is room for improvement in the algorithm. However, currently the algorithm is totally brute force - it compares the start of one file with the end of another etc. and I don't think that this makes sense when examining file formats. A more intelligent approach would be to divide a file into zones, maybe based on already detected Grams, and then to look for commonalities between those zones.
Posts
