Random Thoughts:

Scripting Languages: Shell..Perl..Python..Ruby..PHP.. and the list goes on .. Proof enough that doing the same things differently, not necessarily more efficiently, safely or quickly, is what drives the IT industry.

Open Source: Giving away the source code to your application doesn't seem to make sense as a business model. Buying software for your enterprise that doesn't come with the source code and locks you into one vendor makes even less sense as a business model.

Character Encoding

Just published a new article that gives a very brief overview of a few character encoding schemes.

[Wed, 1 Mar 2006] The article compares the ASCII, Unicode and ISO 8859 character encoding schemes using two concrete exampes - the Latin alphabets A and the Arabic symbol for the number 8. The idea is to explain the core concepts without getting into the detail - thats what the specifications are for. To read the article, click here.

A new, shiny website

After tiring of a second CMS, the time came to rethink the website and now you can see the results.

[Thur, 26 Jan 2006] A Content Management System is great - you can add new text and pages using your web browser and all the site layout is already there. The problem with CMSs is that to make the site interactive, a lot of functionality must be added which exposes a lot more functionality that can be exploited. Before Christmas I was using b2evolution (v0.9.0.11), a blogging CMS, which I had to remove because of a constant bombardment of robots trying to add links for tracebacks to the site. I believe this vulnerability was fixed in later versions but I used the opportunity to try e107 (v0.6172). e107 is all full featured CMS, built with PHP like b2evolution, that has lots of functionality. However, it has vulberabilities of course and these were expoited before I could upgrade to the latest version.

To me, the problem seems to be that using a CMS requires almost constant upgrading to the very latest version. This is ok if you want to upgrade several hours a week to a website but not ok if the website is a background task. So I've decided to go from using a CMS i.e. GNU/Linux, Apache, PHP and mySQL to using plain old XHTML and CSS i.e. GNU/Linux and Apache. This will definitely reduce the attack vectors and hopefully reduce the attacks. Time will tell and remembering that nothing is 100% secure.

There is a great site called Open Source Web Design where web designers showcase their templates and designs - its like Sourceforge for designers. I found this design that was put together by a guy caled Andreas Viklund. Its pure XHTML and CSS and W3C compliant.

On a last note, if you are interested in trying out some CMSs, this site is great - it lets you try out the CMS without downloading anything.

FRET v0.0.6 - here at last

Arriving some weeks later than planned, the new version includes new functionality, several bug fixes and refactored code.

[Thur, 19 Jan 2006] I've added two new commandine options: --phase and --scan, that allow the user to specify a particular range of tests to run. Several bugs have been fixed and the Group code (and concept) is now gone - this makes the code size more manageable and hasn't removed any currently used functionality.

I also had a chance to analyse the performance of FRET for the first time. As expected, the ScanGrind analysis (it looks for data that is common to two of more files) is the major contributor. Its analysis duration grows exponentially with file size. This can unfortunately lead to very long durations for files over 1 MB - I estimated that it takes 27 hours to analyse two 1.7 MB files. This is not unexpected although I am sure there is room for improvement in the algorithm. However, currently the algorithm is totally brute force - it compares the start of one file with the end of another etc. and I don't think that this makes sense when examining file formats. A more intelligent approach would be to divide a file into zones, maybe based on already detected Grams, and then to look for commonalities between those zones.

FRET v0.0.5 Released

[Wed, 14 Sep 2005] Quoting from the FRET website, " Wednesday 14 September 2005. The worlds press gathered today in front of a small Unix terminal to watch the release of v0.0.5 of FRET into the wild. The tarball was reticent at first, looked back once at the gathered crowd and then swiftly ran away to join its own kind."

Actually, its a bug-fix release with one significant change - the improved ScanGrind function. (As a side note, I'm thinking of spinning off the ScanGrind function as a self-contained C module for other developers to use, maybe even in the Public Domain). Now I need to decide what will go into release 0.0.6. The three main options I am considering at the moment are: - Add a test framework for libfret, probably using CUnit - Add a way of measuring FRET's functionality i.e. how much of a set of test files does it correctly analyse. - Make libfret thread-safe I've also noticed that FRETs been added to the FileFormat site in the tools section - always good to see some interest!

FRET v0.0.5 Design Changes

[Fri, 02 Sep 2005] Currently work is progressing on FRET version 0.0.5 and the major change for this version is a swap in the order of the Phase 4 and Phase 5 Scans. An improvement in ScanGrind was the driving force for this change. Previously, when a data structure, which was common to two Buffers, was identified by ScanGrind, a Group Gram was created with the offset from the first Buffer. This caused the loss of valuable information about the structures location in the second Buffer. The new version of ScanGrind now creates two Buffer Grams, one for each Buffer and no Group Gram is created. A Phase 5 Scan will then notice that two Buffer Grams are identical and will create a Group Gram with frequency 2.

There is an article available describing the design changes in more detail.

mick mccarthy / software engineer