Macbook vs. Powerbook: Boxing Matches and Big Ugly Tables
Posted by pookzilla on July 12, 2006
I should be ashamed of how long it’s taken me to get these results out of the database. I should be, but I’m not – life happens. 
First, the tale of the tape.
In the red corner wearing silver trunks with silver detail and weighing in as a 1.67Ghz PPC, 1 GB RAM, 5400RPM HHD – Dishwasher! In the blue corner also wearing silver trunks with silver detail and weighing in as a 2.16Ghz Intel Core Duo, 1 GB RAM, 7200RPM HHD – Dishwasher II! The referee this evening will be a 1.4GHz Mac Mini running a Cloudscape database server.
Each machine was configured with a modified (as per previous posts) version of the Eclipse SDK (eclipse-SDK-I20060602-1317-macosx-carbon.tar.gz) and matching JUnit zip (eclipse-junit-tests-I20060602-1317.zip). Each machine was hardwired to the network and set up to send the results of tests to the database server. Each suite was run one time and for the most part all tests passed. As you’ll see in the table below some tests are missing results. The reasons were varied and to my mind irrelevant to the purpose I was trying to achieve. In a perfect world (with more time) I’d find a way to isolate the tests and try running them on their own but I couldn’t find an easy way to do that with the performance framework. Also in this hypothetical “perfect world” the entire suite would’ve been run multiple times. In that perfect world I’d have three laptops, not only two, to sate my internet addictions. But I digress… 
Using a heavily mangled version of View.java I was able to produce the following table. As you can see the average gain is somewhere between 2x and 3x – not quite the 5x Apple has been advertising for the MacBook but impressive and welcome nonetheless. Most of these numbers weren’t too surprising but I was expecting the SWT gains to be more given how much more responsive Eclipse feels. This could be a consequence of the SWT test coverage, however – much of the speed difference I felt was concentrated in text manipulation and the gains in the jdt.text packages seem to be the best of the lot.
The tests on OS X record Elapsed Process, CPU Time, and Java Heap. Presented here are the results for Elapsed Process in big ugly table form. Note the big and ugly!
As I’ve mentioned before getting the tests to run on the Mac wasn’t as easy as I had hoped. There were numerous changes to different script and build files as I have already mentioned. If anyone is interested in reproducing these results I can provide a zip of the testing environment that I used as well as the modified View.java although I’m sure someone less lazy than me could have come up with something far more elegant involving Data Tools and BIRT. Mmm. BIRT.
