I read this blog entry today on "This Code Goes to Eleven" regarding a JVM vs CLR benchmark.
I decided to redo the benchmark. The first thing I did is slightly tweak the Java version (found here) to use System.currentTimeMillis() as opposed to creating a new Date object which IMHO is fairer test with the C# .Net version. The C# version was unchanged.
I then ran the program in both the JVM's client profile and server profile ("java -client BubbleSort" and "java -server BubbleSort respectively").
To provide some background. Sun's JVMs run in two profiles: the server profile which is optimised for environments which have loads of memory and CPU time and doesn't really have UI interaction, whereas the client is designed to work with more constrained UI centric environments.
My machine specs were as follows:
- Windows XP SP2 32-bit
- Intel(R) Core(TM)2 Duo CPU 2.33GHz
- 4096MB of memory
This was the outcome (click to enlarge):
The results are noteworthy; for one thing the client profile for the JVM is considerably slower than either the CLR or the JVM server profile. Furthermore the JVM running in server profile is markedly faster than the CLR version.
But what have I actually proved?
Absolutely nothing really.
To illustrate, lets take this another step: I took the same test and I ran it on my Ubuntu 8.04 partition on the same machine (once again click to enlarge):
As it turns out the Linux version of the JVM running the same program in server mode is slightly faster overall than it's Windows counterpart and the client mode is on average slower than it's Windows counterpart:
And that's the problem with this whole benchmark.
You can never simply make a blanket statement and say Java is faster than .Net. There are several reasons for this.
A JVM is only as fast as it's implementation allows it to be. IBM's JVMs will perform differently to Sun's JVMs to Oracle's JVMs to Apple's JVMs since the guts of the JVM are implemented quite differently for each vendor. Each company's has engineers who will approach the JVM design differently and have a different set of priorities which may get dictated by the market segment within which they compete.
Secondly a JVM is also constrained by it's platform. If you have a JVM which makes a system call which is slow, or has to generate native code which is slow it's going make your program slow. This means that a Java program performs differently across different operating systems.
Thirdly .Net is supposed to be Microsoft's pre-eminent development tool for the Windows platform. It's fair to say that MS can tweak their stuff for Windows like no one else can (In fact many people have complained that they purposefully do this). The fact that Java does beat .Net on Windows is quite surprising
Finally the usage of Java and .Net in the industry has taken the development of the CLR and JVM down different directions. Java - at least from Sun's point of view - took the high road and thus has had a lot of work done for big iron type environments, meaning that their JVM performs better in server environments than client environments. MS on the other hands needs to keep both environments as happy as possible without rocking the boat too much.