I decided to redo the benchmark. The first thing I did is slightly tweak the Java version (found here) to use System.currentTimeMillis() as opposed to creating a new Date object which IMHO is fairer test with the C# .Net version. The C# version was unchanged.
I then ran the program in both the JVM's client profile and server profile ("java -client BubbleSort" and "java -server BubbleSort respectively").
To provide some background. Sun's JVMs run in two profiles: the server profile which is optimised for environments which have loads of memory and CPU time and doesn't really have UI interaction, whereas the client is designed to work with more constrained UI centric environments.
My machine specs were as follows:
- Windows XP SP2 32-bit
- Intel(R) Core(TM)2 Duo CPU 2.33GHz
- 4096MB of memory
This was the outcome (click to enlarge):
The results are noteworthy; for one thing the client profile for the JVM is considerably slower than either the CLR or the JVM server profile. Furthermore the JVM running in server profile is markedly faster than the CLR version.
But what have I actually proved?
Absolutely nothing really.
To illustrate, lets take this another step: I took the same test and I ran it on my Ubuntu 8.04 partition on the same machine (once again click to enlarge):
As it turns out the Linux version of the JVM running the same program in server mode is slightly faster overall than it's Windows counterpart and the client mode is on average slower than it's Windows counterpart:
And that's the problem with this whole benchmark.
You can never simply make a blanket statement and say Java is faster than .Net. There are several reasons for this.
A JVM is only as fast as it's implementation allows it to be. IBM's JVMs will perform differently to Sun's JVMs to Oracle's JVMs to Apple's JVMs since the guts of the JVM are implemented quite differently for each vendor. Each company's has engineers who will approach the JVM design differently and have a different set of priorities which may get dictated by the market segment within which they compete.
Secondly a JVM is also constrained by it's platform. If you have a JVM which makes a system call which is slow, or has to generate native code which is slow it's going make your program slow. This means that a Java program performs differently across different operating systems.
Thirdly .Net is supposed to be Microsoft's pre-eminent development tool for the Windows platform. It's fair to say that MS can tweak their stuff for Windows like no one else can (In fact many people have complained that they purposefully do this). The fact that Java does beat .Net on Windows is quite surprising
Finally the usage of Java and .Net in the industry has taken the development of the CLR and JVM down different directions. Java - at least from Sun's point of view - took the high road and thus has had a lot of work done for big iron type environments, meaning that their JVM performs better in server environments than client environments. MS on the other hands needs to keep both environments as happy as possible without rocking the boat too much.
5 comments:
It's quite interesting to see how close the JVM and CLR are in regard to performance, when you take into consideration how much more complex and optimized the JVM is.
I'd like to add to the equation that of cold start and perceived performance. One of the reasons why Java to many is considered to be slow is that it does take time to load up all the classes and do initial interpretation. I have always found .NET applications to come up much quicker, presumably since it always does a JIT and has fewer classes to load (no class loader hierarchy after all).
But isolation and class loader and the right use through OSGI is vital for enterprise class application.
Hard to have an apple to apple comparison between jvm and .net.
Moreover on complex and long running application it would be interesting to see if hotspot engine makes a difference.
jos.
@jos42 Granted I don't know much about OSGI and it has been years since I last did .NET development, but it seems to me the JVM was never really truly meant to host multiple isolated applications. I have certainly seen my share of issues with PermGenSpace over the years on various Java app servers due to class loaders never releasing certain things. To the point that I know prefer to always use a dedicated Tomcat or Jetty to each application - it's just simpler and memory is cheap.
@Casper you can't really compare things like startup time either. Microsoft tends to preload a lot of shared libs which they use for their applications. That's how they get office to load so fast even though it's a resource hog. No doubt they've done something similar for .Net.
@julian
A similar thing happens with Java. The whole rt.jar is expanded and JIT'ed the first time it's run (or installed, can't remember) to a binary form which can be loaded into memory out of the typical class loading and validation scheme. And with JRE6u10, even a Java bootstrap process is kept running at all time.
What I have never understood though, is how come the same isn't done for Java client applications. Applications such as applets and webstart apps obviously live in a cache. It should be easy to serialize the binary image to disk paired with some hash function taking into consideration the amount of RAM and speed of the CPU (if these are different, it might be a good idea to recompile from scratch again). .NET does something along this line and it seems much cleaner than artificially keeping bits and pieces on a hotplace as u10 seems to focus on.
Post a Comment