What I think is that you can get benchmarks to show just about anything you want them to show. Minus maybe graphics cards and game performance (and we've seen vendors release modified drivers to get better performance there!). I'd argue that it's more about looking where your IT groups expertise is, how difficult configuration/setup/etc. is, than a benchmark. If you optimize any platform you're working on, you'll generally get much better results.

Although, I do have to say I've worked on a $20,000+ Sun system, and a $6000 or so Linux box with interesting results. The Sun was a V880, Linux, Dual P3 w/ 2GB Ram and Ultra 160 drives on Raid 5 (hardware). The Sun was using FC-AL drives, 4GB Ram, software raid. The linux box SMOKED the Sun system. Now, it's possible that Sun systems when properly "optimized" can do better - but it's a pain to do so (there's a lot you can configure in Solaris) when compared to Linux or Microsoft (personally, I think Windows is almost as difficult as there's not enough you can control, but I think it comes out of the box better optimized than Solaris). Most of this "speed" impression came from running an Oracle database. And the last personal experience, with default installs of stuff, I've found Linux is easier and better in performance than windows on Oracle work. If nothing else, removing Oracle from a Windows box is a ROYAL pain. Having crap buried everyplace in the registry drives me frigging NUTS. Linux, just remove the files. But, that's a tagent...

With regards to serving web files, from personal experience on straight HTML with some Perl/PHP parsing - I've found that Apache (usually on Linux) is a HECK of a lot easier to clean up, shrink down, etc. than IIS. I've also had better luck securing Apache than IIS. I'll also give definite kudo's to Microsoft for doing a much better job with 2003 server - IIS is MUCH more secure by default - a VERY good thing. .Net vs J2EE - I'm usually more concerned about getting code released. Hardware is cheap, programming time not so cheap. From what I've seen, a lot of times, writing a servlet is a heck of a lot easier than getting a .Net program working fully. And no offense, but the API documentation on J2EE is a HELL OF A LOT BETTER. Everyone I know who tries to write any program on Windows, although C++ and Visual Basic are usually the biggest complaints, complains about a distinct lack of decent API's or documentation - javadocs and the centralized API for the J2SE system are VERY nice.

The last point I'd make on microsoft vs. other systems is for personal use and playing around. I'd argue that for personal use, for an average home user wanting to play, Apache/Java is in many ways an easier system to get up and going. If nothing else, cost wise, it's much cheaper. Someone who's got some old hardware laying around can get linux running on it, with a web server and usually a minimal version of java. It's not usually fast, but it's free - I know a HUGE amount of people who are in this situation. And not all of them are "geeks" - a lot of them are friends who just want to play with web design using HTML and a few CGI's that usually they find on the net.

Last note (ok, second last note ;)) - I thought I'd post this from one of the forums on the article, from Greg Leake at Microsoft (supposedly).
http://www.theserverside.com/news/thread.tss?thread_id=27341#130170

Some interesting comments on this forum I would like to respond to. When Sun published their benchmark numbers, we were taken aback by the very slow performance they showed for .NET, so we decided to investigate. They did not post any code (note in .NET benchmarks we do we do we post code so people can review and replicate if they desire), even though we called and asked them for the code so we could investigate. Nevertheless, if you read Sun's document they actually describe in detail the functional specification for their tests, the hardware/software platform, and their tuning applied to .NET and Java. So we re-created their benchmark, and got startling different results for .NET using just simple code and pretty much out-of-the-box tuning on .NET. In fact, our results were 2-3 times faster than Sun got for .NET, and we wanted to correct Sun's report since it is wrong. If you think any tricky wizards or tuning were used to get our results, check out the code, we published all the code so instead of just flaming or accusing, developers can actually see exactly what was tested and replicate/comment on the results.

We are not sure how Sun got the results they got, probably not intentionally, but somehow they did. Note our Java results closely match Sun's results...its just their .NET results that were off.

Some responses to comments that have been posted:

1) Comment from forum "Why was Tomcat tuned as "Xmx256m" on "2 x 3.06 GHz w/ 2GB RAM"? This matches how Sun tuned Tomcat, on a similar machine with 2GB RAM. So this was the appropriate tuning setting, since we had to precisely match what Sun did in their Java tests to replicate their tests. By the way, the Web Services tested are so simple, changing this setting to a higher number or running multiple instances of Tomcat would not yield better results---but you can verify this by downloading the code and actually running it.

2) Comment from Forum: "VS and .NET have wizards that generate heavy-weight DataSet code by default when creating Web services and while we may have hand-coded for this benchmark, in general VS-generated .NET Web Services are 10 times slower than WebSphere". This is simply not true. VS does nothing to generate heavy-weight code for Web Services (of course you can use notepad and straight C# code if you want!), it simply generates stock WebMethods and you fill in the implementation code. Datasets are NOT used by default, you would have to choose to use them, and they are handy from programming standpoint. However, developer is free to not use datasets as well, its totally a choice. To somehow say MS created 'hand- crafted' code for this benchmark that is out of ordinary or would not be what you got if using Visual Studio properly is simply wrong. Download the code, open it in VS, check it out, and you will see for yourself. Creating Web Services in VS is simple, straightforward, and does not 'generate' overhead code for the implementation. As for Websphere being 10 times faster, well, I have not seen a single benchmark from IBM or other that supports this, but guess what? Anyone can take the code we published and (with some changes to run it in Websphere) test it in Websphere to find out!

At any rate, the purpose of this is not to start a war, but rather to correct results that Sun published becuase they were in error. And it's not all about performance, or .NET vs. J2EE, a single choice. The two technologies are both important, will be used side by side, and need to work together. That's why Web services are so important. But if someone does a benchmark, they should at least publish their code and test harness with their results.

Greg Leake
Microsoft Corporation

************
Wanted to clarify my post real quick - the place from the URL to where my email and website are linked is NOT MY POST, but a REPOST of a comment on another site!!
Thanks!
*************