CTPI Benchmark & Analysis – There is no other mainstream software category that evolves as fast as the web browser. Recent announcements by Mozilla to upgrade its JavaScript and graphics engine were reason enough for us to compare the developer versions of upcoming IE10, Firefox 9, Chromium 15, Opera 12 and Safari 5.1. And boy, were we surprised!
It has been some time since our last browser performance test and since our last update to our CT Performance Index (CTPI). One of the reasons simply was that the browser speed race in JavaScript performance was largely declared over at the time Firefox 4 was released earlier this year and Google indicated that there would only be marginal improvements in the future, even if JavaScript execution remains the most critical component in future web apps as JavaScript will be the interface to hardware acceleration in technologies such as WebGL and WebCL.
Last week, Mozilla announced that it has found new ways to accelerate Firefox in JavaScript again – we found that claim to be accurate as Firefox 9 was up to 32% faster than Firefox 6 in a quick benchmark run. However, new versions of Firefox (starting with version 7) will also get Mozilla’s new Azure graphics backend that promises much better graphics and Canvas 2D performance that can challenge the class-leading graphics engine in IE9 and IE10. Our JavaScript and HTML5 benchmark index revealed that browser performance has shifted within half a year.
A few notes
Our performance index does not compare absolute benchmark result against absolute benchmark result. In the end, what does a score in WebViz of “2823″ really mean? Our index compares the performance of a browser to the performance of Firefox 3.6.11, which we use as our baseline benchmark with a performance level of 100% in each category. A performance of, for example, 150% would mean that this specific browser delivers 50% more performance than Firefox 3.6.11. Rather than giving our readers absolute numbers, we believe that a relative performance index is much more useful. We are providing the absolute performance numbers in a separate table as well.
Keep in mind that all test results heavily depend on the software and hardware test platform. All our browser tests use an off-the-shelf Dell Studio XPS 7100 PC that Dell has donated to us for the purpose of browser performance evaluation. The hardware includes an AMD Phenom II X6 1055T processor with six CPU cores. The PC has 8 GB of memory and integrates an AMD Radeon 5870 graphics card and runs on an up to date Windows 7 OS. Depending on the hardware in your PC, your benchmark results may vary. However, our benchmark parcours is designed to test a wide variety of browser features and balance the strengths of each browser.
Our JavaScript run includes Sunspider 0.9.1 (50 iterations, weighed at 60%), Mozilla Kraken (20%) and Google V8 (20%). Our HTML5 evaluation includes Asteroids (Canvas 2D effects, 20%), WebViz Bench (25%), Mandelbrot Canvas 2D (15%), GUIMark 2 Chart (10%), GUIMark 2 Gaming (10%), GUIMark 2 (10%), Microsoft Pysychedelic Browsing (5%) and Microsoft Mr. Potato Gun (5%).
The browsers included in this test are:
Firefox 3.6.11 (baseline browser)
IE10 PP2
Firefox 9 Nightly (August 31 build)
Chromium 15.0.871.0 Build 99507
Safari 5.1
Safari 5.1 stands out as it is a stable and not a final version. Apple does not provide easy access to developer versions of its browser and the raw Webkit build does not truly reflect what Apple’s next browser may be. However, we thought it would be interesting to see the difference of this browser in comparison to the developer version of the other upcoming Webkit browser, Chromium. That said, it is no secret that Safari is not a competitive browser on Windows anymore and that is in desperate need of a new installation process, much memory management as well as JavaScript and HTML5 enhancements. So think of Safari as a complementary extra, but not as critical information in this test.
Read on the next page: Benchmark Results
Pages: 1 2
You can leave a response, or trackback from your own site.
