Microsoft Internet Explorer 9 Caught Cheating in the SunSpider Benchmark?

Internet-Explorer-9-Cheats-Benchmark SunSpider is one of the commonly used artificial JavaScript benchmarks for measuring the efficiency of a browser’s JavaScript engine. With Internet Explorer 9, Microsoft managed to pull off significant improvements, and leap frogged Firefox (which has already fired back with JagerMonkey) in the SunSpider benchmark. However, it appears that Microsoft might have resorted to cheating to achieve its considerable gains.

Rob Sayre, Mozilla’s Platform Engineer, was the first one to notice that something is fishy. He documented his observations in a bug report, and published it on his blog. The revelation was soon picked up the tech crowd at Hacker News, who further delved into the matter to dig up some compelling proof. It appears that Internet Explorer 9 is employing some optimizations that are specific to the math-cordic test on SunSpider. While it’s not illegal to come up with clever optimizations that improve performance, it’s important that they are applicable to a wide range of cases. Unfortunately in this case, even simple modifications to the benchmark code, like setting return to true, has been found to significantly alter the results returned by Internet Explorer 9.

It’s possible that this is just the result of an unreliable and narrow implementation of dead code optimization. However, if Internet Explorer is specifically targeting the SunSpider benchmark to work around its tests, then that is a probably a case of cheating.

(Image taken from www.ietestdrive.com)

Update: Rob Sayre has penned a new blog-post, in which he makes it pretty clear that he believes that Internet Explorer 9 is targeting the SunSpider benchmark.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>