Google Has Open Sourced Octane, a New JavaScript Benchmark Suite

Posted from: Google Has Open Sourced Octane, a New JavaScript Benchmark Suite

Should now make it easier to test for performance more seamlessly and regularly.

----------------------------

Google has open sourced Octane, a JavaScript benchmarking suite consisting of 13 tests measuring browser performance.

Google has open sourced Octane, a JavaScript benchmarking suite consisting of 13 tests meant to measure the performance of browsers loading and executing complex and large JavaScript applications such as games, interactive and rich web pages and online tools. Octane consists of 8 tests found in the initial V8 Benchmark Suite plus the addition of 5 new ones – pdf.js, Mandreel, GB Emulator, Code Loading, Box2DWeb - that are meant to measure performance areas not covered yet by other tests:

• Richards - OS kernel simulation benchmark, originally written in BCPL by Martin Richards (539 lines).
• Deltablue - One-way constraint solver, originally written in Smalltalk by John Maloney and Mario Wolczko (880 lines).
• Raytrace - Ray tracer benchmark based on code by Adam Burmister (904 lines).
• Regexp - Regular expression benchmark generated by extracting regular expression operations from 50 of the most popular web pages (1761 lines).
• NavierStokes - 2D NavierStokes equations solver, heavily manipulates double precision arrays. Based on Oliver Hunt's code (387 lines).
• Crypto - Encryption and decryption benchmark based on code by Tom Wu (1698 lines).
• Splay - Data manipulation benchmark that deals with splay trees and exercises the automatic memory management subsystem (394 lines).
• EarleyBoyer - Classic Scheme benchmarks, translated to JavaScript by Florian Loitsch's Scheme2Js compiler (4684 lines).
• pdf.js - Mozilla's PDF Reader implemented in JavaScript. It measures decoding and interpretation time (33,056 lines).
• Mandreel - Runs the 3D Bullet Physics Engine ported from C++ to JavaScript via Mandreel (277,377 lines).
• GB Emulator - Emulate the portable console's architecture and runs a demanding 3D simulation, all in JavaScript (11,097 lines).
• Code loading - measures how quickly a JavaScript engine can start executing code after loading a large JavaScript program, social widget being a common example. The source for test is derived from open source libraries (Closure, jQuery) (1,530 lines).
• Box2DWeb - Based on Box2DWeb, the popular 2D physics engine originally written by Erin Catto, ported to JavaScript. (560 lines, 9000+ de-minified)

The benchmark runs in Chrome 14+, Firefox 13+, IE 10, Opera 12 and Safari 5.1.7+ on the desktop, and in the mobile versions of Chrome, Firefox and Opera. It does not run in IE 9 because Microsoft’s browser does not implement WebGL's Typed Arrays, and in several mobile browsers which fail to execute some of the tests: Android Browser, Chrome on iOS 4 (due to iOS restrictions), and Safari on iOS 4.

Octane is more comprehensive than other JavaScript benchmarking tests such as V8, SunSpider, Kraken or Dromaeo. Google mentioned their intent to keep improving the test suite, inviting users to fill in issues reporting performance areas or applications that can be used as a base for more comprehensive JavaScript testing.

The source code of the Octane benchmark is available under a New BSD License.