Message-oriented middleware is at the core of a vast number of financial services and telecommunications applications. It is also gaining traction in industries such as manufacturing, transportation, healthcare and supply chain management. The Standard Performance Evaluation Corp. (SPEC) today released SPECjms2007, a new benchmark that measures the performance of enterprise message-oriented middleware servers. The benchmark uses the JMS (Java Message Service) standard interface supported by all major enterprise messaging vendors.

The SPECjms2007 benchmark uses a real-world, event-driven application scenario that emulates a supermarket supply chain in which the flow of goods is tracked using state-of-the-art RFID technology. The scenario involves the supermarket company and its stores, distribution centers and suppliers. Interactions within the scenario stress different subsets of the functionality offered by messaging servers, including point-to-point and publish/subscribe communication.

SPECjms2007 is the first industry-standard benchmark designed specifically for enterprise messaging servers based on JMS. It provides a consistent workload and performance metrics for competitive product comparisons, as well as a framework for in-depth performance analysis of enterprise JMS messaging.

SPECjms2007 was developed by SPEC’s Java subcommittee, with contributions from Technische Universität Darmstadt, IBM, Sun, Oracle, BEA, Sybase and Apache.

“This new benchmark provides a much-needed capability for organizations seeking to optimize their enterprise JMS messaging infrastructure,” says Ricardo Morin, chair of SPEC’s Java subcommittee. “Thanks to all those who contributed, especially Technische Universität Darmstadt, which led this effort over the past two years.”

The SPECjms2007 workload can be structured using three different topologies: horizontal, vertical and freeform.

The horizontal topology exercises the ability of the server to handle increasing message traffic by scaling the number of destinations while keeping the traffic per destination constant. The vertical topology exercises the ability of the server to handle increasing message traffic by scaling the rate at which messages are injected through a fixed set of destinations. These two topologies are used for official SPEC benchmark results.

The free-form topology allows users to construct their own workload using SPECjms2007 components as building blocks. The benchmark provides flexibility in customizing the workload, making it an ideal tool for in-depth performance analysis of the JMS server in specific areas of interest.

SPECjms2007 measures the end-to-end performance of all components that make up the application environment, including hardware, JMS server software, JVM software, database software if used for message persistence, and the system network.

The benchmark provides two metrics, SPECjms2007@Horizontal for the horizontal topology and SPECjms2007@Vertical for the vertical topology. Since SPECjms2007 results are based on a unique workload, they cannot be compared to any other benchmark results. SPEC requires that any public reporting of results from the SPECjms2007 benchmark follow its fair use rules, available at www.spec.org/osg/fair_use-policy.html.

SPECjms2007 is available immediately from SPEC for $1,800; qualified non-profit and educational institutions can purchase the benchmark for $450.