Welcome!

Java IoT Authors: Yeshim Deniz, Elizabeth White, Liz McMillan, Pat Romanski, Zakia Bouachraoui

Related Topics: Java IoT, IBM Cloud, Weblogic

Java IoT: Article

Java vs C++ "Shootout" Revisited

Java vs C++ "Shootout" Revisited

Keith Lea writes of the benchmark, on his results page, "I was sick of hearing people say Java was slow, when I know it's pretty fast, so I took the benchmark code for C++ and Java from the now outdated Great Computer Language Shootout and ran the tests myself."

Lea used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2 flag (for both i386 and i686). He compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. He ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop "has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk," he notes.

The results he got were that Java is significantly faster than optimized C++ in many cases.

"They also show that no one should ever run the client JVM when given the choice," Lea adds. ("Everyone has the choice," he says. To run the server VM, see instructions in the Using the Server JVM section below.)

JDJ has agreed to post online anyone else's results as long as they use Java 1.4.2 or higher and any version of GCC that produces faster or equivalent code than the 3.3.1 I used. We encourage you to download the source and/or the binaries and perform the tests yourself, with your favorite compiler and on your favorite platform.


Lea's Data and Results

JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.

Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."

Lea also modified one of the tests, the string concatenation test for Java.

"The test was creating a new StringBuffer in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."

(The updated tests at the original shootout use this new method.)

"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."

Several versions of some of the C++ tests (like matrix) were present in the original shootout source, he continues. 

"I used the versions without numbers in them, like matrix.g++ instead of matrix.g++2. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."

"Looking at them again," Lea says, "matrix.g++3 runs faster than the matrix.g++ that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"

He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."

Lea ran th Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).

"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."

Lea says he's been accused of biasing the results by using the -O2 option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.

But this is not what -O2 does, he points out, referring to the GCC -O documentation:

JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.

Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."

Lea also modified one of the tests, the string concatenation test for Java.

"The test was creating a new StringBuffer in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."

(The updated tests at the original shootout use this new method.)

"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."

Several versions of some of the C++ tests (like matrix) were present in the original shootout source, he continues. 

"I used the versions without numbers in them, like matrix.g++ instead of matrix.g++2. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."

"Looking at them again," Lea says, "matrix.g++3 runs faster than the matrix.g++ that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"

He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."

Lea ran the tests many times before running the "official" recorded set of tests, so there was plenty of time for both Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).

"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."

Lea says he's been accused of biasing the results by using the -O2 option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.

But this is not what -O2 does, he points out, referring to the GCC -O documentation:

-O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code.

"On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think O2 was the best choice," Lea concludes.

 

"I don't have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn't run for me)," he continues. "I really do want people to test it on their own machines, but it's going to take some work, I guess."

Lea compiled the C++ code with:

g++ [test].cpp -O2 -march=i386 -o [test]-386

g++ [test].cpp -O2 -march=i686 -o [test]-686

and the Java code with:

javac [test].java

To see how he ran the binaries, see the run log. You can download the source code he used in either .bz2 or .zip format.

Using the Server JVM

Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."

"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."

Lea explains the two ways to run Java applications with the server VM as follows

  1. When launching a Java application from the command line, use java -server [arguments...] instead of java [arguments...]. For example, use java -server -jar beanshell.jar.
  2. Modify the jvm.cfg file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:
    -client KNOWN
    -server KNOWN
    You should change them to:
    -server KNOWN
    -client KNOWN
    This change will cause the server VM to be run for all applications, unless they are run with the -client argument.

He can be contacted at

Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."

"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."

Lea explains the two ways to run Java applications with the server VM as follows

  1. When launching a Java application from the command line, use java -server [arguments...] instead of java [arguments...]. For example, use java -server -jar beanshell.jar.
  2. Modify the jvm.cfg file in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:
    -client KNOWN
    -server KNOWN
    You should change them to:
    -server KNOWN
    -client KNOWN
    This change will cause the server VM to be run for all applications, unless they are run with the -client argument.

He can be contacted at [email protected].

Links

More Stories By Jeremy Geelan

Jeremy Geelan is Chairman & CEO of the 21st Century Internet Group, Inc. and an Executive Academy Member of the International Academy of Digital Arts & Sciences. Formerly he was President & COO at Cloud Expo, Inc. and Conference Chair of the worldwide Cloud Expo series. He appears regularly at conferences and trade shows, speaking to technology audiences across six continents. You can follow him on twitter: @jg21.

Comments (152)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust tha...
Where many organizations get into trouble, however, is that they try to have a broad and deep knowledge in each of these areas. This is a huge blow to an organization's productivity. By automating or outsourcing some of these pieces, such as databases, infrastructure, and networks, your team can instead focus on development, testing, and deployment. Further, organizations that focus their attention on these areas can eventually move to a test-driven development structure that condenses several l...
The graph represents a network of 1,329 Twitter users whose recent tweets contained "#DevOps", or who were replied to or mentioned in those tweets, taken from a data set limited to a maximum of 18,000 tweets. The network was obtained from Twitter on Thursday, 10 January 2019 at 23:50 UTC. The tweets in the network were tweeted over the 7-hour, 6-minute period from Thursday, 10 January 2019 at 16:29 UTC to Thursday, 10 January 2019 at 23:36 UTC. Additional tweets that were mentioned in this...
Over the course of two days, in addition to insightful conversations and presentations delving into the industry's current pressing challenges, there was considerable buzz about digital transformation and how it is enabling global enterprises to accelerate business growth. Blockchain has been a term that people hear but don't quite understand. The most common myths about blockchain include the assumption that it is private, or that there is only one blockchain, and the idea that blockchain is...
Never mind that we might not know what the future holds for cryptocurrencies and how much values will fluctuate or even how the process of mining a coin could cost as much as the value of the coin itself - cryptocurrency mining is a hot industry and shows no signs of slowing down. However, energy consumption to mine cryptocurrency is one of the biggest issues facing this industry. Burning huge amounts of electricity isn't incidental to cryptocurrency, it's basically embedded in the core of "mini...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
The term "digital transformation" (DX) is being used by everyone for just about any company initiative that involves technology, the web, ecommerce, software, or even customer experience. While the term has certainly turned into a buzzword with a lot of hype, the transition to a more connected, digital world is real and comes with real challenges. In his opening keynote, Four Essentials To Become DX Hero Status Now, Jonathan Hoppe, Co-Founder and CTO of Total Uptime Technologies, shared that ...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the competition, or worse, just keep up. Each new opportunity, whether embracing machine learning, IoT, or a cloud migration, seems to bring new development, deployment, and management models. The results are more diverse and federated computing models than any time in our history.