| By Jeremy Geelan | Article Rating: |
|
| June 15, 2004 12:00 AM EDT | Reads: |
217,788 |
Keith Lea writes of the benchmark, on his results page, "I was sick of hearing people say Java was slow, when I know it's pretty fast, so I took the benchmark code for C++ and Java from the now outdated Great Computer Language Shootout and ran the tests myself."
Lea used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2 flag (for both i386 and i686). He compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. He ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop "has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk," he notes.
The results he got were that Java is significantly faster than optimized C++ in many cases.
"They also show that no one should ever run the client JVM when given the choice," Lea adds. ("Everyone has the choice," he says. To run the server VM, see instructions in the Using the Server JVM section below.)
JDJ has agreed to post online anyone else's results as long as they use Java 1.4.2 or higher and any version of GCC that produces faster or equivalent code than the 3.3.1 I used. We encourage you to download the source and/or the binaries and perform the tests yourself, with your favorite compiler and on your favorite platform.
Lea's Data and Results

- Data table and graph (also see above)
- Console run log
- Source:
- Binaries
- Download all source code and binaries: .bz2, (69KB) .zip (189KB)
JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.
Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."
Lea also modified one of the tests, the string concatenation test for Java.
"The test was creating a new StringBuffer in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."
(The updated tests at the original shootout use this new method.)
"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."
Several versions of some of the C++ tests (like matrix) were present in the original shootout source, he continues.
"I used the versions without numbers in them, like matrix.g++ instead of matrix.g++2. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."
"Looking at them again," Lea says, "matrix.g++3 runs faster than the matrix.g++ that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"
He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."
Lea ran th Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).
"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."
Lea says he's been accused of biasing the results by using the -O2 option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.
But this is not what -O2 does, he points out, referring to the GCC -O documentation:
JVM startup time was included in these results. "That means even with JVM startup time, Java is still faster than C++ in many of these tests," says Lea.
Some of the C++ tests would not compile. "I've never been very good at decoding GCC's error messages," he admits, "so if I couldn't fix a test with a trivial modification, I didn't include it in my benchmarks."
Lea also modified one of the tests, the string concatenation test for Java.
"The test was creating a new StringBuffer in each iteration of the loop, which was just silly," he explains. "I updated the code to use a single StringBuffer and appending to it inside the loop."
(The updated tests at the original shootout use this new method.)
"Java lost this benchmark even with the modifications," Lea declares. "So if anyone wants to accuse me of biasing the results, they're going to have to try harder."
Several versions of some of the C++ tests (like matrix) were present in the original shootout source, he continues.
"I used the versions without numbers in them, like matrix.g++ instead of matrix.g++2. I don't know which of these were used in the original benchmarks, but from my quick experimenting, the numberless ones generally ran faster than their numbered counterparts."
"Looking at them again," Lea says, "matrix.g++3 runs faster than the matrix.g++ that I use. However, it still runs slower than the Java version, so I don't plan to modify the graph/data unless someone asks me to, since getting that graph in the first place was sort of a pain.)"
He continues: "I've been told that the C++ code for the Method Call benchmark returns by value while the Java code returns by reference, and that modifying the C++ code to pass a pointer makes that benchmark faster. However, even with the modification, the C++ version still runs slower than the Java version."
Lea ran the tests many times before running the "official" recorded set of tests, so there was plenty of time for both Java and the C++ tests to "warm up" (both the Java and C++ tests got faster after he ran them a few times).
"I've been told that these tests are invalid because they were run with GCC," he concedes, adding: "I have seen both benchmarks that show GCC producing faster code than Visual Studio's VC++ compiler, and benchmarks showing the opposite. If I update the benchmarks with another compiler added, it will be the Intel C++ Compiler, which I'm pretty sure produces faster code than VC++."
Lea says he's been accused of biasing the results by using the -O2 option for GCC, "supposedly because -O2 optimizes for space, thus slowing down the benchmark," he explains.
But this is not what -O2 does, he points out, referring to the GCC -O documentation:
-O2: Optimize even more. GCC performs nearly all supported optimizations that do not involve a space-speed tradeoff. The compiler does not perform loop unrolling or function inlining when you specify -O2. As compared to -O, this option increases both compilation time and the performance of the generated code.
"On the other hand, -O3 performs space-speed tradeoffs, and -O performs fewer optimizations. Thus, for these tests, I think O2 was the best choice," Lea concludes.
"I don't have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn't run for me)," he continues. "I really do want people to test it on their own machines, but it's going to take some work, I guess."
Lea compiled the C++ code with:
g++ [test].cpp -O2 -march=i386 -o [test]-386
g++ [test].cpp -O2 -march=i686 -o [test]-686
and the Java code with:
javac [test].java
To see how he ran the binaries, see the run log. You can download the source code he used in either .bz2 or .zip format.
Using the Server JVM
Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."
"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."
Lea explains the two ways to run Java applications with the server VM as follows
- When launching a Java application from the command line, use
java -server [arguments...]instead ofjava [arguments...]. For example, usejava -server -jar beanshell.jar. - Modify the
jvm.cfgfile in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:
You should change them to:-client KNOWN -server KNOWN
This change will cause the server VM to be run for all applications, unless they are run with the-server KNOWN -client KNOWN
-clientargument.
He can be contacted at
Every form of Sun's Java runtime comes with both the "client VM" and the "server VM."
"Unfortunately, Java applications and applets run by default in the client VM," Lea observes. "The Server VM is much faster than the Client VM, but it has the downside of taking around 10% longer to start up, and it uses more memory."
Lea explains the two ways to run Java applications with the server VM as follows
- When launching a Java application from the command line, use
java -server [arguments...]instead ofjava [arguments...]. For example, usejava -server -jar beanshell.jar. - Modify the
jvm.cfgfile in your Java installation. (It's a text file, so you can use Notepad or Emacs to edit it.) This is located in C:\Program Files\Java\j2reXXX\lib\i386\ on Windows, /usr/java/j2reXXX/lib/i386/ on Linux. You will see two lines:
You should change them to:-client KNOWN -server KNOWN
This change will cause the server VM to be run for all applications, unless they are run with the-server KNOWN -client KNOWN
-clientargument.
He can be contacted at keith@kano.net.
Links
- Java vs. C benchmark - the benchmark is in Microsoft Word format
- FreeTTS performance case study - at Sun Research
- Linux Number Crunching - used 1.4.1 VM, got very different results than Lea did, maybe due to floating point optimizations added in 1.4.2
- SNAP matrix benchmark - Java vs. C++ for matrix multiplication
- Nine-Language Performance Round-Up: math and file I/O performance among nine languages
- Performance comparison C++, C# and Java - similar results as Lea's, mostly
- Java vs. C - for bubble sort, IBM's 1.3.0 client JVM comes out on top over GCC with full optimizations
Published June 15, 2004 Reads 217,788
Copyright © 2004 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Jeremy Geelan
Jeremy Geelan is President & COO of Cloud Expo, Inc. and Conference Chair of the worldwide Cloud Expo series. He appears regularly at conferences and trade shows, speaking to technology audiences both in North America and overseas. He is executive producer and presenter of Cloud Expo's "Power Panels" on SYS-CON.TV.
![]() |
Scott Ellsworth 06/16/04 12:18:33 PM EDT | |||
The point is that many C++ users feel that java is too slow and uses too much memory. Microbenchmarks like this show that this off the cuff response is as silly as claiming that exceptions cost too much just because cfront compilers had that cost. I spent seven years doing C++ in a shrink wrap environment, and five doing Java. In my experience with real world apps, Java code gets done faster, works better out of the box, and is easier to maintain. Tricks like custom memory allocators and carefully crafted templates can make faster C++ code, but I reserve those for time critical parts of the program. Where I do not spend some programmer cycles profiling and optimizing, the two environments are pretty close. Thus - if Java is producing code reasonably close or ahead in fairly straightforward usage, and the code gets written faster and more portably, then Java is a win for my work. I can always choose to use C++, assembly, or FORTRAN for the time critical routines, and then link in. Scott |
||||
![]() |
Larry 06/16/04 12:17:20 PM EDT | |||
Calling System.gc() marks someone as a rank neophyte. Garbage collection does not run in paralllel to the main execution. It runs when the main app is not running Sorry if this is insulting, but there is no real reason to ever call it. This was left for backwards compatibility with older vm''s As someone who has developed a jvm, one of the worse things for performance is to manually call System.gc(). This invokes a full gc only which is the most intensive gc possible. Like killing a rat with an atom bomb. Effective but definitely causes collateral issues. Garbage collection since 1.2, 1.3, 1.4 and currently in 1.4.2 have leapt in technology capabilities tremendously. A lot of people seem to be confused about what is done in garbage collection. There are several types of garbage collection in a VM. These can be categorized as incremental or full. Incremental involves sweeping through a smaller space to remove objects making space. It is very fast, primarily due to the nature of the space it works on. As said before, full is the full monty, quite time consuming and linaerily proportional to the heap size. There are several differently type of spaces where objects are kept in the heap. These are called generations. These are new generation, old generation and permanent generation. Incremental gc is done mostly on the new generations All objects created go into the new gen which is composed of an eden (as in garden of ...) where they are first created and two survivor spaces called 1 and 2 where they are moved on each inc gc if they are still needed. These spaces use very efficient algorithms for allocation, movement, and deletion. Once an object has been around for a while, it is transferred to the old generation where it is eventually tenured. The permanent gen is used for interned string (a quoted string in your code), all classes and methods, etc. full gc compacts the old gen and this can be expensive in time. There may be 100 incremental gcs for every full gc but this depends on your code. Gc is triggered whenever certain way points are reached in the different spaces and generations. Every time eden is filled then an inc gc occurred. Since eden is a smaller fraction of the whole heap, this can be quite fast. There are many reasons why an object is collected but it comes down to the type of reference attadched to it of which there are 4 types. Strong references are never collected. I generally run the application and look at the gc behaviour and further tune this with command line parameters to select optimal algorithms based on what I see. I can generally increase gc performance by a factor of 5 because I know what I am doing. Don''t forget that while a the vm is doing gc it is also compiling the byte code based on performance values captured in the last active phase of the app. Each time the app is running, perfomance measurement and benchmarks are run. When gc happens, those portions of the app that are under the heaviest load are optimized. This sort of capability is what gives a runtime optimizer the teeth to do what a static optimization system like those present in C++ can''t. Oh and the sun vm is written in C++ mostly. There is a fair amount of inline asm. |
||||
![]() |
Swen Tromm 06/16/04 11:33:38 AM EDT | |||
José: I think this shall addressed on an other place: But very briefly: What would I like to see is a programming language may be called "C++ strict" which follows C++, but has a stricter type handling (e.g. ending the muddeling of size_t and integer, the weak implemention of bool etc.) and the end of keyword like "export". |
||||
![]() |
SWagner 06/16/04 11:33:26 AM EDT | |||
As a expirenced C++ developer (and Java as well) I only see that the C++ code is optimized for minimum performance. Especially the methodcall where the class hast virtual constructors or destructors which is seldom useful and significantly hurts performance. Only because you cant turn this off in Java does not mean that you have to make it slow in C++. And that is only one example for all the flaws of this test. In my practical experience normal Java code is 1000 times slower to OPTIMIZED C++. However if you make severe design flaws you can make any program slow. And if you base on a bad benchmark and make it even worse you should better stop any benchmarking you planned for the future. |
||||
![]() |
José 06/16/04 11:33:00 AM EDT | |||
And lets not get into Solaris. I have no problems with it, other than Linux or FreeBSD serves me well, because I haven''t got an enterprise to run. |
||||
![]() |
José 06/16/04 11:29:41 AM EDT | |||
I dont mean to be patronizing, it''s just that you stated clearly that I didnt know anything about Java, so people shouldn''t listen to what I have to say. I know very well about Java''s advantages. I agree, C++ doesn''t do cross-platform, but nor was it designed to be. I think it''s great that Sun likes the whole WORA concept, but is it what people really want? Considering the disadvantages? I much prefer diversity. But it''s certainly admirable. I think maybe its a case of being way ahead of their time. You could argue that if speed is all that matters, then one could program in assembly rather than C/C++, for example, but really there was a huge leap between C/C++ and the older languages and the advantages were clear. It was not seen as an appendage of existing technology or just an alternative. This is what I''m trying to say. Java suffers because it hasn''t got that extra oomph to make it as desirable as it wants to be. Where C++ lacks, you always have alternative technologies. See, I don''t believe in encapsulating every technology into one box. Where technologies win is where it does what it does and it does it very well. But I digress. Java will always lose out because of the history and implications of Microsoft/UNIX/Linux. Sun doesn''t have an operating system or application to truly show off with, and maybe thats important, because I don''t think many serious industry types really trust Sun with the future of Java. It has too much control over it. And many people think that Java is the last peg Sun has to stand on. What happens when Sun goes belly up? There are more issues than just the technical side of things. |
||||
![]() |
Very Funny 06/16/04 11:16:55 AM EDT | |||
The great thing about these comments is that this article brought all two dozen C++ coders out of the wood-work. Hehehehehehehehehehehehehehe. That''s a great comment Cameron |
||||
![]() |
Cameron 06/16/04 11:07:44 AM EDT | |||
The great thing about these comments is that this article brought all two dozen C++ coders out of the wood-work. |
||||
![]() |
Lloyd 06/16/04 10:52:40 AM EDT | |||
Jose said "Also, as I mentioned, this feedback forum is about the article stating that Java is faster than C++, which it isn''''t nor was it designed to be. " AFTER ranting about its various IDEs and its utility in serious projects. yes I''ve know of System.gc() (but thanks for being patronising) but honestly I have never used it in serious code. Relying on the JVM''s GC algorithm has always worked out fine thus far for me (though I can understand that it would be needed a lot more in a mobile phone environment). I think the argument I was trying to make was that whilst java undoubtedly doesn''t run as fast as C++ (assuming both programs are written perfectly) it can and has been used for many ''serious'' applications. Or are you discounting Java''s use in web based apps some companies have used it there in very serious ways? You can complain about applet performance but how well does an applet run thats written in C++? whats that? oh it doesn''t run because there is no cross platform client based support for it. I''d say not being possible counts as infinitely slow. sorry, I realise that post was emotive and I know C++ is lovely and all that just that it is not the be all and end all and don''t ya forget that. |
||||
![]() |
José 06/16/04 10:29:01 AM EDT | |||
Well, some people misunderstand the use and reason for Export, which is one of its main problems. http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1426.pdf |
||||
![]() |
Grzegorz Grasza 06/16/04 10:16:34 AM EDT | |||
Try using printf instead of cin/cout... (it''s significantly faster in GCC, at least version 3.3) |
||||
![]() |
Swen Tromm 06/16/04 09:37:57 AM EDT | |||
José wrote:"The value of a language consists >not in how fast it runs, but in >what it enables the programmer to do.Which means a win-win situation to C++ in any case, ...". This is just not true. The strengh of a programming language lies also in what it does not provides. C++ weaknesses are more to see in key words like "export" which are not needed for a well planned programme, but exist. C++ would be a much better language if such ideas would be ereased. |
||||
![]() |
José 06/16/04 09:20:14 AM EDT | |||
Lloyd, thanks for the personal comments, but dont be sad if people will read my comments. I do know what Im talking about and my opinions are not mine only. Namely, the garbage collector. It''s meant to run as a thread parallel to your code, but in reality, in freeing the programmer from the responsibility of recycling memory, clever algorithms are needed and therefore the garbage collector is only as good as the algorithm (plus the overhead on the CPU). As I stated, my opinions are formed of empirical evidence from my own experience. I can even give you snapshots of memory graphs from my own development experiences. Also, this makes it very inconvenient for critical and/or optimized applications that rely on speed and load balancing. Also, does this line ( System.gc() ) not look familiar? Alas, I''ve seen many times the momery usage go up to hundred or so megabytes and then suddenly drop to 15 for example, and not only on my own programs. As for Java hanging and being slow, trust me... It was a fresh install of Windows. Netbeans, leave it running for a while, maybe an hour or so and notice the slow down... maximize on the desktop and Netbeans slows down to a crawl. In particular when using JavaDocs auto-completion, such that I can make some tea and come back in time to see the combo box appear with accompanied JavaDoc definition. Another example is the Wireless Toolkit for J2ME devices, over time compiling the program and running it in an emulator gets really slow. What you can do with the Wireless Toolkit is tell it to output every line of code possible and you get your screen flooding with code that... well... I think the effect would be stronger if you took a look yourself. And we are just talking about a cut down version of J2SE. If you think I''m trying to say that Java is completely rubbish, I apologize but that''s not what I mean to say. However, I just don''t think Java is as great as everyone makes it out to be, especially Java gurus and I think it encourages messy programming, despite what everyone tries to make me think. Which is why I place it between C++ and Visual Basic, for example. But I don''t use it for any serious work nor would I recommend it for serious projects. But it works quite well for some things, for example, I think its brilliant for mobile phone technology. Unfortunately, most programming still requires optimized code these days. And thats really what I''m trying to say. Also, as I mentioned, this feedback forum is about the article stating that Java is faster than C++, which it isn''t nor was it designed to be. |
||||
![]() |
Jim Slade 06/16/04 09:05:35 AM EDT | |||
Reminds me of the benchmarks M$ paid for showing that Visual Basic was faster than Delphi. |
||||
![]() |
Lloyd 06/16/04 08:18:54 AM EDT | |||
it makes me sad how many people will listen to Jose and think he knows whats hes talking about with regards to java. yes, java is sandboxed _When_Running_In_A_Client_JRE_In_A_Browser but for good security reasons and therefore offers a security model not (as far as I know though I may be wrong) provided in C++. I really don''t know what you were going on about the Java GC needing to be manually called for. I''ve been developing real-time java (and C++ to improve performance in key areas) apps for a couple of years and haven''t seen any GC calls in the code. maybe you were hanging on to object references, I don''t know. I will totally agree that C++ is faster to run like for like programs than Java. I won''t agree with statements saying it always produces painfully slow code and programs that hang (mine don''t : ) .... well, maybe sometimes). C++ can live together quite well with Java providing, in my opinion, an environment where I can get some very complex high level things developed rapidly and providing cross platform support whilst certain system aspects can be left to native C++ implementations that are accessed via JNI (Java Native Interface). I haven''t seen any problems with this approach to rapid development of robust real-time apps. ....and relax. |
||||
![]() |
José 06/16/04 07:52:53 AM EDT | |||
>The value of a language consists Which means a win-win situation to C++ in any case, because Java is no revolution in programming terms really, which is why it is and always has been the vague point in the computer industry and why its not really being adopted that much, despite at its matured stage. And clearly, speed is absolutely a serious consideration when it comes to HCI. In any case, people here are not arguing whether or not Java is useful or not or more useful in some contexts. In case you have forgotten, this article was specifically about Java being faster than C++, which is completely wrong. |
||||
![]() |
Trance 9 06/16/04 07:38:09 AM EDT | |||
The value of a language consists not in how fast it runs, but in what it enables the programmer to do. |
||||
![]() |
Trance 9 06/16/04 07:36:25 AM EDT | |||
A good test of a compiler is whether it can take ordinary code and get good performance out of it. It's much less interesting to take optimized code and see how much more the compiler can optimize it. What you're basically saying is if I want to hand optimize every line of code I write, rather than concentrating on the task at hand, then I should use C++, but if I want the compiler to do the heavy lifting then I should use Java? OK--I''ve made my choice! Realistically each language has its place. The JVM is written in C because C is a good systems programming language. For a similar reason the kernel is written in C, and so forth. However that doesn't mean you want to write business logic of any kind in C--you don't. The question is then whether you turn to C++ or to Java or to some other higher level language. |
||||
![]() |
Patrik Beno 06/16/04 07:35:36 AM EDT | |||
This benchmark was made by Java guy who has only a little expertise in C++. I suggest that now it''s time for C++ expert to optimize C++ code and compilation, run the very same tests again and post the results. I would do that but I''m Java guy too :-) So don''t blame Lea, he did his best, now its your turn, C++ experts :-) |
||||
![]() |
Troy Baer 06/16/04 07:34:44 AM EDT | |||
The author doesn''t really explain why he didn''t compile with -O3 aside from a very slight amount of hand-waving about space-speed tradeoffs, which quite frankly I don''t buy. If you''re benchmarking, why wouldn''t you optimize as heavily as possible? If he was really interested in benchmarking this stuff objectively, he could''ve at least shown that there wasn''t much different between -O2 and -O3. Not to mention the question of whether g++ generates good binary code on a given platform... This didn''t exactly fill me with optimism either: I don''t have an automated means of building and benchmarking these things (and the scripts that came with the original shootout didn''t run for me). I really do want you to test it on your own machine, but it''s going to take some work, I guess. This would seem to imply that the author does not know much about either shell scripting or Makefiles. I''m not sure I''m willing to trust benchmarks from somebody who can''t figure out an automated way to build and run them. |
||||
![]() |
miyako 06/16/04 07:32:39 AM EDT | |||
I wish people would just get over this programming language pissing contest that seems to have been going on, and promote each language based on its own merits, and not try to say that any given language is the end all be all of programming. In my experience, Java is fast to write, and C++ is fast to run. C++ can be hard to debug, and Java can make it hard to eliminate overhead. If one day Java is faster than C++, Then I think it will have come at the expense at many of the things that java gets right. |
||||
![]() |
TheLastUser 06/16/04 06:48:19 AM EDT | |||
Its not impossible that Java would be faster at some things than C++, merely because the VM is written in C++. VM''s have runtime advantages over languages like C++. A VM can optimize code based on how it is currently being used and where it is currently running, whereas C++ optimizations are a strictly compile-time exercise. If I create an app running on Linux/x86 and have it send an object to a Solaris/sparc machine, the object will optimize itself properly on both machines. If my application starts to use a particular segment of code heavilly then the VM can re-optimize for that execution pattern. Java takes a hit by requiring a VM, but, as this article shows, the hit isn''t that big and the payoffs are probably worth it. |
||||
![]() |
rebelcool 06/16/04 06:44:56 AM EDT | |||
Java is faster for some applications than C++, and for others C++ is faster. It depends on what you want to do with it. I think java continues to improve its performance and feature set though, with every new version that comes out every couple of years. C++ is more or less into its mature, sunsetting years and not likely to have any revolutionary, or even major evolutionary, improvements. |
||||
![]() |
One Problem 06/16/04 06:43:35 AM EDT | |||
Java and C++ are language. Languages aren''t "faster" or "slower", but compilers for them might be. |
||||
![]() |
Omnifarious 06/16/04 06:38:41 AM EDT | |||
I think Java is highly overrated. I would prefer that a better C++ (a C-like memory model, powerful generic programming, inheritance, and polymorphism) that lacked C++''s current nightmare of strangely interacting features and syntax. I use Python when I don''t need C++s speed or low-level memory model, and I''m happier for it. It''s more flexible than Java, much quicker to develop in, and faster for running small programs. Java doesn''t play well with others, and it was seemingly designed not to. Besides, I suspect that someone who knew and like C++ really well could tweak his benchmarks to make C++ come out faster again anyway. That''s something I''ve noticed about several benchmarks that compare languages in various ways. |
||||
![]() |
MyName 06/16/04 05:44:36 AM EDT | |||
The server VM is faster but consumes more memory??? But high memory comsumption is one of the main problems with Java! And takes even more time to start? |
||||
![]() |
Dr. Valentin Samko 06/16/04 05:36:50 AM EDT | |||
methcall example, win32, g++ 3.3.3 (mingw special), compiled with -O3, results are quite different from what you have. > time java -client methcall_java 1000000000 |
||||
![]() |
José 06/16/04 05:34:27 AM EDT | |||
>i too, am sick of hearing about how slow java is. >with today''''s absurdly fast computers, just-in-time For 2 reasons: 2. Java''s WORA concept is nice but with heavy flaws. For example, Java apparently interprets floating points differently depending on the machine it was written for. People still joke about using applets for a good reason. Because Java has no native OS, it ultimately depends on how well someone can wrap a layer onto the underlying OS or "if" it is possible to do it well at all. Ultimately, Java is best suited for small applications where speed matters little and memory limits are defined (such as mobile phones). Ironic, considering that Java originated from set-top tv boxes. Java, although finding its niche in some markets, is still struggling to find its purpose as being something between "cool, proven and well-liked as C/C++" and "simple to write for and controlled by one monopolistic company Visual Basic". Yeah, I hear lots of Java gurus go one about how Java is totally object orientated and interfaces and bla-dee-bla, but really, that''s all it is, bla-dee-bla... FYI, the Garbage Collector is terrible. Invoking it directly is not recommended by Sun, but you have to, otherwise you watch your memory fill up wondering how the hell your small application could possibly use so much. Then once it reaches some peak, then the GC decides to do something about it, and suddenly you are back to pretty small memory usage and the cycle continues. Good idea in principle, but its a bit like relying on an elephant to clean up your room after you. I prefer to do that job myself. |
||||
![]() |
Jukian Morrison 06/16/04 05:26:26 AM EDT | |||
C++ is lame, so, for the race to be fair, Java has to have one leg strapped behind its back? Rubbish. Java CG is supposed to defer memory stuff into efficient rare operations rather than faffing about with allocation and deallocation in the antique manual style of C. |
||||
![]() |
GooberToo 06/16/04 05:24:34 AM EDT | |||
The more I read the code, it sure is starting to look more and more like an apples and oranges comparison; which is usually what happens when people do java benchmarks. As typically observed, I''m seeing the benchmarks take serious advantage of java''s GC mechanism, whereby, they never pay the piper. With C++, the piper is constantly being paid. All of the benchmarks which allocate objects and then delete them, are therefore, invalid. To be fair, I think, you either need to add a System.gc() line in the java code where C++ is doing its deletes or you need to implement your own new and delete operators to function more in line with what java is doing. Until you do either one of those things, the comparisons where objects are being allocated and deallocated are invalid. And frankly, I''m still not sure adding System.gc() is even fair, on either side. The reason is, calling System.gc() simply hints that it''s a good idea to collect. There is nothing which requires the collection to take place. So, technically, the call could still be many times faster. On the other hand, I don''t know enough about how they handle their gc-hint logic nor am I aware of exactly how much overhead is involved in the actual collection process. If it occurs too often, the shift in workload may be too unfair. Nonetheless, it''s a point of very serious contention. Just for kicks, I modified objinst.java with, "if( i%(n/1000) == 0 ) System.gc() ;", on the lines that the C++ code had it''s delete. When I timed it, it was over twice as slow as the C++ code (24+s vs 55+s). Worse, when I ran it with a 1:1 ratio of delete:System.gc(), I simply got tired of waiting, having waited over 5 minutes. So, basically, I''m not nearly as impressed as I first was. Simplistically, it''s starting to look like a serious apples and oranges comparison. Elsewhere, you can find other examples of just plain bad code. Where again, with correct C++ code, came in about twice as fast at the Java code, whereby, more optimizations were still possible with the C++ code. So, it looks like we''re seeing a combination of things here. Looks like a combination of bad code, ideal corner cases for java''s hot spot, and invalid comparisons with memory allocation. Sadly, I''m once again seriously disappointed in java. |
||||
![]() |
anon 06/16/04 05:21:15 AM EDT | |||
I looked at the Ackermann and Fibonacci implementations. If you used better algorithms for these computations you could beat both the C++ and Java versions he has using just about any implementation language you chose. Both algorithms are practically textbook examples of how a runtime optimizer can beat compile time only optimization of the same algorithm. In both of these situations you compute the exact same value over and over a ridiculous number of times. If you switch to more intelligent algorithms there becomes less runtime optimization opportunities and you have to rely on the optimization of your compiler. Furthermore, you get overhead from the runtime system which will hurt your performance. I did a brief test of just the computation of Fibonacci(45). Java with -server took 21 seconds, GCC 3.5.0 built from CVS head today with good optimization switches took 15 seconds. Using a better algorithm and implementing in Python, the computation took 0.031 seconds(Hello World in Python took 0.029 seconds). |
||||
![]() |
kcpeppe 06/16/04 05:19:07 AM EDT | |||
Though benchmarks are about the "real world", they themselves are not real world. They must ask a question and then answer it. They cannot take into account all questions, only the question that has been asked. If you are evaluating the validity of a benchmark, you must ask, what was the question that was asked and does the benchmark answer it. The usefulness of a benchmark is a much more subjective measure. If you are not doing any EJB work, then EJB benchmarks would not be very useful to you. When we say one language is faster than another, what is the question we are asking. We are saying that the compiler can take equivalent source code and compile it to a more reduced/efficient set of machine codes. Languages are expressions or a notation convenience to express your thoughts about how to solve a particular problem. Some languages are more expressive in a particular problem domain. So, how do we define "equivalent" source code. An even more fundamental question is, which problem domains to we consider. We can''t do the all and we really can''t create equivalent pieces of source code, there really can be no definitive answer to this question. One last point, doesn''t faster hardware make this point moot? The problem is that we still cannot develop applications at the rate desired by those sponsoring projects. Certainly the largest gains to be had come from being able to express ourselves efficiently with our most important tool, language. Kirk Pepperdine |
||||
![]() |
unwesen 06/16/04 05:16:15 AM EDT | |||
c++ relies heavily on the stl, which, being templates, relies heavily on inlining. gcc switches on inlining of methods with the -O3 option, so -O2 is not necessarily realistic in the first place. |
||||
![]() |
NotZed 06/16/04 04:55:59 AM EDT | |||
Java == modern COBOL. Java works well as a COBOL replacement, a backend application language for boring buisiness apps. Thats what it should be benchmarked against. Comparing it to C is like comparing apples and oranges. |
||||
![]() |
No thank you 06/16/04 04:55:35 AM EDT | |||
But what is the point in a benchmark of code that no one would ever use in a production environment. Surely knowing the best a language can do is more useful than knowing what kind of performance a naive implementation produces? It would be interesting to see this shootout re-run with code submitted by people who know the languages well. |
||||
![]() |
TheLittleJetson 06/16/04 04:54:52 AM EDT | |||
However, C++ is faster to LEARN than java - at least if you already know C. |
||||
![]() |
TheLittleJetson 06/16/04 04:51:29 AM EDT | |||
i too, am sick of hearing about how slow java is. i think this claim had a lot more validity back in the day (''98 or so) but now it''s just echoed amongst the idiots. one thing they''ve beaten into us in computer science courses, is that optimization is really only needed in a few critical areas, and the rest of the program wont depend that much on performance. with today''s absurdly fast computers, just-in-time compiling and runtime optimizations, semi-native environments (OSX''s java, GCJ, JNI, or projects combining multiple languages) -- why must this issue constantly be touted every time java is mentioned? |
||||
![]() |
Thank You 06/16/04 04:49:27 AM EDT | |||
To run benchmarks well a person must be diligent, methodical, and objective. Being a hacker is not a prerequisite. These benchmarks look a lot better than many I''ve seen. |
||||
![]() |
Staf Verhaegen 06/16/04 04:41:41 AM EDT | |||
I read your comment about -O2 but I disagree. I certainly think -O3 should be greets, |
||||
![]() |
Swen Tromm 06/16/04 04:07:34 AM EDT | |||
I only had a look into the matrix handling. The handling of the matrix is the typical example of "fine-in-a-text-book-but-do-not-do-so": It is well known that a the handling of pointer to pointer is in respect of performace a bad idea. Any experience coder would build a one-dimensional array and would calulate the pointer "manually"; Soemthing like: int Matrix::CacululatePos (int iPos_x, int iPos_y) |
||||
![]() |
chocolatei 06/16/04 04:03:24 AM EDT | |||
Oh dear. The -server switch seems to work nice though. |
||||
![]() |
Dr. Valentin Samko 06/16/04 03:54:48 AM EDT | |||
This article is completely useless and only shows the lack of understanding of C++ by the author. This reminds me of a person who clamed that qbasic is faster than C++ because his program which outputs a few million blank lines works faster in qbasic than in C++. Why in half of your files you use char* instead of std::string? You still call that C++? Or take strcat.cpp for example, how usefull is that? std::stringstream, or std::deque would be much more suitable there. Also, consider heapsort.cpp, do you really think, this is C++? This is pure C. This benchmark has nothing to do with C++. Claiming that Java is faster than C is stupid at least, because JVM was written in C. matrix.cpp - allocating memory for each matrix row separately only shows you incompetence. This is NOT how it is done by any math library. And again, what does this code have to do with C++, except from std::cout<< call ? This is C code, you really have to change the title of your article to C vs Java, then just remove this article completely, because saying that C is faster than Java is plain stupid (see above). |
||||
![]() |
boxed 06/16/04 03:49:12 AM EDT | |||
Greetings all commentors. Welcome to 1995. Have a nice day. |
||||
![]() |
Iain 06/16/04 02:13:28 AM EDT | |||
The comparison you are trying to make between C++ and Java is not particularly meaningful. At the end of the day, if a VM is faster, then there is nothing to stop you implementing a VM in C++ and running the "C++ program" within that framework. What times would then be valid? The tools you are comparing are very different. In addition, the C++ code is seriously sub-optimal. To take one example, the "matrix.cpp" is definitely not written for speed. A general purpose C function is being used to repeatedly allocate rows, when a single large block of size "mn" could have been used. For that matter, why is malloc( ) being used at all? (A custom memory manager optimised for small blocks could have been used instead - malloc( ) is slow). If you insist on writing simple C, then at least use memset( ) for the zeromatrix( ) function. Fast template matrix libraries exist for C++ - these could have been used instead - the result would have been shorter, clearer and considerably faster. Aside from perhaps demonstrating that you know Java far better than you know C++, I don''t think these tests were a particularly useful exercise. |
||||
![]() |
Lachlan Stuart 06/16/04 01:44:33 AM EDT | |||
The simple truth of the matter is that java cannot be faster than C++. 1. Use useless benchmarks that dont output. Any loop that doesn''t use the output variable for anything will be removed, especially when there is a recompilation/ interpretation stage... see how Via''s mini-ITX computers treat benchmarks, however many C++ compilers let you make useless loops, or only remove useless loops if you inline any involved functions. Most real world apps dont have useless loops, and if they do, they''re inlined enough to be removed. 2. Neglect resource usage. C++ is relatively resource friendly... so nyah :-P 3. Isolate unlikely situations. 4. Use external timers. Though I could list many more, im quite sure only the first example made a considerable impact. What you must realize is that C++ and java have much different targets, its rare that you will encounter a situation where both are equally suitable. No matter what, unless you forget to turn debug compile off, C++ will beat java in real world situations on the performance front...Try praising something else about java...like its security, or ease of programming |
||||
![]() |
Tom 06/16/04 01:43:29 AM EDT | |||
First i feel bad for any programmer who''s manager just read this obviously erroneous benchmark. Ignoring the obviously bad C++ code there are many issues with these tests. The Intel and Microsoft compilers are the best out there and run circles around G++. Since the Java code isn''t really deleting memory ( as it''s GC''d ) then none of the C++ tests should try to delete memory. None of these tests look at resource usage... how much CPU time was used? How much memory was used? In both instances you''ll find most C++ compilers superior to any Java compiler. It''s in the title of the original article... This is nothing, but a Java lovefest. |
||||
![]() |
Howe 06/15/04 10:21:07 PM EDT | |||
This is one of the most ridiculous article I ever read. I won''t even check how you run your benchmarks and methodology: this article was already a big, fat waste of my time. Go study a bit more and don''t ever write again, please. |
||||
![]() |
Jose 06/15/04 08:03:35 PM EDT | |||
You must be joking? I''ve seen Java slow down to a snails pace on the simplest of tasks. It''s fine if you want to continuously "System.out.println("Hello Beans");", but anything that involves immutable objects and the dodgy garbage collector, its a different situation. I can only say this. I''m judging based on my own experiences. I''ve developed in Java several times and can see it''s good points, but I would never accept it as reliable or optimized. All including its irrational treatment of floating point calculations and even using Netbeans as an IDE was so slow, I ended up using a Delphi written IDE called GEL. Admittedly, using a JVM slows things considerably, but nothing as to be so brave to say its on par with any C++ compiler. And, besides, isn''t Java meant to be sandboxed, hence there is always a form of JVM? Suspicious... Finally, was this tested taking into account you can use inline assembly in GCC/C++? |
||||
![]() |
M. Jung 06/15/04 08:01:16 PM EDT | |||
Those "benchmark" results make me laught. The binaries in the archive are not stripped and the C++ code is pretty ineffective. You didn''t proof that java was faster than C++ but you showed that you didn''t invest any time in reading the G++ manpage and know nothing about C++. This must be a joke - let''s hope nobody believes this nonsense. |
||||
![]() |
Chris 06/15/04 07:37:28 PM EDT | |||
Lets face it, gcc is NOT an overly optimising compiler - so comparing it with a highly optimizing java compiler and interpreter is not a comparison of languages, mearly a comparison of tools. Try doing your comparison using a vendor C++ compiler for a particular platform (such as SunONE, etc). You will then see C++ code that is orders of magnitude more efficient. |
||||
- Cloud People: A Who's Who of Cloud Computing
- New Relic Q1 2013 Blazes Past Growth Targets and Reaches 40,000 Active Customer Accounts
- Cloud Expo New York: Delivering Digital Marketing on the Cloud
- Cloud Expo New York: Rethink IT and Reinvent Business with IBM SmartCloud
- The Accessibility of the Cloud
- Cloud Expo NY: Best Practices for Delivering Oracle Database as a Service
- Cloud Expo New York: Basics of SSD Technology and Its Use in Cloud
- Session Topics: 12th Cloud Expo / Cloud Expo New York
- Cloud Expo New York: The Big Challenge of Big Data & Hadoop Integration
- Measuring the Business Value of Cloud Computing
- What CIOs Need to Know About Enterprise Virtualization
- Cloud Expo New York: Build Modern Business Applications
- Cloud People: A Who's Who of Cloud Computing
- Cloud Expo New York: Best CIO Practices Shared from SHI’s Customers
- Cloud Expo New York: How to Use Google Apps Script
- New Relic Q1 2013 Blazes Past Growth Targets and Reaches 40,000 Active Customer Accounts
- Cloud Expo New York: Why Big Data Is Really About Small Data
- Small Cancers, Big Data, and a Life Examined
- Cloud Expo New York: Delivering Digital Marketing on the Cloud
- Cloud Expo New York: Requirements of a Cloud Database
- Cloud Expo New York: Rethink IT and Reinvent Business with IBM SmartCloud
- Cloudant to Exhibit at Cloud Expo & Big Data Expo New York
- The Accessibility of the Cloud
- Learn How To Use Google Apps Script
- A Cup of AJAX? Nay, Just Regular Java Please
- Java Developer's Journal Exclusive: 2006 "JDJ Editors' Choice" Awards
- JavaServer Faces (JSF) vs Struts
- The i-Technology Right Stuff
- Rich Internet Applications with Adobe Flex 2 and Java
- Java vs C++ "Shootout" Revisited
- Bean-Managed Persistence Using a Proxy List
- Reporting Made Easy with JasperReports and Hibernate
- Creating a Pet Store Application with JavaServer Faces, Spring, and Hibernate
- Why Do 'Cool Kids' Choose Ruby or PHP to Build Websites Instead of Java?
- What's New in Eclipse?
- Where Are RIA Technologies Headed in 2008?




























