YOUR FEEDBACK
andy.mulholland wrote: intriguing !!! We have full scale 'Mashup Factories' in Chicago USA and Utrec...


2008 East
DIAMOND SPONSOR:
Data Direct
Frontiers in Data Access: The Coming Wave in Data Services
PLATINUM SPONSORS:
Red Hat
The Opening of Virtualization
Intel
Virtualization – Path to Predictive Enterprise
Green Hills
IT Security in a Hostile World
JBoss / freedom oss
Practical SOA Approach
GOLD SPONSORS:
Software AG
The Art & Science of SOA: How Governance Enables Adoption
PlateSpin
Effective Planning for Virtual Infrastructure Growth
Fujitsu
Automated Business Process Discovery & Virtualization Service
Ceedo
Workspace Virtualization
Click For 2007 West
Event Webcasts

2008 East
PLATINUM SPONSORS:
Appcelerator
Think Fast: Accelerate AJAX Development with Appcelerator
GOLD SPONSORS:
DreamFace Interactive
The Ultimate Framework for Creating Personalized Web 2.0 Mashups
ICEsoft
AJAX and Social Computing for the Enterprise
Kaazing
Enterprise Comet: Real–Time, Real–Time, or Real–Time Web 2.0?
Nexaweb
Now Playing: Desktop Apps in the Browser!
Sun
jMaki as an AJAX Mashup Framework
POWER PANELS:
The Business Value
of RIAs
What Lies Beyond AJAX?
KEYNOTES:
Douglas Crockford
Can We Fix the Web?
Anthony Franco
2008: The Year of the RIA
Click For 2007 Event Webcasts
SYS-CON.TV
TOP THREE LINKS YOU MUST CLICK ON


Help I'm Out Of Memory! Who Has My Memory?
It's Still Necessary to Watch Your Bytes

Many years ago I saved up for a 16K RAM pack for my tiny Sinclair ZX81 computer. I thought, rather naively, that this was going to be the answer to all my memory issues. I would be able to use increasingly complex programs, okay games, and I could program without the restriction of literally making every byte count. I quickly found out, as you have already discovered if you have been writing Java applications for a while, that adding more memory to your machine is not always the answer to the running out-of-memory problem, the infamous "OutofMemoryError". Dr. Phil, a TV psychologist, likes to use the quote, "You can't solve money problems with money." I believe the same thing applies to the JVM, "You can't always solve memory problems, with more memory." Treating just the symptom doesn't have a long-lasting effect.

So why, when Java was going to save us all from thinking about memory allocation, do we now have to think about memory allocation? If you look around, it's obviously a common issue, and there is no shortage of profiling and diagnostic tools. Carlos E. Perez covers a long list of tools in his manageability blog: www.manageability.org/blog/stuff/open-source-profilers-for-java/view and that's not even the full list Java developers have at hand.

Go to any Java conference and you are bound to hear at least one presentation about out-of-memory errors. There were some good sessions at this year's JavaOne. One was a packed BOF led by some of my old colleagues from the Sun serviceability team discussing the six ways to meet an out-of-memory error and another from Steffan and the JRockit JVM team at BEA.

Memory Leaks in Production
The challenge has always been to provide a memory leak tool that your IT staff will let you try in production. Realistically, many slow memory leaks only occur at deployment time, partly due to the changed environment and partly due to the longer uptime of the application. In many cases an IDE-based tool may not be convenient in those scenarios; the other catch is that you also want the diagnostics to have a minimal affect on the application itself. So recompiling to add profiling hooks is often really only the last resort, if at all. To help ease this barrier to adoption, JDK 5.0 included byte code insertion, this technology has been used by profiling tools to do this 're-compilation' at runtime to make this more palatable One thing that you may not be aware of is that the Sun JVM automatically generates some very lightweight counters, available by default in 1.4.2. Sun JVM tools such as jmap and visualgc can provide a view into the garbage collectors operation. The jconsole demo in JDK 5.0 also provides some insight into the garbage collector providing you started the JVM with the option -Dcom.sun.management.jmxremote.

There is also an improved hprof profile agent in JDK 5.0 that can be used to dump out profiling information in a format that can be used by the hat analysis tool from java.net. I find it more useful for snapshotting a JVM, than letting it run to completion. To generate a hprof file I can use QUIT signal, kill -3 on Unix after starting the JVM with the hprof agent -agentlib:hprof=heap=all,format=b. The hat tool looks for the java.hprof output file by default and starts a mini Web server at port 7000. This allows you to browse the object instances used by the JVM. It's by no means as powerful as the commercial tools, but can give you some clues as to what is going on.

Who Has My Memory?
Everyone generally has the same advice about memory leaks in Java. Providing you have already sized your maximum heap correctly (if heap is the detected out-of-memory condition), e.g., -Xmx512m, then some number of objects is holding onto references to other objects that are no longer required. As these objects are still reachable, and hence live, they will not be marked by the garbage collector as objects that can be deleted.

One of the classic examples given is that you wrote some JNI code and forgot to free the memory. Now, admittedly, that's fairly easy to do if you are using JNI, but what if your application has no JNI code at all?

First, rule out your code or dependent libraries or even the JVM. It's less common than it used to be, but JVMs do still have bugs. One area to check is if there are exceptions at deployment time. Sometimes the unsuccessful code paths are not as diligent at cleaning up non-local instances as the successful code path. Even a simple operation, such as creating a substring from a string, and some StringBuffer operations have caused leaks in a few of the 1.4 releases.

Dependent libraries could be an XML parser or JDBC driver. Both need to frequently build, manipulate, and copy chunks of data. Try using an alternative release number or version of both. Again it may not be your code at fault. If you still strongly suspect you've introduced the leak, what should you look at? Caches are prime suspects, or any place where you store references, whether it's a vector, array or Collection can easily always hang on to reachable objects. Make sure you null the references or use weak references. Other areas to check are custom classloaders if you use them. By their very nature they hold on to many references, and if you're using finalizers. remember they may never get called.

Now that desktop machines have 1Gb of memory installed instead of 1K, it's still necessary to watch your bytes. Tracking down a memory leak is never easy, however, there are certainly more tools and techniques at your disposal than ever before.

About Calvin Austin
A section editor of JDJ since June 2004, Calvin Austin is an engineer at SpikeSource.com. He previously led the J2SE 5.0 release at Sun Microsystems and also led Sun's Java on Linux port.

YOUR FEEDBACK
SYS-CON En Espanol wrote: Help I'm Out Of Memory! Who Has My Memory? Many years ago I saved up for a 16K RAM pack for my tiny Sinclair ZX81 computer. I thought, rather naively, that this was going to be the answer to all my memory issues. I would be able to use increasingly complex programs, okay games, and I could program without the restriction of literally making every byte count. I quickly found out, as you have already discovered if you have been writing Java applications for a while, that adding more memory to your machine is not always the answer to the running out-of-memory problem, the infamous 'OutofMemoryError'.
JDJ News Desk wrote: Java Developer's Journal Editorial - Who Has My Memory? Many years ago I saved up for a 16K RAM pack for my tiny Sinclair ZX81 computer. I thought, rather naively, that this was going to be the answer to all my memory issues. I would be able to use increasingly complex programs, okay games, and I could program without the restriction of literally making every byte count. I quickly found out, as you have already discovered if you have been writing Java applications for a while, that adding more memory to your machine is not always the answer to the running out-of-memory problem, the infamous 'OutofMemoryError'.
Gordon Harding wrote: Any suggestions on how to determine if JNI is the issue on out of memory prolems. We have deployed using JVM 1.4.2 and another group has included a COM object using JNI. I would like to lay at their feet but lack the tools to do so.
Gordon Harding wrote: Any suggestions on how to determine if JNI is the issue on out of memory prolems. We have deployed using JVM 1.4.2 and another group has included a COM object using JNI. I would like to lay at their feet but lack the tools to do so.
Gordon Harding wrote: Any suggestions on how to determine if JNI is the issue on out of memory prolems. We have deployed using JVM 1.4.2 and another group has included a COM object using JNI. I would like to lay at their feet but lack the tools to do so.
LATEST JAVA STORIES & POSTS
The one thing that unifies the distributed computing style known as SOA, in most of its manifestations, is self-describing data via the Extensible Markup Language (XML). The benefits of XML over opaque message formats in data interchange are well established. No matter if your fo...
In the past couple of years, interest in Jetty has surged. Jetty is an open source Java-based web and application server and servlet container, but what else do you know about it? To commemorate the 12th anniversary of Jetty, here are 12 things that might surprise you
JavaScript is one of the most interesting and misunderstood programming languages in common use today. Most developers will go their entire careers without realizing its full potential. It's not often that you get a language that supports the feature set that JavaScript does, whi...
JavaScript 2 is becoming increasingly important. Learn how to take advantage of JavaScript 2 while still running in today's browsers. Leverage your current JavaScript and HTML skills to build applications that run in Flash 7-9, DHTML and more with no code changes! OpenLaszlo 4.2 ...
JavaScript is a language with more than its share of bad parts. It went from non-existence to global adoption in an alarmingly short period of time. It never had an interval in the lab when it could be tried out and polished. JavaScript has some extraordinarily good parts. In Jav...
Cloud computing is an opportunity for businesses to implement low-cost, low-power and high-efficiency systems to deliver scalable infrastructure. But moving to a cloud infrastructure is not necessarily as nice and clean as the providers would want you to think. With cloud infrast...
SUBSCRIBE TO THE WORLD'S MOST POWERFUL NEWSLETTERS
SUBSCRIBE TO OUR RSS FEEDS & GET YOUR SYS-CON NEWS LIVE!
Click to Add our RSS Feeds to the Service of Your Choice:
Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Subscribe in NewsGator Online
myFeedster Add to My AOL Subscribe in Rojo Add 'Hugg' to Newsburst from CNET News.com Kinja Digest View Additional SYS-CON Feeds
Publish Your Article! Please send it to editorial(at)sys-con.com!

Advertise on this site! Contact advertising(at)sys-con.com! 201 802-3021


SYS-CON FEATURED WHITEPAPERS

SPONSORED BY INFRAGISTICS
In every field of design one of the first things students do is learn from the work of others. They ...
There are many forces that influence technological evolution. After a decade of building enterprise ...
2008 is going to be an important year for Rich Internet Applications. Most organizations are deliver...
The OpenAjax Alliance is developing an Ajax industry wishlist for future browsers, using a dedicated...
Infragistics announced the availability of two Community Technology Preview (CTP) User Interface (UI...
The YUI development team has released version 2.5.2; you can download the new release from SourceFor...
ADS BY GOOGLE