Welcome!

Java Authors: Carmen Gonzalez, Pat Romanski, Victoria Livschitz, Elizabeth White, Liz McMillan

Related Topics: Java, SOA & WOA, Oracle

Java: Book Excerpt

Book Excerpt: Java Application Profiling Tips and Tricks - Part 2

Identifying lock contention in a profile

This excerpt is from the book, ‘Java Performance' by Charlie Hunt and Binu John, published in the Addison-Wesley Professional Java Series, ISBN 0137142528, Sept. 2011, Copyright 2012 Oracle America, Inc. For more info please visit: www.informit.com/title/0137142528

In the book Java Performance, Chapter 5, "Java Application Profiling," presented the basic concepts of using a modern Java profiler such as the Oracle Solaris Studio Performance Analyzer and NetBeans Profiler. It did not, however, show any specific tips and tricks in using the tools to identify performance issues and approaches of how to resolve them. This is the purpose of this excerpt. Its intention is to show how to use the tools to identify performance issues and take corrective actions to resolve them. This excerpt looks at several of the more common types of performance issues the authors have observed through many years of working as Java performance engineers.

In Part 1, we discussed Performance Opportunities and System or Kernel CPU Usage. In Part 2 we will discuss identifying lock contention in a profile,  approaches to reducing lock contention, and identifying data structure resizing .

In early JVM releases, it was common to delegate Java monitor operations directly to operating system monitors, or mutex primitives. As a result, a Java application experiencing lock contention would exhibit high values of system CPU utilization since operating system mutex primitives involve system calls. In modern JVMs Java monitors are mostly implemented within the JVM in user code rather than immediately delegating them to operating system locking primitives. This means Java applications can exhibit lock contention yet not consume system CPU. Rather, these applications first consume user CPU utilization when attempting to acquire a lock. Only applications that experience severe lock contention may show high system CPU utilization since modern JVMs tend to delegate to operating system locking primitives as a last resort. A Java application running in a modern JVM that experiences lock contention tends to show symptoms of not scaling to a large number of application threads, CPU cores, or a large number of concurrent users. The challenge is finding the source of the lock contention, that is, where are those Java monitors in the source code and what can be done to reduce the lock contention.

Finding and isolating the location of highly contented Java monitors is one of the strengths of the Oracle Solaris Performance Analyzer. Once a profile has been collected with the Performance Analyzer, finding the highly contented locks is easy.

The Performance Analyzer collects Java monitor and lock statistics as part of an application profile. Hence, you can ask the Performance Analyzer to present the Java methods in your application using Java monitors or locks.

Tip
You can also view locks used within the JVM with the Performance Analyzer, but that requires setting the presentation view mode to Machine Mode.

By selecting the View > Set Data Presentation menu in Performance Analyzer and choosing the Metrics tab, you can ask the Performance Analyzer to present lock statistics, both inclusive or exclusive. Remember that inclusive lock metrics include not only the lock time spent in a given method but also the lock time spent in methods it calls. In contrast, exclusive metrics report only the amount of lock time spent in a given method.

Figure 10 shows the Performance Analyzer's Set Data Presentation form with options selected to present both inclusive and exclusive lock information. Also notice the options selected report both the time value and the percentage spent locking.

After clicking OK, the Performance Analyzer displays the profile's lock inclusive and exclusive metrics in descending order. The arrow in the metric column header indicates how the data is presented. In Figure 11, the lock data is ordered by the exclusive metric (notice the arrow in the exclusive metric header and note the icon indicating an exclusive metric).

Figure 10: Set user lock data presentation

Figure 11: Java monitors/locks ordered by exclusive metric

Tip
Before blindly looking only at lock metrics in Performance Analyzer, an application should be exhibiting scalability symptoms. The classic scaling symptoms occur when executing an application on a system with a large number of CPUs, CPU cores, or hardware threads does not show an expected scaling in performance throughput relative to a system with a smaller number of CPUs, CPU cores, or hardware threads, or leaves CPU utilization unused. In other words, if an application is not showing scaling issues, then there is no need to investigate an application's locking activity.

The screenshot taken in Figure 11 is from a simple example program (complete source code for the remaining examples used in this chapter can be found in Appendix B, "Profiling Tips and Tricks Example Source Code") that uses a java.util.HashMap as a data structure to hold 2 million fictitious tax payer records and performs updates to those records stored in the HashMap. Since this example is multithreaded and the operations performed against the HashMap include adding a new record, removing a new record, updating an existing record, and retrieving a record, the HashMap requires synchronized access, that is, the HashMap is allocated as a synchronized Map using the Collections.synchronizedMap() API. The following list provides more details as to what this example program does:

  • Creates 2 million fictitious tax payer records and places them in an in-memory data store, a java.util.HashMap using a tax payer id as the HashMap key and the tax payer's record as the value.
  • Queries the underlying system for the number of available processors using the Java API Runtime.availableProcessors() to determine the number of simultaneous Java threads to execute concurrently.
  • Uses the number returned from Runtime.availableProcessors() and creates that many java.util.concurrent.Callable objects to execute concurrently in an allocated java.util.concurrent.ExecutorService pool of Executors.
  • All Executors are launched and tax payer records are retrieved, updated, removed, and added concurrently by the Executor threads in the HashMap. Since there is concurrent access to the HashMap through the actions of adding, removing, and updating records, HashMap access must be synchronized. The HashMap is synchronized using the Collections.synchronizedMap() wrapper API at HashMap creation time.

From the preceding description, it should be of little surprise this example program experiences lock contention when a large number of threads are trying to concurrently access the same synchronized HashMap. For example, when this program is run on a Sun SPARC Enterprise T5120 Server configured with an UltraSPARC T2 processor, which has 64 virtual processors (the same value as that returned by the Java API Runtime.availableProcessors()), the performance throughput reported by the program is about 615,000 operations per second. But only 8% CPU utilization is reported due to heavy lock contention. Oracle Solaris mpstat also reports a large number of voluntary thread context switches. In Chapter 2, the "Memory Utilization" section talks about high values of voluntary thread context switches being a potential indicator of high lock contention. In that section, it is said that the act of parking a thread and awaking a thread after being notified both result in an operating system voluntary context switch. Hence, an application experiencing heavy lock contention also exhibits a high number of voluntary context switches. In short, this application is exhibiting symptoms of lock contention.

Capturing a profile of this example program with the Performance Analyzer and viewing its lock statistics, as Figure 11 shows, confirms this program is experiencing heavy lock contention. The application is spending about 59% of the total lock time, about 14,000 seconds, performing a synchronized HashMap.get() operation. You can also see about 38% of the total lock time is spent in an entry labeled <JVM-System>. You can read more about this in the "Understanding JVM-System Locking" sidebar. You can also see the calls to the put() and remove() records in the synchronized HashMap as well.

Figure 12 shows the Callers-Callees of the SynchronizedMap.get() entry. It is indeed called by the TaxPayerBailoutDBImpl.get() method, and the SynchronizedMap.get() method calls a HashMap.get() method.

Understanding JVM-System Locking
A JVM-System entry in Performance Analyzer indicates time spent within the JVM internals. In the context of looking at lock contention statistics in Performance Analyzer, this is the amount or percentage of time spent in locks within the internals of the JVM. This may sound alarming when looking at the amount of time spent in the JVM-System in Figure 11.

Figure 12: Callers-Callees of synchronized HashMap.get()

Hence, this requires a little further explanation and clarification. Recall from Chapter 5 that switching from a Data Presentation Format of User mode to either Expert mode or Machine mode shows the internal operations of the JVM and puts them in the JVM-System entry seen in User mode. Also remember that switching to Expert mode or Machine mode also shows highly contended Java monitors as a form of a _lwp_mutex, __lwp_cond_wait, or __lwp_park type of entry and isolates the locking within Java APIs with those found within the JVM. Figure 13 shows the same profile but is switched from User mode to Expert mode in the Performance Analyzer.

Comparing Figure 11 to Figure 13 suggests the JVM-System entry has resolved into __lwp_condition_wait and __lwp_park operations. The sum of the __lwp_condition_wait and __lwp_park are close to what is reported for JVM-System in Figure 11. Your initial reaction may be the JVM is also experiencing lock contention. However, selecting the __lwp_cond_wait entry and selecting the Callers-Callees tab and walking up the call stack, the source of the locking activity associated with __lwp_cond_wait, in other words the locking activity associated with the JVM-System entry, is shown in Figure 14.

All five of the methods shown in Figure 14 are internal JVM methods. Notice that over 95% of the attributable lock time is spent in GCTaskManager::get_task(unsigned).

Figure 13: Switching from User mode to Expert mode

Figure 14:Traversing up the call stack of callers of __lwp_cond_wait

This method is part of the garbage collection subsystem of the Java HotSpot VM. This garbage collection method blocks and waits on a queue for work to do on behalf of the garbage collector subsystem. Each of the method names listed in Figure 14 represent areas of the Java HotSpot VM that may block and wait for some work to be placed on their respective work queue. For example, the VMThread::loop() method blocks on a queue for work to do on behalf of the Java HotSpot VM. You can think of the VMThread as the "kernel thread" of the Java HotSpot VM. The CompilerBroker::compile_thread_loop() method blocks and waits for work to do on behalf of the JIT compilation subsystem and so on. As a result, the entries reported as the JVM-System entry in User Mode can be ignored as being hot locks in this profile.

Continuing with the example program, the reaction from many Java developers when he or she observes the use of a synchronized HashMap or the use of a java.util.Hashtable, the predecessor to the synchronized HashMap, is to migrate to using a java.util.concurrent.ConcurrentHashMap.1 Following this practice and executing this program using a ConcurrentHashMap instead of a synchronized HashMap showed an increase of CPU utilization of 92%. In other words, the previous implementation that used a synchronized HashMap had a total CPU utilization of 8% while the ConcurrentHashMap implementation had 100% CPU utilization. In addition, the number of voluntary context switches dropped substantially from several thousand to less than 100. The reported number of operations per second performed with the ConcurrentHashMap implementation increased by a little over 2x to 1,315,000, up from 615,000 with the synchronized HashMap. However, seeing only a 2x performance improvement while utilizing 100% CPU utilization compared to just 8% CPU utilization is not quite what was expected.

Tip
When performance testing, observing an unexpected result or observing a result that looks suspicious is a strong indication to investigate performance results and revisit testing methodology.

Capturing a profile and viewing the results with the Performance Analyzer is in order to investigate what happened. Figure 15 shows the hot methods as java.util.Random.next(int) and java.util.concurrent.atomic.AtomicLong.compareAndSet (long, long).

Using the Callers-Callees tab to observe the callers of the java.util.concurrent.atomic.AtomicLong.compareAndSet (long, log) method shows java.util.Random.next(int) as the most frequent callee. Hence, the two hottest methods in the profile are in the same call stack; see Figure 16.

java.util.concurrent.ConcurrentHashMap was introduced in the Java 5 SE class libraries and is available in Java 5 and later Java JDKs/JREs.

Figure 15: Hot methods in the ConcurrentHashMap implementation of the program

Figure 16: Callers of AtomicLong.compareAndSet

Figure 17: Callers and callees of Random.nextInt(int)

Figure 17 shows the result of traversing further up the call stack of the callers of Random.next(int). Traversing upwards shows Random.next(int) is called by Random.nextInt(int), which is called by a TaxCallable.updateTaxPayer(long, TaxPayerRecord) method and six methods from the BailoutMain class with the bulk of the attributable time spent in the TaxCallable.updateTaxPayer(long, TaxPayerRecord) method.

The implementation of TaxCallable.updateTaxPayer(long, TaxPayerRecord)  is shown here:

final private static Random generator = BailoutMain.random;
// these class fields initialized in TaxCallable constructor
final private TaxPayerBailoutDB db;
private String taxPayerId;
private long nullCounter;
private TaxPayerRecord updateTaxPayer(long iterations,
TaxPayerRecord tpr) {
if (iterations % 1001 == 0) {
tpr = db.get(taxPayerId);
} else {
// update a TaxPayer's DB record
tpr = db.get(taxPayerId);
if (tpr != null) {
long tax = generator.nextInt(10) + 15;
tpr.taxPaid(tax);
}
}
if (tpr == null) {
nullCounter++;
}
return tpr;
}

The purpose of TaxCallable.updateTaxPayer(long, TaxPayerRecord) is to update a tax payer's record in a tax payer's database with a tax paid.  The amount of tax paid is randomly generated between 15 and 25. This randomly generated tax is implemented with the line of code, long tax = generator.nextInt(10) + 15. generator is a class instance static Random that is assigned the value of BailoutMain.random which is declared in the BailoutMain class as final public static Random random = new Random(Thread.currentThread().getId()).  In other words, the BailoutMain.random class instance field is shared across all instances and uses of BailoutMain and TaxCallable. The BailoutMain.random serves several purposes in this application.  It generates random fictitious tax payer ids, names, addresses, social security numbers, city names and states which are populated in a tax payer database, a TaxPayerBailoutDB which uses a ConcurrentHashMap in this implementation variant as its storage container. BailoutMain.random is also used, as described earlier, to generate a random tax for a given tax payer.

Since there are multiple instances of TaxCallable executing simultaneously in this application, the static TaxCallable.generator field is shared across all TaxCallable instances. Each of the TaxCallable instances execute in different threads, each sharing the same TaxCallable.generator field and updating the same tax payer database.

This means all threads executing TaxCallable.updateTaxPayer(long, TaxPayerRecord)trying to update the tax payer database must access the same Random object instance concurrently. Since the Java HotSpot JDK distributes the Java SE class library source code in a file called src.zip, it is possible to view the implementation of java.util.Random. A src.zip file is found in the JDK root installation directory. Within the src.zip file, you can find the java.util.Random.java source code. The implementation of the Random.next(int) method follows (remember from the Figure 6-17 that Random.next(int) is the method that calls the hot method java.util.concurrent.atomic.AtomicLong.compareAndSet(int,int)).

private final AtomicLong seed;
private final static long multiplier = 0x5DEECE66DL;
private final static long addend = 0xBL;
private final static long mask = (1L << 48) - 1;
protected int next(int bits) {
long oldseed, nextseed;
AtomicLong seed = this.seed;
do {
oldseed = seed.get();
nextseed = (oldseed * multiplier + addend) & mask;
} while (!seed.compareAndSet(oldseed, nextseed));
return (int)(nextseed >>> (48 - bits))
;}

In Random.next(int), there is a do/while loop that performs an AtomicLong.compareAndSet(int,int) on the old seed and the new seed (this statement is highlighted in the preceding code example in bold). AtomicLong is an atomic concurrent data structure. Atomic and concurrent data structures were two of the features added to Java 5. Atomic and concurrent data structures typically rely on some form of a "compare and set" or "compare and swap" type of operation, also commonly referred to as a CAS, pronounced "kazz".

CAS operations are typically supported through one or more specialized CPU instructions. A CAS operation uses three operands: a memory location, an old value, and a new value. Here is a brief description of how a typical CAS operation works. A CPU atomically updates a memory location (an atomic variable) if the value at that location matches an expected old value. If that property fails to hold, no changes are made. To be more explicit, if the value at that memory location prior to the CAS operation matches a supplied expected old value, then the memory location is updated with the new value. Some CAS operations return a boolean value indicating whether the memory location was updated with the new value, which means the old value matched the contents of what was found in the memory location. If the old value does not match the contents of the memory location, the memory location is not updated and false is returned.

It is this latter boolean form the AtomicLong.compareAndSet(int, int) method uses. Looking at the preceding implementation of the Random.next(int) method, the condition in the do/while loop does not exit until the AtomicLong CAS operation atomically and successfully sets the AtomicLong value to the nextseed value. This only occurs if the current value at the AtomicLong's memory location has a value of the oldseed. If a large number of threads happen to be executing on the same Random object instance and calling Random.next(int), there is a high probability the AtomicLong.compareAndSet(int, int) CAS operation will return false since many threads will observe a different oldseed value at the AtomicLong's value memory location. As a result, many CPU cycles may be spent spinning in the do/while loop found in Random.next(int). This is what the Performance Analyzer profile suggests is the case.

A solution to this problem is to have each thread have its own Random object instance so that each thread is no longer trying to update the same AtomicLong's memory location at the same time. For this program, its functionality does not change with each thread having its own thread local Random object instance. This change can be accomplished rather easily by using a java.lang.ThreadLocal. For example, in BailoutMain, instead of using a static Random object, a static ThreadLocal<Random> could be used as follows:

// Old implementation using a static Random
//final public static Random random =
//                    new Random(Thread.currentThread.getid());
// Replaced with a new ThreadLocal<Random>
final public static ThreadLocal<Random> threadLocalRandom =
new ThreadLocal<Random>() {
@Override
protected Random initialValue() {
return new Random(Thread.currentThread().getId());
}
};

Then any reference to or use of BailoutMain.random should be replaced with threadLocalRandom.get(). A threadLocalRandom.get() retrieves a unique Random object instance for each thread executing code that used to use BailoutMain.random. Making this change allows the AtomicLong's CAS operation in Random.next(int) to succeed quickly since no other thread is sharing the same Random object instance. In short, the do/while in Random.next(int) completes on its first loop iteration execution.

After replacing the java.util.Random in BailoutMain with a ThreadLocal<Random> and re-running the program, there is a remarkable improvement performance. When using the static Random, the program reported about 1,315,000 operations per second being executed. With the static ThreadLocal<Random> the program reports a little over 32,000,000 operations per second being executed. 32,000,000 operations per second is almost 25x more operations per second higher than the version using the static Random object instance. And it is more than 50x faster than the synchronized HashMap implementation, which reported 615,000 operations per second.

A question that may be worthy of asking is whether the program that used the synchronized HashMap, the initial implementation, could realize a performance improvement by applying the ThreadLocal<Random> change. After applying this change, the version of the program that used a synchronized HashMap showed little performance improvement, nor did its CPU utilization improve. Its performance improved slightly from 615,000 operations per second to about 620,000 operations per second. This should not be too much of a surprise. Looking back at the profile, the method having the hot lock in the initial version, the one that used a synchronized HashMap, and shown in Figure 6-11 and Figure 6-12, reveals the hot lock is on the synchronized HashMap.get() method. In other words, the synchronized HashMap.get() lock is masking the Random.next(int) CAS issue uncovered in the first implementation that used ConcurrentHashMap.

One of the lessons to be learned here is that atomic and concurrent data structures may not be the holy grail. Atomic and concurrent data structures rely on a CAS operation, which in general employs a form of synchronization. Situations of high contention around an atomic variable can lead to poor performance or scalability even though a concurrent or lock-free data structure is being used.

Many atomic and concurrent data structures are available in Java SE. They are good choices to use when the need for them exists. But when such a data structure is not available, an alternative is to identify a way to design the application such that the frequency at which multiple threads access the same data and the scope of the data that is accessed is minimized. In other words, try to design the application to minimize the span, size, or amount of data to be synchronized. To illustrate with an example, suppose there was no known implementation of a ConcurrentHashMap available in Java, that is, only the synchronized HashMap data structure was available. The alternative approach just described suggests the idea to divide the tax payer database into multiple HashMaps to lessen the amount or scope of data that needs to be locked. One approach might be to consider a HashMap for tax payers in each state. In such an approach, there would be two levels of Maps. The first level Map would find one of the 50 state Maps. Since the first level Map will always contain a mapping of the 50 states, no elements need to be added to it or removed from it. Hence, the first level Map requires no synchronization. However, the second level state maps require synchronized access per state Map since tax payer records can be added, removed, and updated. In other words, the tax payer database would look something like the following:

public class TaxPayerBailoutDbImpl implements TaxPayerBailoutDB {
private final Map<String, Map<String,TaxPayerRecord>> db;
public TaxPayerBailoutDbImpl(int dbSize, int states) {
db = new HashMap<String,Map<String,TaxPayerRecord>>(states);
for (int i = 0; i < states; i++) {
Map<String,TaxPayerRecord> map =
Collections.synchronizedMap(
new HashMap<String,TaxPayerRecord>(dbSize/states));
db.put(BailoutMain.states[i], map);
}
}
...

In the preceding source code listing you can see the first level Map is allocated as a HashMap in the line db = new HashMap<String, Map<String, TaxPayerRecord>>(dbSize) and the second level Map, one for each of the 50 states is allocated as a synchronized HashMap in the for loop:

for (int i = 0; i < states; i++) {
Map<String,TaxPayerRecord> map =
Collections.synchronizedMap(
new HashMap<String,TaxPayerRecord>(dbSize/states));
db.put(BailoutMain.states[i], map);
}

Modifying this example program with the partitioning approach described here shows about 12,000,000 operations per second being performed and a CPU utilization of about 50%. The number of operations per second is not nearly as good as the 32,000,000 observed with a ConcurrentHashMap. But it is a rather large improvement over the single large synchronized HashMap, which yielded about 620,000 operations per second. Given there is unused CPU utilization, it is likely further partitioning could improve the operations per second in this partitioning approach. In general, with the partitioning approach, you trade-off additional CPU cycles for additional path length, that is, more CPU instructions, to reduce the scope of the data that is being locked where CPU cycles are lost blocking and waiting to acquire a lock.

Volatile Usage
JSR-133, which was introduced in Java 5, addressed many issues in the Java Memory Model. This is well documented at http://jcp.org/jsr/detail?id=133 by the JSR-133 Expert Group with further material at http://www.cs.umd.edu/~pugh/java/memoryModel/ maintained by Dr. Bill Pugh. One of the issues addressed with JSR-133 is the use of the Java keyword volatile. Fields in Java objects that are declared as volatile are usually used to communicate state information among threads. The inclusion of JSR-133 into Java 5 and later Java revisions, ensures that a thread that reads a volatile field in an object is guaranteed to have the value that was last written to that volatile field, regardless of the thread that is doing read or write, or the location of where those two threads are executing, that is, different CPU sockets, or CPU cores. The use of a volatile field does limit optimizations a modern JVM's JIT compiler can perform on such a field. For example, a volatile field must adhere to certain instruction ordering. In short, a volatile field's value must be kept in sync across all application threads and CPU caches. For instance, when a volatile field's value is changed by one thread, whose field might be sitting in a CPU cache, any other thread that might have a copy of that volatile field in its CPU cache, a different CPU cache than the other thread that performed the change, must have its CPU cache updated before its thread reads that volatile field found in its local CPU cache, or it must be instructed to retrieve the updated volatile field's value from memory. To ensure CPU caches are updated, that is, kept in sync, in the presence of volatile fields, a CPU instruction, a memory barrier, often called a membar or fence, is emitted to update CPU caches with a change in a volatile field's value.

In a highly performance sensitive application having multiple CPU caches, frequent updates to volatile fields can be a performance issue. However, in practice, few Java applications rely on frequent updates to volatile fields. But there are always exceptions to the rule. If you keep in mind that frequent updates, changes, or writes to a volatile field have the potential to be a performance issue (i.e., reads of a volatile field are okay, not a cause for performance concern), you will likely not experience performance issues when using volatile fields.

A profiler, such as the Performance Analyzer, that has the capability to gather CPU cache misses and associate them to Java object field access can help isolate whether the use of a volatile field is a performance issue. If you observe a high number of CPU cache misses on a volatile field and the source code suggests frequent writes to that volatile field, you have an application that is experiencing performance issues as a result of its usage of volatile. The solution to such a situation is to identify ways in which less frequent writes are performed to the volatile field, or refactor the application in a way to avoid the use of the volatile field. Never remove the use of a volatile field if it breaks program correctness or introduces a potential race condition. It is much better to have an underperforming application than it is to have an incorrect implementation, or one that has the potential for a race condition.

Data Structure Resizing
Java applications tend to make high use of Java SE's StringBuilder or StringBuffer for assembling Strings and also make high use of Java objects that act as containers of data such as the Java SE Collections classes. Both StringBuilder and StringBuffer use an underlying char[] for their data storage. As elements are added to a StringBuilder or StringBuffer, the underlying char[] data storage, may be subject to resizing. As a result of resizing, a new larger char[] array is allocated, the char elements in the old char[] are copied into the new larger char[] array, and the old char[] discarded, that is, available for garbage collection. Similar resizing can also occur in Java SE Collections classes that use an array for their underlying data store.

This section explores ways to identify data structure resizing, in particular StringBuilder, StringBuffer, and Java SE Collections classes resizing.

StringBuilder/StringBuffer Resizing
When a StringBuilder or StringBuffer becomes large enough to exceed the underlying data storage capacity, a new char array of a larger size, 2x larger in the OpenJDK StringBuilder and StringBuffer implementation (used by Java HotSpot Java 6 JDK/JRE), is allocated, the old char array elements are copied into the new char array, and the old char array is discarded. A version of the implementation used by StringBuilder and StringBuffer follows:

char[] value;
int count;
public AbstractStringBuilder append(String str) {
if (str == null) str = "null";
int len = str.length();
if (len == 0) return this;
int newCount = count + len;
if (newCount > value.length)
expandCapacity(newCount);

str.getChars(0, len, value, count);
count = newCount;
return this;
}
void expandCapacity(int minimumCapacity) {
int newCapacity = (value.length + 1) * 2;
if (newCapacity < 0) {
newCapacity = Integer.MAX_VALUE;
} else if (minimumCapacity > newCapacity) {
newCapacity = minimumCapacity;
}
value = Arrays.copyOf(value, newCapacity);

}

Continuing with the fictitious tax payer program example from the previous section (full listing of the source code used in this section can be found in Appendix B in the section "First Resizing Variant"), StringBuilder objects are used to assemble random Strings representing tax payer names, addresses, cities, states, social security numbers, and a tax payer id. It also uses the no argument StringBuilder constructor. Hence, the program is likely to be subject to StringBuilder's underlying char[] being resized. A capture of a memory or heap profile with a profiler such as NetBeans Profiler confirms that is the case. Figure 6-18 shows a heap profile from NetBeans Profiler.

In Figure 18, you can see that char[], StringBuilder, and String are the most highly allocated objects and also have the largest amount of live objects. In the NetBeans Profiler, selecting and right-clicking on the char[] class name in the far left column as shown in Figure 19 shows the allocation stack traces for all char[] objects.

In the char[] stack allocation traces, shown in Figure 20, you can see an entry for java.lang.AbstractStringBuilder.expandCapacity(int), which is

called from AbstractStringBuilder.append(char) and AbstractStringBuilder.append(String) methods. The expandCapacity(int) method calls java.util.Arrays.copyOf(char[], int). Looking back at the previous source code listing, you can see where AbstractStringBuilder.append(String str) calls expandCapacity(int) and calls Arrays.copyOf(char[] int).

Figure 18: Heap profile

Figure 19: Showing allocation stack traces

Figure 20: char[] allocations from expanding StringBuilders

You can also see from Figure 20, over 11% of the current live char[] objects are from resized StringBuilder char[]. In addition, there are a total of 2,926,048 char[] objects that have been allocated, and of those, 390,988 char[] allocations occurred as a result of StringBuilder char[] resizing. In other words, about 13% (390,988/2,926,048) of all char[] allocations are coming from resized StringBuilder char[]s. Eliminating these char[] allocations from resizing improves the performance of this program by saving the CPU instructions needed to perform the new char[] allocation, copying the characters from the old char[] into the new char[], and the CPU instructions required to garbage collect the old discarded char[].

In the Java HotSpot JDK/JRE distributions, both the StringBuilder and StringBuffer offer no argument constructors that use a default size of 16 for their underlying char array data storage. These no argument constructors are being used in this program. This can be seen in the profile by expanding the java.lang.Ab-stractStringBuilder.<init>(int) entry seen in Figure 20. The expansion of the java.lang.AbstractStringBuilder.<init>(int) entry, shown in Figure 21, shows it is called by a no argument StringBuilder constructor.

In practice, few StringBuilder or StringBuffer object instances result in having consumed 16 or fewer char array elements; 16 is the default size used with the no argument StringBuilder or StringBuffer constructor. To avoid StringBuilder and StringBuffer resizing, use the explicit size
StringBuilder or StringBuffer constructor.

A modification to the example program follows, which now uses explicit sizes for constructing StringBuilder objects. A full listing of the modified version can be found in Appendix B in the section "Second Resizing Variant."

Recent optimizations in Java 6 update releases of the Java HotSpot VM analyze the usage of StringBuilder and StringBuffer and attempt to determine the optimal char array size to use for a given StringBuilder or StringBuffer object allocation as means to reduce unnecessary char[] object allocations resulting from StringBuilder or StringBuffer expansion.

public static String getRandomTaxPayerId() {
StringBuilder sb = new StringBuilder(20);

for (int i = 0; i < 20; i++) {
int index =
threadLocalRandom.get().nextInt(alphabet.length);
sb.append(alphabet[index]);
}
return sb.toString();
}
public static String getRandomAddress() {
StringBuilder sb = new StringBuilder(24);

int size = threadLocalRandom.get().nextInt(14) + 10;
for (int i = 0; i < size; i++) {
if (i < 5) {
int x = threadLocalRandom.get().nextInt(8);
sb.append(x + 1);
}
int index =
threadLocalRandom.get().nextInt(alphabet.length);
char c = alphabet[index];
if (i == 5) {
c = Character.toUpperCase(c);
}
sb.append(c);
}
return sb.toString();
}

Figure 21: Uses of StringBuilder default constructor

In Part 3 we will measure the performance impact after addressing StringBuilder and StringBuffer resizing will be done in combination with addressing any Java Collection classes resizing.

More Stories By Charlie Hunt

Charlie Hunt is the JVM performance lead engineer at Oracle. He is responsible for improving the performance of the HotSpot JVM and Java SE class libraries. He has also been involved in improving the performance of the Oracle GlassFish and Oracle WebLogic Server. A regular JavaOne speaker on Java performance, he also coauthored NetBeans™ IDE Field Guide (Prentice Hall, 2005).

More Stories By Binu John

Binu John is a senior performance engineer at Ning, Inc., where he focuses on improving the performance and scalability of the Ning platform to support millions of page views per month. Before that, he spent more than a decade working on Java-related performance issues at Sun Microsystems, where he served on Sun’s Enterprise Java Performance team. John has contributed to developing industry standard benchmarks such as SPECjms2007 and SPECJEnterprise2010; published several performance whitepapers; and contributed to java.net's XMLTest and WSTest benchmark projects.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...
The security devil is always in the details of the attack: the ones you've endured, the ones you prepare yourself to fend off, and the ones that, you fear, will catch you completely unaware and defenseless. The Internet of Things (IoT) is nothing if not an endless proliferation of details. It's the vision of a world in which continuous Internet connectivity and addressability is embedded into a growing range of human artifacts, into the natural world, and even into our smartphones, appliances, and physical persons. In the IoT vision, every new "thing" - sensor, actuator, data source, data con...
The Internet of Things is tied together with a thin strand that is known as time. Coincidentally, at the core of nearly all data analytics is a timestamp. When working with time series data there are a few core principles that everyone should consider, especially across datasets where time is the common boundary. In his session at Internet of @ThingsExpo, Jim Scott, Director of Enterprise Strategy & Architecture at MapR Technologies, discussed single-value, geo-spatial, and log time series data. By focusing on enterprise applications and the data center, he will use OpenTSDB as an example t...
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends: Exposing the device to a management framework Exposing that management framework to a business centric logic Exposing that business layer and data to end users. This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles ...
An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and assessments, including a decade of leading incident response and digital forensics. He is co-author of t...
The Internet of Things will greatly expand the opportunities for data collection and new business models driven off of that data. In her session at @ThingsExpo, Esmeralda Swartz, CMO of MetraTech, discussed how for this to be effective you not only need to have infrastructure and operational models capable of utilizing this new phenomenon, but increasingly service providers will need to convince a skeptical public to participate. Get ready to show them the money!
The Internet of Things will put IT to its ultimate test by creating infinite new opportunities to digitize products and services, generate and analyze new data to improve customer satisfaction, and discover new ways to gain a competitive advantage across nearly every industry. In order to help corporate business units to capitalize on the rapidly evolving IoT opportunities, IT must stand up to a new set of challenges. In his session at @ThingsExpo, Jeff Kaplan, Managing Director of THINKstrategies, will examine why IT must finally fulfill its role in support of its SBUs or face a new round of...
One of the biggest challenges when developing connected devices is identifying user value and delivering it through successful user experiences. In his session at Internet of @ThingsExpo, Mike Kuniavsky, Principal Scientist, Innovation Services at PARC, described an IoT-specific approach to user experience design that combines approaches from interaction design, industrial design and service design to create experiences that go beyond simple connected gadgets to create lasting, multi-device experiences grounded in people's real needs and desires.
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets. According to a recent IDG Research Services Survey this rate of traffic will only grow. What's driving t...
Scott Jenson leads a project called The Physical Web within the Chrome team at Google. Project members are working to take the scalability and openness of the web and use it to talk to the exponentially exploding range of smart devices. Nearly every company today working on the IoT comes up with the same basic solution: use my server and you'll be fine. But if we really believe there will be trillions of these devices, that just can't scale. We need a system that is open a scalable and by using the URL as a basic building block, we open this up and get the same resilience that the web enjoys.
Connected devices and the Internet of Things are getting significant momentum in 2014. In his session at Internet of @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, examined three key elements that together will drive mass adoption of the IoT before the end of 2015. The first element is the recent advent of robust open source protocols (like AllJoyn and WebRTC) that facilitate M2M communication. The second is broad availability of flexible, cost-effective storage designed to handle the massive surge in back-end data in a world where timely analytics is e...
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will want to use their existing identities, but these will have credentials already that are (hopefully) i...
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
P2P RTC will impact the landscape of communications, shifting from traditional telephony style communications models to OTT (Over-The-Top) cloud assisted & PaaS (Platform as a Service) communication services. The P2P shift will impact many areas of our lives, from mobile communication, human interactive web services, RTC and telephony infrastructure, user federation, security and privacy implications, business costs, and scalability. In his session at @ThingsExpo, Robin Raymond, Chief Architect at Hookflash, will walk through the shifting landscape of traditional telephone and voice services ...
Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy. How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at Internet of @ThingsExpo, James Kirkland, Chief Architect for the Internet of Things and Intelligent Systems at Red Hat, described how to revolutioniz...
Bit6 today issued a challenge to the technology community implementing Web Real Time Communication (WebRTC). To leap beyond WebRTC’s significant limitations and fully leverage its underlying value to accelerate innovation, application developers need to consider the entire communications ecosystem.
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from hardware to software, or as we like to say, it’s an Internet of many different things. The difference ...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.