Welcome!

Java IoT Authors: Zakia Bouachraoui, Elizabeth White, Liz McMillan, Pat Romanski, Yeshim Deniz

Related Topics: Java IoT

Java IoT: Article

Escaping Swing's Limitations when Drawing Graphs

...and improving performance

Swing is a general and flexible library for drawing graphical user interfaces. However, when trying to use Swing components to draw nodes in a graph, generality and flexibility have their limitations.

The Problem
We once worked on a project where we had to graphically display the configuration and topology of a telecommunications network. From the user interface programmer's point of view, this task is similar to drawing a graph, that is, nodes connected by links.

The user should be able to display as well as interact with the graph elements - move them, zoom in or out, display a pop-up menu, and so on. Our first thought was to use a JComponent as the base for the graph elements. Swing can take care of all the work we don't want to bother with, such as redrawing overlapping components and distributing the events to the correct graph element.

Since our nodes were nothing more than a label with an icon, we used the JLabel as a base class. Other Swing components made our job easy. A comment could be represented by a JTextArea, a table of attributes by a JTable, and links and other objects were simple JComponents in which we could override the paintComponent method.

This solution was quite easy and effective up to a couple hundred nodes. However, as the number of nodes and the complexity of the network grew, we quickly realized that Swing would not scale to our needs.

The best and most spectacular example to demonstrate Swing's lack of scalability is the moving of a node. If you have a small network of 20 nodes with a couple of links, you can grab a node, move it around, and it will follow the mouse pointer pretty smoothly. However, when you increase the number of nodes, you start to notice a latency time between the moment you initiate the move and the moment the node actually starts to move. When your network reaches the size of a real-life telecom network, the latency lasts over a couple of seconds. If you keep moving the mouse pointer, the node will try to follow it without ever managing to catch it.

Solutions
Use the Memory You Really Need
It turns out that JComponents are so general, they contain a lot of variables and checks that are only overhead in our case. We decided to avoid using JComponents for our graph elements. In the same way that Swing uses the AWT Frame or Dialog as its top-level container and then takes care of everything else within the window, we use JComponent as our top-level component and take care of everything else within its bounds.

Stripping away everything that was unnecessary and keeping only what we really needed, we came up with objects optimized for our needs. We had the attributes we wanted to use, and the methods had less overhead because we had full control. This helped us counter one of the big problems of Swing: in spite of its flexibility, there are a lot of things you have no control over.

Table 1 compares the memory consumption of our graph elements (using JDK 1.4.1).

Most of the objects on a given display were made of links whose size was brought down to 256 bytes, less than a quarter of the size of its Swing counterpart.

Memory can be saved by offering interfaces only to the developer instead of an all-knowing base class. In this way, just the necessary information will be saved in the elements, and we get rid of any unwanted or unnecessary default behavior.

Repaint Only What You Need To
The area in which Swing quickly reaches its limit is the speed of the repaint. In our experience, Swing paints more graph elements than it should. Some objects get repainted even if their appearance was not modified, or if they were not overlapped by an object that triggers a repaint.

When component X needs repainting (its color has changed or it has changed location), Swing looks for other components that also need to be repainted. To do so, it iterates through all the siblings of X and checks if a sibling's bounds are intersecting with X's bounds. If the bounds are intersecting, it repaints this sibling and also checks if it intersects with another one, creating a cascading effect of repaints. You might say that objects are rarely that close to each other, but we are drawing a graph of nodes connected with links. Link objects propagate the repaint to a lot more objects than you might think.

To cut short this cascading effect, simply avoid doing the intersection checks for the siblings. This will modify the Z-order of the elements in the graph, but this usually is an acceptable effect for the users.

The other reason why Swing repaints too much is the use of a rectangular shape as bounds for the components. Although most components you'd find in a user interface are roughly rectangular, when you look at links, you'll notice that nothing is less rectangular than a simple line. A lot of links get repainted because the intersection calculation takes into account the entire rectangle containing the line. It's quite easy for two rectangles to overlap, even if their diagonals do not.

If we are in control of everything, we can simply replace the rectangle with something else, let's say a shape or an array of rectangles. We can even ask the graph element if it intersects with a given rectangle, providing the developer with a higher level of flexibility. This complicates the intersection calculation, but the slow operation here is the repaint, and it's worth the extra effort if we can avoid several repaints.

An example of such a function in a Link element could look like this:

public boolean intersectWith(Rectangle r)
{
return r.intersectsLine(fromX, fromY, toX, toY);
}

Control the Repaint Mechanism
The way Swing repaints is also an obstacle to optimization. A repaint invariably starts with drawing a rectangle the size of the component using the background color. This means that whatever is behind it will be erased and need repainting. In other words, whatever will be deleted will be added to the repaint queue, causing further deletion and repaints. We propose implementing the repaint as a two-step operation: deletion and drawing.

Instead of only one repaint queue, you can implement two queues: a deletion and a drawing. Graph elements can decide for themselves if they need to be deleted, redrawn, or both by adding themselves to one or both queues.

Not all types of repaint operations require deletion. For example, changing the color of a link needs only a repaint of the link, not of the objects it crosses. This will paint the link on top of the object it crosses, but again, modifying the Z-order of the graph elements has to be an acceptable side effect. Also, removing an element from the graph requires a deletion operation only.

The drawing algorithm deletes only those rectangles stored in the deletion queue, adding the objects they overlap to the drawing queue (provided they are not already there). Following that, the drawing queue is processed and all objects are painted only once. These queues can also check for duplicates, so that a deletion or a drawing needs to be processed only once.

The best optimization that Swing offers for limiting repainting is the clipping rectangle, but that also has a limit. If you know the extent of what needs to be repainted, you can give it as a parameter to the overloaded version of repaint. However, in most cases it's quite hard to know what exactly will be repainted. If a node is moved, several links and attribute tables may move too.

It's not the job of the node to calculate the extent of the clipping rectangle. This feature is better implemented inside the Paint Manager, which already knows what it is going to repaint. Going through the deletion and drawing queue, it's easy to find out what really needs to be repainted. Once again, there's extra work in calculating the union of the bounds of all the graph elements, but the savings at the end is worth the game. In some cases, like zooming, when you know that everything will be drawn, you can simply bypass this check and tell the Paint Manager to draw everything without calculating the clipping rectangle.

Repaint Only When You Need To
Another automation of Swing that makes operations such as moving and zooming slower than they should be is the repaint on standard operations. Changing the size or location of a component provokes an immediate repaint. If you consider that zooming requires changing the location and size of all the elements in the graph, you have an idea of how slow this operation can be when each element repaints immediately upon its change.

Every call to repaint costs a lot and these should be kept to a minimum. Removing the Swing component from the chain allows you to reduce this behavior.

Another benefit is the location and size information, which is unnecessary for link objects. Links are just a line drawn between two nodes. Where is the need for location and size information? In the first version of our library, when a node was being moved, it triggered the following piece of code in the link element, which in turn triggered two repaint events:

Rectangle bounds = new Rectangle (from.getX(), from.getY(), 0, 0);
bounds.add(to.getX(), to.getY());
setLocation(bounds.x, bounds.y);
setSize(bounds.width, bounds.height);

Now this piece of code completely disappears. Even better, size and location can completely disappear from some elements such as the links. Throwing away the Swing components allows you to use absolute coordinates when drawing your links, instead of having to draw from (0,0) all the time. What good is this? Look at how simple the code for painting a link becomes:

public void paint(Graphics g)
{
g.drawLine(from.getX(), from.getY(), to.getX(), to.getY());
}

Storing Your Elements
A Swing JComponent stores its children in a private array. There is absolutely no control over what is going on in this array, but sometimes you would like to be able to store these components in another type of structure adapted to your needs.

For example, when the user wants to select an element, it would be faster to look through the graph if it were stored in a quad tree. If you decouple the graph element container from the component, you could choose the container that suits your needs. You could handle ordering, apply filtering, or offer a different search criterion. Once again, you have complete control.

Other Optimizations
We'd like to finish this article with some optimizations that can be made on every graph drawing application, regardless of whether it uses Swing.

First, we mentioned several times the modification of the Z-order of the objects, which is a side effect of the drawing algorithm. Modifying the Z-order can actually be ugly - if your links come in front of your nodes, for example. If there are types of objects between which the ordering matters (like between nodes and links), it's possible to store your graph elements on different layers. Each layer will have its own drawing mechanism, a buffered image on which to draw, and a container of its own graph elements. It not only allows you to group like objects together, but you can also ensure that your links will always appear behind your nodes. Also, repainting a node will not affect the links, as each layer computes its dirty bounds separately.

For moving operations, an effective optimization is to add an extra layer, which I call the glass layer. When moving several graph elements, you can transfer them to the glass layer where they will be the only elements to be repainted during a move. When the move operation terminates, you can transfer the graph elements back to their original layer.

Transferring objects between layers can be a slow operation because it requires a deletion from the original layer. When moving an object, this deletion causes a delay before the graph element can actually be moved on the glass layer. One solution is to leave the elements on their original layer, while a copy of the element is being moved on the glass layer. When the move operation terminates, the deletion will occur on all layers, keeping the slow operation for the end of the move when the user is busy looking for his or her next action.

Finally, some graph elements can have quite complex algorithms to draw. Of course, it's possible to cache some of the values in the renderer so that the drawing happens faster. But the moving usually invalidates those caches, and the algorithm will have to be applied all the time. One solution is to replace the renderer on the fly with a simpler one during a move operation. For example, we had a link that was moving together with an arrow showing in which direction the information was flowing. Calculating the angle of the arrowheads is a complex operation. While moving, the renderer was replaced with another one, which represented the complex link as a simple black line.

Where to Go from Here
In this article we covered some of the improvements that can be made to Swing behavior to draw graphs more efficiently. While you can manage to trick Swing in several ways to get over some of these difficulties, in some cases you'll reach the limits.

Rewriting the painting and event distribution can be a painful and long operation, but it is worth the trip. It provides performance improvement as well as control over what is really going on under the hood.

More Stories By John Hutton

John Hutton is an information specialist for Horizon Technology.

More Stories By David Shay

David Shay is a software developer for Ericsson in Hungary. He has worked for the past five years building GUIs in Java for telecom modelling tools.

Comments (3)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
Bill Schmarzo, Tech Chair of "Big Data | Analytics" of upcoming CloudEXPO | DXWorldEXPO New York (November 12-13, 2018, New York City) today announced the outline and schedule of the track. "The track has been designed in experience/degree order," said Schmarzo. "So, that folks who attend the entire track can leave the conference with some of the skills necessary to get their work done when they get back to their offices. It actually ties back to some work that I'm doing at the University of San...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in development and launches of disruptive technologies to create new market opportunities as well as enhance enterprise product portfolios with emerging technologies. His most recent venture was Octoblu, a cross-protocol Internet of Things (IoT) mesh network platform, acquired by Citrix. Prior to co-founding Octoblu, Chris was founder of Nodester, an open-source Node.JS PaaS which was acquired by AppFog and ...
The deluge of IoT sensor data collected from connected devices and the powerful AI required to make that data actionable are giving rise to a hybrid ecosystem in which cloud, on-prem and edge processes become interweaved. Attendees will learn how emerging composable infrastructure solutions deliver the adaptive architecture needed to manage this new data reality. Machine learning algorithms can better anticipate data storms and automate resources to support surges, including fully scalable GPU-c...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...