According to Moore's Law, the number of transistors on an integrated circuit doubles every 24 months. Can software keep up? To do so, software vendors need to tackle the seismic shift in server technology toward 64-bit multi-core servers with massive memory banks. Software vendors have been slow to adopt the techniques required to get the most out of these new servers, never mind those coming in the future. And customers pay the price, without necessarily realizing it.

This article will cover what the software industry needs to do to keep Moore's Law going, examining what major changes we can expect over the next year, what challenges these pose to software vendors, and how the software industry can keep pace to optimize customer benefits.

Quad-cores are here, and microprocessors with even more of these tiny computers on a single chip aren't far behind. Add advanced memory, and the hardware guys have set the stage for more power and more seamless user experiences on a variety of new devices.

For the past 20 years software developers could count on Moore's Law to increase application performance, but the free ride is over. Higher CPU performance and usually increased clock speed translated into better application performance. But clock speeds have physical limits, so chipmakers have shifted instead to multi-core servers to boost performance.

Software developers need to do things differently. They've been complacent about how they do their work, using time-tested techniques that were good enough for yesterday's hardware but sub-optimal for tomorrow's.

This article describes two methods to write software to take advantage the benefits of multi-core chips, multithreading, and in-memory databases.

How Next-Generation Hardware Is Evolving
As 64-bit multi-core servers emerge, processing power grows rapidly; size, cost, and power consumption drop. The hardware side is moving rapidly. Soon the smallest, cheapest blade server will contain four-eight cores, with this number increasing significantly over time.

Power consumption is key in the new calculus, since faster chips require more power and produce more heat that then requires more cooling. A rule of thumb is that for every dollar spent to power a server, another dollar in electricity is needed to cool it. Many data centers can't add capacity despite empty floor space - local utilities can't provide any more electricity to power and cool server farms. Power-efficient multi-core servers break this power dilemma.

A SOP to Moore's Law
In fact, advances in server hardware may outpace Moore's Law. New packaging technology such as system-on-package (SOP) accelerates processing power, especially for handheld devices.

"SOP leapfrogs Moore's Law by embedding everything into a new type of [integrated circuit] package so small that eventually handhelds will become anything from multi- to mega-function devices," said Rao R. Tummala in the June 2006 issue of IEEE Spectrum Magazine ("Moore's Law Meets its Match," http://spectrum.ieee.org/jun06/3649). These SOP chips will open the way to cheaper, faster consumer devices.

Software Developers Lag on New Methods
In a multi-core environment, a number of software engineering techniques can help software take advantage of this raw power. These methods include multithreading, in-memory databases, and new ways to adapt software in scale-out or blade architectures, where more servers are added to generate more computing muscle.

Why Multi-Core Chips Need Multithreading
Multithreading techniques are a key method to take advantage of multi-core processors. Traditionally, applications were written as processes that are typically limited to running sequentially on a single CPU. If executed on a multi-core CPU, a process will run only on a single core - wasting the extra power because it uses only a quarter of a quad-core processor (see Figure 1).

To take full advantage of multi-core and multiprocessor environments, a programmer should use multithreading techniques. Unlike processes, "threads" share memory, state, and other resources with other threads. Software architects must break down processes into multiple threads that can run in parallel on different cores.

In a multi-core environment, the threads of a multithreaded application are distributed across all available cores to run simultaneously. Because they share memory and resources, they can be synchronized to gain maximum performance.

This technique is especially useful in data-intensive applications, such as databases, that can process data in parallel on different cores. CPU-intensive applications, such as high-performance computing, also benefit from multithreading.

Think of a traditional application running on a single-core CPU with different processes that must wait in line to use the processing power of the single CPU. With a multithreaded application, a multi-core processor gives access to processing without waiting.

Designing the Multithreaded Application
Multithreading requires a new approach to designing an application. The programmer must design the application and write small, tight code segments (or threads) that can run in parallel or simultaneously. Threads can't depend on each other or create conflicting data.

In a database, for example, a thread that runs a transaction if someone deposits money into an account and another thread that withdraws money from that account can't run in parallel. They must be synchronized, meaning that the withdrawal depends on a deposit happening first.

One database technique is to have separate threads doing the same thing on different data. One thread might process data for names beginning with A to C, another D to G, and so on. No conflicts arise because the threads work on different data. This approach is a typical database parallelization or multithreaded method.

Software architects must design applications so threads are discreet, autonomous, and run independently of each other. With the smallest servers soon containing four-eight cores, multithreading will be important even in scale-out environments.