Since the beginning of the information technology era, IT managers have implemented various systems management applications, ranging from enterprise frameworks to open source products, in an attempt to manage the various servers, storage, and network devices currently found in their data centers. For the most part, these costly solutions have met little success because most of these devices cannot communicate with each other, as they are based on different standards and protocols depending on the vendor that created them.

Just as a Rosetta Stone was needed to translate hieroglyphics into a common language, the systems management market needs a way to unify these technologies and attain the holy grail of the automated data center. To achieve this long sought-after vision, a new platform is required that starts before servers, storage, or networking equipment are delivered to the data center. The solution, rather than introducing yet another management framework or technology, would unite all existing technologies under a common infrastructure that could glue any device to any management application.

Service Oriented Architectures are rapidly gaining in popularity for addressing the challenges of data center complexity and heterogeneity, but to date, they have not been widely used to address the infrastructure issues such as systems management. In this paper, we will look at how a service-oriented approach 1) can be applied to the issues surrounding systems management, 2) can help provide solutions that are powerful, flexible, and highly tailored to leverage existing "best-of-breed" solutions, and 3) can bring, for the first time, a solution that can unite disparate systems and systems management applications into one common infrastructure.

The Growing Challenges Faced by Data Center Managers
The life of a data center manager in most organizations is hard and destined to get harder unless some things change. According to Gartner, 85% of data center budgets are spent on operating expenses - mostly labor - and only 15% are spent on capital expenses. For most data centers, the labor is largely dedicated to simply "keeping the lights on," or ensuring that existing and newly acquired systems continue to run and that service levels don't drop. More often than not, strategic projects designed to improve service levels or better drive business improvements fail to get implemented.

For some time now, data centers have been deploying an increasing number of commodity servers with the ratio of system administrators to servers decreasing, forcing each administrator to manage more servers. Furthermore, with the growing importance of Web and networking traffic, and the increased numbers of threats (SPAM, viruses, intrusions, denial of service attacks, etc.), data centers must resort to bringing in additional network-related appliances to handle jobs related to security, routing, load balancing, traffic shaping, fire walling, etc. These appliances all serve very useful purposes but significantly contribute to the increasing complexities of managing the data center.

The challenges faced by data center managers will continue to grow. If you consider what the introduction of Service Oriented Architectures (SOAs) bring, it's obvious that SOA's make creating powerful, distributed, and inter- and intra-company applications much easier. However, SOAs bring new challenges to applications that are loosely coupled and distributed, and parts of them may be behind different company's firewalls. Additionally, some of these loosely coupled components may be running on virtual machines and can potentially be migrated from one server to another.

Because of these challenges, IT administrators are being forced to do more with less: manage more devices and support software applications that have become more granular, more distributed, and partially hidden behind firewalls.

Although SOAs might be partially to blame for an increase in management difficulties, they also offer great promise toward providing useful ways to rescue data center managers from complexity problems.

Systems Management Software that Can't Manage Systems
Before describing how SOA approaches to systems management can help, it's worth taking a look at the current state of systems management today. This will help to illustrate how much an SOA-based approach to systems management can help.

In order to effectively manage a given device such as a computer, a router, a network appliance, etc., that device has to be a good "managee." Just as in everyday life, the best manager in the world can't be fully effective if the people or things he or she is trying to manage are inherently unmanageable or only poorly manageable. Many devices sold today simply have terrible management characteristics. Some devices only offer weak management interfaces, some respond only to a very limited set of management commands, some do not have programmatically accessible management interfaces, and some do not even respond correctly to the management commands they were presumably designed to comprehend. Many devices do not speak any of the standard or de facto standard management protocols like SNMP, IPMI, or WMI. Some devices can only be controlled or queried for status via proprietary means like a proprietary command line interface (CLI) or Web interface. Just as in every day life, a manager who has to manage several "entities" with whom he or she does not share any common languages and who might not be very responsive, even if there is a shared language, can't be very effective. In the systems management world, developers refer to devices or sets of devices that can't be managed in standard ways, but only in idiosyncratic ways as "silos of management." The more silos of management there are, the tougher the system administrator's job is with more to learn, more system differences to comprehend, and more complexity in the data center overall.

Generally, a data center manager can find management software that will perform certain limited but powerful functions. However, there are entire classes of management problems that can't be managed very well today at all. Imagine for a moment that you are a data center manager. Now imagine further that you have gotten over the initial shock of being overworked and underpaid when one of the managers of the group to whom you provide service asks you, 'Why is e-mail so slow today?' On the surface the question seems direct enough; however, you will quickly realize that the root cause for e-mail being slow could be any number of things, or a combination of them such as: routers malfunctioning, a virus invading the network, an overloaded e-mail server, or an attack on your network. The problem is a distributed system problem. Data center managers have gotten used to, or perhaps numb to, the fact that the bulk of systems management software on the market today is terrible at managing a distributed system environment.

Those familiar with SOA-based applications realize that the complexities of managing distributed systems are about to get much worse. Forget about the, 'Why is e-mail slow problem' and now ask yourself how you will answer next week's question, 'Why did so many Web transactions through our new SOA loosely coupled applications fail this afternoon?' And by the way, some of the machines that those loosely coupled SOA components were running on were virtual machines (VMs) which have been moved to different physical servers, different from the ones they were on this afternoon. The SOA management problem is in many ways harder than the e-mail problem since it is more distributed, and some of the system elements involved will be or could be more transient in nature.

Companies that want to effectively manage distributed systems today, whether they are e-mail, Web serving, or SOA-based applications, are in a bind. There are so many un-integrated silos of management that companies must either implement their own very expensive means to manage them, or they must simply rely on their human administrators to understand all the relationships between devices and software components and hope they can monitor for problems and troubleshoot them in acceptable timeframes. Either way, the effort required is extremely expensive requiring costly development or costly human labor. In practice, companies will typically adopt a hybrid approach. They will make use of the management products they have, "siloed" though they may be, and write some of their own management software, perhaps incorporating some good open source management components, and rely on their people to provide whatever additional intelligence that can't easily be automated. Overall the situation is messy and expensive to fix.

SOA Management to the Rescue
SOAs are causing some of the problems, so it is only fitting that they be the ones to provide a significant part of the solution. In fact, this trend is already starting. Some companies and players in the systems management market are drawing on SOA approaches to improve the systems management picture. Both the DMTF and OASIS standards groups are working on Web services-based management standards called WS-MAN and WSDM respectively.

The basic idea behind the standards is simple: to create service-oriented management interfaces for hardware and software resources in the network. The implications for systems builders (ODMs, OEMs, ISVs, system integrators) are huge, as are the implications for data center administrators.

Let's take a look at how the situation should play out, based on what the standards groups are doing, what many systems management companies are doing, and according to what we learned in the world of SOA-based applications when standards started to emerge.

Fewer New Unmanageable or Poorly Manageable Resources
Developers of SOA applications have made giant (and once nearly unthinkable) strides toward creating software components that can be reused and readily integrated with other software components. Software from different companies can interoperate, and loosely coupled applications can span the globe if need be.

System builders are working to make their devices manageable via WS-MAN and WSDM. The management interfaces to these devices will be formally defined using WSDL, and access to those interfaces will be via standard Web services protocols like SOAP. Today's "Towers of Babel" and management silos will start to erode.

In some cases, existing "legacy" management devices will be upgradeable to support the new WS- interfaces. However, in many cases, it will be either impractical or impossible to upgrade such devices in the field. A key question then will be: 'Can anything be done to fully integrate the management of those devices under the SOA management paradigm?' The answer, of course, is yes.

New WS-Based Ways to Manage Legacy Resources
Developers of complex SOA-based applications often want to capture the value of legacy applications rather than rewrite them. Rewrites are typically just not feasible. Capturing the value of legacy apps and integrating them into the SOA world is done through encapsulation: create a formal SOA interface for the legacy application, bind that interface to the application via the most suitable means, and thereby enable other newer SOA components to interact with the services now exposed by new SOA interfaces to the legacy software. Companies like BEA Systems, IBM, and Tibco Software provide enterprise application integration (EAI) products to perform these kinds of feats, although there are many cases where the work required is not terribly complex.

The same basic encapsulate and integrate techniques can be used to bring legacy devices and software into the SOA management world. In some cases the interfaces can be added to the management "agents" running on the devices themselves. In other cases, proxy agents can be created that run off-board from a legacy device but that support a WS-based management interface to that device. The interface can be bound to the device via a number of techniques that basically require the off-board proxy agent to talk to the device's on-board management agent via whatever management language or languages the on-board agent speaks natively. In some cases the native languages are standard (e.g., SNMP, IPMI, WMI, etc.) but in others the native language may be proprietary and idiosyncratic, say a proprietary CLI or proprietary management protocol. Some systems management software vendors can provide a kind of one stop shopping for system developers interested in integrating legacy devices into the WS-management world.

Management Software That Can Manage Anything
If all hardware and software resources to be managed are managed via WS-based interfaces, then it becomes far easier to create management software that can potentially manage anything. Rather than management software having to deal with the "Tower of Babel," enough WS-based consistency (whether native or proxy based) can be introduced to make the management of heterogeneous collections of resources - say everything in a particular data center - possible. Almost by definition, management silos exist today because of the lack of common management languages that managers can use to interact with "managees."

Systems Management Software That Can Truly Manage Systems
If hardware and software resources are consistently manageable it becomes possible to truly manage systems problems like the 'Why is e-mail slow?' problem. All of the components that make up an e-mail delivery system can be comprehended by and managed by, say, extensible management appliances. More sophisticated, higher order services can be built up from lower order component services. The "e-mail performance diagnostic" service, for example, can query the routers, the network load monitors, the file servers, the mail servers, etc., to collect information necessary to resolve the problem. If one type of router is removed from the network and a replacement is installed, the benefits of loose coupling can be appreciated by simply removing consideration of the old router from the diagnostic mix and inserting consideration of the new router.

Real systems management power can be achieved by capitalizing on modularity, reusability, standardization, encapsulation, composition, layering, etc. Note that most of these descriptive terms are not typically used to describe the systems management software widely used today.

Integration of New and Legacy Management Software
When developers create systems management applications or appliances it would be foolish to ignore the value that existing management software can add. Consequently, it makes sense for new SOA management applications to employ the same encapsulate and integrate approach mentioned previously to integrate legacy management software. Consider the interesting case of open source management technology called Nagios. Nagios is an application that can be used to monitor the health of over a thousand different types of hardware and software resources, everything from servers, to routers, to databases, and other popular applications. Each different type of monitoring is performed by a different software plug-in called a (what else) monitor. Given the modular nature of the plug-ins, it is relatively easy to integrate them into a SOA application. Integrating other legacy systems management software is often more difficult but, in most cases, possible.

How Long Is the Wait?
The WS-MAN and WSDM standards are already quite far along. The DMTF has already released version 1.0.0a of its WS-MAN specification and can boast contributions from Microsoft, Intel, Dell, Sun, BEA, HP, and many other major players. The OASIS group has released a suite of 1.0 version standards documents related to WSDM, also with contributions from major players such as IBM, BEA, BMC, Dell, HP, Oracle, and others. Many system and software vendors are already starting to implement the standards.

Additional Reading