Virtualization technology is transforming the IT landscape and holds significant promise for those looking to maximize hardware utilization as well as reduce the time associated with provisioning and administering separate physical systems. According to the Yankee Group, today more than 60% of enterprises have implemented virtualization technology - and by the end of 2007 that figure is expected to increase to 90%.

Software development organizations are also reaping the benefits of virtualization, leveraging virtual lab environments to reduce test equipment costs, slash test cycle times, and increase the quality of the applications and systems they deliver.

The highly dynamic nature of software development often requires operation on multiple technology platforms, operating systems, Web and application servers, and databases. Add to this the many different builds, patches, or regionalized versions that are delivered by development and you get an idea of the immense challenge test teams face providing adequate coverage.

This article explores new ways of integrating virtual environments seamlessly into the software testing process and demonstrates how modern test management technologies can align to enable a quantum leap in a test team's ability to test more efficiently, across a wider range of environments, and with greater coverage of critical requirements to ultimately improve software quality across-the-board.

Addressing the Problem of Testing a Complex Environment
Typically, the goal of a testing team is to ensure that applications are tested in all the common scenarios that reflect the environments that the application will be deployed on and also to ensure that the applications perform as expected with other applications that integrate or interact with them. For multi-tier applications, this means ensuring it renders correctly on a variety of browsers, runs on an assortment of servers, interacts with all supported database systems, integrates with all of the other applications in its ecosystem, and reliably integrates with custom or packaged applications.

To deliver on this tall order, testing teams are continually challenged - often on short notice - to procure and provision a variety of hardware/software configurations. At the end of a test run, the infrastructure is usually torn down and given up to other teams that may require it. Not only does this eat up a great deal of time and resources - time that isn't being spent testing - the transient nature of the hardware and software configurations that are used for testing makes bug-finding and problem reproduction extremely challenging. Often, when the time comes for a developer to actually reproduce and fix the issue, the exact environment isn't available, and the issue may go unresolved.

The result? Testing teams sacrifice core testing time setting up and administering physical systems. Development teams can't accurately reproduce issues or validate fixes in the same environment they were found, and most importantly, quality is sacrificed and end-user satisfaction suffers - with all the associated negative ramifications.

The solution? Technology that synergizes virtual lab automation with quality management.

A virtual lab automation system, when specifically architected for testing teams, substantially mitigates the problems associated with setting up and administering complex testing environments. In this capacity, users can create and assign individual tests to run on many different configurations or platforms without the high cost of hardware or the time needed to administer physical systems. Organizations can also improve test team productivity by optimizing the use of virtualized infrastructure, allowing pooled resources to be shared across testing and development teams. The results are higher-quality software, built faster with lower server and data center costs and operating costs.

The combination of virtual lab automation with quality management can speed even the most complex software development cycles through rapid, automated setup and teardown. In essence, the virtual lab acts as a library of pre-configured multi-machine configurations in a suspended state.

Consider a test configuration that may require several interconnected servers, an application server, and a database server running on Linux machines integrated with a custom application running in a data center in a foreign country. In an automated virtual lab, this multi-machine configuration can be made available for use with ease and in very little time. In essence, the virtual lab acts as a library of pre-configured multi-machine configurations in a suspended state. When a tester, or in more automated scenarios when an automated test execution program requests a particular configuration, virtual lab automation is called on to provide exactly the right virtual machine from its "pool" of available configurations. The capabilities of effective virtual lab automation solutions include:

• Centralized access to a pool of virtualized test and development environments
• The ability to automatically set up and tear down these complex virtual environments
• The ability to interface with test management frameworks and bug tracking systems so test and development engineers can share and access the same virtual images
• The ability to move configurations easily between distributed development and test facilities

The common theme is "It works in my environment." In such cases, the defect is usually closed as "Not Reproducible." If the defect is serious in nature, the developer may spend an unnecessary amount of time trying to rebuild the environment to try to reproduce the defect. This additional time spent could add significant cost overruns to a project.

In an automated virtualized lab, once a test run is started it quickly collects data from the systems, and then shuts down immediately when the test is completed. If a problem is discovered, the system's configuration capture capability enables a QA professional to capture the entire state of the running configuration that the defect was discovered in. Once captured, the library configuration can be referenced in the defect report. When the developer receives the bug report, he can check out the configuration from the system and instantly recreate the exact environment the defect was discovered in. Time and again, we hear of instances were major bugs appear in production systems because they were not reproducible on a developer's system because the exact configuration, or even the right level of details about the environment, was no longer available when the time came to fix the defect.

...But Virtualization Alone Isn't the Answer
Regardless of the development or QA methodology and technologies an organization chooses, if it is to leverage return on its investments, the end goal of any process must be to optimize software quality throughout its lifecycle. That includes the adoption of more effective decision support tools to help users make tradeoff decisions about quality status, scope, resources, schedule, and overall management, including how decisions made in one functional area will impact the quality of another. In addition, quality has to be evaluated based on a set of previously agreed-to metrics that are aligned with the business goals. Finally, all stakeholders must have access to the same set of quality metrics and quality-related information.

This holds true for virtual environments, too. An effective test process must treat virtual infrastructure the same as any other physical environment, which isn't as easy as it sounds. While software testing teams can develop and test new capabilities side-by-side on multiple operating systems - and benefit from faster build/test/rebuild cycles, planning - actually having a solid way to manage the testing process is critical to their success. Testing and test preparation need to be managed processes that everyone on the project team can understand and work with. In this way, it becomes much easier to understand the tradeoffs and make the informed decisions about resources and schedules needed to test a product.

Pairing lab automation with good quality practices helps QA managers take into account the many significant issues that could affect their ability to reap the advantages of virtualization and ultimately help them make better decisions about testing. QA managers are able to plan for and guide testing from the start of project cycles, rather than waiting until the first build rolls out of developments to be tested, often late in the cycle. When the entire project team is kept up-to-date on testing, understands the approach used, and can observe the progress of testing while it is being done, it is easier for the team to function effectively as problems are found and as new versions go through the testing phases of the project.

Summary
Virtualization provides IT organizations clear benefits in streamlining IT infrastructure. Quality teams that embrace virtualization will see immediate boosts in productivity and turnaround time will improve since the infrastructure setup/provisioning time for complex multi-machine or cross-platform configurations is dramatically reduced.

By adopting a process that synergizes virtual lab automation with quality management, organizations will see improved communication and collaboration across its teams, better requirements to test coverage, accurate reproduction of defects leading to shorter defect resolution times and less costly IT infrastructure - all resulting in higher-quality IT products.