Software testing - whether it's automated, manual, or model-based - is a systemic method of discovering variances between how a program was expected to perform and how it actually behaves in use. Every application should be tested to ensure it is both usable and functional - that it doesn't contain bugs or flaws.

Model-based testing has stirred up a significant amount of interest over the past couple of years. For some development and testing teams it's certainly a test method worth exploring but be aware that model-based testing may be a great supplement to the automated testing you already do, but it's really an adjunct and not a replacement for standard automated testing.

A properly constructed automated software quality assurance test aims to discover all of the ways that every feature in a piece of software can be used, and how these features will interact with all other features, exposing the flaws and bugs that lurk in the programming code. Standard tests accomplish this by running a pre-determined series of code examinations. Model-based testing, in contrast, uses algorithms to determine all of the usage paths for an application, pares down that number for maximum coverage and minimal testing, and then generates various test cases to try the application against.

Skilled software testers labor long hours to figure out all of the possible ways a user might interact with an application. But users are almost guaranteed to surprise programmers by figuring out some innovative and exciting way to use a program, ones cause results that may not have been predictable to a logical mind.

Model-based testing can be compared to having many users, all of whom are quite happy to test your application 24/7 for weeks on end, each one interacting with the application in a distinctive way. And because model-based testing generates a broad sampling of test cases, it avoids the fallacies that are potentially inherent in repeating the same narrow run of tests. This can generate a false sense of security as subsequent runs will naturally produce ever smaller numbers of found flaws in programming code that may actually harbor serious problems. That's why model-based testing can sometimes catch flaws that standard tests might miss.

As we all know once you put an application onto the Web, users will inevitably be probing it for flaws or accidentally stumble across whatever bugs managed to evade your software testing process. Spotting and correcting these problems will save money, time, and aggravation. Think of model-based testing as a good way to broaden your quality assurance procedures.

Inherent Issues Concerning Model-Based Testing
Before you even begin to seriously consider bringing model-based testing into your development process it's important to note that this testing method does not suit every application or organization's needs.

As we've discussed, model-based testing examines an application's actual behavior in response to actions. These actions are generated from inputs created specifically for the application that is undergoing testing. To develop the inputs that the tool will use to generate test cases we have to understand, in depth, exactly what the application that we're testing should optimally do, how it should respond to its users, and perhaps how it should interact with other applications. We then need to clearly describe that expected behavior to the testing tool. The process of creating inputs for the testing tool is a highly critical process, since all the results that follow will necessarily be based on those inputs. If the inputs are imperfect, if they don't follow the proper syntax (and so aren't properly processed by the tool) or are incomplete, then your testing process will be flawed.

An issue of model-based testing is the skill set of the people using this approach. Classical manual testers and/or automated scripters will have a difficult time creating the framework needed to execute this kind of testing. Resources with in-depth knowledge of fully functioning programming languages (such as Java) are needed to code this approach. Either testers with exceptional programming skills or developers are needed to implement the model-based testing approach.

It's also difficult and time-consuming to define the correct inputs. Testing and development teams that have complete documentation to draw from will have an easier time developing inputs. Without such documentation, creating proper inputs can be more trouble than it's worth. One side benefit of model-based testing can be that it's an impetus for creating good documentation and coding guidelines. Establishing in-house guidelines for writing code enables others to easily check, work with, maintain, and reuse code. While each coder may prefer his or her own methods, personal programming idiosyncrasies, particularly of the undocumented sort, can baffle other developers - and perhaps even the original developer when they attempt to rework code at a later date.

Standardizing the development process doesn't mean it becomes stagnant. Java programmers have a wealth of best practices information to draw from, created and shared by members of the development community. Regarding security flaws, since most vulnerability attacks take advantage of common and well-known mistakes in programming code, complying with best practice development rules is a useful way to avoid potential problems. Each problem that's prevented by complying with best practice rules is one less problem that the team needs to identify, analyze, and correct later in the development process (or that will impact the deployed application.)

One of the benefits of model-based testing can be more thoughtful programming, since programmers and/or testers must understand in depth how an application is actually intended to work in order to develop model-based tests. Model-based testing can also hone more of an awareness of how actions interact and what effects these interactions have on code - simply thinking through the process of how to test an application can result in better coding.

All that said, only you can determine whether model-based testing is a concept that's likely to suit your project or company's culture. If you or your team already has a tendency to work under tight deployment deadlines, adding model-based testing to the mix may create unnecessary stress.

Input Issues
A good way to get started with model-based testing with your Java applications is to pick a small, easy-to-describe subset of your application to test. Choose five to 10 primary states, generate inputs for them, and run this basic test for at least a few hours - preferably a few days - to see what problems the test turns up. Remember that at this point you're primarily testing and refining your input-creation skills, even if the tests do turn up some bugs that had been hidden and especially if they don't it would be incorrect to assume you have the model-based testing process all worked out. Continue expanding the complexity of the subsets you test until you're confident that your inputs are correct, at that point you can begin to trust your test results.

When first starting to develop your inputs, you'll want to think about all the ways you intend the application to be used, and all the ways it can potentially be used and how you might attempt to abuse the application if you wanted to crack it. And since a big part of a Web-based application's success will be judged by its users on performance issues, consider testing scalability and reliability as well as looking for security flaws.

Thankfully model-based testing doesn't require programmers/testers to predict and fully define every single possible (or bizarre) interaction a user and an application might conjure up. Instead you will define the states, actions, and transitions for your application. So why did you have to think through how someone might use or abuse your application? Because having both a big picture and detailed view of how your application should and could act will help you define those inputs. Once you've defined them, the tool will generate the test cases.

Next you'll need to set initial test objectives. When you're just beginning to work with model-based testing this won't be an issue, since you'll be running very limited tests on small subsets of your application. But if you decide to incorporate this tool into your test process you'll need to clarify objectives before you begin testing. Since there are potentially an infinite number of tests that can be run in model-based testing, the best practice approach is to risk-prioritize the work. Decide which areas are likely to be at highest risk and then proceed to develop objectives for the test in descending order of perceived risk. As you run the tests, analyze the results and fix bugs, you'll continue to modify the model until you've met your testing objectives. Make sure to document what you've discovered and how the problem should be or was addressed.

In general, model-based testing can be a good way to augment the testing process that you already use. You'll want to put your application through other testing processes, obviously, with Java you typically need to ascertain how the application works in tandem with a Web browser, or how it functions in an embedded mobile system. You'll likely want to run scalability tests to see how the application holds up under heavy use. There are, as you've no doubt noticed, dozens of excellent tools - many of them open source - and verification programs available for Java application testing. Make use of whatever suits your application and its intended use. If you're developing an application for a client you may want to confer with them to see if they require or would prefer that you/your team performs other specific analysis of the application during the testing process. If your application is intended for the mobile market you may want to have it tested under the Unified Testing Criteria (www.javaverified.com), a set of mobile industry-approved automated tests that check Application Launch, Functionality, Operation, User Interface, Security, Network, and Localization abilities, so that you can use the "Java Powered" logo in your marketing.

Model-based testing is not a marvelous magic bullet that will quickly and effortlessly spot every single flaw in a program, nor will it necessarily produce results more economically or effectively than standard automated tests. Its ability to generate non-repetitive, real-world user tests is obviously of real value particularly in projects that tend to be more intricate such as real-time and embedded applications, assuming that good inputs can be produced.