Extending the Model Based philosophy to a software
The process of developing model-based software test automation consists of the following steps:
- Exploring the system under test, by developing a model or a map of testable parts of the application.
- Domain definition: enumerating the system inputs and determine verification points in the model
- Developing the model itself.
- Generating test cases by traversing the model
- Execution and evaluation of the test cases
- Note the bugs the model missed and improve the model to catch (maybe build some intelligence)
Model-based testing is very nimble and allows for rapid adaptation to changes to the system under development. As the software under test evolves, static tests would have to be modified whenever there is a change in functionality. With model-based testing, behavioral changes are handled simply by updating the model. This applies especially well to temporary changes in the system under test. For example, if a certain area of the system under test is known to be broken on a given build, static tests would keep running into the same problem, or those particular commands to the test runner would have to be rewritten to keep that from happening. On the other hand, if testing is done using a model, inputs that lead to the known faulty areas can be temporarily disabled from the model. Any test cases based on this new model will avoid the known errors and not a single line of test code has to be changed.
Model-based testing is resistant to the pesticide paradox, where tests become less efficient over time because the bugs they were able to detect have been fixed. With model-based testing, test cases are generated dynamically from the model. Each series of tests can be generated according to certain criteria, as explained later in this paper. In addition, model-based testing can be useful to detect bugs that are sensitive to particular input sequences. It is very important to note the fact that these additional benefits come at no extra cost to the model or the test harness. This means that once the initial investment has been made to create a model and a test harness, they are modified only sporadically. Meanwhile, the same model can generate large numbers of test cases, which can then be applied using the same test harness.
Developing a model is an incremental process that requires the people creating the model to take into consideration many aspects of the system under test simultaneously. A lot of behavioral information can be reused in future testing, even when the specifications change. Furthermore, modeling can begin early in the development cycle. This may lead to discovering inconsistencies in the specification, thus leading to correct code from the very outset.
To a certain extent, the technique of applying inputs randomly offers an alternative to static tests [5]. However, these types of test automation are not aware of the state of the system under test; for example, the test automation does not know what window currently has the focus and what inputs are possible in this window. Because of this, they will often try to do things that are illegal, or they exercise the software in ways it will never be used. In other words, it’s difficult to guide random input test automation in a cost-efficient way precisely because of its purely random nature. Another consequence of this unawareness of state is that random input test automation can only detect crashes, since it doesn’t know how the system works. The test automation is only able to apply inputs, but it does not know what to expect once an input has been applied. For example, the random test runner may execute a sequence of inputs in one particular window that brings up a new window that has a completely different set of possible inputs. Nevertheless, the test runner is unaware of this fact, and keeps on applying random keystrokes and mouse clicks as if nothing had happened. On the other hand, model-based tests know exactly what is supposed to be possible at any point in time, because the model describes the entire behavior of the system under test. This in turn makes it possible to implement oracles of any level of sophistication. It is feasible to build a certain level of intelligence into the random test automation such that it will only try to apply inputs that are physically possible. The test automation can keep track of the window that currently has input focus and constrain the inputs that it will try to execute based on this knowledge. The disadvantage of the random input generation method is that when the system under test changes in behavior, the test automation has to be modified accordingly.
Disadvantages of Model Based Testing
· Requires that testers be able to program
· Effort is needed to develop the model.
Vinay Jagtap
A hard core Technocrat with over a decade of extensive experience in heading complex test projects coupled with real time experience of project management and thought leadership. Extensive experience in Performance, Security and Automation Testing and development of automation frameworks and ability to setup and execute Global service centers and Center of Excellences for testing.
