In most dance forms, the structure of the dance is akin to a core framework, or skeleton. The framework itself is composed of dance patterns, which are mere glorified sequences of dance steps. The sequences of steps, or patterns, reside in the salsero’s tool belt (in the case of salsa), ready to be used when the appropriate time or situation arises. Dancing necessitates an acute sense of stage direction, an awareness unnoticed by casual dancers.
In testing software, we often find ourselves hitting the same situations over and over again. Data verification, multiple interface testing, unclear requirements, a constantly changing application under test, fuzzy result logic (in the case of testing a search engine), system environment setup – all constitute common software quality issues. One way to attack these issues systemically without reinventing the protein shake is to apply classic software design patterns. After all, test automation is development and the same principles in the software development life cycle apply.
- Factory Pattern for testing multiple interfaces using a common schema.
- Builder Pattern for dynamically building a test suite composed of varying test cases specified at runtime
“We need to automate the tests for this app!”
The app happens to be Gmail, my favorite web based mail application. Management wants the first stab of testing to take the form of GUI
automation tests, hoping to mirror the interaction with the app of the end user. So, after deciding upon an open source tool like Selenium or a commercial tool like TestComplete, Rational Functional Tester, WinRunner, or Silk Test, you get down to business. We take a look at the gmail look and feel and we start recording, via the gui testing tool, solely for the purposes of understanding the gui tools interaction’s with the app under test. WE WILL NOT RELY UPON RECORD / REPLAY TECHNOLOGY AS THE BASIS FOR OUR TESTING. Rather, we simply use the script generated as a starting point for creating smarter tests.
Solution 1: Record / replay using a gui tool and custom coding
textPassWord().setText(“************”);
buttonOK().click();
sleep(2);
// set email properties and send email textRecipient().setText(“someone@somewhere.com”);textSubject().setText(“hey, what’s up?”);
textBody().setText(“help me with my interview please”);
// send the email
sendButton().click();
sleep(2);
// print the 1st email
menu().click(atPath(“File->Print”);
sleep(2);
textPrinter().selectComboBox(“Printer 3”);
buttonOK().click();
browser().close();
Ok, so we’ve just massaged a script using our tool of choice after some preliminary recording and replaying. The synopsis…
Pros:
- Good for consistent target application that exhibits minimal interface changes
- Easy to create
Cons:
- Scripts become unnecessarily long
- Doesn’t take advantage of common routines and tool may store separate UI object mappings for the same components (Menu Bar, Buttons, etc) and this results in duplicate code that must be maintained
- Hart to maintain, especially in an agile project
- We’re reliant upon a test script and this takes effort to code
Solution 2: Action based Objected Oriented script. Apply object orientation to create a common library for test scripts to use, thereby removing code duplication and shorter scripts.
browser().open(“www.gmail.com”);
login().login(“sogwiz”, “*****************”);
// set email properties and send email
Email().sendMail(“someone@somwhere.com”, “hey, what’s up”, “help me with my interview please”);
// print the 1st email
Email().print(“Printer 3”);
// tear down
Browser().close();
Above, we’ve reduced the code in our script (please note that we’re still scripting here, NOT data driving) by creating a framework that encapsulates functionality we’ll be using in multiple tests. For example, the login(String username, String password) function will be used by numerous scripts and so it makes sense to refactor the function into a framework class – that way the test script can simply call the login function without having to worry about implementation details. We’ve done this in solution 2, and c’mon now… this is just simple object orientation. The synopsis…
Pros:
- Easier to read and test scripts begin resembling business logic
- Easier to maintain and takes advantage of reuse
Cons:
- Library takes time to create
- We’re still reliant upon a test script and this takes effort to code
As you sip your coffee, green tea, detoxifying smoothie, or alcoholic beverage at work, management rushes into your office once more, this time sweating.
“We’re exposing the API to the public and this needs to be tested as well as the GUI!”
New Requirement – a new interface (GUI or API or web service layer) is exposed and must be tested.
/**
* Access to GMAIL app via command line instead. API Clients and GUIs may use classes that implement
* this interface
* @author SBenjamin
*
*/
public interface IGmail {
public boolean login(String username, String password);
public boolean sendMail(String recipientListl, String subject, String body);
public boolean printEmail(String emailID, String printerName, boolean attachments);
public String getEmail(String emailID);
}
So, after looking at the API and speaking with developers, you conjure up a small framework and write some scripts calling the framework. A script may resemble the following:
login implGmail.login(“sogwiz”, “*****************”);
// set email properties and send email
implGmail.sendMail(“someone@somwhere.com”, “hey, what’s up”, “help me with my interview please”);
// print the 1st email
implGmail.printEmail(“Printer 3”);
This doesn’t seem so bad, you say. There is one caveat – we now must maintain 2 scripts that test different interfaces to the same application. An interesting thing to note is that both scripts (the action based GUI script from solution 2 and the API test) look strikingly similar, both testing the same business logic.
PROBLEM: Maintenance of 2 scripts testing different interfaces to the same business logic
Solution : Design Pattern – Let’s try the Factory. It’s a creational pattern that allows for the creation of objects without strictly specifying the class used to create the object. This is VERY useful as we simply define an interface for creating something and then delegate the creation details to the subclass(es).
So, how does this apply to testing? How does it apply to our problem of testing multiple interfaces (GUI and API) in the web based application?
We can define a test schema or a test interface that maps to our business logic. We’ll then start our automation tool and the it will pick up the test schema, chug it through the factory, and the factory will decide whether or not to generate GUI test interactions or API test interactions. We’ve just alleviated the maintenance of two test scripts. Instead, we’ll define a common schema and write data driven / property tests in accordance to the schema. The factory can then delegate how to execute the test (via GUI or API). Here’s a single data driven test that we can use instead of creating two test scripts.
While the data driven test case definition (shown to the left) may have more lines of code than the action based script, the big benefit here lies in the independence from any one scripting language and no reliance upon the sequential execution of code. See, in the action based script, we had to define the navigational logic of the code. In other words, we had to define the sequence of actions to take. This is not the case with XML as its simply a definition of the business logic and, in this case, inputs to the various business logic components. We could have various test frameworks parse the test definition and then perform custom actions, allowing for reusibility.
So, what does a factory look like? How does the code appear?
// a class that implements the ITest interface will be created with our factory. The factory checks for the type of test desired (GUI or API) and then creates the appropriate objects to commence testing.
public static ITest getTest(String testType, String testName)
{
if (testType == “GUI”)
{
return new GUITest(testName);
}
else if (testType == “API”)
{
return new APITest(testName);
}
return null;
}
}
In synopsis… Pros:
- Reuse. With a common test script format, the same test case can be used to do GUI functional testing or API testing. Dev integration testing is also possible.
- Common Language. Both developers and testers can communicate via the xml schema, as it serves as the common language. This is key, especially for global teams.
- Unified training. If new team members join, developers and testers will not have to be trained separately to learn the usage of the test framework.
Cons:
- Test schema definition. Defining the test schema is a painstaking process that entails a detailed requirements process.
- GUI Verification. If a GUI test wishes to verify a GUI component (eg the ‘Send Mail’ button is disabled until after the recipient field is filled in), then a script would have to be written that interacts directly with the test engine as you don’t want the XML doc to contain info pertaining to a certain type of testing interface.
You’re feeling pretty good and walking around the office with a spring in your step. You’re at the break room and the boss walks in. Small talk commences and as you respond to what you’re up to this weekend, the boss interrupts, “Ok, that’s nice. We should be able to execute any combination of tests within a test suite!”
Up until now, we’ve been statically declaring which test cases (xml definitions) to execute. See, we’ve created a Suite class that statically specifies which tests cases to run. One solution to this is to create multiple Suite classes that statically specify different combinations of test cases to run. Ouch!
STATIC CODE DECLARATION OF A TEST SUITE’S CONTENTS
public void executeTestSuite()
{
Vector
//creation of tests using the Factory
ITest iTest1 = TestFactory.getTest(“GUI”, “test1”);
ITest iTest2 = TestFactory.getTest(“API”, “test2”);
//BAD, BAD, BAD – we’re statically specifying which classes to add to the suite
//this should be dynamically done at Runtime without needing to compile an altered
//representation of the TestSuite. We should use the BuilderPattern to alleviate
//this problem of creating new objects each time we want a different representation
//of the same class (TestSuite.java)
suite.add(iTest1);
suite.add(iTest2); executeSuite();
}
The solution to the problem of creating multiple Suite classes is to use the Builder Pattern.
“The intention is to separate the construction of a complex object from its representation so that the same construction process can create different representations.” – http://en.wikipedia.org/wiki/Builder_pattern
We’ll be able to construct a complex object (a Suite class) so that the same construction process can create different representations (the same physical Suite class whose representation differs in that it can specify varying test cases to execute).dynamically specify which test cases to run at runtime through the use of a properties, text, command line arguments, or xml file
By the way, if you’re in the Bay Area (
Recent Comments