Test Driven Development and GUI Testing: Functional tests

Previous article:Test Driven Development and GUI Testing: Unit tests

Google Docs: This article can be viewed at Test Driven Development and GUI Testing: Functional tests

In the previous article Test Driven Development and GUI Testing: Unit tests, we followed the Test Driven Development approach to go from our Use Case:Convert Temprature to our TemperatureConverter class which is now fully tested and will be the foundation of our application.
We now trust it, because it has passed our tests. Furthermore, these tests give us the confidence to refactor, improve and change it.
This article is about building functional tests as we implement our swing GUI.

Adding functional tests

Functional tests, also know as Acceptance tests, are other fundamental concept in eXtreme Programming. Those tests would help us assure that our application correctly implements what we described in Use Case:Convert Temprature and are always written from a client or user perspective.
Writing functional tests to validate application GUI using Test Driven Development techniques is even more trickier at the beginning.
How can you write a functional test to test a GUI that still doesn't exist ?
Functional testing is approached much like unit testing as we seen before.
Well, let's see how.

Firstly, we are adding the first component of the application GUI.

Create JFrame

1. Select the TDD project node
   
2. Right click and add new JFrame From...
3. Name it TemperatureConverterGUI
4. Select Source Package location and tdd package
5. OK

We now have an empty form.

Synchronize the model

1. In the TemperatureConverterGUI source code editor, right click
2. Select ReverseEngineer...
3. Existing UML: TDD-UML
4. OK

Create test

1. Select Functional Test Packages node
2. Right click and add New File... | Other | JUnit | NbTestCase Test New File.. | Testing Tool | JellyTestCase Test
   
3. Select Next
4. FileName: OverallTest
5. Folder test/qa-functional/src/validation
6. Finish

Add libraries

1. Add Jellytools, Jemmy and NBUnit to Libraries

Synchronize the model

1. Right click on the OverallTest editor
2. Select Reverse Engineer...
3. Existing UML: TDD-UML
4. OK
5. Add OverallTest class from Model to TDD Class Diagram

Finishing all of these steps, we have this TDD Class Diagram

Add Jelly functional tests

Jelly is NetBeans dependent, so if you would like to avoid this you can base your tests directly on Jemmy.

In our TDD Class Diagram

1. In OverallTest class
   
2. Rename placeholder test1 to testCorrectConversion
3. Rename placeholder test2 to testIncorrectConversion
4. Add attribute app type ClassReference and default value initializeApp()
5. Add method initializeApp() returning ClassReference and being private and static
6. Generate Code...
7. OK
   

Let's add this code to initializeApp() method

   private static ClassReference initializeApp () {
 try {
  return new ClassReference("tdd.TemperatureConverterGUI");
 } catch (ClassNotFoundException ex) {
  throw new RuntimeException("Couldn't initialize app", ex);
 }
   }

Then press Ctrl + Shift + I to fix the imports.

Mark tests as not yet implemented

Stub tests, as generated by NetBeans New | JellyTestCase Test are empty, so we need to add these sentences temporarily

    public void testCorrectConversion () {
  fail("This test is not yet implemented");
    }

    public void testIncorrectConversion () {
  fail("This test is not yet implemented");
    }

Add the tests to the suite

These tests should be added to the existing suite. This is not automatically changed when we changed the tests names in the TDD Class diagram.

    public static NbTestSuite suite () throws ClassNotFoundException {
        NbTestSuite suite = new NbTestSuite();
        suite.addTest(new OverallTest("testCorrectConversion"));
        suite.addTest(new OverallTest("testIncorrectConversion"));
        return suite;
    }

Modify build-qa-functional.xml

To be able to compile and run Jelly test cases we have to modify the rules in build-qa-functional.xml

<!-- Path to Jemmy library -->
<path id="jemmy.path" location="/opt/java/netbeans-6.0/testtools/modules/ext/jemmy.jar">

<!-- Path to Jelly library -->
<path id="jelly.path" location="/opt/java/netbeans-6.0/testtools/modules/ext/jelly2-nb.jar">

<!-- ========= -->
<!-- Compilers -->
<!-- ========= -->

<!-- Compile functional tests. This target is used in cfg-qa-functional.xml. -->
<target name="qa-functional-compiler">

<!-- Build application before tests -->
<ant dir=".." target="jar">
<buildTests srcdir="qa-functional/src" compileexcludes="**/data/**">
<classpath>
<!-- Add classpath elements needed to compile tests -->
<path refid="jemmy.path">
<path refid="jelly.path">
<fileset dir="../dist" includes="*.jar"> </fileset>
</path>
</path>

<!-- ========= -->
<!-- Executors -->
<!-- ========= -->
<!-- Run tests in JVM -->
<target name="run-jvm">
<executeTests pluginname="jvm">
<classpath>
<!-- Add classpath elements needed to run tests -->
<path refid="jemmy.path">
<path refid="jelly.path">
<fileset dir="../dist" includes="*.jar"> </fileset>
</path>
</path>

Run the tests

As expected recently added tests will fail because we forced the fail condition until we implement them.

Review Temperature Converter mock-up

In this mock-up we can identify the widgets required by the application.

We need

• a window having "Temperature Converter" title
• two text fields, one editable to enter the temperature and the other not editable to show the conversion result
• two labels corresponding to each text field showing the corresponding temperature units
• one convert button to do the conversion
• one close button to close the windows and exit the application

That's how our application will look like or its structure.

From the behavioral point of view, we can say that every time the user presses the Convert button, a conversion is carried away and the result showed. If there's a problem with the conversion the error is also showed in the conversion text field but to get user attention this is showed in red, for example:

Invalid temperature: -274 below absolute zero

Now that we have identified the required components we can proceed to write the tests.
Yes, we haven't written the application yet but using the knowledge we obtained analyzing this mock-up we are going to write our tests expecting the components to be there.

Implementing the tests

We are using Jelly/Jemmy to implement our swing GUI tests. Jelly/Jemmy use the concept of Operators.

Add Jemmy Operators

Jemmy operators is a set of classes which are test-side agents for application components. Operators provide all possible methods simulating user action with components, methods to find and wait components and windows. Also operators map all components methods through the event queue used for event dispatching. All Jelly operators are subclasses of Jemmy operators.

All of the operators provide access to their subcomponents by "getters" methods. These methods are implemented using the "lazy initialization" technique, so real suboperator instances are not initialized until it's necessary. All of the suboperators are initialized by verify() method invocation, so this method guarantees that all subcomponents are already loaded.

So, our first step is to add such Operators.
We have to add these attributes to our OverallTest class in the TDD Class Diagram

• jfo type JFrameOperator
• jlfo and jlco type JLabelOperator
• jtffo and jtfco type JTextFieldOperator
• jbcvo and jbclo type JButtonOperator

Add a String title attribute with default value "Temperature Converter" to keep window title.

Initialize Operators in findOperators

Let's add the private findOperator method.
Then

1. Right click on the OverallTest class and select Generate Code...
2. Select TDD as target project
3. Select Functional Test Packages as Source Root
4. Check Add Merge Markers to Existing Source Elements
5. OK
   

When finished, select findOperators method, right click on it and Navigate To Source.
Add this code to the method

   private void findOperators () {
 //wait frame
 jfo = new JFrameOperator(title);
 jlco = new JLabelOperator(jfo, "Celsius");
 //
 // Using the getLabelFor is a way to locate JTextFields, the other is by name or by initial text
 //
 jtfco = new JTextFieldOperator((JTextField) jlco.getLabelFor());
 jlfo = new JLabelOperator(jfo, "Fahrenheit");
 jtffo = new JTextFieldOperator((JTextField) jlfo.getLabelFor());
 jbcvo = new JButtonOperator(jfo, "Convert");
 jbclo = new JButtonOperator(jfo, "Close");
   }

Write the actual tests

testCorrectConversion tests some conversions that are know to be correct, and as a double check the actual result is compared against TemperatureConverter.celsiusToFahrenheit() which has passed the unit tests.
After entering the text into the field, the Convert button is pressed and the value in the Fahrenheit text field is checked.

    public void testCorrectConversion () {
 int[] temps = {100, -100, 0, -1, 1};

 for (int t : temps) {
     jtfco.setText(Integer.toString(t));
     jbcvo.clickMouse();

            // verify the result
            int fahrenheit = TemperatureConverter.celsiusToFahrenheit(t);

            assertEquals("conversion", "" + fahrenheit, jtffo.getText());
            assertEquals(Color.BLACK, jtffo.getForeground());
 }
    } 

testIncorrectConversion invokes the conversion with malformed or invalid parameters.

    public void testIncorrectConversion () {
 String[] temps = { "aaa", "0-0", "--1", "1a", "-274" };

 for (String s : temps) {
     jtfco.setText(s);
     jbcvo.clickMouse();

     assertEquals(Color.RED, jtffo.getForeground());
 }
    } 

Application start

Add this code to start the application and to find Jemmy operators. If you add the method calls and then you can use the IDE to complete the surrounding try-catch block.

    public void setUp () {
 try {
     System.out.println("########  " + getName() + "  #######");
     app.startApplication();
     findOperators();
 } catch (InvocationTargetException ex) {
        Logger.getLogger(OverallTest.class.getName()).log(Level.SEVERE, null, ex);
 } catch (NoSuchMethodException ex) {
     Logger.getLogger(OverallTest.class.getName()).log(Level.SEVERE, null, ex);
 }
    }

Running the tests

Finally, our functional test infrastructure is ready. To sum up: we have now two tests that using the swing GUI, that we are going to implement right now, are converting temperatures and verifying the results and obtaining corresponding error messages when the input data is incorrect.
This was defined in our Use Case:Convert Temprature.

We can see an empty window, and after a while the browser is launched and both tests failing with

fail: Frame with title: "Temperature Converter"

Clicking on the Yes link under Workdir a screenshot is saved and can be analyzed to solve the problem.
That's because we need to implement our swing GUI.

UML TDD Class Diagram

This will be our final Class Diagram, after we implement the GUI

Next article

Next article Test Driven Development and GUI Testing: Implementing swing GUI (coming soon) will show how to build the swing based GUI using Test Driven Development approach and we will obtain our working and fully tested application.