Diego Torres Milano's blog: July 2009

Tuesday, July 07, 2009

Android: Testing on the Android platform - Is Toast leaking ?

This document can be read in Google Docs (http://docs.google.com/View?id=ddwc44gs_194s36s83fz), cut and paste link if you have problems accessing it.

A couple of days ago I've found an interesting post by skink on Android Developers group titled **never ever** use Toasts with Activity context. The post speaking about NotifyWithText in ApiDemos, states something like:

"...try to show any Toast, then exit NotifyWithText

demo. run it again - you will see getInstanceCount() increases leaking

Activities. repeat running demo couple of times. counter still

increases."

So the questions are:

Is Toast leaking Context objects ?

Is it a bug in Toast, in ApiDemos, in the documentation ?

Should we use Application Context instead ?

Let's try to find the answers using some Unit Tests as we have been investigating in previous articles in this blog.

ToastActivity

Let's recreate a oversimplified version of the Activity to display just the Toast depending on an extra parameter DISPLAY_TOAST in the Intent starting the Activity.

package com.codtech.android.training.toast;

import android.app.Activity;

import android.app.Application;

import android.content.Context;

import android.content.Intent;

import android.os.Bundle;

import android.util.Log;

import android.widget.Toast;

public class ToastActivity extends Activity {

   private static final String TAG = "ToastActivity";

   public static final String USE_ACTIVITY_CONTEXT =

        "com.codtech.android.training.toast.useActivityContext";

   public static final String DISPLAY_TOAST =

        "com.codtech.android.training.toast.displayToast";

   private Context toastContext;

   private boolean useActivityContext;

   private boolean displayToast;



    /** Called when the activity is first created. */

    @Override

    public void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        final Intent intent = getIntent();

        useActivityContext =

intent.getBooleanExtra(USE_ACTIVITY_CONTEXT, true);

        displayToast =

            intent.getBooleanExtra(DISPLAY_TOAST, true);



       if ( useActivityContext ) {

           toastContext = this;

       }

       else if ( displayToast ) {

           toastContext =

getApplication().getApplicationContext();

       }

    }





    /* (non-Javadoc)

   * @see android.app.Activity#onPause()

   */

   @Override

   protected void onResume() {

       super.onResume();

       if ( displayToast ) {

           Toast.makeText(toastContext,

                "Sample Toast: " + getInstanceCount(),

Toast.LENGTH_SHORT).show();

       }

       else {

           Log.d(TAG, "No toast displayed");

       }

       finish();

   }

   /* (non-Javadoc)

   * @see android.app.Activity#onDestroy()

   */

   @Override

   protected void onDestroy() {

       super.onDestroy();

       Log.d(TAG, "Activity destroyed: " + this);

   }

}

ToastActivity Tests

As usual, let's create our test to.

In this very particular case we want it to run several times specified by ToastActivityTests.N, 50 actually but you can change it if you like, to see if displaying a Toast or not change things in some way.

RepeatedTestSuite

This class implements repeated tests.

package com.codtech.android.training.toast.tests;

import junit.framework.Test;

import junit.framework.TestSuite;

public class ReapeatedTestSuite extends TestSuite {



   public static Test repeatedSuite(Test test, int count) {

       TestSuite suite = new TestSuite("Repeated");



       // there's no RepeatedTest in android's junit

       for (int i=0; i<count; i++) {

           suite.addTest(test);

       }



       return suite;

   }

}

RepeatedActivityContextWithToast

This test will run testSingleActivityContextWithToast starting the activity displaying the Toast several times

/**

*

*/

package com.codtech.android.training.toast.tests;

import junit.framework.Test;

/**

* @author diego

*

*/

public class RepeatedActivityContextWithToast extends ReapeatedTestSuite {

   public static Test suite() {

       return repeatedSuite(

            new ToastActivityTests("testSingleActivityContextWithToast"),

            ToastActivityTests.N);

   }

}

RepeatedActivityContextWithoutToast

This test will run testSingleActivityContextWithoutToast starting the activity, without displaying any Toast, several times

/**

*

*/

package com.codtech.android.training.toast.tests;

import junit.framework.Test;

/**

* @author diego

*

*/

public class RepeatedActivityContextWithoutToast extends ReapeatedTestSuite {

   public static Test suite() {

       return repeatedSuite(

            new ToastActivityTests("testSingleActivityContextWithoutToast"),

            ToastActivityTests.N);

   }

}

ToastActivityTests

We decided to fail the test if the Activity instance count reaches N/4.

/**

*

*/

package com.codtech.android.training.toast.tests;

import java.lang.reflect.Method;

import android.app.Instrumentation;

import android.content.Intent;

import android.os.Bundle;

import android.test.ActivityInstrumentationTestCase2;

import android.test.FlakyTest;

import android.test.suitebuilder.annotation.MediumTest;

import android.util.Log;

import com.codtech.android.training.toast.ToastActivity;

/**

* @author diego

*

*/

public class ToastActivityTests

    extends ActivityInstrumentationTestCase2<ToastActivity> {

   private static final String TAG = "ToastActivityTests";

   public static final int N = 50;

   /**

   * @param name

   */

   public ToastActivityTests(String name) {

       super("com.codtech.android.training.toast",

            ToastActivity.class);

       setName(name);

   }

   /* (non-Javadoc)

   * @see junit.framework.TestCase#setUp()

   */

   protected void setUp() throws Exception {

       super.setUp();

   }

   /* (non-Javadoc)

   * @see android.test.InstrumentationTestCase#tearDown()

   */

   protected void tearDown() throws Exception {

       super.tearDown();

   }

   @MediumTest

    public void testSingleActivityContextWithToast() {

       exersiseActivityLifecycle(intentFactory(true, true),

            "testSingleActivityContextWithToast");

    }



   @MediumTest

    public void testSingleApplicationContextWithToast() {

       exersiseActivityLifecycle(intentFactory(false, true),

            "testSingleApplicationContextWithToast");

    }



   @MediumTest

    public void testSingleActivityContextWithoutToast() {

       exersiseActivityLifecycle(intentFactory(true, false),

            "testSingleActivityContextWithoutToast");

    }



   @MediumTest

    public void testSingleApplicationContextWithoutToast() {

       exersiseActivityLifecycle(intentFactory(false, false),

            "testSingleApplicationContextWithoutToast");

    }



   /**

   * @param intent

   */

   private void exersiseActivityLifecycle(final Intent intent, final String name) {

       setActivityIntent(intent);

       final ToastActivity activity = getActivity();

       final Instrumentation instrumentation = getInstrumentation();

       // At this point, onCreate() has been called, but nothing else

       // Complete the startup of the activity

       instrumentation.callActivityOnStart(activity);

       instrumentation.callActivityOnResume(activity);

       // At this point you could test for various configuration aspects, or you could

       // use a Mock Context to confirm that your activity has made certain calls to the system

       // and set itself up properly.

       instrumentation.callActivityOnPause(activity);

       // At this point you could confirm that the activity has paused properly, as if it is

       // no longer the topmost activity on screen.

       instrumentation.callActivityOnStop(activity);

       Runtime.getRuntime().gc();

       Runtime.getRuntime().runFinalization();

       Runtime.getRuntime().gc();



       long aic = ToastActivity.getInstanceCount();

       assertTrue("instance count reached " + aic, aic < N/4);

       // At this point we are invoking onDestroy explicitly because we are iterating

       // and tearDown() will not be called

       instrumentation.callActivityOnDestroy(activity);



       // run N times, requires FlakyTest

       try {

           Method method = this.getClass().getMethod(name, new Class[] {});

           FlakyTest flakyTest = method.getAnnotation(FlakyTest.class);

           if ( flakyTest != null ) {

               assertTrue(count >= flakyTest.tolerance());

           }

       } catch (SecurityException e) {

           // TODO Auto-generated catch block

           e.printStackTrace();

       } catch (NoSuchMethodException e) {

           // TODO Auto-generated catch block

           e.printStackTrace();

       }

   }



   private static Intent intentFactory(boolean useActivityContext,

      boolean displayToast) {

       final Intent intent = new Intent();

       intent.setAction(Intent.ACTION_MAIN);

       intent.setClassName("com.codtect.android.training.toast",

            "com.codtect.android.training.toast.ToastActivity");

       intent.putExtra(ToastActivity.USE_ACTIVITY_CONTEXT, useActivityContext);

       intent.putExtra(ToastActivity.DISPLAY_TOAST, displayToast);

       return intent;

   }

}

Running the tests

Running RepeatedActivityContextWithoutToast

Running the tests we can verify that everything is fine.

diego@bruce:~$ adb shell am instrument -w -e class com.codtech.android.training.toast.tests.RepeatedActivityContextWithoutToast com.codtech.android.training.toast.tests/android.test.InstrumentationTestRunner

com.codtech.android.training.toast.tests.ToastActivityTests:...................

...............................

Test results for InstrumentationTestRunner=.........................................

.........

Time: 42.473

OK (50 tests)

Running RepeatedActivityContextWithToast

Running the test that displays the Toast we find a different result. Activity instance count reaches the maximum allowed and the test fails

diego@bruce:~$ adb shell am instrument -w -e class com.codtech.android.training.toast.tests.RepeatedActivityContextWithToast com.codtech.android.training.toast.tests/android.test.InstrumentationTestRunner

com.codtech.android.training.toast.tests.ToastActivityTests:...........

Failure in testSingleActivityContextWithToast:

junit.framework.AssertionFailedError: instance count reached 12

   at com.codtech.android.training.toast.tests.ToastActivityTests.

       exersiseActivityLifecycle(ToastActivityTests.java:195)

   at com.codtech.android.training.toast.tests.ToastActivityTests.

       testSingleActivityContextWithToast(ToastActivityTests.java:138)

...

Test results for InstrumentationTestRunner=............F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.F.

F.F.F.F.F.F.F.F.F.F

Time: 56.101

FAILURES!!!

Tests run: 50, Failures: 39, Errors: 0

Conclusion

We can see that the only difference between both tests is the Toast being displayed, and the results are completely different.

While the demonstration given in the Android Developer's thread seems correct, this seems correct too !

Comments, suggestions and corrections are gladly welcome.

If you are interested in the source code or APK just drop me a line or leave a comment in the blog.

Friday, July 03, 2009

Android: Testing on the Android platform - Hamcrest

This document can be read in Google Docs (http://docs.google.com/View?id=ddwc44gs_193f2v698g9), cut and paste link if you have problems accessing it.

We used a custom Comparator in the previous post to be able to compare Strings content in our mock object's method invocations.

While this method is valid a more generic approach would be to introduce hamcrest, a library of matcher objects (also known as constraints or predicates) allowing 'match' rules to be defined declaratively, to be used in other frameworks. Hamcrest also provides adaptors for EasyMock 2.

We will be revisiting our previous example introducing hamcrest for our matchers.

Hamcrest matchers

Hamcrest comes with a library of useful matchers. Here are some of the most important ones.

    · Core
          o anything - always matches, useful if you don't care what the object under test is
          o describedAs - decorator to adding custom failure description
          o is - decorator to improve readability - see "Sugar", below
    · Logical
          o allOf - matches if all matchers match, short circuits (like Java &&)
          o anyOf - matches if any matchers match, short circuits (like Java ||)
          o not - matches if the wrapped matcher doesn't match and vice versa
    · Object
          o equalTo - test object equality using Object.equals
          o hasToString - test Object.toString
          o instanceOf, isCompatibleType - test type
          o notNullValue, nullValue - test for null
          o sameInstance - test object identity
    · Beans
          o hasProperty - test JavaBeans properties
    · Collections
          o array - test an array's elements against an array of matchers
          o hasEntry, hasKey, hasValue - test a map contains an entry, key or value
          o hasItem, hasItems - test a collection contains elements
          o hasItemInArray - test an array contains an element
    · Number
          o closeTo - test floating point values are close to a given value
          o greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo - test ordering
    · Text
          o equalToIgnoringCase - test string equality ignoring case
          o equalToIgnoringWhiteSpace - test string equality ignoring differences in runs of whitespace
          o containsString, endsWith, startsWith - test string matching

hasToString matcher

Let's create a matcher to replace stringCmp. EasyMock2Adapter is an adapter class provided by hamcrest.

import org.hamcrest.integration.EasyMock2Adapter;

import org.hamcrest.object.HasToString;

    /**

   * Create an {@link EasyMock2Adapter} using a
   * {@link HasToString.hasToString}

   *

   * @param <T> The original class of the arguments

   * @param o The argument to the comparison

   * @return o

   */

    public static <T> T hasToString(T o) {

        EasyMock2Adapter.adapt(
            HasToString.hasToString(o.toString()));

        return o;

    }

testTextChanged

Now, the watcher mock object checks will include this matcher:

    watcher.beforeTextChanged(hasToString(sar[i-1]),

   eq(0), eq(sar[i-1].length()), eq(sar[i].length()));

    watcher.onTextChanged(hasToString(sar[i]),

   eq(0), eq(sar[i-1].length()), eq(sar[i].length()));

    watcher.afterTextChanged(

       hasToString(Editable.Factory.getInstance()
       .newEditable(sar[i])));

Conclusion

We have introduced hamcrest that makes a great number of matchers and the ability to create new ones and adapt them to be used by EasyMock.

Thursday, July 02, 2009

Android: Testing on the Android platform - Mock objects

This document can be read in Google Docs (http://docs.google.com/View?id=ddwc44gs_186chdgdwdj), cut and paste link if you have problems accessing it.

In a previous post we created a parallel project to hold our tests. Our project is still empty and has only a TestSuite

Now we will explore how to create the actual tests and in this post we will be introducing mock objects.

If we are Test Driven Development purists we may argue about the use of mock objects and be more inclined to use real ones. Martin Fowler calls these two styles the classical and mockist Test Driven Development dichotomy in his great article Mocks aren't stubs.

Temperature Converter

We will be using our well know Temperature Converter application, the same we have used in many other examples, however we will be introducing some modifications and new tests.

EditNumber

Temperature Converter uses two fields where temperatures, as signed decimal numbers, can be entered. Let's extend EditText to have a specialized View with this behavior.

Test Driven Development

We are starting writing our tests for our yet inexistent class EditNumber. We decided to extend EditText and we know that EditTexts can add a listener, actually a TextWatcher, to provide methods that are called whenever EditText's text changes.
And this is precisely where we are introducing a mock TextWatcher to check method invocations while text changes.
EasyMock will help us achieve this. This is not an EsyMock tutorial, we will just be analyzing its use in Android, so if you are not familiar with it I would recommend you to take a look at the documentation available in its web site.
Add easymock-2.5.1.jar to the Tests project properties.

testTextChanged

This test will excersise EditNumber behavior checking the method calls on the TextWatcher mock and verify the results.
We are using an InstrumentationTestCase because we are interested in testing EditNumber in isolation of other components or Activities. Later on we will be introducing Functional Tests where we test the whole Activity.
sai and sar are two String arrays containing the inputs and results expected.
We will be using a special Comparator, stringCmp, because we are interested in comparing the String content for different classes used by the Android like Editable, CharSequence, String, etc.

/**
* EditNumberTests
*/
package com.codtech.android.training.temperatureconverter.tests.view;

import static com.codtech.android.training.temperatureconverter.tests.TestUtils.stringCmp;
import static org.easymock.EasyMock.createMock;
import static org.easymock.EasyMock.eq;
import static org.easymock.EasyMock.replay;
import static org.easymock.EasyMock.reset;
import static org.easymock.EasyMock.verify;
import android.content.Context;
import android.test.InstrumentationTestCase;
import android.text.Editable;
import android.text.TextWatcher;

import com.codtech.android.training.temperatureconverter.view.EditNumber;

/**
* @author diego
*
*/
public class EditNumberTests extends InstrumentationTestCase {

   private static final String TAG = "EditNumberTests";
   private EditNumber editNumber;

   /**
   * @param name
   */
   public EditNumberTests(String name) {
       super();
       setName(name);
   }

   /* (non-Javadoc)
   * @see junit.framework.TestCase#setUp()
   */
   protected void setUp() throws Exception {
       super.setUp();
       final Context context = getInstrumentation().getTargetContext();
       editNumber = new EditNumber(context);
       editNumber.setFocusable(true);
   }

   /* (non-Javadoc)
   * @see android.test.InstrumentationTestCase#tearDown()
   */
   protected void tearDown() throws Exception {
       super.tearDown();
   }

   /**
   * testTextChanged
   */
   public final void testTextChanged() {
       String[] sai = new String[] {null, "", "1", "123", "-123", "0",
            "1.2", "-1.2", "1-2-3", "+1", "1.2.3" };
       String[] sar = new String[] {"",   "", "1", "123", "-123", "0",
"1.2", "-1.2", "123",   "1", "1.23" };

       // mock
       final TextWatcher watcher = createMock(TextWatcher.class);
       editNumber.addTextChangedListener(watcher);

       for (int i=1; i < sai.length; i++) {
           // record
           watcher.beforeTextChanged(stringCmp(sar[i-1]), eq(0),
eq(sar[i-1].length()), eq(sar[i].length()));
           watcher.onTextChanged(stringCmp(sar[i]), eq(0),
eq(sar[i-1].length()), eq(sar[i].length()));
                        watcher.afterTextChanged(stringCmp(
                Editable.Factory.getInstance().newEditable(sar[i])));

           // replay
           replay(watcher);

                        // exersise
           editNumber.setText(sai[i]);

           // test
           final String actual = editNumber.getText().toString();
           assertEquals(sai[i] + " => " + sar[i] + " => " + actual, sar[i], actual);

           // verify
           verify(watcher);

           // reset
           reset(watcher);
       }
   }
}

stringCmp

This is the Comparator we are using

    public static final class StringComparator<T> implements Comparator<T> {

       /**
       * Constructor
       */
       public StringComparator() {
           super();
       }

       /* (non-Javadoc)
       * @see java.util.Comparator#compare(java.lang.Object, java.lang.Object)
       *
       * Return the {@link String} comparison of the arguments.
       */
       @Override
       public int compare(T object1, T object2) {
           return object1.toString().compareTo(object2.toString());
       }
   }

   /**
   * Return {@link EasyMock.cmp} using a {@link StringComparator} and
   * {@link LogicalOperator.EQUAL}
   *
   * @param <T> The original class of the arguments
   * @param o The argument to the comparison
   * @return {@link EasyMock.cmp}
   */
   public static <T> T stringCmp(T o) {
       return EasyMock.cmp(o, new StringComparator<T>(), LogicalOperator.EQUAL);
   }

Implementing EditNumber

This is what we want for our EditNumber. This is an extremely simple custom View but same techniques can be used in more complicated cases.

/**
* EditNumber
*/
package com.codtech.android.training.temperatureconverter.view;

import android.content.Context;
import android.text.InputFilter;
import android.text.method.DigitsKeyListener;
import android.util.AttributeSet;
import android.widget.EditText;

/**
* @author diego
*
*/
public class EditNumber extends EditText {

   /**
   * @param context
   */
   public EditNumber(Context context) {
       super(context);
       init(null);
   }

   /**
   * @param context
   * @param attrs
   */
   public EditNumber(Context context, AttributeSet attrs) {
       super(context, attrs);
       init(attrs);
   }

   /**
   * @param context
   * @param attrs
   * @param defStyle
   */
   public EditNumber(Context context, AttributeSet attrs, int defStyle) {
       super(context, attrs, defStyle);
       init(attrs);
   }


   private void init(AttributeSet attrs) {
       final InputFilter[] filters = new InputFilter[] {
DigitsKeyListener.getInstance(true, true) };

       setFilters(filters);
   }

}

Running the tests

Run the tests. Surprisingly we obtain an error

java.lang.AssertionError:
Unexpected method call onTextChanged(12.3, 0, 1, 4):
onTextChanged(StringComparator<com.codtech.android.training.temperatureconverter.tests.TestUtils.StringComparator>(1.23) == 0, 0, 1, 4): expected: 1, actual: 0
afterTextChanged(StringComparator<com.codtech.android.training.temperatureconverter.tests.TestUtils.StringComparator>(1.23) == 0): expected: 1, actual: 0
at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:43)
at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:72)
at $Proxy0.onTextChanged(Native Method)
at android.widget.TextView.sendOnTextChanged(TextView.java:5905)
at android.widget.TextView.setText(TextView.java:2634)
at android.widget.TextView.setText(TextView.java:2501)
at android.widget.EditText.setText(EditText.java:71)
at android.widget.TextView.setText(TextView.java:2476)
at com.codtech.android.training.temperatureconverter.tests.view.EditNumberTests.testTextChanged(EditNumberTests.java:151)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:191)
at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:181)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:164)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:151)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:418)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1520)

When we call editNumber.setText("1.2.3") instead of the expected "1.23" we obtain "12.3". What really give us the indication of a potential bug is that if we use the UI and type "1.2.3" we obtain the expected value but it differs if we use setText().
Is it a bug in TextView, in DigitsKeyListener, somewhere else ?
We need to find out.

Conclusion

We have seen how sometimes mock objects and particulary EasyMock is able to help us on our mockist flavor Test Driven Development and quickly find bugs that could take much longer to discover using other techniques.

Android: Testing on the Android platform - Creating tests

This document can be read in Google Docs (http://docs.google.com/View?id=ddwc44gs_186chdgdwdj), cut and paste link if you have problems accessing it.

In a previous post we analyzed the alternatives we have to run our tests on the Android platform and we suggested that the best way to create our tests is into a separate parallel project. As a convention this project will be named as the original project plus the Tests suffix.

For example if we have the MyAndroidProject project tests will be in MyAndroidProjectTests.

Create the project using this name, the same Build Target as the original project, Application name can be empty, for package name use the original project's package name appending the suffix .tests, and you can opt not to create an Activity as it's not needed in most cases.

As usual we will be using a variation of our AndroidTemperatureConverter project and consequently our tests will be in a parallel project named AndroidTemperatureConverterTests.

AndroidManifest.xml

Some changes must be introduced to the default AndroidManifest.xml to run our tests.

First, under Instrumentation, select the Name of the class implementing Instrumentation, usually android.test.InstrumentationTestRunner, the Target package and optionally a Label which is the one that will be displayed in the Instrumentation list as described in Running tests.

Target package is the package containing the code you are testing not your tests.

If you try to run the tests just now, you'll receive

ERROR: Application does not specify a android.test.InstrumentationTestRunner instrumentation or does not declare uses-library android.test.runner

To solve this, let's add android.test.runner as an uses library Application node.

Creating the TestSuite

We need a TestSuite, a Composite of Tests, to simply run all of our tests. Let's name this class AllTests.
Using TestSuiteBuilder we can automatically include all of the tests under the specified package. Obviously, we still don't have any test yet and the TestSuite will be initally empty.

/**
* Test suite
*/
package com.example.tests;

import android.test.suitebuilder.TestSuiteBuilder;

import junit.framework.Test;
import junit.framework.TestSuite;

public class AllTests extends TestSuite {

   public static Test suite() {
       return new TestSuiteBuilder(AllTests.class)
       .includeAllPackagesUnderHere()
       .build();
   }

}

Running the tests

Using Run as... -> Android JUnit Test will run our, still empty, list of tests.

Conclusion

Organizing the tests into a seprate project gives you the advantage of isolating them and its dependencies in a different APK. Shortly we will be introducing mock objects for our tests, using libraries like EasyMock, and in such case we will be adding it to the build path of the tests project only keeping the tidiness of the target untouched.