Android Continuous Integration Guide

Android Application Testing Guide features a whole chapter about Continuous Integration (Chapter 8), however some latest changes and additions to the tools available may require a more in-depth coverage of the subject.
Consequently, I'm preparing an Android Continuous Integration Guide to compile all the information laying around on the subject and to provide concise, working examples that you could use to base your own projects on.
In the creation of these examples using Jenkins and EMMA for code coverage one of the most annoying things is the R class affecting the results of the coverage report, as most probably you are not creating tests for such auto-generated class.

This screenshot show how the coverage report for classes reaches 100% once we filter out the R class.

The problem is that in current Android SDK Tools (Rev 16), there's no way to filter classes from EMMA coverage unless you modify the file /tools/ant/build.xml, changing the emma target to include the filters as showed in this code snippet:

            <!-- It only instruments class files, not any external libs -->
            <emma enabled="true">
               <instr verbosity="${verbosity}"
                               mode="overwrite"
                               instrpath="${out.absolute.dir}/classes"
                               outdir="${out.absolute.dir}/classes">
                    <!-- DTM: 2011-12-23: added filter for R -->
                     <filter excludes="*.R" />
                     <filter excludes="*.R$*" />
                </instr>
                <!-- TODO: exclusion filters on R*.class and allowing custom exclusion from
                             user defined file -->
            </emma>

I hope this help you getting started and stay tuned, Android Continuous Integration Guide is scheduled to be released by the end of January 2012.

Any comments, suggestions and requests are welcome and can be entered using the Google+ pages. 

Android Application Testing Guide: Q&A

Q:  Hi Diego, I wanted to ask that can i write a monkey runner script which controls a web based apk ?
Eg; I install youtube.apk which is nothing but a browser with hardcoded youtube url.
Now my monkeyrunner script shall install this apk and then pass events such as a search string etc on this web based application.
All this i want to do and control externally through the monkey runner script. Is this possible? If yes, then could you please guide me by some pseudo code? 


Comment on Using Android monkeyrunner from Eclipse

Posted by latha

A:  This is an interesting question and a good monkeyrunner example, so here we go. monkeyrunner has the ability of installing APKs after obtaining the connection with the device. Then we start Youtube main activity, sleep for a bit to let things settle down.
Once we have the activity running is time to start our search. To do it, we touch the Search icon, enter the desired search string, 'android' in this particular case and the we touch the Search button again to actually start the action.
Following, is the script that translates our plan to monkeyrunner:

#! /usr/bin/env monkeyrunner

import sys
import os
from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice

YOUTUBE = 'com.google.android.youtube-2.1.6.apk'
prog = os.path.basename(sys.argv[0])

def usage():
        print >>sys.stderr, "usage: %s" % prog
        sys.exit(1)

def main():
        if len(sys.argv) != 1:
                usage()

        print "waiting for connection..."
        device = MonkeyRunner.waitForConnection()

        print "installing youtube"
        device.installPackage(YOUTUBE)

        device.startActivity(component="com.google.android.youtube/.HomeActivity")
        MonkeyRunner.sleep(3)
        # search
        device.touch(450, 80, MonkeyDevice.DOWN_AND_UP)
        MonkeyRunner.sleep(5)
        device.type('android')
        # done
        device.touch(450, 740, MonkeyDevice.DOWN_AND_UP)



if __name__ == '__main__':
    main()

This script covers the case described in the question but it could be easily adapted for other cases and application.

I hope this is the answer you were looking for.

Android Application Testing Guide: Q&A

Q:I followed your example. I set up the test project in a similar way.
But try to write a TestCase for the utility class from the original
project.
Eclipse says "Class under test does not exist in the current project."

It's kind of reasonable to me, since that class is indeed in another
project.
I only have experienced using JUnit to test normal Java Project where
the test directory is inside the project.

Also, I checked out the two example projects and found there is a
build.properties and build.xml. Is that the reason that you can import
the original class:
"import com.example.i2at.tc.TemperatureConverter;"?

Thanks

Best wishes,
Ryan

Posted by Ryan Huang.

A:If you have imported both projects (main & test) into Eclipse you should have no problems because the versions available at github have the required properties set.
However, if for some reason they were not set properly, this is what you should verify in your test project's Java Build Path -> Libraries

The other files you mentioned are used when you build with ant.

Android Application Testing Guide: Q&A

Q:Diego, if we were to test something that is asynchronous, like in my case I'm wanting to test if a webview loads an URL how would I go about waiting for the webpage to finish loading in my webview ?

Posted by Pedro Veloso.

A:Actually this is a very interesting question as there are many ways and you should be cautious about WebView semantics and how some of the WebViewClient methods are called.

For example onPageFinished may be invoked wether the page was successfully loaded or there was an error. So, you may need a different approach if your intention is to test if an url was successfully loaded.

In this code snippet I'm using a MockWebViewClient to detect error conditions and simply waiting some time for the page to load. We could also iterate over a period of time checking if the value has changed instead of just waiting but we are keeping this as simple as possible. This is also assuming you have an Activity holding the WebView and it has the required getters.

/**
 * 
 */
package com.example.aatg.webview.test;

import android.test.ActivityInstrumentationTestCase2;
import android.test.suitebuilder.annotation.Suppress;
import android.webkit.WebView;
import android.webkit.WebViewClient;

import com.example.aatg.webview.AndroidHelloWebViewActivity;

/**
 * @author diego
 *
 */
public class AndroidHelloWebViewActivityTests extends
        ActivityInstrumentationTestCase2<AndroidHelloWebViewActivity> {

    private static final String VALID_URL = "http://developer.android.com";
    private static final String INVALID_URL = "http://invalid.url.doesnotexist987.com";

    private static final long TIMEOUT = 5000;

    private AndroidHelloWebViewActivity mActivity;
    private WebView mWebView;
    private MockWebViewClient mMockWebViewClient;

    /**
     * @param name
     */
    public AndroidHelloWebViewActivityTests() {
        super(AndroidHelloWebViewActivity.class);
    }

    /* (non-Javadoc)
     * @see android.test.ActivityInstrumentationTestCase2#setUp()
     */
    protected void setUp() throws Exception {
        super.setUp();
        mActivity = getActivity();
        mWebView = mActivity.getWebView();
        mMockWebViewClient = new MockWebViewClient();
        mWebView.setWebViewClient(mMockWebViewClient);
    }

    /* (non-Javadoc)
     * @see android.test.ActivityInstrumentationTestCase2#tearDown()
     */
    protected void tearDown() throws Exception {
        super.tearDown();
    }

    public final void testLoadValidUrl() {
        assertLoadUrl(VALID_URL);
        assertFalse(mMockWebViewClient.mError);
    }

    public final void testLoadInvalidUrl() {
        assertLoadUrl(INVALID_URL);
        assertTrue(mMockWebViewClient.mError);
    }

    private void assertLoadUrl(String url) {
        mWebView.loadUrl(url);
        sleep();
        assertTrue(!(mWebView.getProgress() < 100));
    }

    private void sleep() {
        try {
            Thread.sleep(TIMEOUT);
        } catch (InterruptedException e) {
            fail("Unexpected timeout");
        }
    }

    private class MockWebViewClient extends WebViewClient {
        boolean mError;

        @Override
        public void onReceivedError(WebView view, int errorCode,
                String description, String failingUrl) {
            mError = true;
        }
    }
}

Hope this helps.

Android Application Testing Guide: Q&A

Lately I've been receiving some comments or questions about some of the book's subjects in my email. To help the community the best I think that is preferable that you share your questions here, as comments to this post if you cannot find a post dealing with the same topic. If possible I will answer also here for the benefit of all.

Android Application Testing Guide

The wait is almost over and after a year of hard work the book is finished and is expected to be published by PACKT this month (June 2011).

You can Pre-order now !

Approach
Adroid Application Testing Guide is a highly detailed book which gives step-by-step examples for a great variety of real-world cases, providing professional guidelines and recommendations that will be extremely valuable for optimizing your development time and resources. In the chapters you will find an introduction to specific testing techniques, and tools for specific situations.

Overview of Android Application Testing Guide

• The first and only book that focuses on testing Android applications
• Step-by-step approach clearly explaining the most efficient testing methodologies
• Real world examples with practical test cases that you can reuse
• eBook available as PDF and ePub downloads and also on PacktLib

More detailed information can be obtained from its web page at PACKT.