How can we obtain the code coverage of a running application, not just its tests ?
I have been asked this question many times. Recently,
Jonas posted a similar question as comment to
Eclipse, Android and EMMA code coverage. So we will elaborate the solution to this problem.
But firstly, let's do a brief introduction of the concepts.
EMMA: a free Java code coverage tool
EMMA is an open-source toolkit for measuring and reporting Java code coverage. EMMA distinguishes itself from other tools by going after a unique feature combination: support for large-scale enterprise software development while keeping individual developer's work fast and iterative.
Android includes
EMMA v2.0, build 5312, which includes some minor changes introduced by Android to adapt it for the platform specifics.
Android Instrumentation
The instrumentation framework is the foundation of the testing framework. Instrumentation controls the application under test and permits the injection of mock components required by the application to run.
Usually, an
InstrumentationTestRunner, a special class the extends Instrumentation, is used to run various types of TestCases, against an android application.
Typically, this Instrumentation is declared in the test project's AndroidManifest.xml and then run from Eclipse or from the command line using am instrument.
Also, to generate EMMA code coverage -e coverage true option is added to the command line.
Basically, we have all the components but in different places because we want to obtain the code coverage from the running application not from its tests.
EmmaInstrumentation
The first thing we need to do is to create a new Instrumentation that starts the Activity Under Test using EMMA instrumentation and when this Activity is finished the coverage data is saved to a file.
To be notified of this Activity finish we need a listener that we can set extending the AUT because one of our objectives is to keep it unchanged.
To illustrate this technique we will be using the
Temperature Converter application that we have used many times in other
posts. The source code is as usual available through
github.
package com.example.instrumentation;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import com.example.i2at.tc.TemperatureConverterActivity;
import android.app.Activity;
import android.app.Instrumentation;
import android.content.Intent;
import android.os.Bundle;
import android.os.Looper;
import android.util.Log;
public class EmmaInstrumentation extends Instrumentation implements FinishListener {
private static final String TAG = "EmmaInstrumentation";
private static final boolean LOGD = true;
private static final String DEFAULT_COVERAGE_FILE_PATH = "/mnt/sdcard/coverage.ec";
private final Bundle mResults = new Bundle();
private Intent mIntent;
private boolean mCoverage = true;
private String mCoverageFilePath;
Extends the AUT to provide the necessary behavior to invoke the
{@link FinishListener} that may have been provided using
{@link #setFinishListener(FinishListener)}.
<code></code>
{@link EmmaInstrumentation#onStart()}
@author
public static class InstrumentedActivity extends
TemperatureConverterActivity {
private FinishListener mListener;
public void setFinishListener(FinishListener listener) {
mListener = listener;
}
@Override
public void finish() {
if (LOGD)
Log.d(TAG + ".InstrumentedActivity", "finish()");
super.finish();
if (mListener != null) {
mListener.onActivityFinished();
}
}
}
Constructor
public EmmaInstrumentation() {
}
@Override
public void onCreate(Bundle arguments) {
if (LOGD)
Log.d(TAG, "onCreate(" + arguments + ")");
super.onCreate(arguments);
if (arguments != null) {
mCoverage = getBooleanArgument(arguments, "coverage");
mCoverageFilePath = arguments.getString("coverageFile");
}
mIntent = new Intent(getTargetContext(), InstrumentedActivity.class);
mIntent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
start();
}
@Override
public void onStart() {
if (LOGD)
Log.d(TAG, "onStart()");
super.onStart();
Looper.prepare();
InstrumentedActivity activity = (InstrumentedActivity) startActivitySync(mIntent);
activity.setFinishListener(this);
}
private boolean getBooleanArgument(Bundle arguments, String tag) {
String tagString = arguments.getString(tag);
return tagString != null && Boolean.parseBoolean(tagString);
}
private void generateCoverageReport() {
if (LOGD)
Log.d(TAG, "generateCoverageReport()");
java.io.File coverageFile = new java.io.File(getCoverageFilePath());
try {
Class<?> emmaRTClass = Class.forName("com.vladium.emma.rt.RT");
Method dumpCoverageMethod = emmaRTClass.getMethod(
"dumpCoverageData", coverageFile.getClass(), boolean.class,
boolean.class);
dumpCoverageMethod.invoke(null, coverageFile, false, false);
} catch (ClassNotFoundException e) {
reportEmmaError("Is emma jar on classpath?", e);
} catch (SecurityException e) {
reportEmmaError(e);
} catch (NoSuchMethodException e) {
reportEmmaError(e);
} catch (IllegalArgumentException e) {
reportEmmaError(e);
} catch (IllegalAccessException e) {
reportEmmaError(e);
} catch (InvocationTargetException e) {
reportEmmaError(e);
}
}
private String getCoverageFilePath() {
if (mCoverageFilePath == null) {
return DEFAULT_COVERAGE_FILE_PATH;
} else {
return mCoverageFilePath;
}
}
private void reportEmmaError(Exception e) {
reportEmmaError("", e);
}
private void reportEmmaError(String hint, Exception e) {
String msg = "Failed to generate emma coverage. " + hint;
Log.e(TAG, msg, e);
mResults.putString(Instrumentation.REPORT_KEY_STREAMRESULT, "\nError: "
+ msg);
}
@Override
public void onActivityFinished() {
if (LOGD)
Log.d(TAG, "onActivityFinished()");
if (mCoverage) {
generateCoverageReport();
}
finish(Activity.RESULT_OK, mResults);
}
}
We are also implementing the FinishListener interface, which is defined as
package com.example.instrumentation;
Listen for an Activity to finish and invokes {@link #onActivityFinished()} when this happens.
@author
public interface FinishListener {
Invoked when the Activity finishes.
void onActivityFinished();
}
Running the instrumented application
Once we have the EmmaInstrumentation class in place we need a few more adjustments to be able to get the coverage report of the running application.
Firstly, we need to add the new Activity to the manifest. Secondly, we should allow our application to write to the sdcard if this is where we decided to generate the coverage report. To do it you should grant the android.permission.WRITE_EXTERNAL_STORAGE permission.
Then, it's time to build and install the instrumented apk:
$ ant clean
$ ant instrument
$ ant installi
Everything is ready to start the instrumented application
$ adb shell am instrument -e coverage true \
-w com.example.i2at.tc/\
com.example.instrumentation.EmmaInstrumentation
If everything went well, the Temperature Converter application will be running and we can use it for a while
when we exit by pressing the BACK button we can see that the coverage data was written to the file and reflected in the logcat
I/System.out(2453): EMMA: runtime coverage data written to [/mnt/sdcard/coverage.ec] {in 975 ms}
this file can then be moved to the host computer using adb pull.
Hope this helps you obtaining the code coverage for your application to help you understand its usage patterns. As always, comments and questions are always welcome.
Nonetheless, from time to time, I receive some questions or reports about problems with scripts created with `culebra` that are attempted to run with `monkeyrunner` or some other combinations. I take the blame for it. I failed at communicating that AndroidViewClient/culebra is a complete replacement and should not be used together.
