Sunday, November 09, 2008

Android: Testing on the Android platform - ApiDemos tests











This document can be read in Google Docs (http://docs.google.com/Doc?id=ddwc44gs_49stmqs7hb), cut and paste link.





A previous post,Android: Testing on the Android platform - Unit tests, showed how to add your tests and run them from inside Eclipse. Although these are very useful tests in our quest to Test Driven Development, they cover only a small part of our needs.

Android 1.0 introduced a whole new world of tests cases like ActivityUnitTestCase to run isolated unit tests, ServiceTestCase to tests services and even functional test of activities like ActivityInstrumentatonTestCase.



ApiDemos provides a set of tests but you will not be able to easily run them unless in a non convoluted way as the way they are structured it's not supported by current Android Development Tools (ADT) Eclipse plugin and you would need other tools, but the solution is so simple that I wonder why they are not originally provided in the way we will be changing them.

Google documentation is not helping either.



ApiDemos tests


Tests are located in the tests/src folder and they have their corresponding AndroidManifest.xml.

Inside this folder, tests like this one are found










/*

 * Copyright (C) 2008 Google Inc.

 */



package com.example.android.apis;



import android.test.ActivityInstrumentationTestCase;



/**

 * Make sure that the main launcher activity opens up properly, which will be

 * verified by {@link ActivityInstrumentationTestCase#testActivityTestCaseSetUpProperly}.

 */

public class ApiDemosTest extends ActivityInstrumentationTestCase<ApiDemos> {



    public ApiDemosTest() {

        super("com.example.android.apis", ApiDemos.class);

    }



}







but we will not be able to run them not resorting to ant or maven, unless we make the following set of changes.

ApiDemosTest Android project




Create new project


Let's create a new Android project from within Eclipse, as usual New -> Project -> Android -> Android Project






Activity and Application name are set to Dummy because they are not used as we are seeing shortly.



Copy ApiDemos tests




Copy the tests/src folder from te original ApiDemos project and turn it into a source folder.

Fix the project setup to use original ApiDemos project to resolve references.






Copy AndroidManifest.xml


Again, from the original ApiDemos project copy AndroidManifest.xml overwriting the existing one






Build and run



If necessary, remove ApiDemos.apk if it has been already installed in the emulator.

Build ApiDemos and ApiDemosTests projects and install them.

Now, if you execute Dev Tools in the emulator and go to Instrumentation, you'll find Test for Api Demos. in the list.








Select it and you will obtain the tests results. Unfortunatelly by now, the reslts will be only visible in the logcat window









11-09 22:56:35.132: INFO/instrumentation(153): Test results for InstrumentationTestRunner=......................

11-09 22:56:35.132: INFO/instrumentation(153): Time: 6.739

11-09 22:56:35.132: INFO/instrumentation(153): OK (22 tests)






or alternativelly, from the command line

$ adb shell am instrument -w com.example.android.apis.tests/android.test.InstrumentationTestRunner

com.example.android.apis.ApiDemosApplicationTests:....

com.example.android.apis.ApiDemosTest:.

com.example.android.apis.app.ForwardingTest:...

com.example.android.apis.app.LocalServiceTest:.....

com.example.android.apis.os.MorseCodeConverterTest:.

com.example.android.apis.view.Focus2ActivityTest:....

com.example.android.apis.view.Focus2AndroidTest:....

Test results for InstrumentationTestRunner=......................

Time: 7.493



OK (22 tests)


Conclusion


So simple that I can't understand why they are not provided in this way by the original Android ApiDemos project.

Next time we will try to present test results in a more concise way inside the emulator as Electro did (video1, video2, video3).









16 comments:

Matthew said...

I am a little unclear on the section "Copy ApiDemos tests". The end result is an eclipse source folder named "tests" with a package named "com.example.andoid.apis.tests"?

I don't quite follow the steps to set that up.

Diego Torres Milano said...

Yes, that's the idea to split the ApiDemos project into two.

Diego Torres Milano said...

I've received some comments requesting some clarification on the entire process of splitting ApiDemos.

1. Create ApiDemosTests project
2. Copy tests/src folder from ApiDemos to ApiDemosTests
3. Open tests folder to show src
4. Right click on src and select Build path -> Use as Source Folder
5. Open ApiDemosApplicationTests from com.example.android.apis and click the light bulb next to the class definition and select Fix project properties
6. Select ApiDemos project
7. Run

Hope this helps.

NOTE:this process was tested on Linux and Eclipse Ganymede, may be slightly different on other platforms and versions

LukeW said...

Hi Diego, I have justice the sources follow ur instruction, but i got a error "Conversion to Dalvik format failed with error 1"
some solution for this ?

Diego Torres Milano said...

Which step gave you that error ?

LukeW said...

When compile or build the project in eclipse automaticly

Grace Hu said...

Hi, i follow the step you gived. but wht it still can not show in the instrumentation. Is there another way to run the test?

Diego Torres Milano said...

Most probably, if they're not listed in Instrumentation, they are not installed.

Anyway, you can try

$ adb shell am instrument -w com.example.android.apis.tests/android.test.InstrumentationTestRunner

If you can provide more information I will be able to help you better.

hake said...

For "Copy AndroidManifest.xml"

did you mean copy the AndroidManifest.xml in tests/src folder? or other the apidemos folder, I copy and paste the apidemos's one, but build error, which says 42 error in AndroidManifest.xml

Diego Torres Milano said...

I meant to copy the ApiDemos/test/AndroidManifest.xml to the root of the new project.

Unknown said...

I have been unable to make this work on Android 1.0R2, Eclipse 3.4.2, and Windows XP SP3.

1) Open Eclipse
2) Select File -> New Android Project
3) Enter 'ApiDemosTest" for the project name
4) Enter "com.example.android.apis.tests" for the package.
5) Enter "Dummy" for the activity name.
6) Enter "Dummy" for the application name.
7) Click on the Finish button.
8) In the Navigator, select ApiDemos/tests and drag it into
ApiDemosTest.
9) Right click on ApiDemosTest and select Properties.
10) Select Java Build Path.
11) Select the Source tab.
12) Select ApiDemosTest/src and click on the Remove button.
13) Click on the Add Folder button.
14) Select tests/src.
15) Select the Projects tab.
16) Click on the Add button.
17) Select ApiDemos.
18) Click on the Ok button
19) Click on the Ok button
20) Select Project -> Build All.

The result is that I get the following errors:

Description Resource Path Location Type
Class com.example.android.apis.tests.Dummy does not exist
AndroidManifest.xml ApiDemosTest line 8 Android Problem

Description Resource Path Location Type
Package 'com.example.android.apis.tests' does not exist!
AndroidManifest.xml ApiDemosTest Unknown aapt Problem

RodneyB said...

Diego,
I'm having problems getting this demo to work. I've got android-sdk-mac_x86-1.0_r2, Eclipse Ganymede, and JUnit 4.5 on Mac OS X 10.5.

I more or less followed your instructions; won't guarantee I did it all the same, but the .apk's built, and I installed them to the emulator successfully. The "Tests for Api Demos" didn't show up in Instrumentation, so I shut down the emulator and restarted it, and then it did show up.

When I selected" Tests for Api Demos," LogCat says,
ERROR/dalvikvm(288): Unable to resolve Lcom/example/android/apis/ApiDemosApplicationTests; annotation class 164

and a few lines later:
ERROR/AndroidRuntime(288): java.lang.NoClassDefFoundError: android.test.suitebuilder.annotation.SmallTest

When I run from the command line, it produces the same NoClassDefFoundError.

Any idea what's causing this? Is there something I don't have set up right in Eclipse, or installed properly to the emulator?

Diego Torres Milano said...

Strange, it seems like you don't have the Annotations classes.
Try upgrading to Android SDK 1.1 (http://developer.android.com/sdk/download.html?v=android-sdk-mac_x86-1.1_r1.zip) and be sure that you are using the same ApiDemos that came with your SDK.

Also verify that your AndroidManifest.xml has these lines:

<instrumentation android:name="android.test.InstrumentationTestRunner"
android:targetPackage="com.example.android.apis"
android:label="Tests for Api Demos."/>

RodneyB said...

Thanks Diego! I had both 1.0_R2 and 1.1_R1 on my system, and it does appear to have been a mismatch somewhere along the line. Fixed now.

Raja said...

i was trying to run MediaFramework test suite as a part of Android unit testing. The test suite expects a big list of media files (mp3, mp4, wma, ogg, wav, amr, 3gp) to be available in sdcard mounted on the emulator. The list of files can be found in MediaNames.java file, since the names are hardcoded with the file name and params like width, duration, length, wait/pause time etc, it appears they should be available somewhere. But i couldn't find them anywhere in the code repository, so was wondering where to get these media files from. Tests can be probably done with own media, but the files are too many and it would be lengthier procedure to download them and update MediaNames.java file accordingly with the params i mentioned above.
Any clue where we can find them.

Gubatron said...

Here is a ScreenCast I made on How to Unit Test on Android and Eclipse

http://www.gubatron.com/blog/2010/05/02/how-to-do-unit-testing-on-android-with-eclipse/

Watching this will save you a lot of headaches.