Tuesday, February 07, 2012

monkeyrunner: interacting with the Views

The time may come when you want your tests to interact with the Application Under Test (AUT) in a more clever way than just guessing the View's coordinates on the screen and sending the events.


Furthermore, this is sometimes not possible at all because in order to send the events you may need to obtain some View state. Let me give you an example to illustrate this and if you want you can try to solve it using monkeyrunner.


We can use Development Settings as our AUT and our intention could be to activate the Show running processes and Immediately destroy activities settings.
As we may accustom to do, we can obtain the coordinates of these Views on the screen and send the corresponding touch events using MonkeyDevice.touch() as usual. Sonner or later, maybe sooner, we will discover that because these settings are persistent we should know the state before sending the event, otherwise we will be changing its state other than just enabling this settings as is this example's intention.



We introduced a way of doing things like that in 
Automated Android testing using Sikuli using visual comparison and obtaining properties like the text in EditText's in monkeyrunner: testing views properties (which depends on a patch to chimpchat that has not yet been approved) but now we are craving for a more general approach unless we had the intention to patch chimpchat to support all of the properties in the different Views.

This approach is AndroidViewClient that you can download and install from github. Still has its rough edges but I wanted to show its functionality here and be open to comments.
AndroidViewClient adds to monkeyrunner the ability of
  • finding Views by ID, very much like you normally do in your Android Activity (using ViewClient.findViewById())
  • obtaining the value for almost any of the Views properties (using for example View.isChecked() or View.mText())
  • sending touch events to the Views by simply invoking View.touch()
The following script is a case of these abilities in action.


#! /usr/bin/env monkeyrunner
'''
Copyright (C) 2012  Diego Torres Milano
Created on Feb 3, 2012

@author: diego
'''


import re
import sys
import os

# this must be imported before MonkeyRunner and MonkeyDevice,
# otherwise the import fails
try:
    ANDROID_VIEW_CLIENT_HOME = os.environ['ANDROID_VIEW_CLIENT_HOME']
except KeyError:
    print >>sys.stderr, "%s: ERROR: ANDROID_VIEW_CLIENT_HOME not set in environment" % __file__
    sys.exit(1)
sys.path.append(ANDROID_VIEW_CLIENT_HOME + '/src')
from com.dtmilano.android.viewclient import ViewClient

from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice


# Displayed com.android.development/.DevelopmentSettings: +379ms
package = 'com.android.development'
activity = '.DevelopmentSettings'
componentName = package + "/" + activity
device = MonkeyRunner.waitForConnection(60, "emulator-5554")
if not device:
   raise Exception('Cannot connect to device')

device.startActivity(component=componentName)
MonkeyRunner.sleep(5)

vc = ViewClient(device)
vc.dump()

showCpu = vc.findViewById("id/show_cpu")
showLoad = vc.findViewById("id/show_load")
alwaysFinish = vc.findViewById("id/always_finish")

if not showLoad.isChecked():
    showLoad.touch()

if not alwaysFinish.isChecked():
    alwaysFinish.touch()

if not showCpu.isChecked():
    # WARNING: Show CPU usage is de-activated as soon as it's activated, that's why it seems it
    # is never set
    showCpu.touch()

Once we run this script against a running emulator or device we will be able to see how the settings are enabled.


Moreover, if you run the script again you will see how the settings remain unchanged.
AndroidViewClient is a work in progress but it is already useful for many cases where the monkeyrunner scripts are not enough or where the complexity of achieving the desired goals is too high.

Give AndroidViewClient a try, share your comments, suggestions, patches and scripts here to help improve it.

29 comments:

Joe Bowbeer said...

Thanks for the writeup. This same information is available in Java from the full hierarchyviewerlib.

The hierarchy viewer DeviceBridge can be initialized from ChimpChat as follows:

DeviceBridge.acquireBridge(
AndroidDebugBridge.getBridge());

Have you observed that sending DPAD physical button presses does not work in monkeyrunner on android-15 emulators, whereas it worked on android-10?

chimpDevice.press(
PhysicalButton.DPAD_DOWN,
TouchPressType.DOWN_AND_UP);

// nope

Joe Bowbeer said...
This comment has been removed by the author.
Joe Bowbeer said...

Update: DPAD presses are broken on newer emulators.

See Issue 25369 for a workaround.

Canvas said...

@Joe Bowbeer, I'm a freshman to Chimchat. Could you give advice how to use DeviceBridge in chimchat?

Joe Bowbeer said...

ChimpChat is the internals of MonkeyRunner that has recently been refactored so that it can be accessed directly in Java.

A basic sequence is:

ChimpChat cc = ChimpChat.getInstance();
IChimpDevice cd = cc.waitForConnection(5000, ".*");
if (cd != null) {
cd.wake();
cd.press("KEYCODE_DPAD_DOWN",
TouchPressType.DOWN_AND_UP);
cc.shutdown();
}

A simple HierarchyViewer can be obtained from IChimpDevice:

HierarchyViewer hv = cd.getHierarchyViewer();

However, this hv has a bug (see patch submitted by Diego) and it is missing some functionality.

DeviceBridge is a more powerful interface. You can pretty much write your own hierarchyviewer tool using DeviceBridge.

DeviceBridge.acquireBridge(AndroidDebugBridge.getBridge());
IDevice[] devs = DeviceBridge.getDevices();
IDevice dev = devs[0];
Window[] wins = DeviceBridge.loadWindows(dev);
ViewNode vn = DeviceBridge.loadWindowData(win[0]);

Android app development said...

Excellent post.I got a fantastic idea about Android iPhone developer by read this post.This is one of the thoughtful post.
Android app developers

Diego Torres Milano said...

Thnaks Joe Bowbeer for your contributions to this post.

小腳掌 said...

Dear Diego,

Many thanks for your great sharing of Android knowledge.

For MonkeyRunner, I would like to confirm two basic concepts first:
1. Did your MonkeyRunner work on Windows platform? I only success on Ubuntu environment.
2. If MonkeyRunner could work and control GUI event. Once the event was done, how to you measure the result if correct or not? For example, if MonkeyRunner launch a browser with an URL, how to confirm the behavior was correct? This cannot be done by MonkeyRunner.

小腳掌 said...
This comment has been removed by the author.
小腳掌 said...
This comment has been removed by the author.
小腳掌 said...
This comment has been removed by the author.
evgeny9 said...

Hello, Diego!
Thank you for another great post.
Couldn't you help me out with installing your tool AndroidViewClient in Windows?
Where should I place it to have a successful import in my monkeyrunner script?
Thank you in advance.

evgeny9 said...

I opened an issue with an error in your project: https://github.com/dtmilano/AndroidViewClient/issues/2.
Please, see!

Diego Torres Milano said...

@evgeny9: I added an extra debug message to help us find the issue. Check github.

Diego Torres Milano said...

@小腳掌 ,
1) monkeyrunner (lowercase) should run on Windows as well as Linux and OSX, and BTW it's Android's not mine ;-)

2) I added the example (browser-open-url.py) to (github). It's exactly what you are trying to do, so I hope it helps.

sridhar said...

Hi Thanks for the valuable post.

My question is regarding hierarchyviewer.

I am able to get windows name as Joe Bowbeer said.
how to get Children size and children names of ViewNode.

Garry said...

I would like to thank Diego. In this blog, discovered for himself useful information.
I wish to draw attention to the use of AndroidViewClient in landscape mode.
Perhaps someone can help solve this problem?

PP said...

Hi Diego,
thanks for your sharing knowledge.
now i have a question about how to implement a method like
touchByText(self, text)
getXY of the text
device.touch with the (x,y)

I just want to simply use text instead of android id.

Thank you.

bharat said...

Hello,

Why am I getting a ERROR: ANDROID_VIE
W_CLIENT_HOME not set in environment


while running your example script browser-open-url.py

What I am doing is monkeyrunner.bat "C:\path\AndroidViewClient\examples\browser-open-url.py"

Diego Torres Milano said...

ANDROID_VIEW_CLIENT_HOME is mandatory. You must set it up in the environment specifying the installation directory of AndroidViewClient.

ビクトル said...

When I try to load ChimpChat in my Android app I get the following error, any idea?


05-04 16:45:10.488: E/AndroidRuntime(4890): java.lang.NoClassDefFoundError: com.android.chimpchat.ChimpChat

Diego Torres Milano said...

Are you following Using monkey from Java ?

Chimpchat is not intended to be used in your Android application but on a Java one.

Anand P said...

I getting a ERROR: ANDROID_VIE
W_CLIENT_HOME not set in environment

I have set the environment variable in eclipse as well as in the code.

Code:

try:
ANDROID_VIEW_CLIENT_HOME = os.environ['C:/Users/Dell/Desktop/dtmilano-AndroidViewClient-f93204c/AndroidViewClient']
except KeyError:
print >>sys.stderr, "%s: ERROR: ANDROID_VIEW_CLIENT_HOME not set in environment" % __file__
sys.exit(1)
sys.path.append(ANDROID_VIEW_CLIENT_HOME + '/src')


Running the script from command prompt:

monkeyrunner.bat C:\workspace_jython\JythonSample\src\Sample.py

Diego Torres Milano said...

ANDROID_VIEW_CLIENT_HOME is not set in your environment. Try:

set ANDROID_VIEW_CLIENT_HOME='\path\to\androidviewclient'

in your command prompt before running the script.

robin zhou said...

Diego:
The AndroidViewClient is a great module, it give a chance to take monkeyrunner into more testing. and make it possible to reuse the script even the UI screen have diffrent.
the core concept is dump the infomation of current view, and store it into the vc.views. then pick up the position and id informaiton from it. BUT, the data is so large about 2M,each dump action will use more than 10s via the adb port forward.
So I think it is the key trouble.
whether I have something wrong in my test? how about the performance in your test for a dump operation?

robin zhou said...
This comment has been removed by the author.
rilwan said...

Dear Diego,
Excellent tool,Expanding monkeyrunner to new areas.thanks.
Is there any documentation or webpage how it is taking the dump.dump action is taking 8-10 secs,sometimes UI is changing by the time.
Is there any way to improve dump action.?
thanks.

Diego Torres Milano said...

@rilwan, Thanks for your comment. Unfortunately the time it takes to get the dump is mainly due to the ViewServer and not much can be done on the client side. If you run hierarchyviewer you can see that it takes almost the same time because both tools use the same strategy.

rilwan said...

thanks Diego for the reply.