Tuesday, November 27, 2012

AndroidViewClient: UiAutomator support

AndroidViewClient v2.3.1 has been released recently providing UiAutomator compatibility, when supported by the device or emulator. UiAutomator is supported since Android API 16.

This is a great improvement over previous version in two different aspects:

  • it can now be run on non-rooted devices not demanding application modification (as previous version required to use LocalViewServer)
  • the change in the backend now frees the client from port redirection, as ViewServer requests, and at the same time the performance of dumping the View tree is greatly improved
 As an introduction of this new release we will be using a simple example that demonstrates some of the new features. This example us based on the demo application AndroidSampleUi.apk.

This example demonstrates:

  • automatic device connection, handling command line parameters if present
  • automatic View tree dump
  • finding Views using regular expressions or text
  • touching found Views
As a precondition to run this example, install and run AndroidSampleUI.


Zoom buttons let you increase or decrease the margins and consequently move the toggle buttons to demonstrate that they will be found whatever their coordinates are.

#! /usr/bin/env monkeyrunner
'''
Copyright (C) 2012  Diego Torres Milano
Created on Aug 31, 2012

@author: diego
'''


import re
import sys
import os

# This must be imported before MonkeyRunner and MonkeyDevice,
# otherwise the import fails.
# PyDev sets PYTHONPATH, use it
try:
    for p in os.environ['PYTHONPATH'].split(':'):
       if not p in sys.path:
          sys.path.append(p)
except:
    pass

try:
    sys.path.append(os.path.join(os.environ['ANDROID_VIEW_CLIENT_HOME'], 'src'))
except:
    pass
from com.dtmilano.android.viewclient import ViewClient, ViewNotFoundException

vc = ViewClient(*ViewClient.connectToDeviceOrExit())

# Find the 3 toggle buttons, because the first 2 change their text if they are selected
# we use a regex to find them.
# Once found, we touch them changing their state
for t in [re.compile('Button 1 .*'), re.compile('Button 2 .*'), 'Button with ID']:
    try:
        vc.findViewWithTextOrRaise(t).touch()
    except ViewNotFoundException:
        print >>sys.stderr, "Couldn't find button with text=", t

Once this script is run, ViewClient will find a device, which can be specified using its serial number in the command line invoking the script, connects to, automatically dump the tree and then use regular expressions to find two of the ToggleButtons because we couldn't use a fixed text because it changes when the button is clicked. If, for some reason the Button is not found, perhaps because it was move outside the screen using Zoom buttons, a message is printed.

This screenshot show the state of the Buttons after the script has run.



More articles and examples will be coming soon, but I didn't want to miss the opportunity to introduce this new version. One of the most remarkably advantages over plain UiAutomator is the  simplification of the script or test creation and the expressiveness gain of using Python instead of Java.

dump.py

dump.py is also present in AndroidViewClient examples. It now supports several command line options now

usage: dump.py [-u|--uniqueId] [-x|--position] [-d|--content-description] [serialno]

so we can use it to verify the content of the screen. If everything went well running

dump.py --content-description

will show the View tree including also the content descriptions, as given by the following dump


android.widget.FrameLayout id/no_id/1  
   android.widget.LinearLayout id/no_id/2  
      android.widget.FrameLayout id/no_id/3  
         android.view.View id/no_id/4  
            android.widget.FrameLayout id/no_id/5  
               android.widget.ImageView id/no_id/6  
            android.widget.LinearLayout id/no_id/7  
               android.widget.LinearLayout id/no_id/8  
                  android.widget.TextView id/no_id/9 Sample UI v2.0 
      android.widget.FrameLayout id/no_id/10  
         android.widget.RelativeLayout id/no_id/11  
            android.widget.Button id/no_id/12 Show Dialog show_dialog
            android.widget.LinearLayout id/no_id/13  
               android.widget.TextView id/no_id/14  
               android.widget.ToggleButton id/no_id/15 Button 1 OFF button_1
               android.widget.TextView id/no_id/16 v=(75.0,82.0) lw=(115,272) ls=(115,272) wxh=(290,72) margin=(40,80) button_1_info
               android.widget.ToggleButton id/no_id/17 Button 2 OFF button_2
               android.widget.TextView id/no_id/18 v=(75.0,273.0) lw=(115,463) ls=(115,463) wxh=(290,72) margin=(40,80) button_2_info
               android.widget.ToggleButton id/no_id/19 Button with ID button_with_id
            android.widget.ZoomControls id/no_id/20  zoom
               android.widget.ZoomButton id/no_id/21  
               android.widget.ZoomButton id/no_id/22  


UPDATE:
Changed linked version to AndroidViewClient 2.3.1 as some latest commits were not in version 2.3 as found by Durairaj.

61 comments:

  1. Thank you for your continued work on UI automation. Will try this and let you know

    Regards
    Durai

    ReplyDelete
  2. Hi Milano,
    I tried running the sample as it is, but I could not get it to work.

    I got the following error:

    Couldn't find button with text=
    Couldn't find button with text=
    Couldn't find button with text= Button with ID

    I verified window_dump.xml from sdcard, I could see the button id/text dumped properly. I am running in a JB non-rooted device, also tried in emulator with the same results.

    ReplyDelete
  3. @Durairaj,
    Hi, thanks for your comment and reporting back.
    I updated the post including how to obtain the dump and how it should look like.

    Let me know if your is different.

    ReplyDelete
  4. Hi Milano, the problem is solved after applying "Fixed text property use with UiAutomator" from your git repository. Now it is able to identify the views, and sending the touch events, but still it is not sending the touch events at the right co-ordinates. I suspect Line number 451 in viewclient.py "if not self.useUiAutomator:"

    Communicating via comments is difficult :-)

    ReplyDelete
  5. @Durairaj,

    You will need commit 1b37d23dc5543507ea07ab0cc69b96f9c3245818 as well to be able to run the same example as described in the post.

    I strongly agree, this comments are really a pain. Feel free to use Google+ which is a little less painful.

    ReplyDelete
  6. Thank you. It is working correctly now.

    ReplyDelete
  7. @Durairaj,
    Updated the link in the post to include these latest commits. Thanks for reporting it.

    ReplyDelete
  8. Hey Diego + co., thanks for this latest update. I'm making partial progress and then running into a com.android.ddmlib.ShellCommandUnresponsiveException.

    I am able to connect to an initial screen, but when I go to a subsequent screen, which is much more content-rich, I'm getting this exception after about 5 seconds.

    I'm thinking there is a timeout somewhere in the android code that could be extended?

    ReplyDelete
  9. ah, I found this SO post which sheds some light

    http://stackoverflow.com/questions/10502833/androids-monkeyrunner-occasionally-throws-exceptions

    Requires doing a custom Android jar build though. Is there a less painful way to address this?

    ReplyDelete
  10. @etherjoe,
    Are you using ViewClient's UiAutomator backend?
    It should happend if you are. Just to verify this, use the following lines in your script:

    vc = ViewClient(*ViewClient.connectToDeviceOrExit())
    if vc.useUiAutomator:
    print "ViewClient: using UiAutomator backend"

    ReplyDelete
  11. @Diego, looks like I am:

    >>> vc = ViewClient(*ViewClient.connectToDeviceOrExit())
    >>> print vc.useUiAutomator
    True

    still getting the exception though. :(

    ReplyDelete
  12. oops, regarding the com.android.ddmlib.ShellCommandUnresponsiveException, this is the correct SO link:

    http://stackoverflow.com/questions/4264057/android-cts-is-showing-shellcommandunresponsiveexception-on-emulator

    This article provides some insight on extending the default DDMS timout.

    ReplyDelete
  13. I am getting the following error:
    view = vc.findViewWithAttributeOrRaise('ENGLISH', 'English')
    File "D:\Telus\Softwares\AndroidViewClient-master\AndroidViewClient\src\com\dtmilano\android\viewclient.py", line 1409, in findViewWithAttributeOrRaise
    raise ViewNotFoundException("Couldn't find View with %s='%s' in tree with root=%s" % (attr, val, root))
    com.dtmilano.android.viewclient.ViewNotFoundException: Couldn't find View with ENGLISH='English' in tree with root=ROOT

    ReplyDelete
  14. Hi Diego,
    view.touch() not working properly, if the view is part of an activity which is not full screen. getXY() also returns wrong co-ordinates(might be returning relative co-ordinates). Same issue is there with monkeyrunner touch(). Is there any workaround for this in androidviewclient?

    Thanks & Regards,
    Tara

    ReplyDelete
  15. @Tara,
    AndroidViewClient suppports the case you mention in your post, however there are tons of variations.

    Please specify your case properly, with all the details and screenshots if possible. Create the issue at https://github.com/dtmilano/AndroidViewClient/issues so it can be tracked properly.

    ReplyDelete
  16. Is there any way how can I run a adb command in UI Automator

    ReplyDelete
  17. @rohit_blog,
    I don't understand what you are trying to do.
    Can you please provide more information, examples, etc.?

    If your comment is related in any way with AndroidViewClient (which I guess it is) please take a look at this post: http://dtmilano.blogspot.com/2012/10/androidviewclient-google-pages.html

    I really appreciate if you use Google+ for the comments as they are easier to track there.

    ReplyDelete
  18. @Manasa Muchchatti,
    You dind't provide much information about your problem (i.e. source code) but I think I can guess.

    After dragging you have to capture the new screen content again using ViewClient.dump():

    drag(...)
    sleep(...)
    vc.dump()
    vc.findViewById(...)


    Also, please take a look at this post: http://dtmilano.blogspot.com/2012/10/androidviewclient-google-pages.html.
    I would really appreciate if you place your AndroidViewClient comments there as they are much easier to handle.

    ReplyDelete
  19. Wow Seems like you are using the Python programming languages. But Any how I prefer to use the JAVA Programming to make the Android Apps. Looking for some JAVA Codes from you.

    Mobile App Development

    ReplyDelete
  20. HI Milano,
    I got this error very often when dump view:
    Error executing command: uiautomator dump /mnt/sdcard/window_dump.xml

    is there a better way to avoid this error?
    Thanks

    ReplyDelete
  21. @Jerry Chen,
    I moved the discussion to Google+.
    Please visit https://plus.google.com/111731764904697052166/posts/bXEH4a57RWD

    ReplyDelete
  22. Hi Diego,

    I've been attempting to use findViewById('id/idName')

    but have been failing. I have access to the source code and know that the activity I'm loading should have the view with android:id="@+id/idName" but I continue to get an exception.

    When I run dump.py, I see that all my views on the page have ids similar to id/no_id/#

    Does this mean I lose idName as a way to identify my view and have to use id/no_id/# instead? Is there something I'm missing here?

    ReplyDelete
  23. @keenos,
    This discussion was moved to Google+: https://plus.google.com/111731764904697052166/posts/3qWrHsdfJSG
    See the answer to your question there.

    ReplyDelete
  24. Hey, Can you help me?
    That is my problem, When I pass the command
    vc = ViewClient(*ViewClient.connectToDeviceOrExit())

    It raises this error:
    ERROR: UIAutomator doesn't contain 'dumped'

    Someone knows how to solve this?

    ReplyDelete
  25. @Matheus Sousa Faria,
    Latest version (2.3.6) adds a slightly verbose error message that perhaps helps you to detect the problem.

    Do you have an SD card ?

    ReplyDelete
  26. Dear Milano,

    This is a great tool. It make automation much easier. However, there is a bug in findViewWithText. I fails to find the text in the last element of a layout. So if my xml layout looks like ..

    linear layout
    ===> button text="start"
    ===> button text="stop"
    ===> button text="pause"

    It will find start/stop but not pause.

    ReplyDelete
  27. +Octav Chipara,
    Thanks for reporting this problem.
    Can you provide a test case?
    Are all the buttons visible?
    Android version?

    Tracking bug/issues reports is much easier at https://github.com/dtmilano/AndroidViewClient/issues, please consider creating one there and provide the relevant information.

    ReplyDelete
  28. Hi Diego,

    Have you been able to do multi-touch (e.g. pinch/zoom, or two-finger swipe) via monkeyrunner?

    Thanks,
    Andrew

    ReplyDelete
  29. I've been attempting to use findViewById('id/idName')

    but have been failing. I have access to the source code and know that the activity I'm loading should have the view with android:id="@+id/idName" but I continue to get an exception.

    When I run dump.py, I see that all my views on the page have ids similar to id/no_id/#

    Augmented Reality App development

    ReplyDelete
  30. You dind't provide much information about your problem (i.e. source code) but I think I can guess.

    After dragging you have to capture the new screen content again using ViewClient.dump():

    drag(...)
    sleep(...)
    vc.dump()
    vc.findViewById(...)

    Download our free ebook for mobile app development

    ReplyDelete
  31. Very informative article. You are providing such a valuable information about studying..and also have some good key points to every student.

    Free guide for macular degeneration diseases

    ReplyDelete
  32. Wow Seems like you are using the Python programming languages. But Any how I prefer to use the JAVA Programming to make the Android Apps. Looking for some JAVA Codes from you.

    Find Content writers

    ReplyDelete
  33. view.touch() not working properly, if the view is part of an activity which is not full screen. getXY() also returns wrong co-ordinates(might be returning relative co-ordinates). Same issue is there with monkeyrunner touch(). Is there any workaround for this in androidviewclient?

    Eye Health

    ReplyDelete
  34. This is a great tool. It make automation much easier. However, there is a bug in findViewWithText.

    ReplyDelete
  35. Really thanks for sharing such an useful stuff...

    mq tutorial

    ReplyDelete
  36. The language of this blog is very technical and the only android developer can relate to this blog. Can you introduce same blog in simple language so lame person also can relate to the blog and gain some additional knowledge from the same.

    ReplyDelete
  37. A backlink is a link created when one website links to another. Backlinks are important to SEO & impact for higher ranking. In my 7+ years seo Career i see, without backlinks a website doesn't rank higher on google SERP.

    Get Your 300+ High Quality DoFollow Backlinks Here!

    Order Now with Full Confidence & 100% satisfaction.

    ReplyDelete
  38. Thanks for sharing the post.
    We are an experienced team in one of the Best software company and product specialist for software development and implementation. Sovereign provides Website Design, Wordpress Development and Mobile App Development, Digital marketing and SEO Services.

    ReplyDelete
  39. Thanks for sharing a great article.
    You are providing wonderful information, it is very useful to us.
    Keep posting like this informative articles.
    Thank you.

    Get to know about 13377x

    ReplyDelete
  40. Computer networks are the basis of communication in IT. They are used in a huge variety of ways and can include many different types of network. A computer network is a set of computers that are connected together so that they can share information. The earliest examples of computer networks are from the 1960s, but they have come a long way in the half-century since then.

    computer network

    ReplyDelete
  41. Thanks for Writing this informative content it will help me to understand this topic, a long time I looking for informative content.
    To be able to acquire information about some of the best companies around the globe or to
    get contact information about companies' executives, you can refer DataGuru
    Marketing head database to conserve your valuable time wandering here and there.

    ReplyDelete
  42. This is a great post - so clear and easy to read.if you ever done research on Subscription based grocery delivery app solution. This service provides strategies to follow the online delivery Business. After looking over the few articles on your site, I truly like your technique of Hyperlocal Business App Ideas.

    ReplyDelete
  43. Become an EC-Council Certified Ethical Hacker Training(CEH) with this certification training course. Learn tools & techniques to protect your network. Attend onsite, in-class or customized training. Contact Now! +1 905-625-2727

    ReplyDelete
  44. Visit our website flixor to watch full free movies and and Tv shows online.

    ReplyDelete
  45. Check out our website afdahtv and watch online free movies and Tv shows without any membership.

    ReplyDelete
  46. Thank you for your useful blog. I think you should lead twitter blog. You can buy twitter followers to promote your blog.

    ReplyDelete

  47. Your Blog is very informative, seriously I like your blog. If you want to Application for your business visit ourmobile app development company in delhi

    ReplyDelete
  48. Amritsar Digital Academy is a licensed digital marketing academy that offers advanced online business courses in Amritsar. Digital marketing generates a pool of excellent work chances for students while also assisting large corporations in increasing sales. Our professionals will assist you in understanding digital marketing strategies and gaining hands-on experience with various long term projects so that you may obtain in-depth expertise.

    ReplyDelete
  49. Hulexo ERP allows you to effectively streamline all your retail departments. Hulexo ERP helps companies cut down on excessive resources & costs while saving valuable time, money, and manual effort that gets wasted on managing the day-to-day business operations and activities while losing out on the opportunity to grow the business. ERP Dubai

    ReplyDelete
  50. The new kid on the block is flixtor movies. It's one of the few websites like 123movies that don't have any adverts or pop-ups. If you have a look around this website, you'll find a big collection of TV shows and movies that you can watch for free online without having to register.

    ReplyDelete
  51. Hi, I am John Smith I am Web Developer, It is an amazing blog thanks for the sharing the blog. Frantic infotech provide the native mobile app development such as an information about software development for costumer service. Franti infotech also provide the wearable app development. The development of advanced web applications is Orient Software’s specialty and we will successfully fulfill all your web application development requirements, from small-sized to wider-ranged projects. We Also do work multiple platforms like:
    mobile app development services
    android app development on mobile
    ios app development services
    react native app development
    ionic app development company
    flutter app development company
    php web development services in india

    ReplyDelete
  52. We’re a Web Design Bali with over a decade of experience in building user friendly, secure websites that help strengthen your business.

    ReplyDelete
  53. Great Info! I Recently Came Across Your Blog And Have Been Reading Along. I Thought I Would Leave My First Comment. I Don’t Know What To Say Except That I Have Best Hospital Software in India.
    Best Hospital Management Software in India
    Best Hospital Software in India

    ReplyDelete
  54. Thanks for sharing!!!
    Your blog is very nice and we hope you are providing more information in future times.
    Wine Glass
    Bathroom Mirror
    Wall Mirror

    ReplyDelete
  55. Quite possibly of the best cross breed stage that permit making a mixture application is Ionic. Ionic isn't simply a structure, an environment incorporates all that improvement needs, beginning from admittance to portable working elements itself to sending and fabricates a cycle, and even dissemination. Made by the organization Drifty Co. in 2013, Ionic is a genuinely late open-source structure that has since laid out its image as a capable half breed application improvement system. Drifty recently was centered around building UI instruments for web and portable and was ultimately determined by client input to make a full cross breed versatile creation device that would zero in on boosting execution>> ionic developers for hire

    ReplyDelete
  56. elza energy claims that it is one of the biggest green energy suppliers. Its energy comes from a variety of solar, wind, renewable sources, hydro, and geothermal facilities.

    ReplyDelete
  57. This article does a great job of outlining the various benefits of custom software development for businesses. It effectively highlights how tailored software solutions can boost efficiency, improve customer satisfaction, and provide a competitive edge. The clear and concise writing style makes it easy to understand the advantages of investing in custom software.

    ReplyDelete