CulebraTester2 ui preview 202003

Happy New Year

Happy New Year!

As of December 2018, AndroidViewClient/culebra reached 500,000 downloads from PyPi alone.
Thanks to all who made it possible.
One of the most important items in the roadmap for 2019 is the long-awaited migration to Python 3. Stay tuned and happy testing.

Testing Android voice apps automatically

Let’s review AndroidViewClient/culebra concertina mode features, compare them with monkey, and see how we can use those features to test voice based UIs or Alexa Skills.

Unlike monkey, which sends pseudo-random events, Culebra concertina mode analyzes the content of the screen and randomly selects a suitable event or action for the also randomly chosen target, usually a View.
Also, unlike monkey, the generated actions and their parameters are saved in a python script that can be later executed as many times as needed.

Read more at https://medium.com/@dtmilano

Testing Alexa Skills — Autogenerated tests

You almost finished your Amazon Alexa Skill and are now started the quest for the Holy Grail of Alexa Testing. Now, you are desperately searching for a way to automate it. Even, googling it gave no obvious outcome.
Fortunately, your search is over.

Now we will be analyzing how we can automate the code generation of such tests. Because some of the details needed to create the tests are available in the Skill’s Interaction Model, we will leverage this to reduce to a bare minimum the information that you have to provide to create a test.

Check Testing Alexa Skills — Autogenerated tests for more.

Testing Alexa Skills — The grail quest

You almost finished your Amazon Alexa Skill and are now started the quest for the Holy Grail of Alexa Testing. Now, you are desperately searching for a way to automate it. Even, googling it gave no obvious outcome.
Fortunately, your search is over.

Read the article on medium: https://medium.com/@dtmilano/testing-alexa-skills-the-grail-quest-3beba82450bb

Creating conversational AWS Lex Bot tests

There exist several ways of testing Amazon AWS Lex Bots, however, either they are manual or require low-level use of the PostContent method. We will introduce here a way of automating part of the process using python.
Read the article at creating-conversational-aws-lex-bot-tests

AndroidViewClient/culebra reached 300,000 downloads

AndroidViewClient/culebra reached 300.000 downloads this month. After some time flying blind, since PyPi statistic stopped working some time ago, this great surprise was awaiting when the counts were restored, now implemented using BigQuery.

Thanks to all who made it possible.

Analyzing Android UI Performance

Integrating UI Performance into your testing practices guarantees interaction with your applications satisfies users’ demands.

Read the step by step guide: https://medium.com/@dtmilano/analyzing-android-ui-performance-52beb577c421

Your first Android Kotlin test

The best part, you don’t have to know Kotlin as the test will be automatically generated for you.

Read the step by step guide: https://medium.com/@dtmilano/your-first-android-kotlin-test-fe007ec41bc5

CulebraTester: Android Kotlin test generation

Recently, Kotlin was made an official language on Android.

It's very easy to get started using Kotlin as it works side by side with Java and C++ on Android.
However, there are new syntax and new language features to learn, even more, when we talk about testing and creating tests for your application UI.
You can gradually introduce Kotlin code to your project as it can coexist with existing code and Android libraries.

So, what's better than start incrementally adding Kotlin to your workflow starting by the tests?
And what is even better is that CulebraTester v0.9.0 introduces Kotlin code generation as can be seen in the previous screenshot.
You can start learning Kotlin now by taking a look at the generated code.

As always, you can find more information about Open Source AndroidViewClient/culebra in its wiki at https://github.com/dtmilano/AndroidViewClient/wiki, about CulebraTester at http://culebra.dtmilano.com/ and https://github.com/dtmilano/CulebraTester-public/wiki and if you have any question you can ask in Stackoverflow using http://stackoverflow.com/questions/tagged/androidviewclient.

Testing Android UI Performance

In a previous post, we were analyzing how to Detect memory leaks on Android apps. That is a very important step in testing and analyzing your application's performance.
Once you are done, you may want to sure that user interactions with your app are smooth, and they run at a consistent 60 frames per second mark.

We will explain here how AndroidViewClient/culebra can help automating and simplifying some steps that usually require manual interaction. In such way, you can measure UI performance, and produce charts that will help you understand and locate the problems 

Latest AndroidViewClient/culebra versions have introduced the ability to plot some metrics from your device or emulator directly. The Dumpsys module obtains and parses dumpsys information to greatly facilitate its usage converting it to objects.

def __plot_dumpsys_gfxinfo(self, pkg):
    self.plot.append(Dumpsys(self.device,        Dumpsys.GFXINFO, pkg, Dumpsys.FRAMESTATS)) \
        .plot(_type=Dumpsys.FRAMESTATS)

This helper method (that can be found in AndroidViewClient/culebra tests) appends to the plot the framestats information obtained from dumpsys gfxinfo for the given package pkg. Then it creates the histogram chart.

This first chart shows the information obtained by running OPEN LIST VIEW from Automated Performance Testing after scrolling the lists.

 This second chart is then obtained by running OPEN RECYCLER VIEW from the same example.

The vertical lines indicate 60 and 30 FPS respectively.

In this case, the improvements made by the RecyclerView are obvious.

As always, you can find more information about AndroidViewClient/culebra in its wiki at https://github.com/dtmilano/AndroidViewClient/wiki, about CulebraTester at http://culebra.dtmilano.com/ and https://github.com/dtmilano/CulebraTester-public/wiki and if you have any question you can ask in Stackoverflow using http://stackoverflow.com/questions/tagged/androidviewclient.

 

Detecting memory leaks on Android apps

More often than it should you find yourself wondering if your application is leaking memory and if so what, where and how.
A lot of questions that usually don't have enough answers.
There are many well know ways your application can leak memory. To name a few

• static references to Contexts, Views or Activities
• inner classes holding a reference to the outer class
• anonymous classes, commonly used in listeners
• drawables holding a reference to the Context
• and many more cases

If you are interested in more detailed descriptions of these common ways your app can leak memory this article will help you.

Also, as there are pitfalls there are tools to help you find them. Again, to name a few

• Android Studio monitors
• dalvik and art GC log messages
• heap dumps
• dumpsys
• etc.

To do an effective use of these tools and uncover memory leaks you should stress your app, running it for a while, or forcing continuous invocations of Activities, perhaps navigation back to the Home screen and launching the Activity again and again. While you can do this manually and the most obvious cases will be detected, it is always more efficient to automate these steps so they can be run before and after to verify that you actually solved the problem.
And what better than AndroidViewClient/culebra to automate these steps as we have discussed so many times in previous articles. Precisely, one of the latest additions is the ability to capture dumpsys information and create plots with it.

This is an example of a method we can define to exercise our Activities including some extra steps like stopping the application to start clean and forcing garbage collection along the way (see complete source code here)

def __plot_dumpsys_meminfo(self, pkg, activity, method=None):
    self.device.shell("am force-stop %s" % pkg)
    for n in range(20):
        if n % 5 == 0:
            self.device.shell(
                "run-as %s pgrep -L 10 %s" % (pkg, pkg))
        self.device.startActivity("%s/%s" % (pkg, activity))
        time.sleep(2)
        if method:
            method()
        self.plot.append(Dumpsys(self.device, Dumpsys.MEMINFO, pkg))
        self.device.press('BACK')
        time.sleep(0.5)
        self.device.press('BACK')
        time.sleep(0.5)
        self.device.press('HOME')
        time.sleep(0.5)
    self.plot.plot()

With every iteration, we are collecting the dumpsys meminfo for the process, exiting the app by sending BACK and returning to Home, and at the end, we plot the chart.

We can easily see that while we are forcing GC the amount of memory used, the number of Activities and Views are constantly increasing.

As always, you can find more information about AndroidViewClient/culebra in its wiki at https://github.com/dtmilano/AndroidViewClient/wiki, about CulebraTester at http://culebra.dtmilano.com/ and https://github.com/dtmilano/CulebraTester-public/wiki and if you have any question you can ask in Stackoverflow using http://stackoverflow.com/questions/tagged/androidviewclient.

Hope this help you spot some leaks in your app.

CulebraTester Calculator Demo

Android testing can be complicated, time-consuming, and tedious. What if it didn’t have to be?

CulebraTester provides a real-time point and click test recording through a web browser.

We are currently in private beta, so if you are interested in participating of this programme please fill the opt-in form and we will be adding your account as soon as possible. 

Turns your Android smartphone into a complete laptop

I do not usually mention third party products in this blog unless there is a very good reason. The Superbook certainly is one of those.  It is a smart laptop shell that provides a large screen, keyboard and multi-touch trackpad, 8+ hours of battery, and phone charging capabilities. When plugged into your Android smartphone, it launches our app to deliver the full laptop experience.

The idea of using the same core device for your mobile and desktop experiences has been tried before, but this time, the Superbook gives it a twist and uses a full Android desktop. This is mainly possible due to the huge improvement lastest Android versions received in areas as multi-window support,  Android for work and Keyboard Shortcuts screen that surely move  the platform in the desktop direction.

That was not true some years ago. 6 years to be more precise, when one of the projects I was part of implemented the same concept but in the opposite direction I would say.

A mobile computing device with a mobile operating system and desktop operating system running concurrently and independently on a shared kernel without virtualization. The mobile operating system provides a user experience for the mobile computing device that suits the mobile environment. The desktop operating system provides a full desktop user experience when the mobile computing device is docked to a secondary terminal environment. The mobile computing device was a smartphone running the Android mobile OS and a full desktop Linux distribution (Ubuntu) on a modified Android kernel.

The concept included also a Smart book where you could dock your device.

Unfortunately, this project was later discontinued.

As I mentioned, a lot has changed in these last 6 years on Android and devices are much more powerful now to support this approach so take a look at Superbook as it might change your mobile and desktop experiences.

No Espresso Test Recorder in Android Studio 2.2

After all the frustration you may have experienced after discovering that Espresso Test Recorder is not in Android Studio 2.2 as announced in Google IO/16 presentation (see https://code.google.com/p/android/issues/detail?id=210478) you can have something to play with today. Sign up for CulebraTester private beta see opt-in form)

CulebraTester Private Beta Opt-in

Android testing can be complicated, time-consuming, and tedious. 
What if it didn’t have to be?

CulebraTester provides a real-time point and click test recording through a web browser. This browser is connected to the Android device under test. Not sure what we mean?
Visit culebra.dtmilano.com for details.

If you are interested in being part of the Private Beta please fill out this form.


 

AndroidViewClient/culebra vs. MonkeyRunner

More than 2 years ago I took a crucial decision in AndroidViewClient/culebra development plan and that was to free it from `monkeyrunner`, Jython and Chimpchat.

AndroidViewClient/culebra was liberated and starting with version 4.0.0 it does not require any other runtime environment than python 2.x (read announcement). It can be installed and upgraded using the corresponding platform tools like easy_install or pip and can be easily integrated into IDEs like Eclipse PyDev or Pycharm. It also improves speed, solves chimpchat bugs, and even provides a GUI whre you can automatically create tests or scripts without writing a single line of code.

Nonetheless, from time to time, I receive some questions or reports about problems with scripts created with `culebra` that are attempted to run with `monkeyrunner` or some other combinations. I take the blame for it. I failed at communicating that AndroidViewClient/culebra is a complete replacement and should not be used together.

In order to improve the situation I gave a very detailed, easy to follow, step-by-step answer to
Error of Script with MonkeyRunner and AndroidViewClient (Touch) on Stackoverflow, showing how you can create a test case that automatically starts.and Activity (Duolingo) , checks if some Views are on the screen, touches them and finally take the screenshot. All from the GUI.

I hope you find this explanation useful.

culebra: concertina mode

One of the latest additions to culebra is the concertina mode. Named after the concertina movement occurring in snakes and other legless organisms that consists of gripping or anchoring with portions of the body while pulling/pushing other sections in the direction of movement, this mode allows culebra to move across the UI.

You may have used android's monkey before. monkey is a UI/Application exerciser that generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. It is commonly used to stress-test application. While it may serve this purpose, it's overly complicated to be used in other scenarios.


Then, it comes culebra concertina mode that instead of sending pseudo-random events, analyzes the content of the screen and randomly selects a suitable event or action for the also randomly selected target, normally a View.

For example, if the randomly selected view is an EditText, culebra enters some random text. However if the EditText turns to be a password, it sends random passwords, that is instead of just entering a normal sentence it selected sample passwords from a list like the infamous "querty".
Furthermore, if it detects the EditText is expecting an email address, by inspecting the id, the hint or anything else that may suggest that, it sends random email addresses like user@example.com. Of course, other Views, receive other treatment, like Buttons being clicked, Scrollables being scrolled, etc.

Additionally, if the content description suggest the application may be awaiting for you to talk, like 'Tap to Speak' or 'Voice Search', culebra will actually speak random text to the app (this feature is only available on Linux and OSX).

You should check this aspect, it's sometimes hilarious.


Taking advantage of the culebra GUI already described in previous posts, while it's running the randomly selected Views are highlighted. At the same time, the python script is being generated. You can save it, modify it and run the same tests again and again. It's worth to mention that commonly the script eliminates the random selection of components and thus it runs deterministically.

Run culebra in concertina mode

$ culebra --gui --concertina --scale=0.5

and you can obtain something like this, where the nickname of a new contact was randomly selected from the available Views and the random text "chaos reigned" was entered.

BTW, this screenshot was taken via dump as described in android: obtaining beautiful screenshots automatically.

Google Cloud Test Lab

Yesterday, at Google I/O 2015, Cloud Test Lab was announced. One of its promoted features is: "you can run all of your tests across all devices, all at the same time--giving you access to massively parallel testing, bringing you deep and scaled insight".
Wow, that's pretty much the same I described in my previous post,

android: culebra multi-device capabilities, minus the devices of course. You have to provide them.

Wouldn't it be great if Cloud Test Lab allows you to create your personalized tests, in addition to the automatically generated ones, using something like culebra GUI, where you can just point and click?

android: culebra multi-device capabilities

culebra and its GUI both include multi-device capabilities in auto-generated test cases and scripts.

The test case or script is created as usual, but when the --multi-device option is present in the command line, the statements generated will be slightly different, still you can easily recognize them. These statements will include support for several devices using python's list comprehension.

For example, a typical self.vc.dump(window=-1) will become

 [vc.dump(window=-1) for vc in self.allVcs()]

that is, for all the ViewClient's dump the content of the default window. You may wonder where the list of all ViewClient's obtained by self.allVcs() comes from. CulebraTestCase class handles multiple devices in its setUp() method and it's there where the lists of all devices, serial numbers and ViewClient's are handled

self.devices.append(ConnectedDevice(serialno=serialno, device=device, vc=vc))

Everything is handled automatically, so there's nothing to worry about. If only one device is connected to adb or a single serial number is passed to the test case as an argument, then the test case behaves exactly as single-device. However, if more than one device is connected to adb and they are specified on the command line the multi-device capabilities kicks-in and the test is run on all devices at the same time.

The special command line keyword all can be used to mean all the connected devices.

This trivial test for Calculator generated using

$ culebra --gui --multi-device \
    --unit-test-class \
    --do-not-verify-screen-dump \
    -o ~/tmp/multi.py

shows multi-device capabilities.

#! /usr/bin/env python
# -*- coding: utf-8 -*-
'''
Copyright (C) 2013-2014  Diego Torres Milano
Created on 2015-05-19 by Culebra v10.5.1
                      __    __    __    __
                     /  \  /  \  /  \  /  \ 
____________________/  __\/  __\/  __\/  __\_____________________________
___________________/  /__/  /__/  /__/  /________________________________
                   | / \   / \   / \   / \   \___
                   |/   \_/   \_/   \_/   \    o \ 
                                           \_____/--<
@author: Diego Torres Milano
@author: Jennifer E. Swofford (ascii art snake)
'''


import re
import sys
import os


import unittest


try:
    sys.path.insert(0, os.path.join(os.environ['ANDROID_VIEW_CLIENT_HOME'], 'src'))
except:
    pass

from com.dtmilano.android.viewclient import ViewClient, CulebraTestCase

TAG = 'CULEBRA'


class CulebraTests(CulebraTestCase):

    @classmethod
    def setUpClass(cls):
        cls.kwargs1 = {'ignoreversioncheck': False, 'verbose': False, 'ignoresecuredevice': False}
        cls.kwargs2 = {'compresseddump': True, 'startviewserver': True, 'forceviewserveruse': False, 'autodump': False, 'ignoreuiautomatorkilled': True}
        cls.options = {'start-activity': None, 'device-art': None, 'use-jar': False, 'multi-device': True, 'unit-test-class': True, 'save-screenshot': None, 'use-dictionary': False, 'glare': False, 'dictionary-keys-from': 'id', 'scale': 1, 'find-views-with-content-description': True, 'window': -1, 'orientation-locked': None, 'save-view-screenshots': None, 'find-views-by-id': True, 'log-actions': False, 'use-regexps': False, 'null-back-end': False, 'auto-regexps': None, 'do-not-verify-screen-dump': True, 'verbose-comments': False, 'gui': True, 'find-views-with-text': True, 'prepend-to-sys-path': True, 'drop-shadow': False, 'output': '/Users/diego/tmp/multi2.py', 'unit-test-method': None, 'interactive': False}
        cls.sleep = 5

    def setUp(self):
        super(CulebraTests, self).setUp()

    def tearDown(self):
        super(CulebraTests, self).tearDown()

    def preconditions(self):
        if not super(CulebraTests, self).preconditions():
            return False
        return True

    def testSomething(self):
        if not self.preconditions():
            self.fail('Preconditions failed')

        _s = CulebraTests.sleep
        _v = CulebraTests.verbose

        [_vc.dump(window=-1) for _vc in self.allVcs()]
        [_vc.findViewWithTextOrRaise(u'2').touch() for _vc in self.allVcs()]
        [_vc.sleep(_s) for _vc in self.allVcs()]
        [_vc.dump(window=-1) for _vc in self.allVcs()]
        [_vc.findViewWithContentDescriptionOrRaise(u'''plus''').touch() for _vc in self.allVcs()]
        [_vc.sleep(_s) for _vc in self.allVcs()]
        [_vc.dump(window=-1) for _vc in self.allVcs()]
        [_vc.findViewWithTextOrRaise(u'1').touch() for _vc in self.allVcs()]
        [_vc.sleep(_s) for _vc in self.allVcs()]
        [_vc.dump(window=-1) for _vc in self.allVcs()]
        [_vc.findViewWithContentDescriptionOrRaise(u'''equals''').touch() for _vc in self.allVcs()]
        [_vc.sleep(_s) for _vc in self.allVcs()]
        [_vc.dump(window=-1) for _vc in self.allVcs()]


if __name__ == '__main__':
    CulebraTests.main()

Then, when you run the test with several devices connected as

$ ~/tmp/multi.py -s all

you will obtain something like this