python UIAutomator2 is a library that can use Python to automate the UI operation of Android devices. The bottom layer is based on Google UiAutomator (Java library). The principle of python UIAutomator2 is to use http rpc service on mobile phones to open the functions in UiAutomator, and then encapsulate these http interfaces into Python libraries. This article records how to use it.

uiautomator2 installation

The GitHub warehouse address of uiautomator2 is: https://github.com/openatx/uiautomator2

python installation:

pip3 install -U uiautomator2

Check whether the installation is successful

uiautomator2 --help

For viewing UI elements, you can use uiautomatorviewer or Appium inspector. Uiautomatorviewer is often not used, and Appium inspector is troublesome. It is recommended to submit a tool to WebEditor, which is simple to install and easy to use. You can view it directly on the browser. Please refer to this article for the installation and use of these three tools: App control positioning.

App initialization

Connecting equipment

d = u2.connect_usb(serial) # Serial: android device serial number, which is obtained through the adb devices command
d = u2.connect(serial) 
d = u2.connect("10.0.0.1") # Through the wireless connection device, 10.0.0.1 is the IP address of the mobile phone. It is necessary to ensure that the mobile phone and the computer can ping each other (connect to the same LAN).
d = u2.connect("10.0.0.1:7912") # adb connect 10.0.0.1:5555

Set newCommandTimeout and implicit wait

d.set_new_command_timeout(300)  # The maximum idle time of the accessibility service. The timeout will be automatically released
d.implicitly_wait(5)  # Implicit wait, element search wait time (default: 20s)

Install uninstall apk

d.app_install('http://some-domain.com/some.apk')
d.app_uninstall("package_name") # Uninstall APK

Open and stop App

Open APP

d.app_start(appPackage) # appPackage: package name, first parsing the mainActivity of the apk package through atx-agent, then calling am start -n $package/$activity to start.
d.app_start(appPackage, appActivity) # Specifying appPackage and appactivity actually means executing ADB shell am start - n appPackage / appActivity
d.app_start(appPackage, appActivity, wait = False, stop = False, use_monkey = False) # Wait: wait for the app to start. Stop: stop (appPackage and appActivity are required) use before starting the app_ Monkey: use the monkey command to start the app (this parameter can be used when no appActivity is specified)

Stop APP:

d.app_stop(appPackage) # Forcibly stop the application, which is equivalent to ADB shell am force stop < apppackage >
d.app_clear(appPackage) # Stop and clear the application data and cache, which is equivalent to ADB shell PM clear < apppackage >
d.app_stop_all() # Forcibly stop all third-party applications (except 'com.github.uiautomator' and 'com.github.uiautomator.test')
d.app_stop_all(excludes=['com.examples.demo']) # Stop except com github. Uiautomator and com github. uiautomator. Test and com examples. Applications other than demo

Get device application information

Get application information

>>> d.app_info("com.android.settings")
{u'packageName': u'com.android.settings', u'label': u'Param\xe8tres', u'mainActivity': u'com.android.settings.HWSettings', u'versionName': u'10.1.0.300', u'versionCode': 10010300, u'size': 132202855}

# Current application information
>>> d.app_current()

Get device information

>>> d.info
{u'displayRotation': 0, u'displaySizeDpY': 780, u'displaySizeDpX': 360, u'screenOn': True, u'currentPackageName': u'com.android.systemui', u'productName': u'HLK-AL10', u'displayWidth': 1080, u'sdkInt': 29, u'displayHeight': 2224, u'naturalOrientation': True}
>>>

Get detailed device information

>>> d.device_info
{u'product': None, u'udid': u'SNHVB20C18002195-d8:a4:91:4f:5c:1e-HLK-AL10', u'brand': u'HONOR', u'cpu': {u'hardware': u'Hisilicon Kirin810', u'cores': 8}, u'usingBeganAt': u'0001-01-01T00:00:00Z', u'provider': None, u'owner': None, u'display': {u'width': 1080, u'height': 2340}, u'battery': {u'status': 5, u'scale': 100, u'temperature': 340, u'level': 100, u'acPowered': False, u'usbPowered': True, u'health': 2, u'voltage': 4355, u'wirelessPowered': False, u'technology': u'Li-poly', u'present': True}, u'version': u'10', u'presenceChangedAt': u'0001-01-01T00:00:00Z', u'agentVersion': u'0.10.0', u'memory': {u'total': 5810780, u'around': u'6 GB'}, u'hwaddr': u'd8:a4:91:4f:5c:1e', u'model': u'HLK-AL10', u'arch': u'', u'serial': u'SNHVB20C18002195', u'sdk': 29}

Get resolution:

>>> d.window_size()
(1080, 2340)

Get device serial number

>>> d.serial
u'SNHVB20C18002195'

Get mobile IP address

>>> d.wlan_ip
u'192.168.0.191'

other

List all running apps:

d.app_list_running() # adb shell pm list packages

Open web page

d.open_url("https://www.baidu.com") # adb shell am start -a android.intent.action.VIEW -d https://www.baidu.com 

Wait for appActivity to appear

>>> d.wait_activity(".HWSettings", timeout=10)
True

UI element positioning

Basic selector

The following parameters are supported through attribute value positioning:

• text, textContains, textMatches, textStartsWith
• className, classNameMatches
• description, descriptionContains, descriptionMatches, descriptionStartsWith
• checkable, checked, clickable, longClickable
• scrollable, enabled,focusable, focused, selected
• packageName, packageNameMatches
• resourceId, resourceIdMatches
• index, instance

d(className="android.widget.TextView", text="quotation")
d(className="android.widget.TextView", textMatches="^that 's ok.*")

Relative selector

Descendant node location

d(resourceId="android:id/tabs").child(text="quotation")
d(resourceId="android:id/tabs").child_by_text("quotation")
d(resourceId="android:id/tabs").child_by_description("description")
d(resourceId="android:id/tabs").child_by_instance("instance")

Sibling node location

d(className="android.widget.ImageView").sibling(text="quotation")

Relative positioning

• d(A).left(B): left element of a
• d(A).right(B): right element B of a
• d(A).up(B): upper element B of a
• d(A).down(B): the lower element B of a

d(text="snowball").right(text="quotation")
d(text="transaction").left(text="quotation")

d(resourceId="com.xueqiu.android:id/tab_name", instance=1)

multiple instances

View and select instances

print(d(resourceId="com.xueqiu.android:id/tab_name").count)
print(len(d(resourceId="com.xueqiu.android:id/tab_name")))
d(resourceId="com.xueqiu.android:id/tab_name")[1].click() # Click the second element that matches

Execution results:

4
4

You can also use the instance parameter to select:

d(resourceId="com.xueqiu.android:id/tab_name", instance=1) # Use the second element that matches

XPath positioning

Java uiautomator does not support xpath by default. xpath positioning is a function of UIAutomator2 extension.

d.xpath('//*[@ text = "quotation"]) wait(10.0).click()
d.xpath('//*[@ text = "quotation"]) click()
d.xpath('//*[@ text = "quotation"]) exists
d.xpath('//*[@ text = "quotation"]) all()

xpath syntax is available for reference Web automated testing: XPath & CSS selector positioning

Element operation method

click

Click on the UI element

ele = d(text="WeChat")
ele.click(timeout=None, offset=None) # timeout: the unit is second, waiting for the element to appear; offset: the default is center (0.5, 0.5)
ele.long_click(duration = 0.5, timeout=None) # duration: click time; timeout: wait for the element to appear

ele.click_exists(timeout=10.0)
ele.click_gone(timeout=10.0, interval=1.0) # 

Click pixel coordinates

d.click(x,y) # Click pixel coordinates
d.double_click(x, y, duration=0.1) # Double click the pixel coordinates
d.long_click(x, y, duration=0.5) # Long press

Text input

Text value acquisition, input and clearing

d(text="quotation").get_text() # Get element text

d(resourceId="com.xueqiu.android:id/action_search").click() # Click search
d(resourceId="com.xueqiu.android:id/search_input_text").set_text("China Merchants Bank")  # Enter text
d(resourceId="com.xueqiu.android:id/search_input_text").clear_text()  # Clear text entry box

wait

Wait element

d(text="Settings").wait(timeout=3.0) # Wait for the element to appear
d(text="Settings").wait_gone(timeout=1.0) # Wait for the element to disappear

WatchContext

with d.watch_context() as ctx:
    ctx.when("^immediately(download|to update)").when("cancel").click() # Click Cancel when both (install now or cancel now) and Cancel buttons appear at the same time
    ctx.when("agree").click()
    ctx.when("determine").click()
    # The above three lines of code are executed immediately without any waiting
    
    ctx.wait_stable() # Open pop-up monitoring and wait for the interface to stabilize (no pop-up in two pop-up inspection cycles represents stability)

    # Use the call function to trigger a function callback
    # call supports two parameters, d and el. It does not distinguish between parameter positions. You can not pass parameters. If you pass parameter variable names, you can't write them wrong
    # eg: when there are elements matching midsummer night, click the back button
    ctx.when("summer night ").call(lambda d: d.press("back"))
    ctx.when("determine").call(lambda el: el.click())

Toast operation

The phone page displays toast

d.toast.show("Hello world")
d.toast.show("Hello world", 1.0) # Display 1s

Get toast

d.toast.get_message(wait_timeout=5.0, cache_timeout=10.0, "default message") 
assert "Hello world" in d.toast.get_message(5.0, default="") # Assert toast information
d.toast.reset() # Clear toast cache

Sliding swipe

Slide according to pixel coordinates

d.swipe(fx, fy, tx, ty, duration = None, steps = None) # Slide from (fx, fy) to (tx, ty). 1 step is about 5ms. If steps is set, the duration parameter will be ignored

UI object based sliding

ele = d(text="WeChat")
ele.swipe(direction, steps=10) # Slide from the center of the UI element. The direction includes "left", "right", "up" and "down".

Drag drag_to

d(text="Settings").drag_to(x, y, duration=0.5) # Drag from the Settings UI object to the (x,y) position
d(text="Settings").drag_to(text="Clock UI", duration=0.2) # Drag the Settings UI object to the Clock UI object for 200ms

Gesture operation

Gesture zoom in and out

d(text="Settings").pinch_in(percent=100, steps=10) # narrow
d(text="Settings").pinch_out() # enlarge

UI element status and information

Determine whether UI elements exist

d(text="quotation").exists
d.exists(text="quotation")
d(text="quotation").exists(timeout=3)

Get element information

$ d(text="quotation").info
{'bounds': {'bottom': 1274, 'left': 255, 'right': 285, 'top': 1253}, 'childCount': 0, 'className': 'android.widget.TextView', 'contentDescription': None, 'packageName': 'com.xueqiu.android', 'resourceName': 'com.xueqiu.android:id/tab_name', 'text': 'quotation', 'visibleBounds': {'bottom': 1274, 'left': 255, 'right': 285, 'top': 1253}, 'checkable': False, 'checked': False, 'clickable': False, 'enabled': True, 'focusable': False, 'focused': False, 'longClickable': False, 'scrollable': False, 'selected': True}

Get element coordinates

x, y = self.d(text="quotation").center() # Element center point
x, y = self.d(text="quotation").center(offset=(0, 0)) # Coordinate of upper left point of element

screenshot

Intercepting UI objects

im = d(text="quotation").screenshot()
im.save("quotation.jpg")

Device screenshot

d.screenshot("saved.jpg")
d.screenshot().save("saved.png")
cv2.imwrite('saved.jpg', d.screenshot(format='opencv'))

Command line operation

Gets the current package name and activity of the specified device

$ python3 -m uiautomator2 --serial SNHVB20C18002195 current
{
    "package": "com.android.settings",
    "activity": ".HWSettings",
    "pid": 11040
}

# Python 3 -m uiautomator2 can be abbreviated to uiautomator2

$ uiautomator2 --serial SNHVB20C18002195 current
{
    "package": "com.android.settings",
    "activity": ".HWSettings",
    "pid": 11040
}

Screenshot: screenshot

$ uiautomator2 screenshot screenshot.jpg

Uninstall: uninstall

$ uiautomator2 uninstall <package-name> # Uninstall a package
$ uiautomator2 uninstall <package-name-1> <package-name-2> # Uninstall multiple packages
$ uiautomator2 uninstall --all # Uninstall all

Stop: stop application

$ uiautomator2 stop com.example.app # Stop an app
$ uiautomator2 stop --all # Stop all app s

pytest + Uiautomator2 instance

Test steps:

1. Open snowball app
2. Enter the market page
3. Click search
4. Enter "China Merchants Bank"
5. Click stock code 03968
6. Assert stock price

Python code:

#!/usr/bin/python3
# -*-coding:utf-8-*-

import uiautomator2 as u2

class TestU2():
    def setup(self):
        self._device = '127.0.0.1:7555'
        self._appPackage = 'com.xueqiu.android'
        self._appActivity = '.common.MainActivity'

        self.d = u2.connect_usb(self._device)
        self.d.set_new_command_timeout(300)
        self.d.app_start(self._appPackage, self._appActivity)

    def teardown(self):
        # self.d.app_stop(self._appPackage)
        pass

    def test_uiautomator2(self):
        # Click Market
        self.d(className="android.widget.TextView", text="quotation").click()
        search_ele = self.d(resourceId="com.xueqiu.android:id/action_search").wait(timeout=3.0)
        assert search_ele == True
        
        # Click search
        self.d(resourceId="com.xueqiu.android:id/action_search").click()
        
        # input
         self.d(resourceId="com.xueqiu.android:id/search_input_text").set_text("China Merchants Bank")  # set the text

		# Click search results
        self.d.xpath('//*[@text="03968"]').wait(3).click()
        
        # Read stock price
        wait_price = self.d(resourceId="com.xueqiu.android:id/current_price")[0].wait(timeout=3.0)
        if wait_price:
            current_price = self.d(resourceId="com.xueqiu.android:id/current_price")[0].get_text()
            assert float(current_price) < 60
        else:
            assert False

--THE END--