ACTS telephony tests

The Android Comms Test Suite (ACTS) performs automated testing of connectivity stacks, such as Wi-Fi, Bluetooth, and cellular services. The testing tool requires adb and Python. The tool is located in tools/test/connectivity/acts.

This page describes how to run the basic telephony tests available in Android Open Source Project (AOSP) for a user with two phones.

Prerequisites

To run ACTS telephony tests, you should have access to a full Android repository and install userdebug builds on the devices that you want to test.

Download an Android repository

Follow the instructions in Download the source to download an Android repository on a machine that has a USB connection to the devices that you want to test.

  • AOSP is a multigigabyte codebase that continues to grow. Adjust your expectations of download times accordingly.
    • If a machine is size constrained, add --depth=1 to the repo init command.
  • To get future updates on ACTS tooling, go to the Android root directory:
    • Use repo sync tools/test/connectivity
    • And repo sync external/sl4a

Verify userdebug builds on devices

Install a userdebug version of Android on your test devices.

  • Build the repository using the instructions in Build Android.
  • Flash the device as described in Flash a device.
    • The command adb shell getprop ro.build.type should return userdebug.
    • If ro.build.type isn't available, a userdebug build allows running the adb shell as root (terminal #).

Build and install SL4A

To build and install Scripting Layer For Android (SL4A), follow the instructions in the Scripting Layer for Android. The section Adding SL4A Builds to Android Builds by Default is optional.

After building SLA4, the last log message contains the location of the sl4a.apk file, which you need to install on ALL devices. Here's an example of a log message:

Copy: out/target/path/to/sl4a.apk

Configure devices to test

Configure the devices using these ACTS build/tool requirements:

  • If ADB vendor keys are unavailable, tap the Trust this computer popup on the DUT when it appears. If ADB vendor keys are available:

    • ADB vendor keys and drivers must be unique to the DUT.
    • To always enable USB debugging, the ADB vendor keys in your source tree must be added to ~/.bashrc (or equivalent) using export ADB_VENDOR_KEYS=/path/to/directory/with/vendor/keys where /path/to/directory is your fully qualified path on the host. For troubleshooting ADB, refer to Run apps on a hardware device.

  • Allow USB debugging: When adb drivers are installed, adb devices may trigger a prompt on the device to allow USB debugging.

  • Install sl4a.apk using $ adb install -r <ANDROID_ROOT>/path/to/sl4a.apk

  • Make sure that the phones use a USB 2.0 connection to the test server.

  • These tests depend on live cell networks, so put SIM cards in the phones and keep the phones in an area that has good cell service.

  • Add all SIM cards to a JSON file as described in Write a config file.

Set up the environment

Before setting up the environment, verify that Python 3.4+ is installed.

Run:

sudo apt-get install python3-setuptools
sudo apt-get install python3-pip
sudo pip3 install --upgrade pip setuptools
sudo apt-get install protobuf-compiler

Install ACTS

Run:

cd <ANDROID_ROOT>/tools/test/connectivity/acts/framework/
sudo python3 setup.py develop
cd -

To verify the installation, run act.py -h, which prints a list of available commands.

Write a config file

To run tests, you must create a file with configuration information. Here's an example .config file. A similar file is available in the Android repository under tools/test/connectivity/acts/framework/sample_config.json. Replace XXXXXXX with device serial numbers to provide the minimum information that ACTS needs to run a telephony test.

{
  "_description": "Minimum Telephony Config",
  "testbed":
  [
      {
          "_description": "A testbed listing two AndroidDevices for adb.",
          "name": "ExampleTestBed",
          "AndroidDevice": [{"serial": "XXXXXXX", "adb_logcat_param": "-b all"},
                            {"serial": "XXXXXXX", "adb_logcat_param": "-b all"}]
      }
  ],
  "logpath": "/path/to/logs",
  "testpaths": "/path/to/android/repo/tools/test/connectivity",
  "sim_conf_file": "/path/to/simcard_list.json"
}

Use the keys in the table to configure the test parameters. Refer to Additional configuration options for more options.

Key Value
_description Comment on the contents of the entire config file
testbed Specification of an environment that tests can run in
(testbed) _description Comment on each testbed, as ACTS can target individual testbeds
(testbed) name Name of the testbed for calling from the command line
(testbed) AndroidDevice List of specific devices that the test targets
Note: There can be multiple devices listed. A test that runs phones calling each other expects there to be two devices and fails if there are fewer. If there are two devices listed and the test expects only one, that test is run on the first device.
(testbed)(AndroidDevice) serial Serial number of device (printed from adb devices)
(testbed)(AndroidDevice) adb_logcat_param Parameters that are passed when adb logcat is called during testing
logpath The location on the test server where the logs are saved
testpaths The location of test case code
sim_conf_file The location of the .json file that lists available SIM cards
The list of SIM cards includes the ID of the card, capabilities of the card, operator, and phone number. 
{
    "sim-card-ID-here": {
        "capability": [
            "voice",
            "ims",
            "volte",
            "vt",
            "sms",
            "tethering",
            "data"
        ],
        "operator": "tmo",
        "phone_num": "12345678901"
    },
...
}

You can pull the phone number from the device under System > About phone. The SIM card ID is generally printed on the SIM card, or is available from the device under System > About phone > SIM status. The operator string is defined at tools/test/connectivity/acts_tests/acts_contrib/test_utils/tel/tel_defines.py, then search for CARRIER_.

Additional configuration options

The entries.py file located at tools/test/connectivity/acts/framework/acts/config/entries.py in the Android repository has definitions for major command line configuration options.

You can also find test-specific parameters in the code by searching for self.user_params.get.

Run tests

Run tests from the command line after the config file is written. While there are many options, the simplest is using -c filename.config -tc TestCase:name. This example uses a config file named minimum.config:

act.py -c minimum.config -tc TelLiveVoiceTest:test_call_mo_voice_general

A successful test run should have a final output message like this:

INFO Summary for test run ExampleTestBed@yyyy-mm-dd_15-23-43-614: Requested 1,
Executed 1, Passed 1, Failed 0, Skipped 0, Error 0

Run multiple tests

To run specific test cases across multiple files, create a test file and pass it with the flag -tf , as shown in the example regression test suite:

TelLiveVoiceTest:
test_call_mo_voice_general,
test_call_volte_to_volte,
test_call_3g_to_3g

TelLiveDataTest:
test_airplane_mode,
test_4g,
test_3g

TelLiveSmsTest:
test_sms_mo_4g,
test_sms_mt_4g,
test_sms_mo_3g,
test_sms_mt_3g,
test_mms_mo_4g,
test_mms_mt_4g

This example command assumes that the file is named regression_suite.

act.py -c minimum.config -tf regression_suite

Run tests multiple times

To run a test case (-tc) or test file (-tf) multiple times, add -ti # (test iterations) to the command.