Compatibility Test Suite – CTS

What is CTS

From the official documentation:

The Android compatibility program works for the benefit of the entire Android community, including users, developers, and device manufacturers.

Each group depends on the others. Users want a wide selection of devices and great apps; great apps come from developers motivated by a large market for their apps with many devices in users’ hands; device manufacturers rely on a wide variety of great apps to increase their products’ value for consumers.

The CTS is an automated testing harness that includes two major software components:

1. The CTS test harness runs on your desktop machine and manages test execution.
2. Individual test cases are executed on attached mobile devices or on an emulator. The test cases are written in Java as JUnit tests and packaged as Android .apk files to run on the actual device target.

Setup CTS

Downloads

Download and unzip the latest CTS package from Google. You will want the following:

1. Compatibility Test Suite zip
2. CTS Verifier APK
3. Media 1.0

APK Install

From extracted zip go to android-cts/repository/testcases:

adb install CtsDeviceAdmin.apk

Device Preparation

In Developer Options:

1. Enable Stay Awake
2. Enable Android Debugging
3. Enable Allow Mock Locations
4. Enable Allow Mock SMS

In Security:

1. Set Screen Lock to None
2. Tap Device Administrators – check the two check boxes for CTS (requires CtsDeviceAdmin.apk from above):

android.deviceadmin.cts.CtsDeviceAdminReceiver

android.deviceadmin.cts.CtsDeviceAdminReceiver2

• Do NOT check the 3rd box for android.deviceadmin.cts.CtsDeviceAdminDeactivatedReceiver

Media Files:

In Android Media 1.0 zip run the following:

$ bash ./copy_media.sh all

• This will push all media files to the device

Additional tasks:

1. Device should have SIM card with data
2. Device should be connected to Wi-Fi
3. Install GMS packages (gapps)
4. For tests, Default the browser to Chrome (do an activity to launch the chooser, select Chrome and press Always)
   1. Go through Chrome’s brief setup
5. Do not log in to any accounts (Google, CM Account, etc)

Run CTS

In the extracted CTS zip, go to android-cts/tools and run the following command:

$ ./cts-tradefed

This opens the CTS command window. Here are useful commands:

• This runs a full CTS test pass

cts-tf> run cts --plan CTS

• To view all sessions in your /repository/results directory and number them:

cts-tf> l r

Re-run failures:

To re-run failures, create a plan based on results of a previous session:

cts-tf> add derivedplan --plan name_of_plan -s # -r fail

• the -s # is for session. Enter the number that corresponds with the session based on ‘l r’
• name_of_plan is a unique name for the plan you create – example n1_fails0411

Run a derived plan:

cts-tf> run cts --plan name_of_plan