By Simran Basi, Dan Shi, Dan Willemsen, and Clay Murphy
Android Engineering Productivity (Android EngProd) seeks to ease development of the Android operating system for the entire ecosystem. Android EngProd creates tools, processes, and documentation aimed at Android platform development. We are now starting to push the best previously internal development infrastructure into the open for all to benefit.
Although comprehensive, the
Android Compatibility Test Suite (CTS) and
Trade Federation Test Harness can be unwieldy to configure. So we recently publicly released new tooling and associated docs that simplify device configuration and testing in the form of the
Soong build system replacing Make,
Test Mapping for easy configs, and
Atest to run tests locally.
Configuring tests in Soong builds
The
Soong build system was introduced in Android 8.0 (Oreo) to eventually replace the Make-based system (i.e.
Android.mk files) used in previous releases. Soong allows
simple build configuration with support for
android_test declarations arriving in Android Q, now available in the Android Open Source Project (AOSP) master branch.
Soong uses
Android.bp files, which are JSON-like simple declarative descriptions of modules to build. Here is an example test configuration in Soong, from:
/platform_testing/tests/example/instrumentation/Android.bpandroid_test {
name: "HelloWorldTests",
srcs: ["src/**/*.java"],
sdk_version: "current",
static_libs: ["android-support-test"],
certificate: "platform",
test_suites: ["device-tests"],
}
Note the
android_test declaration at the beginning indicates this is a test. Including
android_app instead would conversely indicate this is a build package.
Complex test configuration options still exist for test modules requiring customized setup and tear down that cannot be performed within the test case itself.
Mapping tests in the source tree
Test Mapping allows developers to create pre- and post-submit test rules directly in the Android source tree and leave the decisions of branches and devices to be tested to the test infrastructure itself. Test Mapping definitions are JSON files named
TEST_MAPPING that can be placed in any source directory.
Test Mapping categorizes tests via a
test group. The name of a test group can be any string. For example,
presubmit can be for a group of tests to run when validating changes. And
postsubmit tests can be used to validate the builds after changes are merged.
For the directory requiring test coverage, simply add a
TEST_MAPPING JSON file resembling the example below. These rules will ensure the tests run in presubmit checks when any files are touched in that directory or any of its subdirectories.
Here is a sample
TEST_MAPPING file:
{ "presubmit": [ { "name": "CtsAccessibilityServiceTestCases", "options": [ { "include-annotation": "android.platform.test.annotations.Presubmit" } ] } ], "postsubmit": [ { "name": "CtsWindowManagerDeviceTestCases" } ], "imports": [ { "path": "frameworks/base/services/core/java/com/android/server/am" } ]}Running tests locally with Atest
Atest is a command line tool that allows developers to build, install, and run Android tests locally, greatly speeding test re-runs without requiring knowledge of
Trade Federation Test Harness command line options.
Atest commands take the following form:
atest [optional-arguments] test-to-run
You can run one or more tests by separating test references with spaces, like so:
atest test-to-run-1 test-to-run-2
To run an entire test module, use its module name. Input the name as it appears in the
LOCAL_MODULE or
LOCAL_PACKAGE_NAME variables in that test's
Android.mk or
Android.bp file.
For example:
atest FrameworksServicesTests
atest CtsJankDeviceTestCasesDiscovering tests with Atest and TEST MAPPING
Atest and TEST MAPPING work together to solve the problem of test discovery, i.e. what tests need to be run when a directory of code is edited. For example, to execute all presubmit test rules for a given directory locally:
- Go to the directory containing the TEST_MAPPING file.
- Run the command: atest
All presubmit tests configured in the TEST_MAPPING files of the current directory and its parent directories are run. Atest will locate and run two tests for presubmit.
Finding more testing documentation
Further, introductory testing documents were published on
source.android.com to support Soong and platform testing in general:
In addition to exposing more testing documentation, Android has recently opened up build infrastructure to monitor submissions through
ci.android.com. See the
More visibility into the Android Open Source Project blog post and the
Continuous Integration Dashboard for instructions on viewing build status and downloading build artifacts.
Android EngProd endeavors to bring you more previously internal-only features to make your life easier. Watch this
Google Testing Blog, the
Android Developers Blog, and
source.android.com for future enhancements.