Tag Archives: NDK

Native Dependencies in Android Studio 4.0

By Dan Albert, Software Engineer

One thing that NDK users struggle with is managing native dependencies:

  • Library authors need to maintain support for both ndk-build and CMake (and hope that their users are using one of those two options and not something else).
  • Libraries don’t always distribute prebuilt binaries for Android, so users must either build the library themselves or rely on (potentially untrustworthy) prebuilt binaries found elsewhere on the web.
  • Android-specific build scripts are often out of date and no longer work.
  • Libraries are sometimes built by a build system that Android doesn’t support.
  • Libraries may not build on the user’s machine. For example, Unix shell scripts won’t run on Windows.
  • Libraries often depend on other libraries, leaving users to chase them down and start the process again.

With version 4.0 of the Android Gradle Plugin, we’ve addressed these issues by adding support for distributing and exposing native libraries through the same mechanism that you do for Java libraries: Android Archives (AARs).

Here’s how you’d use curl and jsoncpp for example (and automatically pull in the implicit OpenSSL dependency that curl has):

// build.gradle
dependencies {
    implementation 'com.android.ndk.thirdparty:curl:7.68.0-alpha-1'
    implementation 'com.android.ndk.thirdparty:jsoncpp:1.8.4-alpha-1'
}

Note: With AGP 4.0 this is still experimental, so to enable this functionality you must set the following properties in your project's gradle.properties file:

# Enables Prefab
android.enablePrefab=true
# Work around https://issuetracker.google.com/149575364
android.enableParallelJsonGen=false
# 4.0.0 canary 9 defaults to Prefab 1.0.0-alpha3, which is not the latest.
android.prefabVersion=1.0.0-alpha5

Importing packages into your build

Declaring the dependencies in your build.gradle will cause Gradle to download those dependencies from Maven, but you must still instruct CMake or ndk-build how those dependencies should be used. Fortunately, the necessary CMake package config or ndk-build module will be automatically generated on your behalf. All you need to do is import and use them.

Here’s an example with CMake:

    cmake_minimum_required(VERSION 3.6)
    project(app VERSION 1.0.0 LANGUAGES CXX)

    find_package(curl REQUIRED CONFIG)
    find_package(jsoncpp REQUIRED CONFIG)

    add_library(app SHARED app.cpp)
    target_link_libraries(app curl::curl jsoncpp::jsoncpp)

And here’s the same example with ndk-build:

    LOCAL_PATH := $(call my-dir)

    include $(CLEAR_VARS)
    LOCAL_MODULE := libapp
    LOCAL_SRC_FILES := app.cpp
    LOCAL_SHARED_LIBRARIES := jsoncpp curl
    include $(BUILD_SHARED_LIBRARY)

    $(call import-module,prefab/curl)
    $(call import-module,prefab/jsoncpp)

And that’s it. In app.cpp you can now do the following:

#include "curl/curl.h"
#include "json/json.h"

A very common issue that people have is building OpenSSL to use with curl. While not explicitly mentioned in the build scripts above, the curl package itself depends on OpenSSL so this support is available automatically.

For the complete example, see the curl-ssl sample.

Prefab

The tool that facilitates all of this is called Prefab. Each AAR that exposes C++ libraries to its consumers packages their libraries, headers, and a small amount of metadata into the prefab directory in the AAR. If the prefab directory is found in an AAR dependency, the Android Gradle Plugin automatically runs Prefab to generate build system scripts from the contained information.

Each AAR might contain a large number of prebuilts for different configurations, so Prefab will perform a number of compatibility checks to find a suitable library for your build configuration. The selected library will match your build’s ABI, minSdkVersion, STL choice, and be the best fit for the version of the NDK that you’re using.

What libraries are available?

We’ve already published the following libraries:

  • com.android.ndk.thirdparty:curl:7.68.0-alpha-1
  • com.android.ndk.thirdparty:jsoncpp:1.8.4-alpha-1
  • com.android.ndk.thirdparty:openssl:1.1.1d-alpha-1
  • com.google.oboe:oboe:1.3.0

For an up to date list, search https://maven.google.com/web/index.html for “com.android.ndk.thirdparty”.

How can I distribute my own libraries?

For the libraries we currently distribute, we wrote ndkports. This is a good fit if the library you’re building is a typical Linux or cross-platform project that doesn’t fit naturally into a typical Android build. If that’s a fit for the library you want, feel free to use ndkports for it, and consider sending us the patch!

If you’d like to request that Google maintain and publish an open source library in Prefab, use the “Package request” bug template on https://github.com/google/prefab/issues. Please keep in mind that each package does come with an ongoing cost, and we will be adding support on a limited basis so we will not be able to support everything.

Coming next is support for exposing your libraries in AARs using the existing Android Library publishing workflow.

Links

For more information about using native dependencies with the Android Gradle Plugin, see the documentation. For more examples, see the NDK samples.

If you’d like to learn more about Prefab itself, see its documentation on GitHub.

If you encounter any issues, file bugs in our Issue Tracker.

Android Game SDK

Posted by Dan Galpin, Developer Advocate

With over 2.5 billion monthly active devices, the Android Platform gives incredible reach for game developers. Taking advantage of that opportunity can be a challenge, particularly if your game really tries to push the limits of what mobile can do. We've spent years working with game developers to try to both capture and address the biggest issues, and we're just beginning to see the fruits of that effort with the launch of the Android Game SDK. The Android Game SDK is a set of libraries that you can use to enhance your Android game.

The first library we are launching in the Android Game SDK helps developers with frame pacing, the synchronization of a game's rendering loop with the OS display subsystem and underlying display hardware. Android's display subsystem is designed to avoid tearing that occurs when the display hardware switches to a new frame in the middle of an update. To this end, it buffers past frames, detects late frame submissions, and repeats the display of past frames when late frames are detected. When a game render loop renders at a different rate than the native display hardware, such as a game running at 30 frames-per-second attempting to render on a device that natively supports 60 FPS, the optimal display flow involves synchronization between the game render loop, the system compositor, and the display hardware.

Optimal Display Flow

Optimal Display Flow

Any mismatch in synchronization can create substantial inconsistencies in frame times. If a frame takes substantially less time to render, it can shorten the presentation of the previous frame, causing something like a 33ms, 16ms, and a 50ms sequence.

Synchronization Mismatch: Rendering too Fast

Synchronization Mismatch: Rendering too Fast

If a frame takes too long to render, a similar problem occurs. The frame will be presented for an extra frame, causing something like a 50ms, 16ms, and 33ms sequence.

Synchronization Mismatch: Slow Frame

Synchronization Mismatch: Slow Frame

In either of these two scenarios, the game player will experience inconsistent delays between game input and screen updates. Visually, things will look less smooth and polished. Both visuals and game play can be impacted.

The Frame Pacing library uses Android's Choreographer API for synchronization with the display subsystem, using presentation timestamp extensions on both OpenGL and Vulkan APIs to make sure frames are presented at the proper time, and sync fences to avoid buffer stuffing. Multiple refresh rates are handled if supported by the device, giving a game more flexibility in presenting a frame. For a device that supports a 60 Hz refresh rate as well as 90 Hz, a game that cannot produce 60 frames per second can drop to 45 FPS instead of 30 FPS to remain smooth. The library detects the expected game frame rate and auto-adjusts frame presentation times accordingly. The Frame Pacing library allows games to take advantage of higher refresh rate 90 and 120 Hz displays, while also making it easy to lock the refresh rate to a desired value, regardless of the underlying display refresh rate.

The Frame Pacing library is built into Unity versions 2019.2 and beyond. Just select the optimized Frame Pacing checkbox under Android Settings to enable smoother frame rates for your game. If you have source to your game engine, it's straightforward to integrate the library into your OpenGL or Vulkan renderer. We've just added library binaries for download at developer.android.com/games/sdk/, or you can download the source code from the Android Open Source Project.

To learn more about Frame Pacing, check out the documentation at developer.android.com, along with the Frame Pacing section of the Optimizing Android Games Performance talk from Google I/O 2019. Be sure to subscribe to our Twitter channel and stay tuned for our announcements at GDC 2020 for more on how we're working to make Android game development better, so you can bring the best game experience to billions of devices.

Introducing NDK r21: our first Long Term Support release

Posted by Dan Albert, Android NDK Tech Lead

Android NDK r21 is now in beta! It’s been a longer than usual development cycle (four months since NDK r20), so there’s quite a lot to discuss for this release.

We have the usual toolchain updates, improved defaults for better security and performance, and are making changes to our release process to better accommodate users that need stability without hindering those that want new features.

New Minimum System Requirements

This release comes with new minimum system requirements. Following Android Studio and the SDK, 32-bit Windows is no longer supported. While this change will not affect most developers, this change does have an impact if you use 32-bit versions of Microsoft® Windows®. Linux users must have glibc 2.17 or newer.

Release Cycle Changes and LTS

One release a year will be our Long Term Support (LTS) release for users that want stability more than they need new features. The release will undergo a longer beta cycle before being released, and will receive bug fixes as backports until next year’s LTS release. Generally releasing in Q4, our first LTS release will be NDK r21.

The non-LTS releases each year, which we call the “rolling” release, will be similar to our current process. These will be approximately quarterly releases of our latest set of features, which will only be patched later for critical toolchain fixes. If you want the latest features from Clang and libc++, this is the release for you.

More detail, including the criteria we will use to determine what will be backported, what kinds of bugs will trigger a point release, and the bar we hold each release to can be found documented on our GitHub Wiki.

New Features and Updates

There are quite a lot of new things in this release, resolving bugs and helping you write better, safer code.

We’ve updated GNU Make to version 4.2, which enables --output-sync to avoid interleaving output with error messages. This is enabled by default with ndk-build. This also includes a number of bug fixes, including fixing the pesky CreateProcess errors on Windows.

GDB has been updated to version 8.3, which includes fixes for debugging modern Intel CPUs.

Updated LLVM

As always, we’ve updated LLVM and all of its components (Clang, lld, libc++, etc) which includes many improvements.

The toolchain has been updated to r365631 (the master branch as of 10 July 2019). This includes fixes for quite a few bugs in the previous release, perhaps most importantly LLD no longer hangs when using multithreaded linking on Windows. OpenMP is now available as a dynamic library (and this is the new default behavior, so link with -static-openmp if you want to stick with the static runtime).

A handful of driver improvements have been made to reduce the amount of compiler configuration required by each build system as well. Build system owners should check the updated Build System Maintainers guide.

libc++ has been updated to r369764.

Fortify

Fortify is now enabled by default when using ndk-build or the CMake toolchain file (this includes ExternalNativeBuild users). Fortify enables additional checks in the standard library that can help catch bugs sooner and mitigate security issues. For example, without fortify the following code compiles fine:

    const char src[] = "this string is too long";
    char dst[10];
    strcpy(dst, src);

With fortify, the buffer overflow is diagnosed at compile-time:

    test.cpp:10:18: error: 'strcpy' called with string bigger than buffer
      strcpy(dst, src);
                     ^

It is not always possible for the compiler to detect this issue at compile-time. In those cases, a run-time check will be used instead that will cause the program to abort rather than continue unsafely.

If you’re using a build system other than ndk-build or CMake via the NDK’s toolchain file, this will not be enabled by default. To enable, simply define _FORTIFY_SOURCE=2. The most reliable way to do this is by adding -D_FORTIFY_SOURCE=2 to your compiler flags.

Clang can also statically detect some of these issues by using -Wfortify-source (also new in r21). This is on by default, but it’s recommended to enforce fixing issues with -Werror=fortify-source. Use this in addition to the C library features, not instead of, since the warnings do not cover the same cases as the C library extension, nor can it add run-time checks.

Note that because run-time support is required for fortify and the feature was gradually added to Android over time, the exact set of APIs protected by fortify depends on your minSdkVersion. Fortify is an improvement, but it is not a replacement for good tests, ASan, and writing safe code.

See FORTIFY in Android for an in-depth explanation of fortify.

Other updates

64-bit requirement

Since August 2019, all existing and new apps are now required to support 64-bit before they can be released to Google Play; there’s an extension for a limited set of apps. For more information and help on adding support for 64-bit, see our guide.

Neon by default

Arm code is now built with Neon by default. In a previous release we enabled it conditionally based on minSdkVersion, but given the very small number of devices that don’t support Neon we now enable it unconditionally. This offers improved performance on all 32-bit Arm devices (64-bit Arm always had this, and it does not affect Intel ABIs).

As before, this behavior can be disabled for apps that need to continue supporting devices without Neon. Alternatively, those devices can be blacklisted in the Play Console. See https://developer.android.com/ndk/guides/cpu-arm-neon for more information.

Up Next

Have a look at our roadmap to see what we’re working on next. The next few big things coming up are package management and better CMake integration.

Introducing Android Native Development Kit r16

Posted by Dan Albert, Android NDK Tech Lead

The latest version of the Android Native Development Kit (NDK), Android NDK r16 Beta 1, is now available for download. It is also available in the SDK manager via Android Studio.

NDK r16 is a big milestone for us, because it's the first release that we're ready to recommend that people start migrating to libc++! More on this later.

We've also updated libc++ and its related projects, so this release has improved support for C++1z. Keep in mind that until C++1z becomes C++17, everything included is subject to change.

You can find the release notes for this release here.

libc++ and libandroid_support

The NDK has a library called libandroid_support that backports libc APIs that libc++ depends on that weren't available on older releases. The reason we've been unable to endorse libc++ (as implemented in the NDK) until now has been a lack of confidence in this library. The focus of r16 was to rewrite this library for improved stability.

Since libandroid_support is now a smaller library, your app's behavior should more closely match the behavior of the system. As an example, libandroid_support previously included an alternative implementation of part of stdio. While some features got backported to ICS, it also meant that any bugs in the alternate implementation would be present on all OS releases since the bug was baked into your app. In the new version of libandroid_support, we've removed this so you'll be missing some features on older devices (almost exclusively things that no one uses, like %a support in format strings), but your apps using libc++ will be smaller and more reliable for not having these features.

Switching to libc++

So, why should you switch to libc++? First and foremost, the other STLs will not be supported going forward (this has been noted in our roadmap for quite some time). We've been using libc++ for the Android platform since Lollipop, and that's been a change that our engineers have been overwhelmingly happy with. We were able to make this transition in the platform earlier than we could in the NDK because we didn't need libandroid_support, and could instead just update libc in place.

In contrast to the other STLs currently available in the NDK, libc++ fully supports C++11, C++14, and most of C++1z! Stlport hasn't had an update since 2008, and gnustl (what we call GNU's libstdc++, to avoid confusion with Bionic's libstdc++, which isn't an STL) historically hasn't worked very well with Clang, particularly in headers that are closely tied to compiler builtins like and .

We'll most likely be making libc++ the default in the next NDK release, but for now you can opt-in if you're not using it already by following the instructions below.

Like the other STLs, libc++ is available as both a static and shared library. Which one you should use depends on your specific circumstances as described in our docs, but tl;dr use the static version if you have one and only one shared library in your application, and use the shared one in all other cases.

ndk-build

Add the following to your Application.mk file:

APP_STL := c++_shared
CMake

Pass the following when invoking CMake:

-DANDROID_STL=c++_shared

If you're using CMake via Gradle, add the following to your build.gradle:

externalNativeBuild {
    cmake {
        arguments "-DANDROID_STL=c++_shared"
    }
}
Standalone Toolchain

When you create your standalone toolchain, pass --stl=libc++.

The Future of libandroid_support

If you've read our roadmap, you've seen that we've planned to expand libandroid_support to backport as much of libc/libm as possible. Whenever we've spoken with people about this, we've received lukewarm responses at best. Given that this doesn't seem to be a thing that people are interested in, and that it would be something that increases library size (and therefore APK size, which is something everyone seems very interested in), we no longer plan to do this.

If we've misinterpreted your response or if we haven't heard from you and this is something you want, please let us know!

_FILE_OFFSET_BITS=64

tl;dr: Don't set _FILE_OFFSET_BITS=64 if you want to keep the behavior present in old NDKs.

Historically, setting _FILE_OFFSET_BITS=64 in the NDK did nothing. This feature was not present in the deprecated headers at all. With unified headers, the NDK now has up to date headers with support for this feature.

_FILE_OFFSET_BITS=64 is a macro you can define in your application to get support for a 64-bit off_t in 32-bit code. This works by both making off_t 64-bit (by default it is 32-bit in 32-bit code) and by implicitly replacing calls to APIs like lseek with calls to lseek64.

Support for _FILE_OFFSET_BITS=64 was not added to Android in a single release. One API, lseek64, has always been in bionic. Most APIs were added in Lollipop, and a few more were not added until later releases.

If you're targeting a release that does not support the 64-bit off_t variant of a function you are using and have set _FILE_OFFSET_BITS=64, the function will not be available. This is in contrast to the behavior for r15 and r15b (but matches r15c) where the functions were wrongly exposed with a 32-bit off_t that would be silently truncated.

Note that the 64-bit off_t APIs are still available without _FILE_OFFSET_BITS=64 under different names. For example, instead of lseek, call lseek64. Instead of off_t, use off64_t.

Finally, since this feature is new to the NDK with unified headers, if you just want to return to the pre-unified headers behavior, all you need to do is stop setting _FILE_OFFSET_BITS=64.

For more information about off_t ABI details in bionic, see the Bionic 32-bit ABI bugs doc.

Introducing Android Native Development Kit r14

Posted by Dan Albert, Android NDK Tech Lead

Android NDK r14


The latest version of the Android Native Development Kit (NDK), Android NDK r14, is now available for download. It is also available in the SDK manager via Android Studio.

So what's new in r14? The full changelog can be seen here, but the highlights include the following:
  • Updated all the platform headers to unified headers (covered in detail below)
  • LTO with Clang now works on Darwin and Linux
  • libc++ has been updated. You can now use thread_local for statics with non-trivial destructors (Clang only)
  • RenderScript is back!

Unified Headers

We've completely redone how we ship platform header files in the NDK. Rather than having one set of headers for every target API level, there's now a single set of headers. The availability of APIs for each Android platform is guarded in these headers by #if __ANDROID_API__ >= __ANDROID_API_FOO__ preprocessor directives.

The prior approach relied on periodically-captured snapshots of the platform headers. This meant that any time we fixed a header-only bug, the fix was only available in the latest version aside from the occasional backport. Now bugfixes are available regardless of your NDK API level.

Aside from bugfixes, this also means you'll have access to modern Linux UAPI headers at every target version. This will mostly be important for people porting existing Linux code (especially low-level things). Something important to keep in mind: just because you have the headers doesn't mean you're running on a device with a kernel new enough to support every syscall. As always with syscalls, ENOSYS is a possibility.

Beyond the Linux headers, you'll also have modern headers for OpenGL, OpenSLES, etc. This should make it easier to conditionally use new APIs when you have an older target API level. The GLES3 headers are now accessible on Ice Cream Sandwich even though that library wasn't available until KitKat. You will still need to use all the API calls via dlopen/dlsym, but you'll at least have access to all the constants and #defines that you would need for invoking those functions.
Note that we'll be removing the old headers from the NDK with r16, so the sooner you file bugs, the smoother the transition will go.

Caveats

The API #ifdef guards do not exist in third-party headers like those found in OpenGL. In those cases you'll receive a link time error (undefined reference) rather than a compile time error if you use an API that is not available in your targeted API level.

Standalone toolchains using GCC are not supported out of the box (nor will they be). To use GCC, pass -D__ANDROID_API__=$API when compiling.

Enabling Unified Headers in Your Build

To ease the transition from the legacy headers to the unified headers, we haven't enabled the new headers by default, though we'll be doing this in r15. How you opt-in to unified headers will depend on your build system.

ndk-build


In your Application.mk:

    APP_UNIFIED_HEADERS := true
You can also set this property from the command-line like this:

    $ ndk-build APP_UNIFIED_HEADERS=true

If you're using ndk-build via Gradle with externalNativeBuild, specify the following configuration settings in build.gradle:

    android {
      ...
      defaultConfig {
        ...
        externalNativeBuild {
          ndkBuild {
            ...
            arguments "APP_UNIFIED_HEADERS=true"
          }
        }
      }
    }

CMake

When configuring your build, set ANDROID_UNIFIED_HEADERS=ON. This will usually take the form of invoking CMake with cmake -DANDROID_UNIFIED_HEADERS=ON $OTHER_ARGS.

If you're using CMake via Gradle with externalNativeBuild, you can use:

    android {
      ...
      defaultConfig {
        ...
        externalNativeBuild {
          cmake {
            ...
            arguments "-DANDROID_UNIFIED_HEADERS=ON"
          }
        }
      }
    }

Standalone Toolchains

When creating your standalone toolchain, pass --unified-headers. Note that this option is not currently available in the legacy script, make-standalone-toolchain.sh, but only in make_standalone_toolchain.py.

Experimental Gradle Plugin

Coming soon! Follow along here.

Custom Build System?

We've got you covered. Instructions on adding support for unified headers to your build system can be found here.

For additional information about unified headers, see our docs and the tracking bug. If you're looking ahead to future releases, the most up-to-date version of the documentation is in the master branch.

Make and ndk-build support in Android Studio 2.2

Posted by Kathryn Shih, Android Product Manager

In addition to supporting the experimental Gradle plugin, Android Studio 2.2 enables you to build C/C++ components of Android projects using CMake and ndk-build.

The Android Studio team plans to continue to support the experimental Gradle plugin. This will eventually replace the current Gradle plugin, providing additional tightly-integrated benefits to C/C++ developers such as smarter dependency management. So if you're interested in someday having the smartest possible interface between your IDE and your build system, you shouldn't ignore the experimental plugin.

CMake and ndk-build are useful alternatives to Gradle in several cases:

  • Projects that are already using CMake or ndk-build, such as legacy Eclipse ndk projects
  • Projects that are unable to assume the risk of using an experimental plugin for their C/C++ builds
  • Projects that will share a C/C++ build system across multiple platforms
  • C/C++ projects that need to use advanced features currently unavailable in experimental Gradle such as NEON support

For new projects, we recommend using CMake or experimental Gradle. For new Android projects with limited C++, we recommend trying the experimental Gradle plugin. For projects with substantial amounts of C++, or where you want the maximally stable build configuration, we recommend using a CMake build. Android Studio intends CMake to be a permanently supported solution.

While we think that there are substantial advantages to having a single build system able to handle all parts of an Android application, stabilizing the experimental plugin is not an option for us because it relies on Gradle APIs that are still a work in progress. Until the Gradle APIs are stabilized, the experimental plugin will keep changing, particularly in its Domain Specific Language, and will be strictly tied to a very specific version of Gradle itself.

Note that the the old, undocumented ndkCompile integration is deprecated. If you are using it, you need to move away from it as we'll remove it completely in the near future. We recommend migrating to gradle+cmake via our migration guide.

Migrating from Eclipse to Android Studio

We no longer support the Eclipse ADT. To get started migrating, download and install Android Studio. For most projects, migration is as simple as importing your existing Eclipse ADT projects in Android Studio with the File → New→ Import Project menu option. For more details on the migration process, check out the migration guide.

Feedback and Open Source Contributions

We're dedicated to making Android Studio the best possible integrated development environment for building Android apps, so if there are missing features or other challenges preventing you from using Android Studio, we want to hear about it [please take our survey]. You can also file bugs or feature requests directly with the team, and let us know via our Twitter or Google+ accounts.

Android Studio is an open source project, available to all at no cost. Check out our Open Source project page if you're interested in contributing or learning more.

Android changes for NDK developers

Posted by Dmitry Malykhanov, Developer Advocate

Related to other improvements to the Android platform, the dynamic linker in Android M and N has stricter requirements for writing clean, cross-platform compatible native code in order to load. It is necessary that an application’s native code follows the rules and recommendations in order to ensure a smooth transition to recent Android releases.

Below we outline in detail each individual change related to native code loading, the consequences and steps you can take to avoid issues.

Required tools: there is an <arch>-linux-android-readelf binary (e.g. arm-linux-androideabi-readelf or i686-linux-android-readelf) for each architecture in the NDK (under toolchains/), but you can use readelf for any architecture, as we will be doing basic inspection only. On Linux you need to have the “binutils” package installed for readelf, and “pax-utils” for scanelf.

Private API (Enforced since API 24)

Native libraries must use only public API, and must not link against non-NDK platform libraries. Starting with API 24 this rule is enforced and applications are no longer able to load non-NDK platform libraries. The rule is enforced by the dynamic linker, so non-public libraries are not accessible regardless of the way code tries to load them: System.loadLibrary(...), DT_NEEDED entries, and direct calls to dlopen(...) will fail in exactly the same way.

Users should have a consistent app experience across updates, and developers shouldn’t have to make emergency app updates to handle platform changes. For that reason, we recommend against using private C/C++ symbols. Private symbols aren’t tested as part of the Compatibility Test Suite (CTS) that all Android devices must pass. They may not exist, or they may behave differently. This makes apps that use them more likely to fail on specific devices, or on future releases --- as many developers found when Android 6.0 Marshmallow switched from OpenSSL to BoringSSL.

In order to reduce the user impact of this transition, we’ve identified a set of libraries that see significant use from Google Play’s most-installed apps, and that are feasible for us to support in the short term (including libandroid_runtime.so, libcutils.so, libcrypto.so, and libssl.so). In order to give you more time to transition, we will temporarily support these libraries; so if you see a warning that means your code will not work in a future release -- please fix it now!

$ readelf --dynamic libBroken.so | grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libnativehelper.so]
 0x00000001 (NEEDED)                     Shared library: [libutils.so]
 0x00000001 (NEEDED)                     Shared library: [libstagefright_foundation.so]
 0x00000001 (NEEDED)                     Shared library: [libmedia_jni.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [libz.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]

Potential problems: starting from API 24 the dynamic linker will not load private libraries, preventing the application from loading.

Resolution: rewrite your native code to rely only on public API. As a short term workaround, platform libraries without complex dependencies (libcutils.so) can be copied to the project. As a long term solution the relevant code must be copied to the project tree. SSL/Media/JNI internal/binder APIs should not be accessed from the native code. When necessary, native code should call appropriate public Java API methods.

A complete list of public libraries is available within the NDK, under platforms/android-API/usr/lib.

Note: SSL/crypto is a special case, applications must NOT use platform libcrypto and libssl libraries directly, even on older platforms. All applications should use GMS Security Provider to ensure they are protected from known vulnerabilities.

Missing Section Headers (Enforced since API 24)

Each ELF file has additional information contained in the section headers. These headers must be present now, because the dynamic linker uses them for sanity checking. Some developers try to strip them in an attempt to obfuscate the binary and prevent reverse engineering. (This doesn’t really help because it is possible to reconstruct the stripped information using widely-available tools.)

$ readelf --header libBroken.so | grep 'section headers'
  Start of section headers:          0 (bytes into file)
  Size of section headers:           0 (bytes)
  Number of section headers:         0
$

Resolution: remove the extra steps from your build that strip section headers.

Text Relocations (Enforced since API 23)

Starting with API 23, shared objects must not contain text relocations. That is, the code must be loaded as is and must not be modified. Such an approach reduces load time and improves security.

The usual reason for text relocations is non-position independent hand-written assembler. This is not common. Use the scanelf tool as described in our documentation for further diagnostics:

$ scanelf -qT libTextRel.so
  libTextRel.so: (memory/data?) [0x15E0E2] in (optimized out: previous simd_broken_op1) [0x15E0E0]
  libTextRel.so: (memory/data?) [0x15E3B2] in (optimized out: previous simd_broken_op2) [0x15E3B0]
[skipped the rest]

If you have no scanelf tool available, it is possible to do a basic check with readelf instead, look for either a TEXTREL entry or the TEXTREL flag. Either alone is sufficient. (The value corresponding to the TEXTREL entry is irrelevant and typically 0 --- simply the presence of the TEXTREL entry declares that the .so contains text relocations). This example has both indicators present:

$ readelf --dynamic libTextRel.so | grep TEXTREL
 0x00000016 (TEXTREL)                    0x0
 0x0000001e (FLAGS)                      SYMBOLIC TEXTREL BIND_NOW
$

Note: it is technically possible to have a shared object with the TEXTREL entry/flag but without any actual text relocations. This doesn’t happen with the NDK, but if you’re generating ELF files yourself make sure you’re not generating ELF files that claim to have text relocations, because the Android dynamic linker trusts the entry/flag.

Potential problems: Relocations enforce code pages being writable, and wastefully increase the number of dirty pages in memory. The dynamic linker has issued warnings about text relocations since Android K (API 19), but on API 23 and above it refuses to load code with text relocations.

Resolution: rewrite assembler to be position independent to ensure no text relocations are necessary. Check the Gentoo documentation for cookbook recipes.

Invalid DT_NEEDED Entries (Enforced since API 23)

While library dependencies (DT_NEEDED entries in the ELF headers) can be absolute paths, that doesn’t make sense on Android because you have no control over where your library will be installed by the system. A DT_NEEDED entry should be the same as the needed library’s SONAME, leaving the business of finding the library at runtime to the dynamic linker.

Before API 23, Android’s dynamic linker ignored the full path, and used only the basename (the part after the last ‘/’) when looking up the required libraries. Since API 23 the runtime linker will honor the DT_NEEDED exactly and so it won’t be able to load the library if it is not present in that exact location on the device.

Even worse, some build systems have bugs that cause them to insert DT_NEEDED entries that point to a file on the build host, something that cannot be found on the device.

$ readelf --dynamic libSample.so | grep NEEDED
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library:
[C:\Users\build\Android\ci\jni\libBroken.so]
$

Potential problems: before API 23 the DT_NEEDED entry’s basename was used, but starting from API 23 the Android runtime will try to load the library using the path specified, and that path won’t exist on the device. There are broken third-party toolchains/build systems that use a path on a build host instead of the SONAME.

Resolution: make sure all required libraries are referenced by SONAME only. It is better to let the runtime linker to find and load those libraries as the location may change from device to device.

Missing SONAME (Used since API 23)

Each ELF shared object (“native library”) must have a SONAME (Shared Object Name) attribute. The NDK toolchain adds this attribute by default, so its absence indicates either a misconfigured alternative toolchain or a misconfiguration in your build system. A missing SONAME may lead to runtime issues such as the wrong library being loaded: the filename is used instead when this attribute is missing.

$ readelf --dynamic libWithSoName.so | grep SONAME
 0x0000000e (SONAME)                     Library soname: [libWithSoName.so]
$

Potential problems: namespace conflicts may lead to the wrong library being loaded at runtime, which leads to crashes when required symbols are not found, or you try to use an ABI-incompatible library that isn’t the library you were expecting.

Resolution: the current NDK generates the correct SONAME by default. Ensure you’re using the current NDK and that you haven’t configured your build system to generate incorrect SONAME entries (using the -soname linker option).

Please remember, clean, cross-platform code built with a current NDK should have no issues on Android N. We encourage you to revise your native code build so that it produces correct binaries.

Android Support Library 23.1

Posted by Ian Lake, Developer Advocate

The Android Support Library is a collection of libraries available on a wide array of API levels that help you focus on the unique parts of your app, providing pre-built components, new functionality, and compatibility shims.

With the latest release of the Android Support Library (23.1), you will see improvements across the Support V4, Media Router, RecyclerView, AppCompat, Design, Percent, Custom Tabs, Leanback, and Palette libraries. Let’s take a closer look.

Support V4

The Support V4 library focuses on making supporting a wide variety of API levels straightforward with compatibility shims and backporting specific functionality.

NestedScrollView is a ScrollView that supports nested scrolling back to API 4. You’ll now be able to set a OnScrollChangeListener to receive callbacks when the scroll X or Y positions change.

There are a lot of pieces that make up a fully functioning media playback app, with much of it centered around MediaSessionCompat. A media button receiver, a key part to handling playback controls from hardware or bluetooth controls, is now formalized in the new MediaButtonReceiver class. This class makes it possible to forward received playback controls to a Service which is managing your MediaSessionCompat, reusing the Callback methods already required for API 21+, centralizing support on all API levels and all media control events in one place. A simplified constructor for MediaSessionCompat is also available, automatically finding a media button receiver in your manifest for use with MediaSessionCompat.

Media Router

The Media Router Support Library is the key component for connecting and sending your media playback to remote devices, such as video and audio devices with Google Cast support. It also provides the mechanism, via MediaRouteProvider, to enable any application to create and manage a remote media playback device connection.

In this release, MediaRouteChooserDialog (the dialog that controls selecting a valid remote device) and MediaRouteControllerDialog (the dialog to control ongoing remote playback) have both received a brand new design and additional functionality as well. You’ll find the chooser dialog sorts devices by frequency of use and includes a device type icon for easy identification of different devices while the controller dialog now shows current playback information (including album art).

To feel like a natural part of your app, the content color for both dialogs is now based on the colorPrimary of your alert dialog theme:

<!-- Your app theme set on your Activity -->
<style name="AppTheme" parent="Theme.AppCompat.Light.DarkActionBar">
  <item name="colorPrimary">@color/primary</item>
  <item name="colorPrimaryDark">@color/primaryDark</item>
  <item name="alertDialogTheme">@style/AppTheme.Dialog</item>
</style>
<!-- Theme for the dialog itself -->
<style name="AppTheme.Dialog" parent="Theme.AppCompat.Light.Dialog.Alert">
  <item name="colorPrimary">@color/primary</item>
  <item name="colorPrimaryDark">@color/primaryDark</item>
</style>

RecyclerView

RecyclerView is an extremely powerful and flexible way to show a list, grid, or any view of a large set of data. One advantage over ListView or GridView is the built in support for animations as items are added, removed, or repositioned.

This release significantly changes the animation system for the better. By using the new ItemAnimator’s canReuseUpdatedViewHolder() method, you’ll be able to choose to reuse the existing ViewHolder, enabling item content animation support. The new ItemHolderInfo and associated APIs give the ItemAnimator the flexibility to collect any data it wants at the correct point in the layout lifecycle, passing that information into the animate callbacks.

Note that this new API is not backward compatible. If you previously implemented an ItemAnimator, you can instead extend SimpleItemAnimator, which provides the old API by wrapping the new API. You’ll also notice that some methods have been entirely removed from ItemAnimator. For example, if you were calling recyclerView.getItemAnimator().setSupportsChangeAnimations(false), this code won’t compile anymore. You can replace it with:

ItemAnimator animator = recyclerView.getItemAnimator();
if (animator instanceof SimpleItemAnimator) {
  ((SimpleItemAnimator) animator).setSupportsChangeAnimations(false);
}

AppCompat

One component of the AppCompat Support Library has been in providing a consistent set of widgets across all API levels, including the ability to tint those widgets to match your branding and accent colors.

This release adds tint aware versions of SeekBar (for tinting the thumb) as well as ImageButton and ImageView (providing backgroundTint support) which will automatically be used when you use the platform versions in your layouts. You’ll also find that SwitchCompat has been updated to match the styling found in Android 6.0 Marshmallow.

Design

The Design Support Library includes a number of components to help implement the latest in the Google design specifications.

TextInputLayout expands its existing functionality of floating hint text and error indicators with new support for character counting.

AppBarLayout supports a number of scroll flags which affect how children views react to scrolling (e.g. scrolling off the screen). New to this release is SCROLL_FLAG_SNAP, ensuring that when scrolling ends, the view is not left partially visible. Instead, it will be scrolled to its nearest edge, making fully visible or scrolled completely off the screen. You’ll also find that AppBarLayout now allows users to start scrolling from within the AppBarLayout rather than only from within your scrollable view - this behavior can be controlled by adding a DragCallback.

NavigationView provides a convenient way to build a navigation drawer, including the ability to creating menu items using a menu XML file. We’ve expanded the functionality possible with the ability to set custom views for items via app:actionLayout or using MenuItemCompat.setActionView().

Percent

The Percent Support Library provides percentage based dimensions and margins and, new to this release, the ability to set a custom aspect ratio via app:aspectRatio. By setting only a single width or height and using aspectRatio, the PercentFrameLayout or PercentRelativeLayout will automatically adjust the other dimension so that the layout uses a set aspect ratio.

Custom Tabs

The Custom Tabs Support Library allows your app to utilize the full features of compatible browsers including using pre-existing cookies while still maintaining a fast load time (via prefetching) and a custom look and actions.

In this release, we’re adding a few additional customizations such as hiding the url bar when the page is scrolled down with the new enableUrlBarHiding() method. You’ll also be able to update the action button in an already launched custom tab via your CustomTabsSession with setActionButton() - perfect for providing a visual indication of a state change.

Navigation events via CustomTabsCallback#onNavigationEvent() have also been expanded to include the new TAB_SHOWN and TAB_HIDDEN events, giving your app more information on how the user interacts with your web content.

Leanback

The Leanback library makes it easy to build user interfaces on TV devices. This release adds GuidedStepSupportFragment for a support version of GuidedStepFragment as well as improving animations and transitions and allowing GuidedStepFragment to be placed on top of existing content.

You’ll also be able to annotate different types of search completions in SearchFragment and staggered slide transition support for VerticalGridFragment.

Palette

Palette, used to extract colors from images, now supports extracting from a specific region of a Bitmap with the new setRegion() method.

SDK available now!

There’s no better time to get started with the Android Support Library. You can get started developing today by updating the Android Support Repository from the Android SDK Manager.

For an in depth look at every API change in this release, check out the full API diff.

To learn more about the Android Support Library and the APIs available to you through it, visit the Support Library section on the Android Developer site.

Get your hands on Android Studio 1.3

Posted by Jamal Eason, Product Manager, Android

Previewed earlier this summer at Google I/O, Android Studio 1.3 is now available on the stable release channel. We appreciated the early feedback from those developers on our canary and beta channels to help ship a great product.

Android Studio 1.3 is our biggest feature release for the year so far, which includes a new memory profiler, improved testing support, and full editing and debugging support for C++. Let’s take a closer look.

New Features in Android Studio 1.3

Performance & Testing Tools

  • Android Memory (HPROF) Viewer

    Android Studio now allows you to capture and analyze memory snapshots in the native Android HPROF format.

  • Allocation Tracker

    In addition to displaying a table of memory allocations that your app uses, the updated allocation tracker now includes a visual way to view the your app allocations.

  • APK Tests in Modules

    For more flexibility in app testing, you now have the option to place your code tests in a separate module and use the new test plugin (‘com.android.test’) instead of keeping your tests right next to your app code. This feature does require your app project to use the Gradle Plugin 1.3.

Code and SDK Management

  • App permission annotations

    Android Studio now has inline code annotation support to help you manage the new app permissions model in the M release of Android. Learn more about code annotations.

  • Data Binding Support

    New data brinding features allow you to create declarative layouts in order to minimize boilerplate code by binding your application logic into your layouts. Learn more about data binding.

  • SDK Auto Update & SDK Manager

    Managing Android SDK updates is now a part of the Android Studio. By default, Android Studio will now prompt you about new SDK & Tool updates. You can still adjust your preferences with the new & integrated Android SDK Manager.

  • C++ Support

    As a part of the Android 1.3 stable release, we included an Early Access Preview of the C++ editor & debugger support paired with an experimental build plugin. See the Android C++ Preview page for information on how to get started. Support for more complex projects and build configurations is in development, but let us know your feedback.

Time to Update

An important thing to remember is that an update to Android Studio does not require you to change your Android app projects. With updating, you get the latest features but still have control of which build tools and app dependency versions you want to use for your Android app.

For current developers on Android Studio, you can check for updates from the navigation menu. For new users, you can learn more about Android Studio on the product overview page or download the stable version from the Android Studio download site.

We are excited to launch this set of features in Android Studio and we are hard at work developing the next set of tools to make develop Android development easier on Android Studio. As always we welcome feedback on how we can help you. Connect with the Android developer tools team on Google+.

Game Performance: Geometry Instancing

Posted by Shanee Nishry, Games Developer Advocate

Imagine a beautiful virtual forest with countless trees, plants and vegetation, or a stadium with countless people in the crowd cheering. If you are heroic you might like the idea of an epic battle between armies.

Rendering a lot of meshes is desired to create a beautiful scene like a forest, a cheering crowd or an army, but doing so is quite costly and reduces the frame rate. Fortunately this is possible using a simple technique called Geometry Instancing.

Geometry instancing can be used in 2D games for rendering a large number of sprites, or in 3D for things like particles, characters and environment.

The NDK code sample More Teapots demoing the content of this article can be found with the ndk inside the samples folder and in the git repository.

Support and Extensions

Geometry instancing is available from OpenGL ES 3.0 and to OpenGL 2.0 devices which support the GL_NV_draw_instanced or GL_EXT_draw_instanced extensions. More information on how to using the extensions is shown in the More Teapots demo.

Overview

Submitting draw calls causes OpenGL to queue commands to be sent to the GPU, this has an expensive overhead which may affect performance. This overhead grows when changing states such as alpha blending function, active shader, textures and buffers.

Geometry Instancing is a technique that combines multiple draws of the same mesh into a single draw call, resulting in reduced overhead and potentially increased performance. This works even when different transformations are required.

The algorithm

To explain how Geometry Instancing works let’s quickly overview traditional drawing.

Traditional Drawing

To a draw a mesh you’d usually prepare a vertex buffer and an index buffer, bind your shader and buffers, set your uniforms such as a World View Projection matrix and make a draw call.

To draw multiple instances using the same mesh you set new uniform values for the transformations and other data and call draw again. This is repeated for every instance.

Drawing with Geometry Instancing

Geometry Instancing reduces CPU overhead by reducing the sequence described above into a single buffer and draw call.

It works by using an additional buffer which contains custom per-instance data needed by your shader, such as transformations, color, light data.

The first change to your workflow is to create the additional buffer on initialization stage.

To put it into code let’s define an example per-instance data that includes a world view projection matrix and a color:

C++

struct PerInstanceData
{
 Mat4x4 WorldViewProj;
 Vector4 Color;
};

You also need to the structure to your shader. The easiest way is by creating a Uniform Block with an array:

GLSL

#define MAX_INSTANCES 512

layout(std140) uniform PerInstanceData {
    struct
    {
        mat4      uMVP;
        vec4      uColor;
    } Data[ MAX_INSTANCES ];
};

Note that uniform blocks have limited sizes. You can find the maximum number of bytes you can use by querying for GL_MAX_UNIFORM_BLOCK_SIZE using glGetIntegerv.

Example:

GLint max_block_size = 0;
glGetIntegerv( GL_MAX_UNIFORM_BLOCK_SIZE, &max_block_size );

Bind the uniform block on the CPU in your program’s initialization stage:

C++

#define MAX_INSTANCES 512
#define BINDING_POINT 1
GLuint shaderProgram; // Compiled shader program

// Bind Uniform Block
GLuint blockIndex = glGetUniformBlockIndex( shaderProgram, "PerInstanceData" );
glUniformBlockBinding( shaderProgram, blockIndex, BINDING_POINT );

And create a corresponding uniform buffer object:

C++

// Create Instance Buffer
GLuint instanceBuffer;

glGenBuffers( 1, &instanceBuffer );
glBindBuffer( GL_UNIFORM_BUFFER, instanceBuffer );
glBindBufferBase( GL_UNIFORM_BUFFER, BINDING_POINT, instanceBuffer );

// Initialize buffer size
glBufferData( GL_UNIFORM_BUFFER, MAX_INSTANCES * sizeof( PerInstanceData ), NULL, GL_DYNAMIC_DRAW );

The next step is to update the instance data every frame to reflect changes to the visible objects you are going to draw. Once you have your new instance buffer you can draw everything with a single call to glDrawElementsInstanced.

You update the instance buffer using glMapBufferRange. This function locks the buffer and retrieves a pointer to the byte data allowing you to copy your per-instance data. Unlock your buffer using glUnmapBuffer when you are done.

Here is a simple example for updating the instance data:

const int NUM_SCENE_OBJECTS = …; // number of objects visible in your scene which share the same mesh

// Bind the buffer
glBindBuffer( GL_UNIFORM_BUFFER, instanceBuffer );

// Retrieve pointer to map the data
PerInstanceData* pBuffer = (PerInstanceData*) glMapBufferRange( GL_UNIFORM_BUFFER, 0,
                NUM_SCENE_OBJECTS * sizeof( PerInstanceData ),
                GL_MAP_WRITE_BIT | GL_MAP_INVALIDATE_RANGE_BIT );

// Iterate the scene objects
for ( int i = 0; i < NUM_SCENE_OBJECTS; ++i )
{
    pBuffer[ i ].WorldViewProj = ... // Copy World View Projection matrix
    pBuffer[ i ].Color = …               // Copy color
}

glUnmapBuffer( GL_UNIFORM_BUFFER ); // Unmap the buffer

And finally you can draw everything with a single call to glDrawElementsInstanced or glDrawArraysInstanced (depending if you are using an index buffer):

glDrawElementsInstanced( GL_TRIANGLES, NUM_INDICES, GL_UNSIGNED_SHORT, 0,
                NUM_SCENE_OBJECTS );

You are almost done! There is just one more step to do. In your shader you need to make use of the new uniform buffer object for your transformations and colors. In your shader main program:

void main()
{
    …
    gl_Position = PerInstanceData.Data[ gl_InstanceID ].uMVP * inPosition;
    outColor = PerInstanceData.Data[ gl_InstanceID ].uColor;
}

You might have noticed the use gl_InstanceID. This is a predefined OpenGL vertex shader variable that tells your program which instance it is currently drawing. Using this variable your shader can properly iterate the instance data and match the correct transformation and color for every vertex.

That’s it! You are now ready to use Geometry Instancing. If you are drawing the same mesh multiple times in a frame make sure to implement Geometry Instancing in your pipeline! This can greatly reduce overhead and improve performance.