Tag Archives: Memory Safety

MTE – The promising path forward for memory safety

Posted by Andy Qin, Irene Ang, Kostya Serebryany, Evgenii Stepanov

Since 2018, Google has partnered with ARM and collaborated with many ecosystem partners (SoCs vendors, mobile phone OEMs, etc.) to develop Memory Tagging Extension (MTE) technology. We are now happy to share the growing adoption in the ecosystem. MTE is now available on some OEM devices (as noted in a recent blog post by Project Zero) with Android 14 as a developer option, enabling developers to use MTE to discover memory safety issues in their application easily.

The security landscape is changing dynamically, new attacks are becoming more complex and costly to mitigate. It’s becoming increasingly important to detect and prevent security vulnerabilities early in the software development cycle and also have the capability to mitigate the security attacks at the first moment of exploitation in production.

The biggest contributor to security vulnerabilities are memory safety related defects and Google has invested in a set of technologies to help mitigate memory safety risks. These include but are not limited to:

MTE is a hardware based capability that can detect unknown memory safety vulnerabilities in testing and/or mitigate them in production. It works by tagging the pointers and memory regions and comparing the tags to identify mismatches (details). In addition to the security benefits, MTE can also help ensure integrity because memory safety bugs remain one of the major contributors to silent data corruption that not only impact customer trust, but also cause lost productivity for software developers.

At the moment, MTE is supported on some of the latest chipsets:

  • Focusing on security for Android devices, the MediaTek Dimensity 9300 integrates support for MTE via ARM's latest v9 architecture (which is what Cortex-X4 and Cortex-A720 processors are based on). This feature can be switched on and off in the bootloader by users and developers instead of having it always on or always off.
  • Tensor G3 integrates support for MTE only within the developer mode toggle. Feature can be activated by developers.

For both chipsets, this feature can be switched on and off by developers, making it easier to find memory-related bugs during development and after deployment. MTE can help users stay safe while also improving time to market for OEMs.

Application developers will be the first to leverage this feature as a way to improve their application security and reliability in the software development lifecycle. MTE can effectively help them to discover hard-to-detect memory safety vulnerabilities (buffer overflows, user-after-free, etc.) with clear & actionable stack trace information in integration testing or pre-production environments. Another benefit of MTE is that the engineering cost of memory-safety testing is drastically reduced because heap bug detection (which is majority of all memory safety bugs) does not require any source or binary changes to leverage MTE, i.e. advanced memory-safety can be achieved with just a simple environment or configuration change.

We believe that MTE will play a very important role in detecting and preventing memory safety vulnerabilities and provide a promising path towards improving software security.

Notes

  1. ASAN = Address Sanitizer; HWASAN = HW based ASAN;GWP-ASAN = sampling based ASAN ↩

Rust/C++ interop in the Android Platform

Posted by Joel Galenson and Matthew Maurer, Android Team

One of the main challenges of evaluating Rust for use within the Android platform was ensuring we could provide sufficient interoperability with our existing codebase. If Rust is to meet its goals of improving security, stability, and quality Android-wide, we need to be able to use Rust anywhere in the codebase that native code is required. To accomplish this, we need to provide the majority of functionality platform developers use. As we discussed previously, we have too much C++ to consider ignoring it, rewriting all of it is infeasible, and rewriting older code would likely be counterproductive as the bugs in that code have largely been fixed. This means interoperability is the most practical way forward.

Before introducing Rust into the Android Open Source Project (AOSP), we needed to demonstrate that Rust interoperability with C and C++ is sufficient for practical, convenient, and safe use within Android. Adding a new language has costs; we needed to demonstrate that Rust would be able to scale across the codebase and meet its potential in order to justify those costs. This post will cover the analysis we did more than a year ago while we evaluated Rust for use in Android. We also present a follow-up analysis with some insights into how the original analysis has held up as Android projects have adopted Rust.

Language interoperability in Android

Existing language interoperability in Android focuses on well defined foreign-function interface (FFI) boundaries, which is where code written in one programming language calls into code written in a different language. Rust support will likewise focus on the FFI boundary as this is consistent with how AOSP projects are developed, how code is shared, and how dependencies are managed. For Rust interoperability with C, the C application binary interface (ABI) is already sufficient.

Interoperability with C++ is more challenging and is the focus of this post. While both Rust and C++ support using the C ABI, it is not sufficient for idiomatic usage of either language. Simply enumerating the features of each language results in an unsurprising conclusion: many concepts are not easily translatable, nor do we necessarily want them to be. After all, we’re introducing Rust because many features and characteristics of C++ make it difficult to write safe and correct code. Therefore, our goal is not to consider all language features, but rather to analyze how Android uses C++ and ensure that interop is convenient for the vast majority of our use cases.

We analyzed code and interfaces in the Android platform specifically, not codebases in general. While this means our specific conclusions may not be accurate for other codebases, we hope the methodology can help others to make a more informed decision about introducing Rust into their large codebase. Our colleagues on the Chrome browser team have done a similar analysis, which you can find here.

This analysis was not originally intended to be published outside of Google: our goal was to make a data-driven decision on whether or not Rust was a good choice for systems development in Android. While the analysis is intended to be accurate and actionable, it was never intended to be comprehensive, and we’ve pointed out a couple of areas where it could be more complete. However, we also note that initial investigations into these areas showed that they would not significantly impact the results, which is why we decided to not invest the additional effort.

Methodology

Exported functions from Rust and C++ libraries are where we consider interop to be essential. Our goals are simple:

  • Rust must be able to call functions from C++ libraries and vice versa.
  • FFI should require a minimum of boilerplate.
  • FFI should not require deep expertise.

While making Rust functions callable from C++ is a goal, this analysis focuses on making C++ functions available to Rust so that new Rust code can be added while taking advantage of existing implementations in C++. To that end, we look at exported C++ functions and consider existing and planned compatibility with Rust via the C ABI and compatibility libraries. Types are extracted by running objdump on shared libraries to find external C++ functions they use1 and running c++filt to parse the C++ types. This gives functions and their arguments. It does not consider return values, but a preliminary analysis2 of those revealed that they would not significantly affect the results.

We then classify each of these types into one of the following buckets:

Supported by bindgen

These are generally simple types involving primitives (including pointers and references to them). For these types, Rust’s existing FFI will handle them correctly, and Android’s build system will auto-generate the bindings.

Supported by cxx compat crate

These are handled by the cxx crate. This currently includes std::string, std::vector, and C++ methods (including pointers/references to these types). Users simply have to define the types and functions they want to share across languages and cxx will generate the code to do that safely.

Native support

These types are not directly supported, but the interfaces that use them have been manually reworked to add Rust support. Specifically, this includes types used by AIDL and protobufs.

We have also implemented a native interface for StatsD as the existing C++ interface relies on method overloading, which is not well supported by bindgen and cxx3. Usage of this system does not show up in the analysis because the C++ API does not use any unique types.

Potential addition to cxx

This is currently common data structures such as std::optional and std::chrono::duration and custom string and vector implementations.

These can either be supported natively by a future contribution to cxx, or by using its ExternType facilities. We have only included types in this category that we believe are relatively straightforward to implement and have a reasonable chance of being accepted into the cxx project.

We don't need/intend to support

Some types are exposed in today’s C++ APIs that are either an implicit part of the API, not an API we expect to want to use from Rust, or are language specific. Examples of types we do not intend to support include:

  • Mutexes - we expect that locking will take place in one language or the other, rather than needing to pass mutexes between languages, as per our coarse-grained philosophy.
  • native_handle - this is a JNI interface type, so it is inappropriate for use in Rust/C++ communication.
  • std::locale& - Android uses a separate locale system from C++ locales. This type primarily appears in output due to e.g., cout usage, which would be inappropriate to use in Rust.

Overall, this category represents types that we do not believe a Rust developer should be using.

HIDL

Android is in the process of deprecating HIDL and migrating to AIDL for HALs for new services.We’re also migrating some existing implementations to stable AIDL. Our current plan is to not support HIDL, preferring to migrate to stable AIDL instead. These types thus currently fall into the “We don't need/intend to support'' bucket above, but we break them out to be more specific. If there is sufficient demand for HIDL support, we may revisit this decision later.

Other

This contains all types that do not fit into any of the above buckets. It is currently mostly std::string being passed by value, which is not supported by cxx.

Top C++ libraries

One of the primary reasons for supporting interop is to allow reuse of existing code. With this in mind, we determined the most commonly used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed all of the external C++ functions used by these libraries and their arguments to determine how well they would interoperate with Rust.

Overall, 81% of types are in the first three categories (which we currently fully support) and 87% are in the first four categories (which includes those we believe we can easily support). Almost all of the remaining types are those we believe we do not need to support.

Mainline modules

In addition to analyzing popular C++ libraries, we also examined Mainline modules. Supporting this context is critical as Android is migrating some of its core functionality to Mainline, including much of the native code we hope to augment with Rust. Additionally, their modularity presents an opportunity for interop support.

We analyzed 64 binaries and libraries in 21 modules. For each analyzed library we examined their used C++ functions and analyzed the types of their arguments to determine how well they would interoperate with Rust in the same way we did above for the top 10 libraries.

Here 88% of types are in the first three categories and 90% in the first four, with almost all of the remaining being types we do not need to handle.

Analysis of Rust/C++ Interop in AOSP

With almost a year of Rust development in AOSP behind us, and more than a hundred thousand lines of code written in Rust, we can now examine how our original analysis has held up based on how C/C++ code is currently called from Rust in AOSP.4

The results largely match what we expected from our analysis with bindgen handling the majority of interop needs. Extensive use of AIDL by the new Keystore2 service results in the primary difference between our original analysis and actual Rust usage in the “Native Support” category.

A few current examples of interop are:

  • Cxx in Bluetooth - While Rust is intended to be the primary language for Bluetooth, migrating from the existing C/C++ implementation will happen in stages. Using cxx allows the Bluetooth team to more easily serve legacy protocols like HIDL until they are phased out by using the existing C++ support to incrementally migrate their service.
  • AIDL in keystore - Keystore implements AIDL services and interacts with apps and other services over AIDL. Providing this functionality would be difficult to support with tools like cxx or bindgen, but the native AIDL support is simple and ergonomic to use.
  • Manually-written wrappers in profcollectd - While our goal is to provide seamless interop for most use cases, we also want to demonstrate that, even when auto-generated interop solutions are not an option, manually creating them can be simple and straightforward. Profcollectd is a small daemon that only exists on non-production engineering builds. Instead of using cxx it uses some small manually-written C wrappers around C++ libraries that it then passes to bindgen.

Conclusion

Bindgen and cxx provide the vast majority of Rust/C++ interoperability needed by Android. For some of the exceptions, such as AIDL, the native version provides convenient interop between Rust and other languages. Manually written wrappers can be used to handle the few remaining types and functions not supported by other options as well as to create ergonomic Rust APIs. Overall, we believe interoperability between Rust and C++ is already largely sufficient for convenient use of Rust within Android.

If you are considering how Rust could integrate into your C++ project, we recommend doing a similar analysis of your codebase. When addressing interop gaps, we recommend that you consider upstreaming support to existing compat libraries like cxx.

Acknowledgements

Our first attempt at quantifying Rust/C++ interop involved analyzing the potential mismatches between the languages. This led to a lot of interesting information, but was difficult to draw actionable conclusions from. Rather than enumerating all the potential places where interop could occur, Stephen Hines suggested that we instead consider how code is currently shared between C/C++ projects as a reasonable proxy for where we’ll also likely want interop for Rust. This provided us with actionable information that was straightforward to prioritize and implement. Looking back, the data from our real-world Rust usage has reinforced that the initial methodology was sound. Thanks Stephen!

Also, thanks to:

  • Andrei Homescu and Stephen Crane for contributing AIDL support to AOSP.
  • Ivan Lozano for contributing protobuf support to AOSP.
  • David Tolnay for publishing cxx and accepting our contributions.
  • The many authors and contributors to bindgen.
  • Jeff Vander Stoep and Adrian Taylor for contributions to this post.

  1. We used undefined symbols of function type as reported by objdump to perform this analysis. This means that any header-only functions will be absent from our analysis, and internal (non-API) functions which are called by header-only functions may appear in it. ↩

  2. We extracted return values by parsing DWARF symbols, which give the return types of functions. ↩

  3. Even without automated binding generation, manually implementing the bindings is straightforward. ↩

  4. In the case of handwritten C/C++ wrappers, we analyzed the functions they call, not the wrappers themselves. For all uses of our native AIDL library, we analyzed the types used in the C++ version of the library. ↩

Detecting Memory Corruption Bugs With HWASan

Posted by Evgenii Stepanov, Staff Software Engineer, Dynamic Tools

Native code in memory-unsafe languages like C and C++ is often vulnerable to memory corruption bugs. Our data shows that issues like use-after-free, double-free, and heap buffer overflows generally constitute more than 65% of High & Critical security bugs in Chrome and Android.

In previous years our memory bug detection efforts were focused on Address Sanitizer (ASan). ASan catches these errors but causes your app to use 2x-3x extra memory and to run slower.

To better tackle these problems we’ve developed Hardware-Assisted Address Sanitizer (HWASan). HWASan typically only requires 15% more memory. It’s also a lot faster than ASan. HWASan’s performance makes it usable not only for unit testing, but also for interactive human-driven testing. We use this to find memory issues in the Android OS itself, and now we've made it easy for app developers to use it too. HWASan is fast enough that some Android developers use it on their development devices for everyday tasks.

Under the hood

HWASan is based on memory tagging and depends on the Top Byte Ignore feature present in all 64-bit ARM CPUs and the associated kernel support. Every memory allocation is assigned a random 8-bit tag that is stored in the most significant byte (MSB) of the address, but ignored by the CPU. As a result, this tagged pointer can be used in place of a regular pointer without any code changes.

Under the hood, HWASan uses shadow memory - a sparse map that assigns a tag value to each 16 byte block of program memory. Compile time code instrumentation is used to insert checks that compare pointer and memory tags for every memory access, and raise an error if they do not match.

This approach allows us to detect both use-after-free and buffer-overflow types of bugs. The memory tag in the shadow is changed to a random value during allocation and deallocation. As a result, attempting to access deallocated memory with a dangling pointer will almost certainly fail due to a tag mismatch. The same is true for an attempt to access memory outside of the allocated region, which is very likely to have a different tag. Stack and global variables are similarly protected.

Use-after-free bug detection with memory tagging.

Use-after-free bug detection with memory tagging.

This approach is non deterministic: because of the limited number of possible tags, an invalid memory access has 1 chance out of 256 (approximately 0.4%) to pass undetected. We have not observed this as a problem in practice, but, due to the tag randomness, running the program the second time is very likely to find any bugs that the first run has missed.

An advantage of HWASan over ASan is its ability to find bugs that happen far from their origination point - for example, a use-after-free where the memory is accessed long after it has been deallocated, or a buffer overflow with a large offset. This is not the case with ASan, which uses red zones around memory allocations, and a quarantine for the temporary storage of recently deallocated memory blocks. Both redzones and the quarantine are of limited size, and error detection is unlikely beyond that. HWASan uses a different approach that does not have these limitations.

Usage

When a bug is discovered the process is terminated and a crash dump is printed to logcat. The “Abort message” field contains a HWASan report, which shows the access type (read or write), access address, thread id and the stack trace of the bad memory access. This is followed by a stack trace for the original allocation, and, for use-after-free bugs, a stack trace showing where the deallocation took place. Advanced users can find extra debugging information below this, including a map of memory tags for nearby locations. 

signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: '==21586==ERROR: HWAddressSanitizer: tag-mismatch on address 0x0042a0807af0 at pc 0x007b23b8786c
WRITE of size 1 at 0x0042a0807af0 tags: db/19 (ptr/mem) in thread T0
    #0 0x7b23b87868 (/data/app/com.example.myapp/lib/arm64/native.so+0x2868)
    #1 0x7b8f1e4ccc (/apex/com.android.art/lib64/libart.so+0x198ccc)
[...]

0x0042a0807af0 is located 0 bytes to the right of 16-byte region [0x0042a0807ae0,0x0042a0807af0)
allocated here:
    #0 0x7b92a322bc (/path/to/libclang_rt.hwasan-aarch64-android.so+0x212bc)
    #1 0x7b23b87840 (/data/app/com.example.myapp/lib/arm64/native.so+0x2840)
[...]

An example snippet from a HWASan crash report.

Google uses HWASan extensively in Android development, and now you can too. Find out more -- including the details of how to rebuild your app for use with HWASan -- at https://developer.android.com/ndk/guides/hwasan. Prebuilt HWASan system images are available on the AOSP build server (or you can build your own). They can be easily flashed onto a compatible device using the recently announced web flash tool.