Tag Archives: Treble

Treble Plus One Equals Four

Posted by Iliyan Malchev (Project Treble Architect), Amith Dsouza (Technical Account Manager) , and Veerendra Bhora (Strategic Partnerships Manager)

Illustration of phone with settings logo in the screen

Extending Android updates on Qualcomm’s Mobile Platforms

In the past few years, the latest Android OS has been adopted earlier by OEMs and deployed in larger numbers to our users. The growth in adoption has been driven by OEMs delivering faster OS updates, taking advantage of the architecture introduced by Project Treble.

At the time Android 11 launched there were 667M active users on Android 10, 82% of whom got their Android 10 build via an over the air (OTA) update. Despite the events throughout 2020, there is a continued momentum among our partners to either launch their devices on Android 11 or offer Android 11 OTAs on their devices earlier.

Line graph comparing Android Pie, Android 10, and Android 11

Our efforts till now have been focussed on making OS updates easier and faster to deploy. The other side of this coin is supporting updates for a longer period of time, and today we’d like to provide an overview of the changes we are making to help our partners achieve this.

Project Treble was an ambitious re-architecture of Android that created a split between the OS framework and device-specific low-level software (called the vendor implementation) through a well-defined, stable vendor interface. As a part of this split, the Android OS framework guarantees backward compatibility with the vendor implementation, which is checked through a standardized compliance test suite - VTS. With each Android release, Project Treble publishes Generic System Images (GSIs) that are built from AOSP sources, and are guaranteed to be backwards-compatible with the previous 3 versions of vendor implementations, in addition of course to the current release—for a total span of four years. Devices launching with the new Android release must have vendor implementations compatible with that GSI. This is the primary vehicle for reducing fragmentation within the OS framework. While we allow and encourage our partners to modify the framework itself, the modifications post-Treble must be done in a way that reduces upgrade costs from one version to the next.

Besides the reuse of a vendor implementation across OS updates, the Treble architecture also facilitates the re-use of the same OS framework code across different vendor implementations.

Chart comparing Original OS framework to Updated OS framework

Another important change introduced by Project Treble is that new vendor-impacting requirements for Android devices are never retroactive. They apply only to devices launching on that Android version and not to devices upgrading from an older version. The term vendor-impacting here refers to requirements for new HALs, or for the shipping of a newer Linux kernel, to the device's vendor implementation. A good example might be a new revision of the camera HAL to support multiple rear camera sensors. Since the Android framework guarantees compatibility with the older HALs, we enable older vendor implementations to be reused by OEMs for upgrades without the considerable cost of updating them with new requirements.

This principle, combined with the backwards-compatibility guarantee, gives device manufacturers (OEMs) the flexibility to support upgrades both faster (since they have to upgrade just the framework, which would cover all of their devices, including those with older versions of the vendor implementation), as well as at a lower cost (since they do not have to touch the older vendor implementations).

However, seen from a System-on-Chip manufacturers’ perspective, this design introduces additional complexity. For each SoC model, the SoC manufacturers now needed to create multiple combinations of vendor implementations to support OEMs who would use that chipset to launch new devices and deploy OS upgrades on previously launched devices.

The result is that three years beyond the launch of a chipset, the SoC vendor would have to support up to 6 combinations of OS framework software and vendor implementations. The engineering costs associated with this support limited the duration for which SoC vendors offered Android OS software support on a chipset. For every single chipset, the software support timeline would look like this:

Timeline of OS framework

Considering that SoC providers have dozens of SoC models at any point of time, the full picture looks closer to this:

More accurate support timeline

The crux of the problem was that, while device requirements were never retroactive, the requirements for SoCs were. For example on Android Pie, SoCs had to support two versions of the Camera HAL API on a chipset if it was used to support new device launches and upgrades.

From this perspective, the solution was simple: we had to extend the no-retroactivity principle to the SoCs as well as to devices. With this change, the SoC provider would be able to support Android with the same vendor implementations on their SoCs for device launches as well as upgrades.

During the past year, we have been working hard to implement this solution. Building on our deep collaboration with our colleagues at Qualcomm, today we’re announcing the results of this work. Going forward, all new Qualcomm mobile platforms that take advantage of the no-retroactivity principle for SoCs will support 4 Android OS versions and 4 years of security updates. All Qualcomm customers will be able to take advantage of this stability to further lower both the costs of upgrades as well as launches and can now support their devices for longer periods of time.

Going one step further, we’re also reusing the same OS framework software across multiple Qualcomm chipsets. This dramatically lowers the number of OS framework and vendor implementation combinations that Qualcomm has to support across their mobile platforms and results in lowered engineering, development, and deployment costs. The diagram below indicates how significant the simplification is. From a software-support perspective, it's an altogether different situation:

Framework timeline with simplification

This change is taking effect with all SoCs launching with Android 11 and later. By working closely with Qualcomm to offer an extended period of OS and security updates, we are looking forward to delivering the best of Android to our users faster, and with greater security for an extended period of time.

Accelerating Android Updates

Posted by Eddie Hsu (Technical Program Manager), Brent VerWeyst (Product Manager), Maya Ben Ari (Product Manager), Amith Dsouza (Technical Account Manager), Iliyan Malchev (Project Treble Architect)

Over the past few years we’ve introduced new capabilities that enable us to deliver updates more uniformly, quickly, and efficiently to Android devices. These capabilities include:

  • Oreo’s introduction of Project Treble created a system/vendor split for a much cleaner separation of OEM and SoC dependencies from the rest of the code base. This effort sped up the adoption of Android Pie by 2.5X. Every Android device that preloads the Google Play Store has been Treble compliant since that point.
  • In Pie, we started publishing the Generic System Images (GSI) so that developers can use them for app-compat testing on real hardware. Treble compliance means that every device is compatible with our GSIs, even if it does not ship with them. We also worked with our major partners to launch an OEM developer preview program. As a result, we saw a further 1.5X increase in the adoption of Android 10.
  • In Android 10, we started updating components of the OS directly via Google Play system updates (Project Mainline). Mainline provides security and privacy updates for the OS in a way that’s similar to apps – through Google Play. For example, in our most recent deployment, we directly updated 285 million devices with fixes for security vulnerabilities.
  • Google Play is also responsible for updating critical applications and services, such as authentication, push notifications, and Google Play Protect. A good example is the launch of the Exposure Notification API. Exposure Notifications are a tool to help public-health agencies in the fight against COVID-19. The API was deployed in May via Google Play to over 2 billion devices in the space of just 4 weeks.

Android 10 Adoption

Thanks to these efforts, the adoption of Android 10 has been faster than any previous versions of Android. Android 10 was running on 100 million devices 5 months post launch – 28% faster than Android Pie.

Updatability in Android 11

Below are the major themes in updatability this year:

OEM Developer Previews: In Android 11, device makers (OEMs) are continuing their developer previews ahead of the official launch. Seven OEMs have released Developer Preview builds on 13 devices to provide app developers with diverse hardware as they test for compatibility.

Google Play system update: 21 OS components are now updatable, including 9 additions in Android 11 focused on improving privacy, security, and developer consistency across devices. Highlights include an enhanced permissions component that standardizes user and developer access to critical privacy controls on Android devices, a Neural Networks API (NNAPI) component that optimizes performance and guarantees consistent APIs across devices, and a Tethering component for improved interoperability. The new updatable OS components in Android 11 are: Tethering, NNAPI, Cell Broadcast Receiver, adbd, Internet Key Exchange, Media Provider, statsd, WiFi, and SDK extension.

Generic Kernel Image: Our ongoing updatability work extends to the Linux kernel itself, with initiatives such as 6-year LTS support. In Android 11, we are further isolating common code in the Android Linux kernel to create a Generic Kernel Image (GKI) that works across all Android devices, as well as to enable faster security deployments. Stay tuned for a more detailed post on GKI in the coming months.

Virtual A/B: Most OS updates are not delivered via Google Play. Instead, they use separate third-party Over-the-Air (OTA) services that differ among the various OEMs. These services use a mechanism that, while very space efficient, has the disadvantage of being slow to apply, rendering the device inoperable for the duration. To solve this problem, in Android Nougat we launched a mechanism called "A/B OTA" (aka Seamless Updates). A/B OTAs have the advantage of appearing to be near-instant from the user's perspective, since they apply in the background and become active on the next reboot. However, they doubled the amount of storage reserved for the OS itself, limiting adoption among OEMs.

We’ve developed a new OTA mechanism – Virtual A/B – that combines the benefits of the previous two: being seamless from the user's perspective while requiring less storage. We are working closely with our OEM partners to begin implementing Virtual A/B in Android 11 devices, making OTAs as frictionless as possible. Going forward, Virtual A/B will be the only supported OTA mechanism in Android.

Looking to the Future

We’re excited by the increased adoption of Android and are grateful for the close collaborations with our chipset and OEM partners to deploy updates earlier. We continue to work on a number of enhancements in the platform and infrastructure to make it easier for developers and users to benefit from the latest versions of Android.

Reminder that the Android engineering team will host a Reddit AMA today at 12:00PM PST to answer your technical questions about Android 11. See this post for details and to submit your questions.

An Update on Project Treble

Posted by Iliyan Malchev, Project Treble Architect

Last week at the 2018 Android Dev Summit, we demonstrated the benefits of Project Treble by showing the same Generic System Image (GSI) running on devices from different OEMs. We highlighted the availability of GSI for Android 9 Pie that app developers can use to develop and test their apps with Android 9 on any Treble-compliant device.

Launched with Android Oreo in 2017, Project Treble has enabled OEMs and silicon vendors to develop and deploy Android updates faster than what was previously possible. Since then, we've been working with device manufacturers to define Vendor Interfaces (VINTF) and draw a clear separation between vendor and framework code on Android devices.

Going forward, all devices launching with Android 9 Pie or later will be Treble-compliant and take full advantage of the Treble architecture to deliver faster upgrades. Thanks to Treble, we expect to see more devices from OEMs running Android 9 Pie at the end of 2018 as compared to the number of devices that were running Android Oreo at the end of 2017.

The GSI is built from the latest available AOSP source code, including the latest bug fixes contributed by OEMs. Device manufacturers already use GSI to validate the implementation of the vendor interface on their devices, and Android app developers can now harness the power of the GSI to test their apps across different devices. With GSI, you can test your apps on a pure AOSP version of the latest Android dessert, including the latest features and behavior changes, on any Treble-compliant device that's unlocked for flashing.

We're continuing to work on making GSI even more accessible and useful for app developers. For example, the GSI could enable early access to future Android platform builds that you can run on a Treble-compliant Android 9 device, so you could start app development and validation before the AOSP release.

If you are interested in trying GSI today, check out the documentation for full instructions on how to build GSI yourself and flash it to your Treble-compliant device.

Shut the HAL Up

Posted by Jeff Vander Stoep, Senior Software Engineer, Android Security

Updates are essential for security, but they can be difficult and expensive for device manufacturers. Project Treble is making updates easier by separating the underlying vendor implementation from the core Android framework. This modularization allows platform and vendor-provided components to be updated independently of each other. While easier and faster updates are awesome, Treble's increased modularity is also designed to improve security.

Isolating HALs

A Hardware Abstraction Layer (HAL) provides an interface between device-agnostic code and device-specific hardware implementations. HALs are commonly packaged as shared libraries loaded directly into the process that requires hardware interaction. Security boundaries are enforced at the process level. Therefore, loading the HAL into a process means that the HAL is running in the same security context as the process it's loaded into.

The traditional method of running HALs in-process means that the process needs all the permissions required by each in-process HAL, including direct access to kernel drivers. Likewise, all HALs in a process have access to the same set of permissions as the rest of the process, including permissions required by other in-process HALs. This results in over-privileged processes and HALs that have access to permissions and hardware that they shouldn't.

Figure 1. Traditional method of multiple HALs in one process.

Moving HALs into their own processes better adheres to the principle of least privilege. This provides two distinct advantages:

  1. Each HAL runs in its own sandbox and is permitted access to only the hardware driver it controls and the permissions granted to the process are limited to the permissions required to do its job.
  2. Similarly, the process loses access to hardware drivers and other permissions and capabilities needed by the HALs.
Figure 2. Each HAL runs in its own process.

Moving HALs into their own processes is great for security, but it comes at the cost of increased IPC overhead between the client process and the HAL. Improvements to the binder driver made IPC between HALs and clients practical. Introducing scatter-gather into binder improves the performance of each transaction by removing the need for the serialization/deserialization steps and reducing the number of copy operations performed on data from three down to one. Android O also introduces binder domains to provide separate communication streams for vendor and platform components. Apps and the Android frameworks continue to use /dev/binder, but vendor-provided components now use /dev/vndbinder. Communication between the platform and vendor components must use /dev/hwbinder. Other means of IPC between platform and vendor are disallowed.

Case study: System Server

Many of the services offered to apps by the core Android OS are provided by the system server. As Android has grown, so has system server's responsibilities and permissions, making it an attractive target for an attacker. As part of project Treble, approximately 20 HALs were moved out of system server, including the HALs for sensors, GPS, fingerprint, Wi-Fi, and more. Previously, a compromise in any of those HALs would gain privileged system permissions, but in Android O, permissions are restricted to the subset needed by the specific HAL.

Case study: media frameworks

Efforts to harden the media stack in Android Nougat continued in Android O. In Nougat, mediaserver was split into multiple components to better adhere to the principle of least privilege, with audio hardware access restricted to audioserver, camera hardware access restricted to cameraserver, and so on. In Android O, most direct hardware access has been entirely removed from the media frameworks. For example HALs for audio, camera, and DRM have been moved out of audioserver, cameraserver, and drmserver respectively.

Reducing and isolating the attack surface of the kernel

The Linux kernel is the primary enforcer of the security model on Android. Attempts to escape sandboxing mechanisms often involve attacking the kernel. An analysis of kernel vulnerabilities on Android showed that they overwhelmingly occurred in and were reached through hardware drivers.

De-privileging system server and the media frameworks is important because they interact directly with installed apps. Removing direct access to hardware drivers makes bugs difficult to reach and adds another layer of defense to Android's security model.

Here comes Treble: A modular base for Android

Posted by Iliyan Malchev, Project Treble team lead

On the Android team, we view each dessert release as an opportunity to make Android better for our users and our ecosystem partners. One thing we've consistently heard from our device-maker partners is that updating existing devices to a new version of Android is incredibly time consuming and costly.

With Android O, we've been working very closely with device makers and silicon manufacturers to take steps toward solving this problem, and we're excited to give you a sneak peek at Project Treble, the biggest change to the low-level system architecture of Android to date.

Life of an Android Release

First, it's helpful to understand the "life of an Android release". There are several steps a new Android release goes through before getting into the hands of users:

  1. The Android team publishes the open-source code for the latest release to the world.
  2. Silicon manufacturers, the companies that make the chips that power Android devices, modify the new release for their specific hardware.
  3. Silicon manufacturers pass the modified new release to device makers -- the companies that design and manufacture Android devices. Device makers modify the new release again as needed for their devices.
  4. Device makers work with carriers to test and certify the new release.
  5. Device makers and carriers make the new release available to users.

    With Project Treble, we're re-architecting Android to make it easier, faster and less costly for manufacturers to update devices to a new version of Android.

    The Vendor Interface

    Android was unveiled in 2007 as a free, open-source mobile operating system. From the beginning, we intended Android to be scaled across a variety of manufacturers. We knew that consistency of API was important for developers, so we created a compatibility program for the Developer API specified by the Compatibility Definition Document (CDD) and its associated Compatibility Test Suite (CTS), now comprising over a million tests.

    The result today is that app developers can write a single app that works across over a billion devices running on different hardware from different manufacturers.

    Project Treble aims to do what CTS did for apps, for the Android OS framework. The core concept is to separate the vendor implementation - the device-specific, lower-level software written in large part by the silicon manufacturers - from the Android OS Framework. This is achieved by the introduction of a new vendor interface between the Android OS framework and the vendor implementation. The new vendor interface is validated by a Vendor Test Suite (VTS), analogous to the CTS, to ensure forward compatibility of the vendor implementation.

    Benefits of Project Treble

    Today, with no formal vendor interface, a lot of code across Android needs to be updated when a device moves to a newer version of Android:

    With a stable vendor interface providing access to the hardware-specific parts of Android, device makers can choose to deliver a new Android release to consumers by just updating the Android OS framework without any additional work required from the silicon manufacturers:

    Project Treble will be coming to all new devices launched with Android O and beyond. In fact, the new Project Treble architecture is already running on the Developer Preview of O for Pixel phones.

    In addition to the architectural changes, we're working with our silicon and device partners to take their code changes, such as features for a carrier network in a specific country, and move them into the common Android Open Source Project (AOSP) codebase. For example, Sony and Qualcomm contributed dozens of features and hundreds of bugfixes to Android O so they no longer need to rework these patches with each new release of Android.

    We plan to publish the full documentation for Project Treble on source.android.com with the launch of O later this summer.