Tag Archives: privacy

Fresher OS with Projects Treble and Mainline

Posted by Anwar Ghuloum, Engineering Director and Maya Ben Ari, Product Manager, Android

With each new OS release, we are making efforts to deliver the latest OS improvements to more Android devices.

Thanks to Project Treble and our continuous collaboration with silicon manufacturers and OEM partners, we have improved the overall quality of the ecosystem and accelerated Android 9 Pie OS adoption by 2.5x compared to Android Oreo. Moreover, Android security updates continue to reach more users, with an 84% increase in devices receiving security updates in Q4, when compared to a year before.

This year, we have increased our overall beta program reach to 15 devices, in addition to Pixel, Pixel 2 and Pixel 3/3a running Android Q beta: Huawei Mate 20 Pro, LGE G8, Sony Xperia XZ3, OPPO Reno, Vivo X27, Vivo NEX S, Vivo NEX A, OnePlus 6T, Xiaomi Mi Mix 3 5G, Xiaomi Mi 9, Realme 3 Pro, Asus Zenfone 5z, Nokia 8.1, Tecno Spark 3 Pro, and Essential PH-1.

But our work hasn’t stopped there. We are continuing to invest in efforts to make Android updates available across the ecosystem.

Safer and more secure devices with Project Mainline

Project Mainline builds on our investment in Treble to simplify and expedite how we deliver updates to the Android ecosystem. Project Mainline enables us to update core OS components in a way that's similar to the way we update apps: through Google Play. With this approach we can deliver selected AOSP components faster, and for a longer period of time – without needing a full OTA update from your phone manufacturer. Mainline components are still open sourced. We are closely collaborating with our partners for code contribution and for testing, e.g., for the initial set of Mainline components our partners contributed many changes and collaborated with us to ensure they ran well on their devices.

Project Mainline updates via Google Play infrastructure components in the Android OS Framework. The Framework components updated are located above the Treble Interface and Hardware-specific implementation, and below the Apps layer.

As a result, we can accelerate the delivery of security fixes, privacy enhancements, and consistency improvements across the ecosystem.

Project Mainline has security, privacy and consistency benefits. Security: Accelerate pushes and remove OEM dependency for critical security bugs. Privacy: Better protection for user’s data; increased privacy standards. Consistency: Device stability and compatibility; developer consistency.

Security: With Project Mainline, we can deliver faster security fixes for critical security bugs. For example, by modularizing media components, which accounted for nearly 40% of recently patched vulnerabilities, and by allowing us to update Conscrypt, the Java Security Provider, Project Mainline will make your device safer.

Privacy: Privacy has been a major focus for us, and we are putting a lot of effort into better protecting users’ data and increasing privacy standards. With Project Mainline, we have the ability to make improvements to our permissions systems to safeguard user data.

Consistency: Project Mainline helps us quickly address issues affecting device stability, compatibility, and developer consistency. We are standardizing time-zone data across devices. Also, we are delivering a new OpenGL driver implementation, ANGLE, designed to help decrease device-specific issues encountered by game developers.

Our initial set of components supported on devices launching on Android Q:

  • Security: Media Codecs, Media Framework Components, DNS Resolver, Conscrypt
  • Privacy: Documents UI, Permission Controller, ExtServices
  • Consistency: Timezone data, ANGLE (developers opt-in), Module Metadata, Networking components, Captive Portal Login, Network Permission Configuration

How does this work?

Mainline components are delivered as either APK or APEX files. APEX is a new file format we developed, similar to APK but with the fundamental difference that APEX is loaded much earlier in the booting process. As a result, important security and performance improvements that previously needed to be part of full OS updates can be downloaded and installed as easily as an app update. To ensure updates are delivered safely, we also built new failsafe mechanisms and enhanced test processes. We are also closely collaborating with our partners to ensure devices are thoroughly tested.

APEX file format. At the top level, an APEX file is a zip file in which files are stored uncompressed. The four files in an APEX file are: apex_manifest.json, AndroidManifest.xml, 
Apex_payload.img, apex_pubkey

Project Mainline enables us to keep the OS on devices fresher, improve consistency, and bring the latest AOSP code to users faster. Users will get these critical fixes and enhancements without having to take a full operating system update. We look forward to extending the program with our OEM partners through our joint work on mainline AOSP.

What’s New in Android: Q Beta 3 & More

Posted by Dave Burke, VP, Engineering

Today Android is celebrating two amazing milestones. It’s Android’s version 10! And today, Android is running on more than 2.5B active Android devices.

With Android Q, we’ve focused on three themes: innovation, security and privacy, and digital wellbeing. We want to help you take advantage of the latest new technology -- 5G, foldables, edge-to-edge screens, on-device AI, and more -- while making sure users' security, privacy, and wellbeing are always a top priority.

Earlier at Google I/O we highlighted what’s new in Android Q and unveiled the latest update, Android Q Beta 3. Your feedback continues to be extremely valuable in shaping today’s update as well as our final release to the ecosystem in the fall.

This year, Android Q Beta 3 is available on 15 partner devices from 12 OEMs -- that’s twice as many devices as last year! It’s all thanks to Project Treble and especially to our partners who are committed to accelerating updates to Android users globally -- Huawei, Xiaomi, Nokia, Sony, Vivo, OPPO, OnePlus, ASUS, LGE, TECNO, Essential, and realme.

Visit android.com/beta to see the full list of Beta devices and learn how to get today’s update on your device. If you have a Pixel device, you can enroll here to get Beta 3 -- if you’re already enrolled, watch for the update coming soon. To get started developing with Android Q Beta, visit developer.android.com/preview.

Privacy and security

As we talked about at Google I/O, privacy and security are important to our whole company and in Android Q we’ve added many more protections for users.

Privacy

In Android Q, privacy has been a central focus, from strengthening protections in the platform to designing new features with privacy in mind. It’s more important than ever to give users control -- and transparency -- over how information is collected and used by apps, and by our phones.

Building on our work in previous releases, Android Q includes extensive changes across the platform to improve privacy and give users control -- from improved system UI to stricter permissions to restrictions on what data apps can use.

For example, Android Q gives users more control over when apps can get location. Apps still ask the user for permission, but now in Android Q the user has greater choice over when to allow access to location -- such as only while the app is in use, all the time, or never. Read the developer guide for details on how to adapt your app for the new location controls.

Outside of location, we also introduced the Scoped Storage feature to give users control over files and prevent apps from accessing sensitive user or app data. Your feedback has helped us refine this feature, and we recently announced several changes to make it easier to support. These are now available in Beta 3.

Another important change is restricting app launches from the background, which prevents apps from unexpectedly jumping into the foreground and taking over focus. In Beta 3 we’re transitioning from toast warnings to actually blocking these launches.

To prevent tracking we're limiting access to non-resettable device identifiers, including device IMEI, serial number, and similar identifiers. Read the best practices to choose the right identifiers for your use case. We're also randomizing MAC address when your device is connected to different Wi-Fi networks and gating connectivity APIs behind the location permission. We’re bringing these changes to you early, so you can have as much time as possible to prepare your apps.

Security

To keep users secure, we’ve extended our BiometricPrompt authentication framework to support biometrics at a system level. We're extending support for passive authentication methods such as face, and we’ve added implicit and explicit authentication flows. In the explicit flow, the user must explicitly confirm the transaction. The new implicit flow is designed for a lighter-weight alternative for transactions with passive authentication, and there’s no need for users to explicitly confirm.

Android Q also adds support for TLS 1.3, a major revision to the TLS standard that includes performance benefits and enhanced security. Our benchmarks indicate that secure connections can be established as much as 40% faster with TLS 1.3 compared to TLS 1.2. TLS 1.3 is enabled by default for all TLS connections made through Android’s TLS stack, called Conscrypt, regardless of target API level. See the docs for details.

Project Mainline

Today we also announced Project Mainline, a new approach to keeping Android users secure and their devices up-to-date with important code changes, direct from Google Play. With Project Mainline, we’re now able to update specific internal components within the OS itself, without requiring a full system update from your device manufacturer. This means we can help keep the OS code on devices fresher, drive a new level of consistency, and bring the latest AOSP code to users faster -- and for a longer period of time.

We plan to update Project Mainline modules in much the same way as app updates are delivered today -- downloading the latest versions from Google Play in the background and loading them the next time the phone starts up. The source code for the modules will continue to live in the Android Open Source Project, and updates will be fully open-sourced as they are released. Also, because they’re open source, they’ll include improvements and bug fixes contributed by our many partners and developer community worldwide.

For users, the benefits are huge, since their devices will always be running the latest versions of the modules, including the latest updates for security, privacy, and consistency. For device makers, carriers, and enterprises, the benefits are also huge, since they can optimize and secure key parts of the OS without the cost of a full system update.

For app and game developers, we expect Project Mainline to help drive consistency of platform implementation in key areas across devices, over time bringing greater uniformity that will reduce development and testing costs and help to make sure your apps work as expected. All devices running Android Q or later will be able to get Project Mainline, and we’re working closely with our partners to make sure their devices are ready.

Innovation and new experiences

Android is shaping the leading edge of innovation. With our ecosystem partners, we’re enabling new experiences through a combination of hardware and software advances.

Foldables

This year, display technology will take a big leap with foldable devices coming to the Android ecosystem from several top device makers. When folded these devices work like a phone, then you unfold a beautiful tablet-sized screen.

We’ve optimized Android Q to ensure that screen continuity is seamless in these transitions, and apps and games can pick up right where they left off. For multitasking, we’ve made some changes to onResume and onPause to support multi-resume and notify your app when it has focus. We've also changed how the resizeableActivity manifest attribute works, to help you manage how your app is displayed on large screens.

Our partners have already started showing their innovative foldable devices, with more to come. You can get started building and testing today with our foldables emulator in canary release of Android Studio 3.5.

5G networks

5G networks are the next evolution of wireless technology -- providing consistently faster speeds and lower latency. For developers, 5G can unlock new kinds of experiences in your apps and supercharge existing ones.

Android Q adds platform support for 5G and extends existing APIs to help you transform your apps for 5G. You can use connectivity APIs to detect if the device has a high bandwidth connection and check whether the connection is metered. With these your apps and games can tailor rich, immersive experiences to users over 5G.

With Android’s open ecosystem and range of partners, we expect the Android ecosystem to scale to support 5G quickly. This year, over a dozen Android device makers are launching 5G-ready devices, and more than 20 carriers will launch 5G networks around the world, with some already broad-scale.

Live Caption

On top of hardware innovation, we’re continuing to see Android’s AI transforming the OS itself to make it smarter and easier to use, for a wider range of people. A great example is Live Caption, a new feature in Android Q that automatically captions media playing on your phone.

Many people watch videos with captions on -- the captions help them keep up, even when on the go or in a crowded place. But for 466 million Deaf and Hard of Hearing people around the world, captions are more than a convenience -- they make content accessible. We worked with the Deaf community to develop Live Caption.

Live Caption brings real-time captions to media on your phone - videos, podcasts, and audio messages, across any app—even stuff you record yourself. Best of all, it doesn’t even require a network connection -- everything happens on the device, thanks to a breakthrough in speech recognition that we made earlier this year. The live speech models run right on the phone, and no audio stream ever leaves your device.

For developers, Live Caption expands the audience for your apps and games by making digital media more accessible with a single tap. Live Caption will be available later this year.

Suggested actions in notifications

In Android Pie we introduced smart replies for notifications that let users engage with your apps direct from notifications. We provided the APIs to attach replies and actions, but you needed to build those on your own.

Now in Android Q we want to make smart replies available to all apps right now, without you needing to do anything. Starting in Beta 3, we’re enabling system-provided smart replies and actions that are inserted directly into notifications by default.

You can still supply your own replies and actions if you want -- such as if you are using ML Kit or other ML frameworks. Just opt out of the system-provided replies or actions on a per-notification basis using setAllowGeneratedReplies() and setAllowSystemGeneratedContextualActions().

Android Q suggestions are powered by an on-device ML service built into the platform -- the same service that backs our text classifier entity recognition service. We’ve built it with user privacy in mind, and the ML processing happens completely on the device, not on a backend server.

Because suggested actions are based on the TextClassifier service, they can take advantage of new capabilities we’ve added in Android Q, such as language detection. You can also use TextClassifier APIs directly to generate system-provided notifications and actions, and you can mix those with your own replies and actions as needed.

Dark theme

Many users prefer apps that offer a UI with a dark theme they can switch to when light is low, to reduce eye strain and save battery. Users have also asked for a simple way to enable dark theme everywhere across their devices. Dark theme has been a popular request for a while, and in Android Q, it’s finally here.

Starting in Android Q Beta 3, users can activate a new system-wide dark theme by going to Settings > Display, using the new Quick Settings tile, or turning on Battery Saver. This changes the system UI to dark, and enables the dark theme of apps that support it. Apps can build their own dark themes, or they can opt-in to a new Force Dark feature that lets the OS create a dark version of their existing theme. All you have to do is opt-in by setting android:forceDarkAllowed="true" in your app’s current theme.

You may also want to take complete control over your app’s dark styling, which is why we’ve also been hard at work improving AppCompat’s DayNight feature. By using DayNight, apps can offer a dark theme to all of their users, regardless of what version of Android they’re using on their devices. For more information, see here.

Gestural navigation

Many of the latest Android devices feature beautiful edge-to-edge screens, and users want to take advantage of every bit of them. In Android Q we’re introducing a new fully gestural navigation mode that eliminates the navigation bar area and allows apps and games to use the full screen to deliver their content. It retains the familiar Back, Home, and recents navigation through edge swipes rather than visible buttons.

Users can switch to gestures in Settings > System > Gestures. There are currently two gestures: Swiping up from the bottom of the screen takes the user to the Home screen, holding brings up Recents. Swiping from the screen’s left or right edge triggers the Back action.

To blend seamlessly with gestural navigation, apps should go edge-to-edge, drawing behind the navigation bar to create an immersive experience. To implement this, apps should use the setSystemUiVisibility() API to be laid out fullscreen, and then handle WindowInsets as appropriate to ensure that important pieces of UI are not obscured. More information is here.

Digital wellbeing

Digital wellbeing is another theme of our work on Android -- we want to give users the visibility and tools to find balance with the way they use their phones. Last year we launched Digital Wellbeing with Dashboards, App Timers, Flip to Shush, and Wind Down mode. These tools are really helping. App timers helped users stick to their goals over 90% of the time, and users of Wind Down had a 27% drop in nightly usage.

This year we’re continuing to expand our features to help people find balance with digital devices, adding Focus Mode and Family Link.

Focus Mode

Focus Mode is designed for all those times you’re working or studying, and you want to to focus to get something done. With focus mode, you can pick the apps that you think might distract you and silence them - for example, pausing email and the News while leaving maps and text message apps active. You can then use Quick Tiles to turn on Focus Mode any time you want to focus. Under the covers, these apps will be paused - until you come out of Focus Mode! Focus Mode is coming to Android 9 Pie and Android Q devices this Fall.

Family Link

Family Link is a new set of controls to help parents. Starting in Android Q, Family Link will be built right into the Settings on the device. When you set up a new device for your child, Family Link will help you connect it to you. You’ll be able to set daily screen time limits, see the apps where your child is spending time, review any new apps your child wants to install, and even set a device bedtime so your child can disconnect and get to sleep. And now in Android Q you can also set time limits on specific apps… as well as give your kids Bonus Time if you want them to have just 5 more minutes at bedtime. Family Link is coming to Android P and Q devices this Fall. Make sure to check out the other great wellbeing apps in the recent Google Play awards.

Family link lets parents set device bedtime and even give bonus minutes.

Android Foundations

We’re continuing to extend the foundations of Android with more capabilities to help you build new experiences for your users -- here are just a few.

Improved peer-to-peer and internet connectivity

In Android Q we’ve refactored the Wi-Fi stack to improve privacy and performance, and also to improve common use-cases like managing IoT devices and suggesting internet connections -- without requiring the location permission. The network connection APIs make it easier to manage IoT devices over local Wi-Fi, for peer-to-peer functions like configuring, downloading, or printing. The network suggestion APIs let apps surface preferred Wi-Fi networks to the user for internet connectivity.

Wi-Fi performance modes

In Android Q apps can now request adaptive Wi-Fi by enabling high performance and low latency modes. These will be of great benefit where low latency is important to the user experience, such as real-time gaming, active voice calls, and similar use-cases. The platform works with the device firmware to meet the requirement with the lowest power consumption. To use the new performance modes, call WifiManager.WifiLock.createWifiLock().

Full support for Wi-Fi RTT accurate indoor positioning

In Android 9 Pie we introduced RTT APIs for indoor positioning to accurately measure distance to nearby Wi-Fi Access Points (APs) that support the IEEE 802.11mc protocol, based on measuring the round-trip time of Wi-Fi packets. Now in Android Q, we’ve completed our implementation of the 802.11mc standard, adding an API to obtain location information of each AP being ranged, configured by their owner during installation.

Audio playback capture

You saw how Live Caption can take audio from any app and instantly turn it into on-screen captions. It’s a seamless experience that shows how powerful it can be for one app to share its audio stream with another. In Android Q, any app that plays audio can let other apps capture its audio stream using a new API. In addition to enabling captioning and subtitles, the API lets you support popular use-cases like live-streaming games, all without latency impact on the source app or game.

We’ve designed this new capability with privacy and copyright protection in mind, so the ability for an app to capture another app's audio is constrained, giving apps full control over whether their audio streams can be captured. Read more here.

Dynamic depth for photos

Apps can now request a Dynamic Depth image which consists of a JPEG, XMP metadata related to depth related elements, and a depth and confidence map embedded in the same file on devices that advertise support. Requesting a JPEG + Dynamic Depth image makes it possible for you to offer specialized blurs and bokeh options in your app. You can even use the data to create 3D images or support AR photography use-cases. Dynamic Depth is an open format for the ecosystem -- the latest version of the spec is here. We're working with our device-maker partners to make it available across devices running Android Q and later.

With Dynamic Depth image you can offer specialized blurs and bokeh options in your app

New audio and video codecs

Android Q adds support for the open source video codec AV1, which allows media providers to stream high quality video content to Android devices using less bandwidth. In addition, Android Q supports audio encoding using Opus - a codec optimized for speech and music streaming, and HDR10+ for high dynamic range video on devices that support it. The MediaCodecInfo API introduces an easier way to determine the video rendering capabilities of an Android device. For any given codec, you can obtain a list of supported sizes and frame rates.

Vulkan 1.1 and ANGLE

We're continuing to expand the impact of Vulkan on Android, our implementation of the low-overhead, cross-platform API for high-performance 3D graphics. We’re working together with our device manufacturer partners to make Vulkan 1.1 a requirement on all 64-bit devices running Android Q and higher, and a recommendation for all 32-bit devices. For game and graphics developers using OpenGL, we’re also working towards a standard, updateable OpenGL driver for all devices built on Vulkan. In Android Q we're adding experimental support for ANGLE on top of Vulkan on Android devices. See the docs for details.

Neural Networks API 1.2

In NNAPI 1.2 we've added 60 new ops including ARGMAX, ARGMIN, quantized LSTM, alongside a range of performance optimisations. This lays the foundation for accelerating a much greater range of models -- such as those for object detection and image segmentation. We are working with hardware vendors and popular machine learning frameworks such as TensorFlow to optimize and roll out support for NNAPI 1.2.

Thermal API

When devices get too warm, they may throttle the CPU and/or GPU, and this can affect apps and games in unexpected ways. Now in Android Q, apps and games can use a thermal API to monitor changes on the device and take action to help restore normal temperature. For example, streaming apps can reduce resolution/bit rate or network traffic, a camera app could disable flash or intensive image enhancement, or a game could reduce frame rate or polygon tesselation. Read more here.

ART optimizations

Android Q introduces several improvements to the ART runtime to help your apps start faster, consume less memory, and run smoother -- without requiring any work from you. To help with initial app startup, Google Play is now delivering cloud-based profiles along with APKs. These are anonymized, aggregate ART profiles that let ART pre-compile parts of your app even before it's run. Cloud-based profiles benefit all apps and they're already available to devices running Android P and higher.

We’re also adding Generational Garbage Collection to ART's Concurrent Copying (CC) Garbage Collector. Generational CC collects young-generation objects separately, incurring much lower cost as compared to full-heap GC. It makes garbage collection more efficient in terms of time and CPU, reduces jank, and helps apps run better on lower-end devices.

More Android Q Beta devices, more Treble momentum than ever

In 2017 we launched Project Treble as part of Android Oreo, with a goal of accelerating OS updates. Treble provides a consistent, testable interface between Android and the underlying device code from device makers and silicon manufacturers, which makes porting a new OS version much simpler and more modular.

In 2018 we worked closely with our partners to bring the first OS updates to their Treble devices. The result: last year at Google I/O we had 8 devices from 7 partners joining our Android P Beta program, together with our Pixel and Pixel 2 devices. Fast forward to today -- we’re seeing updates to Android Pie accelerating strongly, with 2.5 times the footprint compared to Android Oreo's at the same time last year.

This year with Android Q we’re seeing even more momentum, and we have 21 devices from 12 top global partners joining us to release Android Q Beta 3 -- in addition all Pixel devices. We’re also providing Q Beta 3 Generic System Images (GSI), a testing environment for other supported Treble devices. All of these offer the same behaviors, APIs, and features -- giving you an incredible variety of devices for testing your apps, and more ways for you to get an early look at Android Q.

You can see the full list of supported partner and Pixel devices at android.com/beta. Try Android Q Beta on your favorite device today and let us know your feedback!

Explore the new features and APIs

When you're ready, dive into Android Q and learn about the new features and APIs you can use in your apps. Take a look at the API diff report for an overview of what's changed in Beta 3, and see the Android Q Beta API reference for details. Visit the Android Q Beta developer site for more resources, including release notes and how to report issues.

To build with Android Q, download the Android Q Beta SDK and tools into Android Studio 3.3 or higher, and follow these instructions to configure your environment. If you want the latest fixes for Android Q related changes, we recommend you use Android Studio 3.5 or higher.

How do I get Beta 3?

It's easy! Just enroll any Pixel device here to get the update over-the-air. If you're already enrolled, you'll receive the update soon, and, no action is needed on your part. Downloadable system images are also available.

You can also get Beta 3 on any of the other devices participating in the Android Q Beta program, from some of our top device maker partners. You can see the full list of supported partner and Pixel devices at android.com/beta. For each device you'll find specs and links to the manufacturer's dedicated site for downloads, support, and to report issues.

For even broader testing on supported devices, you can also get Android GSI images, and if you don’t have a device you can test on the Android Emulator -- just download the latest emulator system images via the SDK Manager in Android Studio.

As always, your input is critical, so please let us know what you think. You can use our hotlists for filing platform issues (including privacy and behavior changes), app compatibility issues, and third-party SDK issues. You've shared great feedback with us so far and we're working to integrate as much of it as possible in the next Beta release.

We're looking forward to seeing your apps on Android Q!

Android Q Scoped Storage: Best Practices and Updates

Posted by Jeff Sharkey, Software Engineer, and Seb Grubb, Product Manager

Application Sandboxing is a core part of Android’s design, isolating apps from each other. In Android Q, taking the same fundamental principle from Application Sandboxing, we introduced Scoped Storage.

Since the Beta 1 release, you’ve given us a lot of valuable feedback on these changes -- thank you for helping shape Android! Because of your feedback, we've evolved the feature during the course of Android Q Beta. In this post, we'll share options for declaring your app’s support for Scoped Storage on Android Q devices, and best practices for questions we've heard from the community.

Updates to help you adopt Scoped Storage

We expect that Scoped Storage should have minimal impact to apps following current storage best practices. However, we also heard from you that Scoped Storage can be an elaborate change for some apps and you could use more time to assess the impact. Being developers ourselves, we understand you may need some additional time to ensure your app’s compatibility with this change. We want to help.

In the upcoming Beta 3 release, apps that target Android 9 Pie (API level 28) or lower will see no change, by default, to how storage works from previous Android versions. As you update your existing app to work with Scoped Storage, you’ll be able to use a new manifest attribute to enable the new behavior for your app on Android Q devices, even if your app is targeting API level 28 or lower.

The implementation details of these changes will be available with the Beta 3 release, but we wanted to share this update with you early, so you can better prepare your app for Android Q devices. Scoped Storage will be required in next year’s major platform release for all apps, independent of target SDK level, so we recommend you add support to your app well in advance. Please continue letting us know your feedback and how we can better align Scoped Storage with your app’s use cases. You can give us input through this survey, or file bugs and feature requests here.

Best practices for common feedback areas

Your feedback is incredibly valuable and has helped us shape these design decisions. We also want to take a moment to share some best practices for common questions we’ve heard:

  • Storing shared media files. For apps that handle files that users expect to be sharable with other apps (such as photos) and be retained after the app has been uninstalled, use the MediaStore API. There are specific collections for common media files: Audio, Video, and Images. For other file types, you can store them in the new Downloads collection. To access files from the Downloads collection, apps must use the system picker.
  • Storing app-internal files. If your app is designed to handle files not meant to be shared with other apps, store them in your package-specific directories. This helps keep files organized and limit file clutter as the OS will manage cleanup when the app is uninstalled. Calls to Context.getExternalFilesDir() will continue to work.
  • Working with permissions and file ownership. For MediaStore, no permissions are necessary for apps that only access their own files. Your app will need to request permission to access media contributed by other apps. However, if your app is uninstalled and then reinstalled later, you’ll need to request permission from the user in order to be able to access media your app previously contributed.
  • Working with native code or libraries. The recommended pattern is to begin your media file discovery in your Java-based or Kotlin-based code, then pass the file's associated file descriptor into your native code.
  • Working with many files efficiently. If you need to perform bulk file operations in a single transaction, consider using ContentProvider.applyBatch(). Learn more about ContentProvider batch processing here.
  • Integrating with the system file picker.
    • Documents apps, such as a word processor, can use the ACTION_OPEN_DOCUMENT or ACTION_GET_CONTENT action to open a system file picker. You can learn more about the differences here.
    • File management apps typically work with collections of apps in a directory hierarchy. Use ACTION_OPEN_DOCUMENT_TREE to let the user pick a directory subtree. The app can further manipulate files available in the returned directory. Through this support, users can access files from any installed DocumentsProvider instance, which can be supported by any cloud-based or locally-backed storage solutions.

We’ve also provided a detailed Scoped Storage developer guide with additional information.

What’s ahead

It’s been amazing to see the community engagement on Android Q Beta so far. As we finalize the release in the next several months, please continue testing and keep the feedback coming. Join us at Google I/O 2019 for more details on Scoped Storage and other Android Q features. We’re giving a ”What’s new on Shared Storage” talk on May 8, and you’ll be able to find the livestream and recorded video on the Google I/O site.

Android Q Beta 2 update

Posted by Dave Burke, VP of Engineering

A few weeks ago we Introduced Android Q Beta, a first look at the next version of Android. Along with new privacy features for users, Android Q adds new capabilities for developers - like enhancements for foldables, new APIs for connectivity, new media codecs and camera capabilities, NNAPI extensions, Vulkan 1.1 graphics, and more.

Android's program of early, open previews is driven by our core philosophy of openness, and collaboration with our community. Your feedback since Beta 1 proves yet again the value of that openness - it's been loud, clear, and incredibly valuable. You've sent us thousands of bug reports, giving us insights and directional feedback, changing our plans in ways that make the platform better for users and developers. We're taking your feedback to heart, so please stay tuned. We're fortunate to have such a passionate community helping to guide Android Q toward the final product later this year.

Today we're releasing Android Q Beta 2 and an updated SDK for developers. It includes the latest bug fixes, optimizations, and API updates for Android Q, along with the April 2019 security patches. You'll also notice isolated storage becoming more prominent as we look for your wider testing and feedback to help us refine that feature.

We're still in early Beta with Android Q so expect rough edges! Before you install, check out the Known Issues. In particular, expect the usual transitional issues with apps that we typically see during early Betas as developers get their app updates ready. For example, you might see issues with apps that access photos, videos, media, or other files stored on your device, such as when browsing or sharing in social media apps.

You can get Beta 2 today by enrolling any Pixel device here. If you're already enrolled, watch for the Beta 2 update coming soon. Stay tuned for more at Google I/O in May.

What's new in Beta 2?

Privacy features for testing and feedback

As we shared at Beta 1, we're making significant privacy investments in Android Q in addition to the work we've done in previous releases. Our goals are improving transparency, giving users more control, and further securing personal data across platform and apps. We know that to reach those goals, we need to partner with you, our app developers. We realize that supporting these features is an investment for you too, so we'll do everything we can to minimize the impact on your apps.

For features like Scoped Storage, we're sharing our plans as early as possible to give you more time to test and give us your input. To generate broader feedback, we've also enabled Scoped Storage for new app installs in Beta 2, so you can more easily see what is affected.

With Scoped Storage, apps can use their private sandbox without permission, but they need new permissions to access shared collections for photos, videos and audio. Apps using files in shared collections -- for example, photo and video galleries and pickers, media browsing, and document storage -- may behave differently under Scoped Storage.

We recommend getting started with Scoped Storage soon -- the developer guide has details on how to handle key use-cases. For testing, make sure to enable Scoped Storage for your app using the adb command. If you discover that your app has a use-case that's not supported by Scoped Storage, please let us know by taking this short survey. We appreciate the great feedback you've given us already, it's a big help as we move forward with the development of this feature.

Bubbles: a new way to multitask

In Android Q we're adding platform support for bubbles, a new way for users to multitask and re-engage with your apps. Various apps have already built similar interactions from the ground up, and we're excited to bring the best from those into the platform, while helping to make interactions consistent, safeguard user privacy, reduce development time, and drive innovation.

Bubbles will let users multitask as they move between activities.

Bubbles help users prioritize information and take action deep within another app, while maintaining their current context. They also let users carry an app's functionality around with them as they move between activities on their device.

Bubbles are great for messaging because they let users keep important conversations within easy reach. They also provide a convenient view over ongoing tasks and updates, like phone calls or arrival times. They can provide quick access to portable UI like notes or translations, and can be visual reminders of tasks too.

We've built bubbles on top of Android's notification system to provide a familiar and easy to use API for developers. To send a bubble through a notification you need to add a BubbleMetadata by calling setBubbleMetadata. Within the metadata you can provide the Activity to display as content within the bubble, along with an icon (disabled in beta 2) and associated person.

We're just getting started with bubbles, but please give them a try and let us know what you think. You can find a sample implementation here.

Foldables emulator

As the ecosystem moves quickly toward foldable devices, new use-cases are opening up for your apps to take advantage of these new screens. With Beta 2, you can build for foldable devices through Android Q enhanced platform support, combined with a new foldable device emulator, available as an Android Virtual device in Android Studio 3.5 available in the canary release channel.

7.3" Foldable AVD switches between the folded and unfolded states

On the platform side, we've made a number of improvements in onResume and onPause to support multi-resume and notify your app when it has focus. We've also changed how the resizeableActivity manifest attribute works, to help you manage how your app is displayed on foldable and large screens. You can read more in the foldables developer guide.

To set up a runtime environment for your app, you can now configure a foldable emulator as a virtual device (AVD) in Android Studio. The foldable AVD is a reference device that lets you test with standard hardware configurations, behaviors, and states, as will be used by our device manufacturer partners. To ensure compatibility, the AVD meets CTS/GTS requirements and models CDD compliance. It supports runtime configuration change, multi-resume and the new resizeableActivity behaviors.

Use the canary release of Android Studio 3.5 to create a foldable virtual device to support either of two hardware configurations 7.3" (4.6" folded) and 8" (6.6" folded) with Beta 2. In each configuration, the emulator gives you on-screen controls to trigger fold/unfold, change orientation, and quick actions.

Android Studio - AVD Manager: Foldable Device Setup

Try your app on the foldable emulator today by downloading the canary release of Android Studio 3.5 and setting up a foldable AVD that uses the Android Q Beta 2 system image.

Improved sharesheet

Following on the initial Sharing Shortcuts APIs in Beta 1, you can now offer a preview of the content being shared by providing an EXTRA_TITLE extra in the Intent for the title, or by setting the Intent's ClipData for a thumbnail image. See the updated sample application for the implementation details.

Directional, zoomable microphones

Android Q Beta 2 gives apps more control over audio capture through a new MicrophoneDirection API. You can use the API to specify a preferred direction of the microphone when taking an audio recording. For example, when the user is taking a "selfie" video, you can request the front-facing microphone for audio recording (if it exists) by calling setMicrophoneDirection(MIC_DIRECTION_FRONT).

Additionally, this API introduces a standardized way of controlling zoomable microphones, allowing your app to have control over the recording field dimension using setMicrophoneFieldDimension(float).

Compatibility through public APIs

In Android Q we're continuing our long-term effort to move apps toward only using public APIs. We introduced most of the new restrictions in Beta 1, and we're making a few minor updates to those lists in Beta 2 to minimize impact on apps. Our goal is to provide public alternative APIs for valid use-cases before restricting access, so if an interface that you currently use in Android 9 Pie is now restricted, you should request a new public API for that interface.

Get started with Android Q Beta

Today's update includes Beta 2 system images for all Pixel devices and the Android Emulator, as well updated SDK and tools for developers. These give you everything you need to get started testing your apps on the new platform and build with the latest APIs.

First, make your app compatible and give your users a seamless transition to Android Q, including your users currently participating in the Android Beta program. To get started, just install your current app from Google Play onto a device or emulator running Beta 2 and work through the user flows. The app should run and look great, and handle the Android Q behavior changes for all apps properly. If you find issues, we recommend fixing them in the current app, without changing your targeting level. See the migration guide for steps and a recommended timeline.

With important privacy features that are likely to affect your apps, we recommend getting started with testing right away. In particular, you'll want to test against scoped storage, new location permissions, restrictions on background Activity starts, and restrictions on device identifiers. See the privacy checklist as a starting point.

Next, update your app's targetSdkVersion to 'Q' as soon as possible. This lets you test your app with all of the privacy and security features in Android Q, as well as any other behavior changes for apps targeting Q.

Explore the new features and APIs

When you're ready, dive into Android Q and learn about the new features and APIs you can use in your apps. Here's a video highlighting many of the changes for developers in Beta 1 and Beta 2. Take a look at the API diff report for an overview of what's changed in Beta 2, and see the Android Q Beta API reference for details. Visit the Android Q Beta developer site for more resources, including release notes and how to report issues.

To build with Android Q, download the Android Q Beta SDK and tools into Android Studio 3.3 or higher, and follow these instructions to configure your environment. If you want the latest fixes for Android Q related changes, we recommend you use Android Studio 3.5 or higher.

How do I get Beta 2?

It's easy - you can enroll here to get Android Q Beta updates over-the-air, on any Pixel device (and this year we're supporting all three generations of Pixel -- Pixel 3, Pixel 2, and even the original Pixel!). If you're already enrolled, you'll receive the update to Beta 2 soon, no action is needed on your part. Downloadable system images are also available. If you don't have a Pixel device, you can use the Android Emulator -- just download the latest emulator system images via the SDK Manager in Android Studio.

As always, your input is critical, so please let us know what you think. You can use our hotlists for filing platform issues (including privacy and behavior changes), app compatibility issues, and third-party SDK issues. You've shared great feedback with us so far and we're working to integrate as much of it as possible in the next Beta release.

Giving users more control over their location data

Posted by Jen Chai, Product Manager

Location data can deliver amazing, rich mobile experiences for users on Android such as finding a restaurant nearby, tracking the distance of a run, and getting turn-by-turn directions as you drive. Location is also one of the most sensitive types of personal information for a user. We want to give users simple, easy-to-understand controls for what data they are providing to apps, and yesterday, we announced in Android Q that we are giving users more control over location permissions. We are delighted by the innovative location experiences you provide to users through your apps, and we want to make this transition as straightforward for you as possible. This post dives deeper into the location permission changes in Q, what it may mean for your app, and how to get started with any updates needed.

Previously, a user had a single control to allow or deny an app access to device location, which covered location usage by the app both while it was in use and while it wasn't. Starting in Android Q, users have a new option to give an app access to location only when the app is being used; in other words, when the app is in the foreground. This means users will have a choice of three options for providing location to an app:

  • "All the time" - this means an app can access location at any time
  • "While in use" - this means an app can access location only while the app is being used
  • "Deny" - this means an app cannot access location

Some apps or features within an app may only need location while the app is being used. For example, if a feature allows a user to search for a restaurant nearby, the app only needs to understand the user's location when the user opens the app to search for a restaurant.

However, some apps may need location even when the app is not in use. For example, an app that automatically tracks the mileage you drive for tax filing, without requiring you to interact with the app.

The new location control allows users to decide when device location data is provided to an app and prevents an app from getting location data that it may not need. Users will see this new option in the same permissions dialog that is presented today when an app requests access to location. This permission can also be changed at any time for any app from Settings-> Location-> App permission.

Here's how to get started

We know these updates may impact your apps. We respect our developer community, and our goal is to approach any change like this very carefully. We want to support you as much as we can by (1) releasing developer-impacting features in the first Q Beta to give you as much time as possible to make any updates needed in your apps and (2) providing detailed information in follow-up posts like this one as well as in the developer guides and privacy checklist. Please let us know if there are ways we can make the guides more helpful!

If your app has a feature requiring "all the time" permission, you'll need to add the new ACCESS_BACKGROUND_LOCATION permission to your manifest file when you target Android Q. If your app targets Android 9 (API level 28) or lower, the ACCESS_BACKGROUND_LOCATION permission will be automatically added for you by the system if you request either ACCESS_FINE_LOCATION or ACCESS_COARSE_LOCATION. A user can decide to provide or remove these location permissions at any time through Settings. To maintain a good user experience, design your app to gracefully handle when your app doesn't have background location permission or when it doesn't have any access to location.

Users will also be more likely to grant the location permission if they clearly understand why your app needs it. Consider asking for the location permission from users in context, when the user is turning on or interacting with a feature that requires it, such as when they are searching for something nearby. In addition, only ask for the level of access required for that feature. In other words, don't ask for "all the time" permission if the feature only requires "while in use" permission.

To learn more, read the developer guide on how to handle the new location controls.

Introducing Android Q Beta

Posted by Dave Burke, VP of Engineering

In 2019, mobile innovation is stronger than ever, with new technologies from 5G to edge to edge displays and even foldable screens. Android is right at the center of this innovation cycle, and thanks to the broad ecosystem of partners across billions of devices, Android's helping push the boundaries of hardware and software bringing new experiences and capabilities to users.

As the mobile ecosystem evolves, Android is focused on helping users take advantage of the latest innovations, while making sure users' security and privacy are always a top priority. Building on top of efforts like Google Play Protect and runtime permissions, Android Q brings a number of additional privacy and security features for users, as well as enhancements for foldables, new APIs for connectivity, new media codecs and camera capabilities, NNAPI extensions, Vulkan 1.1 support, faster app startup, and more.

Today we're releasing Beta 1 of Android Q for early adopters and a preview SDK for developers. You can get started with Beta 1 today by enrolling any Pixel device (including the original Pixel and Pixel XL, which we've extended support for by popular demand!) Please let us know what you think! Read on for a taste of what's in Android Q, and we'll see you at Google I/O in May when we'll have even more to share.

Building on top of privacy protections in Android

Android was designed with security and privacy at the center. As Android has matured, we've added a wide range of features to protect users, like file-based encryption, OS controls requiring apps to request permission before accessing sensitive resources, locking down camera/mic background access, lockdown mode, encrypted backups, Google Play Protect (which scans over 50 billion apps a day to identify potentially harmful apps and remove them), and much more. In Android Q, we've made even more enhancements to protect our users. Many of these enhancements are part of our work in Project Strobe.

Giving users more control over location

With Android Q, the OS helps users have more control over when apps can get location. As in prior versions of the OS, apps can only get location once the app has asked you for permission, and you have granted it.

One thing that's particularly sensitive is apps' access to location while the app is not in use (in the background). Android Q enables users to give apps permission to see their location never, only when the app is in use (running), or all the time (when in the background).

For example, an app asking for a user's location for food delivery makes sense and the user may want to grant it the ability to do that. But since the app may not need location outside of when it's currently in use, the user may not want to grant that access. Android Q now offers this greater level of control. Read the developer guide for details on how to adapt your app for this new control. Look for more user-centric improvements to come in upcoming Betas. At the same time, our goal is to be very sensitive to always give developers as much notice and support as possible with these changes.

More privacy protections in Android Q

Beyond changes to location, we're making further updates to ensure transparency, give users control, and secure personal data.

In Android Q, the OS gives users even more control over apps, controlling access to shared files. Users will be able to control apps' access to the Photos and Videos or the Audio collections via new runtime permissions. For Downloads, apps must use the system file picker, which allows the user to decide which Download files the app can access. For developers, there are changes to how your apps can use shared areas on external storage. Make sure to read the Scoped Storage changes for details.

We've also seen that users (and developers!) get upset when an app unexpectedly jumps into the foreground and takes over focus. To reduce these interruptions, Android Q will prevent apps from launching an Activity while in the background. If your app is in the background and needs to get the user's attention quickly -- such as for incoming calls or alarms -- you can use a high-priority notification and provide a full-screen intent. See the documentation for more information.

We're limiting access to non-resettable device identifiers, including device IMEI, serial number, and similar identifiers. Read the best practices to help you choose the right identifiers for your use case, and see the details here. We're also randomizing the device's MAC address when connected to different Wi-Fi networks by default -- a setting that was optional in Android 9 Pie.

We are bringing these changes to you early, so you can have as much time as possible to prepare. We've also worked hard to provide developers detailed information up front, we recommend reviewing the detailed docs on the privacy changes and getting started with testing right away.

New ways to engage users

In Android Q, we're enabling new ways to bring users into your apps and streamlining the experience as they transition from other apps.

Foldables and innovative new screens

Foldable devices have opened up some innovative experiences and use-cases. To help your apps to take advantage of these and other large-screen devices, we've made a number of improvements in Android Q, including changes to onResume and onPause to support multi-resume and notify your app when it has focus. We've also changed how the resizeableActivity manifest attribute works, to help you manage how your app is displayed on foldable and large screens. To you get started building and testing on these new devices, we've been hard at work updating the Android Emulator to support multiple-display type switching -- more details coming soon!

Sharing shortcuts

When a user wants to share content like a photo with someone in another app, the process should be fast. In Android Q we're making this quicker and easier with Sharing Shortcuts, which let users jump directly into another app to share content. Developers can publish share targets that launch a specific activity in their apps with content attached, and these are shown to users in the share UI. Because they're published in advance, the share UI can load instantly when launched.

The Sharing Shortcuts mechanism is similar to how App Shortcuts works, so we've expanded the ShortcutInfo API to make the integration of both features easier. This new API is also supported in the new ShareTarget AndroidX library. This allows apps to use the new functionality, while allowing pre-Q devices to work using Direct Share. You can find an early sample app with source code here.

Settings Panels

You can now also show key system settings directly in the context of your app, through a new Settings Panel API, which takes advantage of the Slices feature that we introduced in Android 9 Pie.

A settings panel is a floating UI that you invoke from your app to show system settings that users might need, such as internet connectivity, NFC, and audio volume. For example, a browser could display a panel with connectivity settings like Airplane Mode, Wi-Fi (including nearby networks), and Mobile Data. There's no need to leave the app; users can manage settings as needed from the panel. To display a settings panel, just fire an intent with one of the new Settings.Panel actions.

Connectivity

In Android Q, we've extended what your apps can do with Android's connectivity stack and added new connectivity APIs.

Connectivity permissions, privacy, and security

Most of our APIs for scanning networks already require COARSE location permission, but in Android Q, for Bluetooth, Cellular and Wi-Fi, we're increasing the protection around those APIs by requiring the FINE location permission instead. If your app only needs to make peer-to-peer connections or suggest networks, check out the improved Wi-Fi APIs below -- they simplify connections and do not require location permission.

In addition to the randomized MAC addresses that Android Q provides when connected to different Wi-Fi networks, we're adding new Wi-Fi standard support, WP3 and OWE, to improve security for home and work networks as well as open/public networks.

Improved peer-to-peer and internet connectivity

In Android Q we refactored the Wi-Fi stack to improve privacy and performance, but also to improve common use-cases like managing IoT devices and suggesting internet connections -- without requiring the location permission.

The network connection APIs make it easier to manage IoT devices over local Wi-Fi, for peer-to-peer functions like configuring, downloading, or printing. Apps initiate connection requests indirectly by specifying preferred SSIDs & BSSIDs as WiFiNetworkSpecifiers. The platform handles the Wi-Fi scanning itself and displays matching networks in a Wi-Fi Picker. When the user chooses, the platform sets up the connection automatically.

The network suggestion APIs let apps surface preferred Wi-Fi networks to the user for internet connectivity. Apps initiate connections indirectly by providing a ranked list of networks and credentials as WifiNetworkSuggestions. The platform will seamlessly connect based on past performance when in range of those networks.

Wi-Fi performance mode

You can now request adaptive Wi-Fi in Android Q by enabling high performance and low latency modes. These will be of great benefit where low latency is important to the user experience, such as real-time gaming, active voice calls, and similar use-cases.

To use the new performance modes, call WifiManager.WifiLock.createWifiLock() with WIFI_MODE_FULL_LOW_LATENCY or WIFI_MODE_FULL_HIGH_PERF. In these modes, the platform works with the device firmware to meet the requirement with lowest power consumption.

Camera, media, graphics

Dynamic depth format for photos

Many cameras on mobile devices can simulate narrow depth of field by blurring the foreground or background relative to the subject. They capture depth metadata for various points in the image and apply a static blur to the image, after which they discard the depth metadata.

Starting in Android Q, apps can request a Dynamic Depth image which consists of a JPEG, XMP metadata related to depth related elements, and a depth and confidence map embedded in the same file on devices that advertise support.

Requesting a JPEG + Dynamic Depth image makes it possible for you to offer specialized blurs and bokeh options in your app. You can even use the data to create 3D images or support AR photography use-cases in the future. We're making Dynamic Depth an open format for the ecosystem, and we're working with our device-maker partners to make it available across devices running Android Q and later.

With Dynamic Depth image you can offer specialized blurs and bokeh options in your app.

New audio and video codecs

Android Q introduces support for the open source video codec AV1. This allows media providers to stream high quality video content to Android devices using less bandwidth. In addition, Android Q supports audio encoding using Opus - a codec optimized for speech and music streaming, and HDR10+ for high dynamic range video on devices that support it.

The MediaCodecInfo API introduces an easier way to determine the video rendering capabilities of an Android device. For any given codec, you can obtain a list of supported sizes and frame rates using VideoCodecCapabilities.getSupportedPerformancePoints(). This allows you to pick the best quality video content to render on any given device.

Native MIDI API

For apps that perform their audio processing in C++, Android Q introduces a native MIDI API to communicate with MIDI devices through the NDK. This API allows MIDI data to be retrieved inside an audio callback using a non-blocking read, enabling low latency processing of MIDI messages. Give it a try with the sample app and source code here.

ANGLE on Vulkan

To enable more consistency for game and graphics developers, we are working towards a standard, updateable OpenGL driver for all devices built on Vulkan. In Android Q we're adding experimental support for ANGLE on top of Vulkan on Android devices. ANGLE is a graphics abstraction layer designed for high-performance OpenGL compatibility across implementations. Through ANGLE, the many apps and games using OpenGL ES can take advantage of the performance and stability of Vulkan and benefit from a consistent, vendor-independent implementation of ES on Android devices. In Android Q, we're planning to support OpenGL ES 2.0, with ES 3.0 next on our roadmap.

We'll expand the implementation with more OpenGL functionality, bug fixes, and performance optimizations. See the docs for details on the current ANGLE support in Android, how to use it, and our plans moving forward. You can start testing with our initial support by opting-in through developer options in Settings. Give it a try today!

Vulkan everywhere

We're continuing to expand the impact of Vulkan on Android, our implementation of the low-overhead, cross-platform API for high-performance 3D graphics. Our goal is to make Vulkan on Android a broadly supported and consistent developer API for graphics. We're working together with our device manufacturer partners to make Vulkan 1.1 a requirement on all 64-bit devices running Android Q and higher, and a recommendation for all 32-bit devices. Going forward, this will help provide a uniform high-performance graphics API for apps and games to use.

Neural Networks API 1.2

Since introducing the Neural Networks API (NNAPI) in 2017, we've continued to expand the number of operations supported and improve existing functionality. In Android Q, we've added 60 new ops including ARGMAX, ARGMIN, quantized LSTM, alongside a range of performance optimisations. This lays the foundation for accelerating a much greater range of models -- such as those for object detection and image segmentation. We are working with hardware vendors and popular machine learning frameworks such as TensorFlow to optimize and roll out support for NNAPI 1.2.

Strengthening Android's Foundations

ART performance

Android Q introduces several new improvements to the ART runtime which help apps start faster and consume less memory, without requiring any work from developers.

Since Android Nougat, ART has offered Profile Guided Optimization (PGO), which speeds app startup over time by identifying and precompiling frequently executed parts of your code. To help with initial app startup, Google Play is now delivering cloud-based profiles along with APKs. These are anonymized, aggregate ART profiles that let ART pre-compile parts of your app even before it's run, giving a significant jump-start to the overall optimization process. Cloud-based profiles benefit all apps and they're already available to devices running Android P and higher.

We're also continuing to make improvements in ART itself. For example, in Android Q we've optimized the Zygote process by starting your app's process earlier and moving it to a security container, so it's ready to launch immediately. We're storing more information in the app's heap image, such as classes, and using threading to load the image faster. We're also adding Generational Garbage Collection to ART's Concurrent Copying (CC) Garbage Collector. Generational CC is more efficient as it collects young-generation objects separately, incurring much lower cost as compared to full-heap GC, while still reclaiming a good amount of space. This makes garbage collection overall more efficient in terms of time and CPU, reducing jank and helping apps run better on lower-end devices.

Security for apps

BiometricPrompt is our unified authentication framework to support biometrics at a system level. In Android Q we're extending support for passive authentication methods such as face, and adding implicit and explicit authentication flows. In the explicit flow, the user must explicitly confirm the transaction in the TEE during the authentication. The implicit flow is designed for a lighter-weight alternative for transactions with passive authentication. We've also improved the fallback for device credentials when needed.

Android Q adds support for TLS 1.3, a major revision to the TLS standard that includes performance benefits and enhanced security. Our benchmarks indicate that secure connections can be established as much as 40% faster with TLS 1.3 compared to TLS 1.2. TLS 1.3 is enabled by default for all TLS connections. See the docs for details.

Compatibility through public APIs

Another thing we all care about is ensuring that apps run smoothly as the OS changes and evolves. Apps using non-SDK APIs risk crashes for users and emergency rollouts for developers. In Android Q we're continuing our long-term effort begun in Android P to move apps toward only using public APIs. We know that moving your app away from non-SDK APIs will take time, so we're giving you advance notice.

In Android Q we're restricting access to more non-SDK interfaces and asking you to use the public equivalents instead. To help you make the transition and prevent your apps from breaking, we're enabling the restrictions only when your app is targeting Android Q. We'll continue adding public alternative APIs based on your requests; in cases where there is no public API that meets your use case, please let us know.

It's important to test your apps for uses of non-SDK interfaces. We recommend using the StrictMode method detectNonSdkApiUsage() to warn when your app accesses non-SDK APIs via reflection or JNI. Even if the APIs are exempted (grey-listed) at this time, it's best to plan for the future and eliminate their use to reduce compatibility issues. For more details on the restrictions in Android Q, see the developer guide.

Modern Android

We're expanding our efforts to have all apps take full advantage of the security and performance features in the latest version of Android. Later this year, Google Play will require you to set your app's targetSdkVersion to 28 (Android 9 Pie) in new apps and updates. In line with these changes, Android Q will warn users with a dialog when they first run an app that targets a platform earlier than API level 23 (Android Marshmallow). Here's a checklist of resources to help you migrate your app.

We're also moving the ecosystem toward readiness for 64-bit devices. Later this year, Google Play will require 64-bit support in all apps. If your app uses native SDKs or libraries, keep in mind that you'll need to provide 64-bit compliant versions of those SDKs or libraries. See the developer guide for details on how to get ready.

Get started with Android Q Beta

With important privacy features that are likely to affect your apps, we recommend getting started with testing right away. In particular, you'll want to enable and test with Android Q storage changes, new location permission states, restrictions on background app launch, and restrictions on device identifiers. See the privacy documentation for details.

To get started, just install your current app from Google Play onto a device or Android Virtual Device running Android Q Beta and work through the user flows. The app should run and look great, and handle the Android Q behavior changes for all apps properly. If you find issues, we recommend fixing them in the current app, without changing your targeting level. Take a look at the migration guide for steps and a recommended timeline.

Next, update your app's targetSdkVersion to 'Q' as soon as possible. This lets you test your app with all of the privacy and security features in Android Q, as well as any other behavior changes for apps targeting Q.

Explore the new features and APIs

When you're ready, dive into Android Q and learn about the new features and APIs you can use in your apps. Take a look at the API diff report, the Android Q Beta API reference, and developer guides as a starting point. Also, on the Android Q Beta developer site, you'll find release notes and support resources for reporting issues.

To build with Android Q, download the Android Q Beta SDK and tools into Android Studio 3.3 or higher, and follow these instructions to configure your environment. If you want the latest fixes for Android Q related changes, we recommend you use Android Studio 3.5 or higher.

How do I get Android Q Beta?

It's easy - you can enroll here to get Android Q Beta updates over-the-air, on any Pixel device (and this year we're supporting all three generations of Pixel -- Pixel 3, Pixel 2, and even the original Pixel!). Downloadable system images for those devices are also available. If you don't have a Pixel device, you can use the Android Emulator, and download the latest emulator system images via the SDK Manager in Android Studio.

We plan to update the preview system images and SDK regularly throughout the preview. We'll have more features to share as the Beta program moves forward.

As always, your feedback is critical, so please let us know what you think — the sooner we hear from you, the more of your feedback we can integrate. When you find issues, please report them here. We have separate hotlists for filing platform issues, app compatibility issues, and third-party SDK issues.

Android Pie à la mode: Security & Privacy

Posted by Vikrant Nanda and René Mayrhofer, Android Security & Privacy Team

There is no better time to talk about Android dessert releases than the holidays because who doesn't love dessert? And what is one of our favorite desserts during the holiday season? Well, pie of course.

In all seriousness, pie is a great analogy because of how the various ingredients turn into multiple layers of goodness: right from the software crust on top to the hardware layer at the bottom. Read on for a summary of security and privacy features introduced in Android Pie this year.

Strengthening Android

Making Android more secure requires a combination of hardening the platform and advancing anti-exploitation techniques.

Platform hardening

With Android Pie, we updated File-Based Encryption to support external storage media (such as, expandable storage cards). We also introduced support for metadata encryption where hardware support is present. With filesystem metadata encryption, a single key present at boot time encrypts whatever content is not encrypted by file-based encryption (such as, directory layouts, file sizes, permissions, and creation/modification times).

Android Pie also introduced a BiometricPrompt API that apps can use to provide biometric authentication dialogs (such as, fingerprint prompt) on a device in a modality-agnostic fashion. This functionality creates a standardized look, feel, and placement for the dialog. This kind of standardization gives users more confidence that they're authenticating against a trusted biometric credential checker.

New protections and test cases for the Application Sandbox help ensure all non-privileged apps targeting Android Pie (and all future releases of Android) run in stronger SELinux sandboxes. By providing per-app cryptographic authentication to the sandbox, this protection improves app separation, prevents overriding safe defaults, and (most significantly) prevents apps from making their data widely accessible.

Anti-exploitation improvements

With Android Pie, we expanded our compiler-based security mitigations, which instrument runtime operations to fail safely when undefined behavior occurs.

Control Flow Integrity (CFI) is a security mechanism that disallows changes to the original control flow graph of compiled code. In Android Pie, it has been enabled by default within the media frameworks and other security-critical components, such as for Near Field Communication (NFC) and Bluetooth protocols. We also implemented support for CFI in the Android common kernel, continuing our efforts to harden the kernel in previous Android releases.

Integer Overflow Sanitization is a security technique used to mitigate memory corruption and information disclosure vulnerabilities caused by integer operations. We've expanded our use of Integer Overflow sanitizers by enabling their use in libraries where complex untrusted input is processed or where security vulnerabilities have been reported.

Continued investment in hardware-backed security

One of the highlights of Android Pie is Android Protected Confirmation, the first major mobile OS API that leverages a hardware-protected user interface (Trusted UI) to perform critical transactions completely outside the main mobile operating system. Developers can use this API to display a trusted UI prompt to the user, requesting approval via a physical protected input (such as, a button on the device). The resulting cryptographically signed statement allows the relying party to reaffirm that the user would like to complete a sensitive transaction through their app.

We also introduced support for a new Keystore type that provides stronger protection for private keys by leveraging tamper-resistant hardware with dedicated CPU, RAM, and flash memory. StrongBox Keymaster is an implementation of the Keymaster hardware abstraction layer (HAL) that resides in a hardware security module. This module is designed and required to have its own processor, secure storage, True Random Number Generator (TRNG), side-channel resistance, and tamper-resistant packaging.

Other Keystore features (as part of Keymaster 4) include Keyguard-bound keys, Secure Key Import, 3DES support, and version binding. Keyguard-bound keys enable use restriction so as to protect sensitive information. Secure Key Import facilitates secure key use while protecting key material from the application or operating system. You can read more about these features in our recent blog post as well as the accompanying release notes.

Enhancing user privacy

User privacy has been boosted with several behavior changes, such as limiting the access background apps have to the camera, microphone, and device sensors. New permission rules and permission groups have been created for phone calls, phone state, and Wi-Fi scans, as well as restrictions around information retrieved from Wi-Fi scans. We have also added associated MAC address randomization, so that a device can use a different network address when connecting to a Wi-Fi network.

On top of that, Android Pie added support for encrypting Android backups with the user's screen lock secret (that is, PIN, pattern, or password). By design, this means that an attacker would not be able to access a user's backed-up application data without specifically knowing their passcode. Auto backup for apps has been enhanced by providing developers a way to specify conditions under which their app's data is excluded from auto backup. For example, Android Pie introduces a new flag to determine whether a user's backup is client-side encrypted.

As part of a larger effort to move all web traffic away from cleartext (unencrypted HTTP) and towards being secured with TLS (HTTPS), we changed the defaults for Network Security Configuration to block all cleartext traffic. We're protecting users with TLS by default, unless you explicitly opt-in to cleartext for specific domains. Android Pie also adds built-in support for DNS over TLS, automatically upgrading DNS queries to TLS if a network's DNS server supports it. This protects information about IP addresses visited from being sniffed or intercepted on the network level.

We believe that the features described in this post advance the security and privacy posture of Android, but you don't have to take our word for it. Year after year our continued efforts are demonstrably resulting in better protection as evidenced by increasing exploit difficulty and independent mobile security ratings. Now go and enjoy some actual pie while we get back to preparing the next Android dessert release!

Acknowledgements: This post leveraged contributions from Chad Brubaker, Janis Danisevskis, Giles Hogben, Troy Kensinger, Ivan Lozano, Vishwath Mohan, Frank Salim, Sami Tolvanen, Lilian Young, and Shawn Willden.

Outline: secure access to the open web

Censorship and surveillance are challenges that many journalists around the world face on a daily basis. Some of them use a virtual private network (VPN) to provide safer access to the open internet, but not all VPNs are equally reliable and trustworthy, and even fewer are open source.

That’s why Jigsaw created Outline, a new open source, independently audited platform that lets any organization easily create and operate their own VPN.

Outline’s most striking feature is arguably how easy it is to use. An organization starts by downloading the Outline Manager app, which lets them sign in to DigitalOcean, where they can host their own VPN, and set it up with just a few clicks. They can also easily use other cloud providers, provided they have shell access to run the installation script. Once an Outline server is set up, the server administrator can create access credentials and share with their network of contacts, who can then use the Outline clients to connect to it.


A core element to any VPN’s security is the protocol that the server and clients use to communicate. When we looked at the existing protocols, we realized that many of them were easily identifiable by network adversaries looking to spot and block VPN traffic. To make Outline more resilient against this threat, we chose Shadowsocks, a secure, handshake-less, and open source protocol that is known for its strength and performance, and enjoys the support of many developers worldwide. Shadowsocks is a combination of a simplified SOCKS5-like routing protocol, running on top of an encrypted channel. We chose the AEAD_CHACHA20_POLY1305 cipher, which is an IETF standard and provides the security and performance users need.

Another important component to security is running up-to-date software. We package the server code as a Docker image, enabling us to run on multiple platforms, and allowing for automatic updates using Watchtower. On DigitalOcean installations, we also enable automatic security updates on the host machine.

If security is one of the most critical parts of creating a better VPN, usability is the other. We wanted Outline to offer a consistent, simple user experience across platforms, and for it to be easy for developers around the world to contribute to it. With that in mind, we use the cross-platform development framework Apache Cordova for Android, iOS, macOS and ChromeOS, and Electron for Windows. The application logic is a web application written in TypeScript, while the networking code had to be written in native code for each platform. This setup allows us to reutilize most of code, and create consistent user experiences across diverse platforms.

In order to encourage a robust developer community we wanted to strike a balance between simplicity, reproducibility, and automation of future contributions. To that end, we use Travis for continuous builds and to generate the binaries that are ultimately uploaded to the app stores. Thanks to its cross-platform support, any team member can produce a macOS or Windows binary with a single click. We also use Docker to package the build tools for client platforms, and thanks to Electron, developers familiar with the server's Node.js code base can also contribute to the Outline Manager application.

You can find our code in the Outline GitHub repositories and more information on the Outline website. We hope that more developers join the project to build technology that helps people connect to the open web and stay more safe online.

By Vinicius Fortuna, Jigsaw

Double Stuffed Security in Android Oreo

Posted by Gian G Spicuzza, Android Security team

Android Oreo is stuffed full of security enhancements. Over the past few months, we've covered how we've improved the security of the Android platform and its applications: from making it safer to get apps, dropping insecure network protocols, providing more user control over identifiers, hardening the kernel, making Android easier to update, all the way to doubling the Android Security Rewards payouts. Now that Oreo is out the door, let's take a look at all the goodness inside.

Expanding support for hardware security

Android already supports Verified Boot, which is designed to prevent devices from booting up with software that has been tampered with. In Android Oreo, we added a reference implementation for Verified Boot running with Project Treble, called Android Verified Boot 2.0 (AVB). AVB has a couple of cool features to make updates easier and more secure, such as a common footer format and rollback protection. Rollback protection is designed to prevent a device to boot if downgraded to an older OS version, which could be vulnerable to an exploit. To do this, the devices save the OS version using either special hardware or by having the Trusted Execution Environment (TEE) sign the data. Pixel 2 and Pixel 2 XL come with this protection and we recommend all device manufacturers add this feature to their new devices.

Oreo also includes the new OEM Lock Hardware Abstraction Layer (HAL) that gives device manufacturers more flexibility for how they protect whether a device is locked, unlocked, or unlockable. For example, the new Pixel phones use this HAL to pass commands to the bootloader. The bootloader analyzes these commands the next time the device boots and determines if changes to the locks, which are securely stored in Replay Protected Memory Block (RPMB), should happen. If your device is stolen, these safeguards are designed to prevent your device from being reset and to keep your data secure. This new HAL even supports moving the lock state to dedicated hardware.

Speaking of hardware, we've invested support in tamper-resistant hardware, such as the security module found in every Pixel 2 and Pixel 2 XL. This physical chip prevents many software and hardware attacks and is also resistant to physical penetration attacks. The security module prevents deriving the encryption key without the device's passcode and limits the rate of unlock attempts, which makes many attacks infeasible due to time restrictions.

While the new Pixel devices have the special security module, all new GMS devices shipping with Android Oreo are required to implement key attestation. This provides a mechanism for strongly attesting IDs such as hardware identifiers.

We added new features for enterprise-managed devices as well. In work profiles, encryption keys are now ejected from RAM when the profile is off or when your company's admin remotely locks the profile. This helps secure enterprise data at rest.

Platform hardening and process isolation

As part of Project Treble, the Android framework was re-architected to make updates easier and less costly for device manufacturers. This separation of platform and vendor-code was also designed to improve security. Following the principle of least privilege, these HALs run in their own sandbox and only have access to the drivers and permissions that are absolutely necessary.

Continuing with the media stack hardening in Android Nougat, most direct hardware access has been removed from the media frameworks in Oreo resulting in better isolation. Furthermore, we've enabled Control Flow Integrity (CFI) across all media components. Most vulnerabilities today are exploited by subverting the normal control flow of an application, instead changing them to perform arbitrary malicious activities with all the privileges of the exploited application. CFI is a robust security mechanism that disallows arbitrary changes to the original control flow graph of a compiled binary, making it significantly harder to perform such attacks.

In addition to these architecture changes and CFI, Android Oreo comes with a feast of other tasty platform security enhancements:

  • Seccomp filtering: makes some unused syscalls unavailable to apps so that they can't be exploited by potentially harmful apps.
  • Hardened usercopy: A recent survey of security bugs on Android revealed that invalid or missing bounds checking was seen in approximately 45% of kernel vulnerabilities. We've backported a bounds checking feature to Android kernels 3.18 and above, which makes exploitation harder while also helping developers spot issues and fix bugs in their code.
  • Privileged Access Never (PAN) emulation: Also backported to 3.18 kernels and above, this feature prohibits the kernel from accessing user space directly and ensures developers utilize the hardened functions to access user space.
  • Kernel Address Space Layout Randomization (KASLR): Although Android has supported userspace Address Space Layout Randomization (ASLR) for years, we've backported KASLR to help mitigate vulnerabilities on Android kernels 4.4 and newer. KASLR works by randomizing the location where kernel code is loaded on each boot, making code reuse attacks probabilistic and therefore more difficult to carry out, especially remotely.

App security and device identifier changes

Android Instant Apps run in a restricted sandbox which limits permissions and capabilities such as reading the on-device app list or transmitting cleartext traffic. Although introduced during the Android Oreo release, Instant Apps supports devices running Android Lollipop and later.

In order to handle untrusted content more safely, we've isolated WebView by splitting the rendering engine into a separate process and running it within an isolated sandbox that restricts its resources. WebView also supports Safe Browsing to protect against potentially dangerous sites.

Lastly, we've made significant changes to device identifiers to give users more control, including:

  • Moving the static Android ID and Widevine values to an app-specific value, which helps limit the use of device-scoped non-resettable IDs.
  • In accordance with IETF RFC 7844 anonymity profile, net.hostname is now empty and the DHCP client no longer sends a hostname.
  • For apps that require a device ID, we've built a Build.getSerial() API and protected it behind a permission.
  • Alongside security researchers1, we designed a robust MAC address randomization for Wi-Fi scan traffic in various chipsets firmware.

Android Oreo brings in all of these improvements, and many more. As always, we appreciate feedback and welcome suggestions for how we can improve Android. Contact us at security@android.com.

_____________________________________________________________________

1: Glenn Wilkinson and team at Sensepost, UK, Célestin Matte, Mathieu Cunche: University of Lyon, INSA-Lyon, CITI Lab, Inria Privatics, Mathy Vanhoef, KU Leuven

Identifying Intrusive Mobile Apps using Peer Group Analysis

Posted by Martin Pelikan, Giles Hogben, and Ulfar Erlingsson of Google's Security and Privacy team

Mobile apps entertain and assist us, make it easy to communicate with friends and family, and provide tools ranging from maps to electronic wallets. But these apps could also seek more device information than they need to do their job, such as personal data and sensor data from components, like cameras and GPS trackers.

To protect our users and help developers navigate this complex environment, Google analyzes privacy and security signals for each app in Google Play. We then compare that app to other apps with similar features, known as functional peers. Creating peer groups allows us to calibrate our estimates of users' expectations and set adequate boundaries of behaviors that may be considered unsafe or intrusive. This process helps detect apps that collect or send sensitive data without a clear need, and makes it easier for users to find apps that provide the right functionality and respect their privacy. For example, most coloring book apps don't need to know a user's precise location to function and this can be established by analyzing other coloring book apps. By contrast, mapping and navigation apps need to know a user's location, and often require GPS sensor access.

One way to create app peer groups is to create a fixed set of categories and then assign each app into one or more categories, such as tools, productivity, and games. However, fixed categories are too coarse and inflexible to capture and track the many distinctions in the rapidly changing set of mobile apps. Manual curation and maintenance of such categories is also a tedious and error-prone task.

To address this, Google developed a machine-learning algorithm for clustering mobile apps with similar capabilities. Our approach uses deep learning of vector embeddings to identify peer groups of apps with similar functionality, using app metadata, such as text descriptions, and user metrics, such as installs. Then peer groups are used to identify anomalous, potentially harmful signals related to privacy and security, from each app's requested permissions and its observed behaviors. The correlation between different peer groups and their security signals helps different teams at Google decide which apps to promote and determine which apps deserve a more careful look by our security and privacy experts. We also use the result to help app developers improve the privacy and security of their apps.

Apps are split into groups of similar functionality, and in each cluster of similar apps the established baseline is used to find anomalous privacy and security signals.

These techniques build upon earlier ideas, such as using peer groups to analyze privacy-related signals, deep learning for language models to make those peer groups better, and automated data analysis to draw conclusions.

Many teams across Google collaborated to create this algorithm and the surrounding process. Thanks to several, essential team members including Andrew Ahn, Vikas Arora, Hongji Bao, Jun Hong, Nwokedi Idika, Iulia Ion, Suman Jana, Daehwan Kim, Kenny Lim, Jiahui Liu, Sai Teja Peddinti, Sebastian Porst, Gowdy Rajappan, Aaron Rothman, Monir Sharif, Sooel Son, Michael Vrable, and Qiang Yan.

For more information on Google's efforts to detect and fight potentially harmful apps (PHAs) on Android, see Google Android Security Team's Classifications for Potentially Harmful Applications.

References

S. Jana, Ú. Erlingsson, I. Ion (2015). Apples and Oranges: Detecting Least-Privilege Violators with Peer Group Analysis. arXiv:1510.07308 [cs.CR].

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean (2013). Distributed Representations of Words and Phrases and their Compositionality. Advances in Neural Information Processing Systems 26 (NIPS 2013).

Ú. Erlingsson (2016). Data-driven software security: Models and methods. Proceedings of the 29th IEEE Computer Security Foundations Symposium (CSF'16), Lisboa, Portugal.