Tag Archives: latest

Bringing modern storage to Viber’s users

This blogpost is a collaboration between Google and Viber. Authored by Kseniia Shumelchyk from Google and Anton Novikov, Sergey Kozlov from Viber.

As a messaging app, Viber needs to store, process and share a significant amount of data. Viber aims to give its users an easy, fast, reliable and secure communication platform by providing an intuitive interface and operating with files in a privacy-preserving way. We believe the modern scoped storage paradigm provides this foundation for app developers and users.

Scoped storage was introduced in Android 10 with further improvements in Android 11 to provide better protection to app and user data on a platform level. Due to Viber's complexity, the team opted to incrementally implement the changes that were required to comply with scoped storage.

In this article, we’ll share how Viber handled the migration to scoped storage, focusing on what they did to optimize working with media files and other data in the app.

Managing storage across Android versions

Android’s storage model has evolved to adapt to changing privacy considerations, leading to the changes in the storage system APIs. Let’s take a look at key platform changes that affected the legacy Viber implementation.

Media directories

Scoped storage changes the way that apps store and access files on a device's external storage. Viber needed to evaluate the differences between the existing app's storage model and updated platform guidelines, followed by gradual application changes to work with files in scoped storage. Therefore Viber invoked the requestLegacyExternalStorage flag to temporarily opt-out of scoped storage on Android 10 until the app was fully compatible.

In order to adjust their app experience to scoped storage, Viber now contributes public media files to well-defined media collections using the MediaStore API. This way, the files are accessible in a device gallery, and can be read by other apps with the storage permission. Private media files are stored in the app-specific directory on external storage and are accessed via the internal ContentProvider.

Storage permissions

The other notable update is related to changes in the storage permissions model: Apps in scoped storage have unrestricted access to their app-specific directories on external storage and can contribute to well-defined media collections without requesting a runtime permission. This change will help Viber provide more granular control to their users:

“This addition supports our efforts to provide our users with the best security and privacy solutions we can provide supported by the Android OS, users will benefit from this added security later without needing to opt-in. We also added a new ‘Save to gallery’ option allowing users to choose to make their photos readable by other apps or not. Because chats may contain private images or videos, it’s important to give users the ability to hide these files from the gallery. This change gives users additional control over the content included in their Viber messages.“ said Anton Novikov and Sergey Kozlov from Viber.

Accessing files outside of app-specific directory

Previously, Viber created and consumed files in a custom top level directory and depended on file path access. With scoped storage, saving app files to a top level directory became an anti-pattern, so Viber has followed best practices to update their implementation to store media files from the chats only in locations that are accessible in scoped storage.

However, to reduce the complexity of migration, Viber decided to keep their own top level directory for Android 10 and below, storing only the media files that are not exposed to the device’s Gallery app, while for Android 11 and above this directory is used in read-only mode to provide backward compatibility.

Another use case that Viber has been refining is sharing files in the chats. The updated storage runtime permission gives read access only to the images, videos and audio files that are available through MediaProvider. Starting from Android 11, the only way for Viber to access non-media files created by other apps is by using the Storage Access Framework document picker, which they had already utilized in a different part of their app.

App-specific files within external storage

In the scoped storage environment, app-specific directories on external storage are becoming private from other apps. This change has helped Viber leverage its use of external storage for storing private user files:

”We find change to app-specific directories to be useful, because it will help to ensure that personal chats are protected and backed with platform security.” said Anton Novikov from Viber. Learn more about how to access app-specific files.

Single interface to access storage

Because Viber targets a large audience running on Android 4.2 and above, they introduced an abstraction layer that aids them in managing storage access efficiently across all supported Android versions and with their use cases in mind.

Previously, Viber heavily used File API to access files, including files in legacy storage locations. Further, they stored absolute file paths for entries in the local database to keep the user's conversation history.

To standardize access to this conversation history and thus ensure that users don’t lose access to their files, Viber replaced absolute file paths with content URIs. In the new implementation, the app is accessing files only via content providers:

  • Internal FileProvider for Viber app-specific directories.
  • External file providers available in the Android framework, such as MediaStore or Storage Access Framework, or those belong to another app that shares files with Viber through Intent.ACTION_SEND.

By using a consistent ContentProvider layer, the ContentResolver gives the app a unified interface to access the file content.

This approach has also helped Viber optimize the network layer and define a universal Loader abstraction to upload/fetch and to read/store different types of media files like voice messages, chat images and stickers.

Summary

Android 11 further enhances scoped storage, which provides better protection of app and user data and makes the transition easier for developers. It’s amazing to see many apps like Viber are migrating to take advantage of scoped storage since Android 10.

We hope Viber’s story is useful and will inspire you to modernize your Android apps as well. Learn more about Android storage use cases and best practices.

11 Weeks of Android: Privacy and Security

Posted by:
Charmaine D’Silva, Product Lead, Android Privacy and Framework
Narayan Kamath, Engineering Lead, Android Privacy and Framework
Stephan Somogyi, Product Lead, Android Security
Sudhi Herle, Engineering Lead, Android Security

This blog post is part of a weekly series for #11WeeksOfAndroid. For each #11WeeksOfAndroid, we’re diving into a key area so you don’t miss anything. This week, we spotlighted Privacy and Security; here’s a look at what you should know.

mobile security illustration

Privacy and security is core to how we design Android, and with every new release we increase our investment in this space. Android 11 continues to make important strides in these areas, and this week we’ll be sharing a series of updates and resources about Android privacy and security. But first, let’s take a quick look at some of the most important changes we’ve made in Android 11 to protect user privacy and make the platform more secure.

As shared in the “All things privacy in Android 11” video, we’re giving users even more control over sensitive permissions. Throughout the development of this release, we have engaged deeply and frequently with our developer community to design these features in a balanced way - amplifying user privacy while minimizing developer impact. Let’s go over some of these features:

One-time permission: In Android 10, we introduced a granular location permission that allows users to limit access to location only when an app is in use (aka foreground only). When presented with the new runtime permissions options, users choose foreground only location more than 50% of the time. This demonstrated to us that users really wanted finer controls for permissions. So in Android 11, we’ve introduced one time permissions that let users give an app access to the device microphone, camera, or location, just that one time. As an app developer, there are no changes that you need to make to your app for it to work with one time permissions, and the app can request permissions again the next time the app is used. Learn more about building privacy-friendly apps with these new changes in this video.

Background location: In Android 10 we added a background location usage reminder so users can see how apps are using this sensitive data on a regular basis. Users who interacted with the reminder either downgraded or denied the location permission over 75% of the time. In addition, we have done extensive research and believe that there are very few legitimate use cases for apps to require access to location in the background.

In Android 11, background location will no longer be a permission that a user can grant via a run time prompt and it will require a more deliberate action. If your app needs background location, the system will ensure that the app first asks for foreground location. The app can then broaden its access to background location through a separate permission request, which will cause the system to take the user to Settings in order to complete the permission grant.

In February, we announced that Google Play developers will need to get approval to access background location in their app to prevent misuse. We're giving developers more time to make changes and won't be enforcing the policy for existing apps until 2021. Check out this helpful video to find possible background location usage in your code.

Permissions auto-reset: Most users tend to download and install over 60 apps on their device but interact with only a third of these apps on a regular basis. If users haven’t used an app that targets Android 11 for an extended period of time, the system will “auto-reset” all of the granted runtime permissions associated with the app and notify the user. The app can request the permissions again the next time the app is used. If you have an app that has a legitimate need to retain permissions, you can prompt users to turn this feature OFF for your app in Settings.

Data access auditing APIs: Android encourages developers to limit their access to sensitive data, even if they have been granted permission to do so. In Android 11, developers will have access to new APIs that will give them more transparency into their app’s usage of private and protected data. The APIs will enable apps to track when the system records the app’s access to private user data.

Scoped Storage: In Android 10, we introduced scoped storage which provides a filtered view into external storage, giving access to app-specific files and media collections. This change protects user privacy by limiting broad access to shared storage in many ways including changing the storage permission to only give read access to photos, videos and music and improving app storage attribution. Since Android 10, we’ve incorporated developer feedback and made many improvements to help developers adopt scoped storage, including: updated permission UI to enhance user experience, direct file path access to media to improve compatibility with existing libraries, updated APIs for modifying media, Manage External Storage permission to enable select use cases that need broad files access, and protected external app directories. In Android 11, scoped storage will be mandatory for all apps that target API level 30. Learn more in this video and check out the developer documentation for further details.

Google Play system updates: Google Play system updates were introduced with Android 10 as part of Project Mainline. Their main benefit is to increase the modularity and granularity of platform subsystems within Android so we can update core OS components without needing a full OTA update from your phone manufacturer. Earlier this year, thanks to Project Mainline, we were able to quickly fix a critical vulnerability in the media decoding subsystem. Android 11 adds new modules, and maintains the security properties of existing ones. For example, Conscrypt, which provides cryptographic primitives, maintained its FIPS validation in Android 11 as well.

BiometricPrompt API: Developers can now use the BiometricPrompt API to specify the biometric authenticator strength required by their app to unlock or access sensitive parts of the app. We are planning to add this to the Jetpack Biometric library to allow for backward compatibility and will share further updates on this work as it progresses.

Identity Credential API: This will unlock new use cases such as mobile drivers licences, National ID, and Digital ID. It’s being built by our security team to ensure this information is stored safely, using security hardware to secure and control access to the data, in a way that enhances user privacy as compared to traditional physical documents. We’re working with various government agencies and industry partners to make sure that Android 11 is ready for such digital-first identity experiences.

Thank you for your flexibility and feedback as we continue to build an increasingly more private and secure platform. You can learn about more features in the Android 11 Beta developer site. You can also learn about general best practices related to privacy and security.

Please follow Android Developers on Twitter and Youtube to catch helpful content and materials in this area all this week.

Resources

You can find the entire playlist of #11WeeksOfAndroid video content here, and learn more about each week here. We’ll continue to spotlight new areas each week, so keep an eye out and follow us on Twitter and YouTube. Thanks so much for letting us be a part of this experience with you!

11 Weeks of Android: Privacy and Security

Posted by:
Charmaine D’Silva, Product Lead, Android Privacy and Framework
Narayan Kamath, Engineering Lead, Android Privacy and Framework
Stephan Somogyi, Product Lead, Android Security
Sudhi Herle, Engineering Lead, Android Security

This blog post is part of a weekly series for #11WeeksOfAndroid. For each #11WeeksOfAndroid, we’re diving into a key area so you don’t miss anything. This week, we spotlighted Privacy and Security; here’s a look at what you should know.

mobile security illustration

Privacy and security is core to how we design Android, and with every new release we increase our investment in this space. Android 11 continues to make important strides in these areas, and this week we’ll be sharing a series of updates and resources about Android privacy and security. But first, let’s take a quick look at some of the most important changes we’ve made in Android 11 to protect user privacy and make the platform more secure.

As shared in the “All things privacy in Android 11” video, we’re giving users even more control over sensitive permissions. Throughout the development of this release, we have engaged deeply and frequently with our developer community to design these features in a balanced way - amplifying user privacy while minimizing developer impact. Let’s go over some of these features:

One-time permission: In Android 10, we introduced a granular location permission that allows users to limit access to location only when an app is in use (aka foreground only). When presented with the new runtime permissions options, users choose foreground only location more than 50% of the time. This demonstrated to us that users really wanted finer controls for permissions. So in Android 11, we’ve introduced one time permissions that let users give an app access to the device microphone, camera, or location, just that one time. As an app developer, there are no changes that you need to make to your app for it to work with one time permissions, and the app can request permissions again the next time the app is used. Learn more about building privacy-friendly apps with these new changes in this video.

Background location: In Android 10 we added a background location usage reminder so users can see how apps are using this sensitive data on a regular basis. Users who interacted with the reminder either downgraded or denied the location permission over 75% of the time. In addition, we have done extensive research and believe that there are very few legitimate use cases for apps to require access to location in the background.

In Android 11, background location will no longer be a permission that a user can grant via a run time prompt and it will require a more deliberate action. If your app needs background location, the system will ensure that the app first asks for foreground location. The app can then broaden its access to background location through a separate permission request, which will cause the system to take the user to Settings in order to complete the permission grant.

In February, we announced that Google Play developers will need to get approval to access background location in their app to prevent misuse. We're giving developers more time to make changes and won't be enforcing the policy for existing apps until 2021. Check out this helpful video to find possible background location usage in your code.

Permissions auto-reset: Most users tend to download and install over 60 apps on their device but interact with only a third of these apps on a regular basis. If users haven’t used an app that targets Android 11 for an extended period of time, the system will “auto-reset” all of the granted runtime permissions associated with the app and notify the user. The app can request the permissions again the next time the app is used. If you have an app that has a legitimate need to retain permissions, you can prompt users to turn this feature OFF for your app in Settings.

Data access auditing APIs: Android encourages developers to limit their access to sensitive data, even if they have been granted permission to do so. In Android 11, developers will have access to new APIs that will give them more transparency into their app’s usage of private and protected data. The APIs will enable apps to track when the system records the app’s access to private user data.

Scoped Storage: In Android 10, we introduced scoped storage which provides a filtered view into external storage, giving access to app-specific files and media collections. This change protects user privacy by limiting broad access to shared storage in many ways including changing the storage permission to only give read access to photos, videos and music and improving app storage attribution. Since Android 10, we’ve incorporated developer feedback and made many improvements to help developers adopt scoped storage, including: updated permission UI to enhance user experience, direct file path access to media to improve compatibility with existing libraries, updated APIs for modifying media, Manage External Storage permission to enable select use cases that need broad files access, and protected external app directories. In Android 11, scoped storage will be mandatory for all apps that target API level 30. Learn more in this video and check out the developer documentation for further details.

Google Play system updates: Google Play system updates were introduced with Android 10 as part of Project Mainline. Their main benefit is to increase the modularity and granularity of platform subsystems within Android so we can update core OS components without needing a full OTA update from your phone manufacturer. Earlier this year, thanks to Project Mainline, we were able to quickly fix a critical vulnerability in the media decoding subsystem. Android 11 adds new modules, and maintains the security properties of existing ones. For example, Conscrypt, which provides cryptographic primitives, maintained its FIPS validation in Android 11 as well.

BiometricPrompt API: Developers can now use the BiometricPrompt API to specify the biometric authenticator strength required by their app to unlock or access sensitive parts of the app. We are planning to add this to the Jetpack Biometric library to allow for backward compatibility and will share further updates on this work as it progresses.

Identity Credential API: This will unlock new use cases such as mobile drivers licences, National ID, and Digital ID. It’s being built by our security team to ensure this information is stored safely, using security hardware to secure and control access to the data, in a way that enhances user privacy as compared to traditional physical documents. We’re working with various government agencies and industry partners to make sure that Android 11 is ready for such digital-first identity experiences.

Thank you for your flexibility and feedback as we continue to build an increasingly more private and secure platform. You can learn about more features in the Android 11 Beta developer site. You can also learn about general best practices related to privacy and security.

Please follow Android Developers on Twitter and Youtube to catch helpful content and materials in this area all this week.

Resources

You can find the entire playlist of #11WeeksOfAndroid video content here, and learn more about each week here. We’ll continue to spotlight new areas each week, so keep an eye out and follow us on Twitter and YouTube. Thanks so much for letting us be a part of this experience with you!

Announcing the winners of the #AndroidDevChallenge, powered by on-device machine learning

Posted by Jacob Lehrbaum, Director of Developer Relations, Android

Developers like you have always played an important role in Android innovation. Over 10 years ago, when we first launched the Android SDK, we also announced the Android Developer Challenge to reward model apps and highlight new ways of solving user problems. As Android pushes the boundaries of machine learning, 5G, foldables, and more, developers continue to help shape these new frontiers. To celebrate this work, we revived the challenge in 2019, with a focus on “Helpful Innovation,” powered by on-device machine learning.

We received hundreds of creative projects, and at the end of last year, picked 10 winners who each combined a strong idea and a thirst to bring it to life. Since then, we’ve been working with those winners to help turn their ideas into reality. And today, we’re announcing the 10 winners. Some are still at the beginning of their journey but but their apps are now ready for you to download and try out! !

  • AgroDoc helps farmers diagnose plant disease and make treatment plans. [Navneet Krishna; Kochi, India]
  • AgriFarm helps farmers detect plant diseases and prevent major damage in fruits and vegetables such as tomatoes, corn and potatoes. [Balochisan, Pakistan]
  • Eskke streamlines mobile money management for people in the Congo, letting them transfer money, pay bills, buy subscriptions and essential airtime through SMS. [David Mumbere Kathoh; Goma, Democratic Republic of Congo]
  • Leepi helps students learn hand gestures and symbols for American Sign Language. [Prince Patel; Bengaluru, India]
  • MixPose is a live streaming platform that gives yoga teachers and fitness professionals the opportunity to teach, track alignment, and give feedback in real-time. [Peter Ma; San Francisco, California, USA]
  • Pathfinder could help people with visual impairments navigate complex situations by identifying and calculating the trajectories of objects moving in their path. [Colin Shelton; Addison, Texas, USA]
  • Snore & Cough helps you identify and analyze snoring and coughing, to help provide info to users seeking assistance from a medical professional. [Ethan Fan; Mountain View, California, USA]
  • Stila pairs with a wearable device, like the Fitbit wristband or a device running on Wear OS by Google to monitor and track the body’s stress levels. By monitoring stress levels over time, you have the chance to better understand and manage stress in your life. [Yingdin Wing; Munich, Germany]
  • Trashly makes recycling easier. Just point the on-device camera at an item, and through object detection, the app identifies and classifies plastic and paper cups, bags, bottles, etc. [Elvin Rakhmankulov; Chicago, Illinois, USA]
  • UnoDogs helps owners better support their pet’s wellness, providing customized information and fitness programs. [Chinmany Mishra; New Delhi, India]

Making on-device machine learning more accessible, with ML Kit and TensorFlow Lite

Increasingly, machine learning is becoming a more accessible tool to developers with limited to no background in the technology. In fact, for most of the winners of the Android Developer Challenge, this was their first foray into machine learning. That’s thanks in part to two key offerings from Google, which bring on-device machine learning into reach for millions of developers around the world.

The first is ML Kit. ML Kit brings Google’s on-device machine learning technologies to mobile app developers, so they can build customized and interactive experiences into their apps. This includes tools such as language translation, text recognition, object detection and more. Eskke, for instance, uses offline text recognition and barcode scanning from ML Kit so users can scan the QR code at a mobile money kiosk and quickly withdraw money. And MixPose uses ML Kit's forthcoming Pose detection API to detect each user’s yoga positions and movements, so teachers can provide feedback.

The other Google resource that many of the Android Dev Challenge winners used was TensorFlow Lite. This powerful machine learning framework can help run machine learning models on Android, iOS and IoT devices that would never normally be able to support them. Its set of tools can be used for all kinds of powerful neural network-related applications, from image detection to speech recognition, bringing the latest cutting-edge technology to the devices we carry around with us wherever we go. Trashly, for instance, uses a custom TensorFlow Lite model to report if an object is recyclable and how to recycle it.

Helpful innovation, such as the 10 winning apps in the Android Developer Challenge, has the potential to change the way we access, use, and interpret information, making it available when we need it, where we need it most. By working with these developers focused on helpful innovation, we hope to inspire the next wave of developers to unlock what’s possible with this new technology.

#11WeeksOfAndroid Week 2 Machine Learning with Android logo head

What’s next in Android Machine Learning week?

As we kick off the second week of #11WeeksOfAndroid, focused on Machine Learning, we will highlight new tools and resources available to Android developers. Here’s a taste of the rest of this week:

  • Tuesday - ML Kit, the turnkey ML SDK went through a major overhaul with its new on-device offering this month. Check out the substantial improvement in developer usability, CameraX support and where the platform is going next.
  • Wednesday - Custom Models. When prepackaged SDK doesn’t quite satisfy your need, tools from Android Studio, TensorFlow Lite and ML Kit might just be the answer. Aside from individual offerings, we will also highlight how they can be used together.
  • Thursday - ML design. Learn some best practices for making ML product decisions from the People + AI Guidebook. We will go behind the scenes of the Read Along app, an on-device ML app that helps grow universal literacy. Bring your whole team because everyone, including UXers, engineers, and product managers are invited!

On Tuesday and Wednesday, we will also have a “codelab of the day” so get your Android Studio 4.1 beta today, block off an hour in your schedule and take this ML journey with us!

*The apps presented here are the projects of the developers individually, and not Google.

Announcing the winners of the #AndroidDevChallenge, powered by on-device machine learning

Posted by Jacob Lehrbaum, Director of Developer Relations, Android

Developers like you have always played an important role in Android innovation. Over 10 years ago, when we first launched the Android SDK, we also announced the Android Developer Challenge to reward model apps and highlight new ways of solving user problems. As Android pushes the boundaries of machine learning, 5G, foldables, and more, developers continue to help shape these new frontiers. To celebrate this work, we revived the challenge in 2019, with a focus on “Helpful Innovation,” powered by on-device machine learning.

We received hundreds of creative projects, and at the end of last year, picked 10 winners who each combined a strong idea and a thirst to bring it to life. Since then, we’ve been working with those winners to help turn their ideas into reality. And today, we’re announcing the 10 winners. Some are still at the beginning of their journey but but their apps are now ready for you to download and try out! !

  • AgroDoc helps farmers diagnose plant disease and make treatment plans. [Navneet Krishna; Kochi, India]
  • AgriFarm helps farmers detect plant diseases and prevent major damage in fruits and vegetables such as tomatoes, corn and potatoes. [Balochisan, Pakistan]
  • Eskke streamlines mobile money management for people in the Congo, letting them transfer money, pay bills, buy subscriptions and essential airtime through SMS. [David Mumbere Kathoh; Goma, Democratic Republic of Congo]
  • Leepi helps students learn hand gestures and symbols for American Sign Language. [Prince Patel; Bengaluru, India]
  • MixPose is a live streaming platform that gives yoga teachers and fitness professionals the opportunity to teach, track alignment, and give feedback in real-time. [Peter Ma; San Francisco, California, USA]
  • Pathfinder could help people with visual impairments navigate complex situations by identifying and calculating the trajectories of objects moving in their path. [Colin Shelton; Addison, Texas, USA]
  • Snore & Cough helps you identify and analyze snoring and coughing, to help provide info to users seeking assistance from a medical professional. [Ethan Fan; Mountain View, California, USA]
  • Stila pairs with a wearable device, like the Fitbit wristband or a device running on Wear OS by Google to monitor and track the body’s stress levels. By monitoring stress levels over time, you have the chance to better understand and manage stress in your life. [Yingdin Wing; Munich, Germany]
  • Trashly makes recycling easier. Just point the on-device camera at an item, and through object detection, the app identifies and classifies plastic and paper cups, bags, bottles, etc. [Elvin Rakhmankulov; Chicago, Illinois, USA]
  • UnoDogs helps owners better support their pet’s wellness, providing customized information and fitness programs. [Chinmany Mishra; New Delhi, India]

Making on-device machine learning more accessible, with ML Kit and TensorFlow Lite

Increasingly, machine learning is becoming a more accessible tool to developers with limited to no background in the technology. In fact, for most of the winners of the Android Developer Challenge, this was their first foray into machine learning. That’s thanks in part to two key offerings from Google, which bring on-device machine learning into reach for millions of developers around the world.

The first is ML Kit. ML Kit brings Google’s on-device machine learning technologies to mobile app developers, so they can build customized and interactive experiences into their apps. This includes tools such as language translation, text recognition, object detection and more. Eskke, for instance, uses offline text recognition and barcode scanning from ML Kit so users can scan the QR code at a mobile money kiosk and quickly withdraw money. And MixPose uses ML Kit's forthcoming Pose detection API to detect each user’s yoga positions and movements, so teachers can provide feedback.

The other Google resource that many of the Android Dev Challenge winners used was TensorFlow Lite. This powerful machine learning framework can help run machine learning models on Android, iOS and IoT devices that would never normally be able to support them. Its set of tools can be used for all kinds of powerful neural network-related applications, from image detection to speech recognition, bringing the latest cutting-edge technology to the devices we carry around with us wherever we go. Trashly, for instance, uses a custom TensorFlow Lite model to report if an object is recyclable and how to recycle it.

Helpful innovation, such as the 10 winning apps in the Android Developer Challenge, has the potential to change the way we access, use, and interpret information, making it available when we need it, where we need it most. By working with these developers focused on helpful innovation, we hope to inspire the next wave of developers to unlock what’s possible with this new technology.

#11WeeksOfAndroid Week 2 Machine Learning with Android logo head

What’s next in Android Machine Learning week?

As we kick off the second week of #11WeeksOfAndroid, focused on Machine Learning, we will highlight new tools and resources available to Android developers. Here’s a taste of the rest of this week:

  • Tuesday - ML Kit, the turnkey ML SDK went through a major overhaul with its new on-device offering this month. Check out the substantial improvement in developer usability, CameraX support and where the platform is going next.
  • Wednesday - Custom Models. When prepackaged SDK doesn’t quite satisfy your need, tools from Android Studio, TensorFlow Lite and ML Kit might just be the answer. Aside from individual offerings, we will also highlight how they can be used together.
  • Thursday - ML design. Learn some best practices for making ML product decisions from the People + AI Guidebook. We will go behind the scenes of the Read Along app, an on-device ML app that helps grow universal literacy. Bring your whole team because everyone, including UXers, engineers, and product managers are invited!

On Tuesday and Wednesday, we will also have a “codelab of the day” so get your Android Studio 4.1 beta today, block off an hour in your schedule and take this ML journey with us!

*The apps presented here are the projects of the developers individually, and not Google.

11 Weeks of Android: People & Identity

Posted by Dr. Stefan Frank, Senior Product Manager, Android System UI

#11 weeks of Android with Android logo

This blog post is part of a weekly series for #11WeeksOfAndroid. Each week we’re diving into a key area of Android so you don’t miss anything. This week, we spotlight people & identity. Here's a look at what you should know.

The big news

One of the goals of Android 11 was making our phones more people-centric. Because nothing matters more to people than connecting to loved ones. It is a core human need especially during our current physical distancing constraints. We have the need to be more social than ever before. Android 11 reimagines how we have conversations on our phones by adding new capabilities to help you maintain your identity across multiple devices.

We are announcing some new features in Android 11 that allows you to easily connect with your loved ones, friends and business colleagues. At the center of this release is the Android Conversation Shortcut API and Identity Services Library. These new tools empower instant connections to your best friend, sharing funny pictures of your dogs, telling your aunt about a tasty seafood recipe that you discovered, or congratulating an office colleague about her promotion. And they also provide a new level of password management that makes it easier for your users to sign up and sign in.

One of our favorite features brings conversations from people that matter most to you right to the lock screen of your phone. You’ll easily recognize them by their avatar and instantly respond to your family, friends, or colleagues. These are people you truly want to connect to. We knew this feature would be useful, but the responses from our beta-testers made us smile. The decision to include the Conversation Shortcut API to improve the lives of our users was one of the easiest decisions for us to make in the release of Android 11.

Creating a bubble from an incoming notification and accessing the conversation from the bubble.

Creating a bubble from an incoming notification and accessing the conversation from the bubble.

One of the new features building on top of shortcuts is the new conversation space at the top of your notifications. It focuses your attention on what matters most - your conversations. Right from these notifications the user can trigger another new feature in Android 11 - Bubbles. Bubbles are small representations of conversations floating over other content on the side of the screen, which can be expanded to allow quick access to conversations, without changing what you were doing on the device. They are super handy for carrying on conversations while using the device for other tasks.

The new conversations space showing how a conversation is marked as priority and will be displayed on the lock screen.

The new conversations space showing how a conversation is marked as priority and will be displayed on the lock screen.

A long tap on conversation notifications enables the user to mark priority conversations in order to give special prominence to the most important people. Priority conversations will be displayed with their individual avatar right on the lock screen and move to the top of your notifications. They can even be set to break through do-not-disturb. Another use of the conversation shortcuts is that they are used as share targets in the system share sheet, which was already launched in Android 10.

Another focus of this week was identity. To tackle user and developer complexity that makes identity a challenge for developers, we've been working on One Tap and Block Store, part of our new Google Identity Services Library. One Tap is our new cross-platform sign-in mechanism for Web and Android, supporting and streamlining multiple types of credentials. Block Store is our new token-based sign-in mechanism that’s built on top of Backup and Restore. It allows you to keep your user signed in across Android devices.

We’re super excited about all these features [since] they help all of us connect, communicate and express ourselves to the people that we care about and to the apps we are dealing with -- which is as important now as it’s ever been.”

What to watch

For a high level overview of the people centric functionality, we recommend that you check out the Android 11 launch highlight video on People. Earlier this week, we also launched a new talk on ‘Conversation Notifications’, where Artur describes how to implement the conversation shortcut and bubbles. There is also a great overview talk on the conversation additions and other System UI news from Dan. Finally, you can also listen to Chet’s podcasts where he interviews us on People and Bubbles.

If you’re interested in learning more about Identity, we also published “In Identity on Android: What’s new in sign-in”, this week. In this video, Vishal explains the new libraries in the Google Identity System: One Tap and Block Store."

Two of the teams that worked very early with us on these conversation specific topics are the Messenger team from Facebook and the direct messaging team from Twitter. Read the stories around both of these implementations here and here.

Learning path

If you’re looking for an easy way to pick up the highlights of this week, check out the People and Identity pathway. A pathway is an ordered tutorial that allows users to complete a pre-defined module that culminates in a quiz. It may include codelabs, videos, articles and blog posts. A virtual badge is awarded to each user who passes the quiz. Test your knowledge of key takeaways about People and Identity to earn a limited edition badge.

Key takeaways

Android 11 is the starting point of an ongoing focus on what matters most to users, people and conversations. Many of our partners in our ecosystem introduced amazing apps and services enabling these connections with people and conversations. We in Android want to elevate and surface these partners more prominently in order to support this goal. Thus if you are working on an app that fosters real time communication between people, we strongly encourage you to adopt the conversation shortcut based APIs for notifications, bubbles and sharing when targeting API 30 in order to put users’ conversations front and center and give them quick access to your app. The developer documentation can be found here.

For apps that handle user accounts, we encourage you to help our users avoid messy password hunting and forgotten credential processes by integrating One Tap to streamline credential management and Block Store to handle device updates. These integrations will work on phones back to Android M.

In the spirit of this week, I wish you meaningful and joyful connections with the people that matter to you and seamless experiences with your favorite apps. We hope you will help us in our journey of supporting these goals.

Resources

You can find the entire playlist of #11WeeksOfAndroid video content here, and learn more about each week here. We’ll continue to spotlight new areas each week, so keep an eye out and follow us on Twitter and YouTube. Thanks so much for letting us be a part of this experience with you!

11 Weeks of Android: People & Identity

Posted by Dr. Stefan Frank, Senior Product Manager, Android System UI

#11 weeks of Android with Android logo

This blog post is part of a weekly series for #11WeeksOfAndroid. Each week we’re diving into a key area of Android so you don’t miss anything. This week, we spotlight people & identity. Here's a look at what you should know.

The big news

One of the goals of Android 11 was making our phones more people-centric. Because nothing matters more to people than connecting to loved ones. It is a core human need especially during our current physical distancing constraints. We have the need to be more social than ever before. Android 11 reimagines how we have conversations on our phones by adding new capabilities to help you maintain your identity across multiple devices.

We are announcing some new features in Android 11 that allows you to easily connect with your loved ones, friends and business colleagues. At the center of this release is the Android Conversation Shortcut API and Identity Services Library. These new tools empower instant connections to your best friend, sharing funny pictures of your dogs, telling your aunt about a tasty seafood recipe that you discovered, or congratulating an office colleague about her promotion. And they also provide a new level of password management that makes it easier for your users to sign up and sign in.

One of our favorite features brings conversations from people that matter most to you right to the lock screen of your phone. You’ll easily recognize them by their avatar and instantly respond to your family, friends, or colleagues. These are people you truly want to connect to. We knew this feature would be useful, but the responses from our beta-testers made us smile. The decision to include the Conversation Shortcut API to improve the lives of our users was one of the easiest decisions for us to make in the release of Android 11.

Creating a bubble from an incoming notification and accessing the conversation from the bubble.

Creating a bubble from an incoming notification and accessing the conversation from the bubble.

One of the new features building on top of shortcuts is the new conversation space at the top of your notifications. It focuses your attention on what matters most - your conversations. Right from these notifications the user can trigger another new feature in Android 11 - Bubbles. Bubbles are small representations of conversations floating over other content on the side of the screen, which can be expanded to allow quick access to conversations, without changing what you were doing on the device. They are super handy for carrying on conversations while using the device for other tasks.

The new conversations space showing how a conversation is marked as priority and will be displayed on the lock screen.

The new conversations space showing how a conversation is marked as priority and will be displayed on the lock screen.

A long tap on conversation notifications enables the user to mark priority conversations in order to give special prominence to the most important people. Priority conversations will be displayed with their individual avatar right on the lock screen and move to the top of your notifications. They can even be set to break through do-not-disturb. Another use of the conversation shortcuts is that they are used as share targets in the system share sheet, which was already launched in Android 10.

Another focus of this week was identity. To tackle user and developer complexity that makes identity a challenge for developers, we've been working on One Tap and Block Store, part of our new Google Identity Services Library. One Tap is our new cross-platform sign-in mechanism for Web and Android, supporting and streamlining multiple types of credentials. Block Store is our new token-based sign-in mechanism that’s built on top of Backup and Restore. It allows you to keep your user signed in across Android devices.

We’re super excited about all these features [since] they help all of us connect, communicate and express ourselves to the people that we care about and to the apps we are dealing with -- which is as important now as it’s ever been.”

What to watch

For a high level overview of the people centric functionality, we recommend that you check out the Android 11 launch highlight video on People. Earlier this week, we also launched a new talk on ‘Conversation Notifications’, where Artur describes how to implement the conversation shortcut and bubbles. There is also a great overview talk on the conversation additions and other System UI news from Dan. Finally, you can also listen to Chet’s podcasts where he interviews us on People and Bubbles.

If you’re interested in learning more about Identity, we also published “In Identity on Android: What’s new in sign-in”, this week. In this video, Vishal explains the new libraries in the Google Identity System: One Tap and Block Store."

Two of the teams that worked very early with us on these conversation specific topics are the Messenger team from Facebook and the direct messaging team from Twitter. Read the stories around both of these implementations here and here.

Learning path

If you’re looking for an easy way to pick up the highlights of this week, check out the People and Identity pathway. A pathway is an ordered tutorial that allows users to complete a pre-defined module that culminates in a quiz. It may include codelabs, videos, articles and blog posts. A virtual badge is awarded to each user who passes the quiz. Test your knowledge of key takeaways about People and Identity to earn a limited edition badge.

Key takeaways

Android 11 is the starting point of an ongoing focus on what matters most to users, people and conversations. Many of our partners in our ecosystem introduced amazing apps and services enabling these connections with people and conversations. We in Android want to elevate and surface these partners more prominently in order to support this goal. Thus if you are working on an app that fosters real time communication between people, we strongly encourage you to adopt the conversation shortcut based APIs for notifications, bubbles and sharing when targeting API 30 in order to put users’ conversations front and center and give them quick access to your app. The developer documentation can be found here.

For apps that handle user accounts, we encourage you to help our users avoid messy password hunting and forgotten credential processes by integrating One Tap to streamline credential management and Block Store to handle device updates. These integrations will work on phones back to Android M.

In the spirit of this week, I wish you meaningful and joyful connections with the people that matter to you and seamless experiences with your favorite apps. We hope you will help us in our journey of supporting these goals.

Resources

You can find the entire playlist of #11WeeksOfAndroid video content here, and learn more about each week here. We’ll continue to spotlight new areas each week, so keep an eye out and follow us on Twitter and YouTube. Thanks so much for letting us be a part of this experience with you!

Bringing @Twitter’s DMs into Android 11’s Conversation API

Twitter logo

This blogpost is a collaboration between Google and Twitter. Authored by Aaron Labiaga with support from Fred Lohner, Suzanne Xie, and Alex Ackerman-Greenberg from Twitter.

Direct Messages for the conversation space

Twitter is a social media app and source for what's happening in the world. And with Android 11’s Conversation APIs, surfacing Twitter’s Direct Messages to the conversation space makes perfect sense as it features real people talking in real time bidirectional conversations.

Conversation Notification API

To surface notifications in the conversation space, developers need to use Messaging style notifications set with a published long-lived shortcut ID. The shortcut ID allows for surfacing the conversation throughout various surfaces in the UI, including as a shortcut in the launcher. For details and sample code on how this is done, please see the conversation API guidelines.

A Q&A with the Twitter team

Frederik Lohner, tech lead of Twitter’s notifications team and Suzanne Xie, Product Management Director for Conversations on Twitter, share their experience with the migration and future prospect of its impact:

How was the migration to messaging style notifications, the technical scope and its impact on codebase?

Due to legacy reasons we first had to migrate to use MessagingStyle notifications for our DM pushes. This offered a great opportunity to clean up a bunch of technical debt, whilst allowing us the opportunity to begin testing things like auto generated replies for notifications with very little additional work on our side.

Were there any challenges in the implementation of shortcuts? As it is a requirement for the conversation space.

We found it challenging to manage the max number of published dynamic shortcuts since there is a limited number, requiring us to manually remove older shortcuts as new ones were created. The first Developer Preview didn't include a great way for managing shortcuts but this changed in Developer Preview 2, with the addition of the pushDynamicShortcut method that handled the limit for you.The addition of this method is a testament to the importance of exploring the APIs in the early stages and the impact one can have on shaping the API through testing and feedback.

What do think is the future prospect on the impact of migration, e.g. more engagement given visibility of DMs in conversation space?

We think these changes will make DM's much easier to use and look forward to getting feedback from our users. The conversation space also fits the purpose of DMs and it is important that these messages are categorized in the right space.

A people-centric experience in Android 11

Android 11 reimagines the way we have conversations on our phones, building an OS that can recognize and prioritize the most important people in your life. If your application has any notion of messaging between people then consider following the guidelines in the Android developer docs to ensure that you're taking advantage of the new conversation space. We’re excited to work with developers like Twitter to help showcase the importance of a people-centric experience on mobile phones with Android 11.

Bringing @Twitter’s DMs into Android 11’s Conversation API

Twitter logo

This blogpost is a collaboration between Google and Twitter. Authored by Aaron Labiaga with support from Fred Lohner, Suzanne Xie, and Alex Ackerman-Greenberg from Twitter.

Direct Messages for the conversation space

Twitter is a social media app and source for what's happening in the world. And with Android 11’s Conversation APIs, surfacing Twitter’s Direct Messages to the conversation space makes perfect sense as it features real people talking in real time bidirectional conversations.

Conversation Notification API

To surface notifications in the conversation space, developers need to use Messaging style notifications set with a published long-lived shortcut ID. The shortcut ID allows for surfacing the conversation throughout various surfaces in the UI, including as a shortcut in the launcher. For details and sample code on how this is done, please see the conversation API guidelines.

A Q&A with the Twitter team

Frederik Lohner, tech lead of Twitter’s notifications team and Suzanne Xie, Product Management Director for Conversations on Twitter, share their experience with the migration and future prospect of its impact:

How was the migration to messaging style notifications, the technical scope and its impact on codebase?

Due to legacy reasons we first had to migrate to use MessagingStyle notifications for our DM pushes. This offered a great opportunity to clean up a bunch of technical debt, whilst allowing us the opportunity to begin testing things like auto generated replies for notifications with very little additional work on our side.

Were there any challenges in the implementation of shortcuts? As it is a requirement for the conversation space.

We found it challenging to manage the max number of published dynamic shortcuts since there is a limited number, requiring us to manually remove older shortcuts as new ones were created. The first Developer Preview didn't include a great way for managing shortcuts but this changed in Developer Preview 2, with the addition of the pushDynamicShortcut method that handled the limit for you.The addition of this method is a testament to the importance of exploring the APIs in the early stages and the impact one can have on shaping the API through testing and feedback.

What do think is the future prospect on the impact of migration, e.g. more engagement given visibility of DMs in conversation space?

We think these changes will make DM's much easier to use and look forward to getting feedback from our users. The conversation space also fits the purpose of DMs and it is important that these messages are categorized in the right space.

A people-centric experience in Android 11

Android 11 reimagines the way we have conversations on our phones, building an OS that can recognize and prioritize the most important people in your life. If your application has any notion of messaging between people then consider following the guidelines in the Android developer docs to ensure that you're taking advantage of the new conversation space. We’re excited to work with developers like Twitter to help showcase the importance of a people-centric experience on mobile phones with Android 11.

Performance insights for Games, powered by Android Performance Tuner

Posted by Dan Galpin, Developer Advocate

Android vitals is the destination for managing your app's technical quality. Over 80,000 developers take advantage of its performance and stability metrics every month.

As part of our work to help you deliver better game experiences to more Android users, we're introducing Android Performance Tuner - a new library in the Android Game SDK that unlocks game performance insights in Android Vitals. This gives you a scalable way to measure and optimize your frame rate and graphical fidelity across the whole Android device ecosystem.

Unity Boat Attack Sample with Different Optimizations

Unity Boat Attack Sample with Different Optimizations

Once you have integrated Android Performance Tuner into your game and published it on Play, you'll be able to see how it performs across real users and devices with the following new features in Android vitals.

Frame rate performance

Frame Rate Performance by Quality Level and Device Model

Frame Rate Performance by Quality Level and Device Model

We chart the frame time distribution across your users’ devices, broken down by quality levels that you have implemented in your game, so you can see how specific device models or hardware specifications are performing on each quality level.

Performance issues

We also analyze your performance data to help determine the likely cause of issues, so you can differentiate between problems associated with specific hardware and problems with specific screens or levels in your game. You annotate your code to give contextual information about what your game is doing at that point. This gives you full control over the granularity of the insights.

Top Device Model/Annotation Issues

Top Device Model/Annotation Issues

We call out the top device model issue as well as the top game-specific issue to give you clear guidance on what's most important.

Underperforming Device Models by GPU

Underperforming Device Models by GPU

You can drill down to see a breakdown of underperforming device models by different specs, such as GPU and SoC. This allows you to decide whether you can work at the GPU or SoC-level to optimize performance. Alternatively, you may decide to change quality levels, rather than work at the device model level.

Device Model Impact, User Impact, GPU time

Device Model Impact, User Impact, GPU Time

You can also see the full list of device models, along with the number of affected user sessions and frame time, to help you prioritize device-specific changes. As well as total frame time, we also show you GPU time to help determine whether the device is GPU bound or has another performance problem, such as being CPU or I/O bound. All data in the device model table can be exported for further analysis and action planning.

Opportunities to make a good experience great

We can also help identify opportunities — places where you could potentially provide users with a better experience by giving them a higher quality level, enabling more advanced graphical features. Frame Time Performance with Opportunities

Frame Time Performance with Opportunities

The devices on the far left are more than meeting the frame times for smooth performance. You can drill down to see stats by device model and specification to see if there is an opportunity to improve the graphical fidelity across a wide range of devices.

Available (almost) everywhere

The Android Performance Tuner is intended to work across over 99% of the Android device ecosystem. You can get these insights on any Android devices around the world, from Android 4.1 (API 16) onwards.

Integrating Android Performance Tuner

Whether you have your own game engine or are using a third-party game engine, we're doing our best to make integration easy. The Android Performance Tuner relies on tick functions being called each frame. Within the library, this tick information is aggregated into histograms, which are periodically uploaded through an HTTP endpoint, so your game will need to have the internet access permission.

With our plugin for the Unity platform, you can collect frame ticks from Unity 2017.4 onwards. Unity 2019.3.14+ enables the collection of higher-fidelity performance information.

If you're doing a native source code level engine integration, we strongly recommend integrating the Frame Pacing API from the Android Game SDK to get the highest quality information. The Frame Pacing API will give you smoother frame rates and improved support for high-refresh rate displays, so it's worth integrating on its own.

Unreal 4.25+ integrates the Frame Pacing API. You enable it by adding a.UseSwappyForFramePacing=1 to the Android_Default profile to activate it for all Android devices.

Within Unreal or your native engine integration, you pass in the Swappy_injectTracer function from the Frame Pacing API at initialization to enable automatic frame time recording.

void InitTf(JNIEnv* env, jobject activity) {
   SwappyGL_init(env, activity);
   swappy_enabled = SwappyGL_isEnabled();
   TFSettings settings {};
   if (swappy_enabled) {
       settings.swappy_tracer_fn = &SwappyGL_injectTracer;
   }
…
}

Enabling Frame Time Recording in the Android Performance Tuner for your Engine

In Unity, we recommend activating Optimized Frame Pacing within the Unity settings (Unity 2019.3+ only), but Frame Pacing isn't required in Unity to use the Android Performance Tuner.

Activating Optimized Frame Pacing in Unity 2019.3

Activating Optimized Frame Pacing in Unity 2019.3

Providing contextual information

Next, you want to define annotations to give contextual information about what your game is doing when a tick is recorded, such as:

  • Current game level
  • Loading a specific scene
  • A "big boss" or other complex rendered item is on the screen
  • Relevant game state information

Annotation Parameters

If you're using Unity with the Android Performance Tuner plugin, you'll automatically get a scene annotation that maps to the current scene being played. The LoadingState annotation can be easily hooked up to your scripts, and you can define additional annotations within the plugin editor UI.

Annotation Parameters within the Unity Editor from the Android Performance Tuner Plugin

Annotation Parameters within the Unity Editor from the Android Performance Tuner Plugin

To pass annotation parameters from within your own game engine, you define a protocol buffer message that contains all of these annotations, such as loading state, level or scene, etc.

Fidelity Parameters and Quality Levels

You also define fidelity parameters and associate them with quality levels that your game reports back. These can be used for anything that you use in your game to reduce the complexity of the scene, such as texture quality, draw distance, particle count, post-processing effects, shadow resolution, etc. In the native integration, you define these parameters using a protocol buffer.

import "tuningfork.proto"
message FidelityParams {
  int32 texture_quality_level = 1;
  int32 shadow_resolution = 2;
  float terrain_details_percent = 3;
  int32 post_processing_effects_level = 4;
}

Example FidelityParams Proto Definition for an In-house Engine

Then, you create up to fifteen sets of quality levels as a set of values defined by the FidelityParams message, which allows the Android Performance Tuner to track its metrics against your quality data. You can create both fidelity parameters and quality levels in the Unity editor interface provided by the Android Performance Tuner for Unity plugin.

Testing your integration

We've created the Tuning Fork Monitor app to act as a local server and display data from an Android Performance Tuner-enabled app. You can call EnableLocalEndpoint() in the Android Performance Tuner Unity plugin on a development build to enable local testing. In your native integration, you set the endpoint_uri_override in the Android Performance Tuner settings.

Once local tests look great, you then enable the Android Performance Parameters API in the Google Cloud Console to test end-to-end.

Available Now

We're committed to helping you bring the best version of your game to the widest number of users and devices in the Android ecosystem. Android Performance Tuner within the Android Game SDK, the Unity plugin, and Performance Insights within Android Vitals are all available now. You can refer to our documentation for a walk through of the process for native and Unity integrations.