Category Archives: Android Developers Blog

An Open Handset Alliance Project

Build smarter Android apps with on-device Machine Learning

Posted by Thomas Ezan, Developer Relations

In the past year, the Android team made significant improvements to on-device machine learning to help developers create smarter apps with more features to process images, sound, and text. In the Google I/O talk Build smarter Android apps with on-device Machine Learning, David Miro-Llopis PM on ML Kit and Thomas Ezan Android Developer Relation Engineer review new Android APIs and solutions and showcase applications using on-device ML.

Running ML processes on-device enables low-latency, increases data-privacy, enables offline support and potentially reduces cloud bill. Applications such as Lens AR Translate or the document scanning feature available in Files in India, benefit from the advantages of on-device ML.

To deploy ML features on Android, developers have two options:

  • ML Kit: which offers production-ready ML solutions to common user flows, via easy-to-use APIs.
  • Android’s custom ML stack: which is built on top of Tensorflow Lite, and provides control over the inference process and the user experience.

ML Kit released new APIs and improved existing features

Over the last year, the ML Kit team worked on both improving existing APIs and launching new ones: face mesh and document scanner. ML Kit is launching a new document scanner API in Q3 2023, that will provide a consistent scanning experience across apps in Android. Developers will be able to use it only with a few lines of code, without needing camera permission and with low apk size impact (given that it will be distributed via Google Play Services. In a similar fashion, Google code scanner is now generally available and provides a consistent scanning experience across apps, without needing camera permission, via Google Play Services.

Image a series of three photos of two girls smiling to show how face mesh improves facial recognition

Additionally, ML Kit improved the performance of the following APIs: barcode detection (by 17%), text recognition, digital ink recognition, pose detection, translation, and smart reply. ML Kit also integrated some APIs to Google Play Services so you don’t have to bundle the models to your application. Many developers are using ML Kit to easily integrate machine learning into their apps; for example, WPS uses ML Kit to translate text in 43 languages and save $65M a year.


Acceleration Service in Android’s custom ML stack is now in public beta

To support custom machine learning, the Android ML team is actively developing Android’s custom ML stack. Last year, TensorFlow Lite and GPU delegates were added to the Google Play Services which lets developers use TensorFlow Lite without bundling it to their app and provides automatic updates. With improved inference performance, hardware acceleration can in turn also significantly improve the user experience of your ML-enabled Android app. This year, the team is also announcing Acceleration Service, a new API enabling developers to pick the optimal hardware acceleration configuration at runtime. It is now in public beta and developers can learn more and get started here.

To learn more, watch the video:

Introducing the Watch Face Format for Wear OS

Posted by Anna Bernbaum, Product Manager

We are excited to announce the launch of the Watch Face Format! We worked in partnership with Samsung to introduce a new way for you to build watch faces for Wear OS smartwatches.

The Watch Face Format is a declarative XML format to design the appearance and behavior of watch faces. This means that there is no executable code involved in creating a watch face, and there will be no code embedded in your watch face APK.

The Wear OS platform takes care of the logic needed to render the watch face so you no longer have to worry about code optimizations or battery performance.

Watch faces that are built with this new format require less maintenance and fewer updates than the ones built using the Jetpack Watch Face library. For example, you no longer need to update your watch face to benefit from improvements in performance or battery consumption, or to get the latest bug fixes.

Starting today, you can build watch faces in this new format and publish them on Google Play, ready for when the first Wear OS 4 watches are available.

The Watch Face Format lets you create…

Analog and digital watch faces:

Three watch faces illustrating different analogue and digital styles.

Watch faces with complications:

Three watch faces displaying different complication formats.

Customizable watch faces:

Three instances of the same watch face displayed in different color choices.

And more...

A planets-based watch face, a rainbow-inspired watch face and one based on the sun and moon

Watch Face Editing

With the Watch Face Format, we have included the watch face editor as part of Wear OS itself, so users can customize every watch face using the same editor UI. You no longer need to build your own watch face editor for users to customize their watch face.

Customizing the watch face with the in-built watch face editor
Wear OS 4’s editor for watch faces made using the Watch Face Format

Build Watch Faces, or Watch Face Tools

The new Watch Face Format can be used to build watch faces directly, or it can be integrated into creation tools, allowing designers to create watch faces without having to write any executable code.


Watch Face Studio

Today, Samsung has released the latest version of Watch Face Studio, ready for you to try now. As an alternative to directly writing XML using the Watch Face Format, Watch Face Studio makes it easy for designers to create watch faces without any coding experience.

Watch faces made in the latest version of Watch Face Studio use the Watch Face Format by default when they are run on a Wear OS 4 watch, or they run as traditional watch faces when the watch face runs on a Wear OS 3 watch.

Using Watch Face Studio to create a watch face.
Using Watch Face Studio to create a watch face

Learn more

Build watch faces using the Watch Face Format today:

What’s new with Android for Cars: I/O 2023

Posted by Jennifer Tsau, Product Management Lead and David Dandeneau, Engineering Lead

For more than a decade, Google has been committed to bringing safe and seamless connected experiences to cars. We’re continuing to see strong momentum and adoption across Android for Cars. Android Auto is supported by nearly every major car maker, and will be in nearly 200 million cars by the end of this year. And the number of cars powered by Android Automotive OS with Google built-in — which includes top brands like Chevrolet, Volvo, Polestar, Honda, Renault and more — is expected to nearly double by the end of this year.

With cars becoming more connected and equipped with immersive displays, there’s more opportunities for developers to bring app experiences to cars. We’re excited to share updates and new ways for developers to reach more users in the car.


Apps designed for driving experiences

Helping drivers while on the road - whether they are navigating, listening to music, or checking the weather - is a top priority. We’re continuing to invest in tools and resources, including the Android for Cars App Library, to make it even easier for developers to build new apps or port existing Android apps over to cars.

New capabilities for navigation apps

Today, we announced Waze rolling out globally on the Google Play Store for all cars with Google built-in, expanding its availability beyond Android Auto. As a part of this launch, we created more templates in Android for Cars App Library to help speed up development time across a number of app categories, including navigation.

For navigation apps, it’s also now possible to integrate with the instrument cluster, providing turn-by-turn directions right in the driver's line of sight. And developers can also access car sensor data to surface helpful information like range, fuel level, and speed to provide more contextual assistance to drivers.

A car dashboard shows the Waze app open on the display panel
The Waze app is coming to all cars with Google built-in, including the first-ever Chevrolet Blazer EV launching this year.

Tools to easily port your media apps across Android for Cars

Media apps continue to be a top use case in the car, and it’s quicker than ever to bring your media apps to Android Auto and Android Automotive OS. Audible recently joined popular streaming audio apps like Deezer, Soundcloud, and Spotify to offer their apps across both Android Auto and cars with Google built-in. If you have a media app on mobile, port it over to reach new users in the car.

New app categories for driving experiences

The Android for Cars App Library now allows developers to bring new apps to cars including internet of things (IoT) and weather apps to cars. The IoT category is available for all developers, while weather is in an early access program. In the weather category, The Weather Channel app will join other weather apps like Weather & Radar later this year.

We’re also working with messaging apps like Zoom, Microsoft Teams, and Webex by Cisco to allow you to join meetings by audio from your car display in the coming months.

A car display shows a Zoom meeting schedule next to a route in Google Maps.
Coming soon, join meetings by audio from your car display.

Apps designed for parked and passenger experiences

With screens expanding in size and more being added for passengers, there is growing demand for parked and passenger experiences in cars.

Video, gaming, and browsing in cars

Now, video and gaming app categories are available in the car, with an early access program for browsing apps coming soon. YouTube is now available for car makers to offer in cars with Google built-in. And drivers of cars with Google built-in will soon have access to popular titles like Beach Buggy Racing 2, Solitaire FRVR, and My Talking Tom Friends from publishers like Vector Unit, FRVR and Outfit7 Limited. Developers can now port their large screen optimized apps to cars to take advantage of this opportunity.

A car display shows a YouTube video of an animated character singing.
YouTube is coming to cars with Google built-in, like the Polestar 2.

More screens in cars allows for new experiences between drivers and passengers, including individual and shared entertainment experiences. We're excited to announce multi-screen support is coming to Android Automotive OS 14 — stay tuned for more updates.

A car with a panoramic front display and screens in headrests showing apps and video content.
Support for multiple screens is coming to Android Automotive OS 14.

Start developing apps for cars today

To learn how to bring your apps to cars, check out the technical session, codelab and documentation on the Android for Cars developer site. With all the opportunities across car screens, there has never been a better time to bring your apps and experiences to cars. Thanks for all the contributions to the Android ecosystem. See you on the road!

Building pixel-perfect living room experiences with Compose for TV

Posted by Paul Lammertsma, Developer Relations Engineer

Over the past year, we’ve continued to see significant growth on Android TV OS, now with over 150 million monthly active devices. In fact, according to Strategy Analytics, the Android TV streaming platform shipped on more devices worldwide than any other streaming TV platform in 2022.

Today, we’re launching the Alpha release of Compose for TV, the latest UI framework for developing beautiful and functional apps for Android TV.


Building pixel-perfect living room experiences with Compose for TV

Compose for TV unlocks all the benefits of Jetpack Compose for your TV apps, allowing you to build apps with less code, easier maintenance and a modern Material 3 look straight out of the box:

  • Less code: Do more with less code and avoid entire classes of bugs, so code is simple and easy to maintain. 
  • Intuitive: Describe your UI, and Compose takes care of the rest. As the app state changes, your UI automatically updates. 
  • Accelerate development: Compose for TV is compatible with all your existing code so you can adopt when and where you want. Iterate fast with live previews and full Android Studio support. 
  • Powerful & flexible: Create beautiful apps with direct access to the Android platform APIs that can be easily reused between other form factors, including your existing mobile, tablet, foldable, wearable and TV interfaces.

TV design guidelines

We're also excited to announce the launch of our new TV Design Guidelines for Android TV. This comprehensive guide gives you the tools you need to create TV apps that are visually appealing, intuitive, and immersive. The guidelines cover everything from typography and color to navigation and layout. Follow these guidelines to create high-quality TV apps that are easy to use.

image of a wall mounted, flat screen television in a modern home. The screen is showing the preview for a show titled 'Paws' with an adorable puppy as the show's star, and a Watch Now button

Components you can use today

Here are some components from the TV library that are optimized for the living room experience. You can use them alongside the Material components in Compose you’re already familiar with.


Scroll containers

TvLazyColumn {   items(contentList) { content ->     TvLazyRow { items(content) { cardItem -> Card(cardItem) }   } }

moving image of a grid of content cards
A grid of content cards

TvLazyRow( pivotOffsets = PivotOffsets(0.0f) ) { items(movie) { movie -> MyContentCard(movie) } }

moving image of a grid of content cards
Adjusting the pivot of a TvLazyRow


Immersive List

ImmersiveList( modifier = Modifier.height(130.dp).fillMaxWidth(), background = { index, _ -> AnimatedContent(targetState = index) { MyImmersiveListBackground(it) } }, ) { TvLazyRow { items(featuredContentList.size) { index -> MyCard( Modifier.focusableItem(index), featuredContentList[index] ) } } }

moving image of a grid of content cards
ImmersiveList allows TvLazyRows to be combined with content featuring

Featured carousel

Carousel( itemCount = featuredContentList.size, ) { index -> CarouselItem( background = { MyBackground(index) }, content = { MyFeaturedContent(featuredContentList[index]) } ) }

moving image of a grid of content cards
Carousel features content with custom content and backgrounds

Navigation

var selectedTabIndex by remember { mutableStateOf(0) } TabRow(selectedTabIndex = selectedTabIndex) { tabs.forEachIndexed { index, tab -> Tab( selected = selectedTabIndex == index, onFocus = { selectedTabIndex = index }, ) { Text(tab) } } } MyContentBody(selectedTabIndex)

moving image of a grid of content cards
TabRows can be placed at the top of the screen to provide top navigation

Side navigation with navigation drawer


NavigationDrawer( drawerContent = { if (DrawerValue.Open == it) { MyExpandedSideMenu() } else { MyCompactSideMenu() } } ){ MyContentBody() }

moving image of a grid of content cards
NavigationDrawer makes it easy to implement side navigation that expands and collapses

TV-optimized components

Subtle focus hints that work on phones and tablets might not be optimal for TVs, due to environmental factors such as distance from the screen and contrast ratio. To address this, we’ve built dedicated Material3 inspired components that provide big, bold focus for selected elements like Buttons and Cards, designed with accessibility in mind. You can use these Indications for your own custom surfaces as well.


moving image of a grid of content cards
Component focus can be customized through different indication types: Scale, Border, Glow and Color

Built with developers

We worked closely with a group of early adopters to get their feedback on Compose for TV. Here’s what they have to say:


Quote card with headshot of Dai Williams, Plex, smiling and text reads,'TV focus and scrolling support on Compose from Google has greatly improved our developer productivity and app performance. We are excited to launch more and more features using Compose this year.'

Quote card with headshot of Danny Preussler, Android Platform Lead, Soundcloud, smiling and text reads,'Thanks to Compose for TV, we are able to reuse components and move much faster than the old Leanback View APIs would have ever allowed us to'

Quote card with headshot of Petr Introvič, Showmax, smiling and text reads,'Dev-friendly components like ImmersiveList or Carousel helped us put front the top content, and NavigationDrawer added a top-level navigation—the final piece for our TV app migration to Compose.'

Quote card with headshot of Kishore AK, CTO, Zee5, smiling, and text reads 'We are constantly striving to ensure our users have the best possible experience. We started early on COmpose for TV and are confident that its implementation will help in making our app rendering faster and smoother.'

Learning more

To get started , check out the developer guides, design reference, our new codelab and sample code. Be sure to check the latest release notes to keep up to date with the latest updates to Compose for TV.


Feedback from developers & designers like you

We’ve heard your feedback about the Leanback API and your desire to use a modern UI framework that looks great out of the box, but also lends itself to be thoroughly themed and customized. Please continue to give us feedback so we can continue shaping Compose for TV to fit your needs.

Watch the Wear OS updates at I/O 2023

Posted by Kseniia Shumelchyk, Android Developer Relations Engineer

As we continue to evolve the Wear OS platform, we're excited to share with you some of the newest features and improvements that have been added to help you create innovative and engaging experiences for your users.

Partners like Peloton and Todoist have been building exceptional experiences for Wear OS - and seeing the impact on their feature-adoption and engagement. Hear directly from Peloton engineers about how they built a differentiated experience for the watch with Compose for Wear OS.


In this blog post, we’ll be highlighting some of the key updates we announced at Google I/O this year, so let’s dive in and explore the latest advancements in Wear OS!

Wear OS 4 Developer Preview

Today we’re releasing the first Developer Preview of Wear OS 4, the next version of Google’s smartwatch platform arriving later this year. It has enhancements to security, user customization, and power optimizations.

This preview introduces several new tools to help enhance your Wear OS app experience:

Watch Face Format

We are launching the Watch Face Format, a new way to create watch faces for Wear OS. The format makes it easier to create customizable and more power-efficient watch faces for Wear OS 4. Developed in partnership with Samsung, the Watch Face Format is a declarative XML format, so there is no executable code involved in creating a watch face and there will be no code embedded in your watch face APK. Read more.

Watch faces created using the new Format

Tiles

Wear OS tiles give users fast, predictable access to the information and actions they rely on most. Version 1.2 of the Jetpack Tiles library introduces support for platform data bindings, so if your tile uses platform data sources such as heart rate, step count, or time, your tile is updated once per second.

The new version of tiles also adds support for animations. You can use tween animations to create smooth transitions on changes to part of your layout, and transition animations can animate new or disappearing elements from the tile.

Image showing examples of animated Tiles
Examples of animated Tiles

Get your app ready

Wear OS 4 is based on Android 13, which is several versions newer than the current Wear OS version, so your app will need to handle the system behavior changes that took effect in Android 12 and Android 13. We recommend you start by testing your app and releasing a compatible update first – as devices get upgraded to Wear OS 4, it’s a basic but a critical level of quality that provides a good app experience for users.

Download the Wear OS 4 emulator in Android Studio Hedgehog to explore new features and test your app on Wear OS 4 Developer Preview.


Tooling and library updates

Wear OS support in Firebase Test Lab

Firebase Test Lab will support running tests for your standalone app on physical Google Pixel Watches in the next few weeks. You can run your automated tests on the Google Pixel Watch via Gradle Managed Devices, or use the Firebase Console to also run Robo tests. To learn more, check out available devices.

Wear OS support in the Pre-launch reports

Today we are also excited to announce Wear OS support in Google Play Pre-launch reports for standalone apps. The Pre-launch report helps to identify issues proactively before your app reaches users, so it’s an important tool to help you launch a high-quality app. You can test for stability, accessibility, security and trust, and screenshot previews! At the moment the analysis runs on Wear emulators and it is soon launching on Google Pixel Watches.

Emulator improvements

The Wear OS 4 emulator brings support for emulated Bluetooth, which lets you test more use cases, for example Bluetooth audio.

The new Wear OS 4 emulator doesn’t support unmanaged 32-bit code, so if your app uses native code, make sure that it includes both 32-bit and 64-bit native libraries. This will also prepare your app for upcoming 64-bit only hardware.

In Android Studio Hedgehog we also added capabilities for capturing screenshots and Logcat snapshots in the Wear OS emulator, so it is now much easier to generate screenshots for your app’s store listing.

Jetpack libraries

Since the latest stable Compose for Wear OS 1.1 release, we continue to bring new features and improvements to the toolkit. Version 1.2 already has a number of alpha releases – check out release notes to find out more.

Health Services version 1.0 has introduced a few new features in latest beta releases. Most notably, it includes BatchingMode to deliver batched exercise data at a configured interval instead of the default interval, as well as an ExerciseTypeConfig API which enables updates during ongoing exercises, such as golfing. If you are interested to learn what's new in Android Health, check out this blog.


Start building for Wear OS now

Wear OS active devices have grown 5x since launching Wear OS 3, and it's the fastest growing smartwatch platform.

We’re excited to share our brand new Wear OS Gallery, where you can find even more guidance with proven design and development patterns for messaging, media, and health & fitness apps!

With the latest updates, you'll have even more tools at your disposal to create beautiful, high-quality wearable experiences.


Learn more

Get started building for Wear OS with hands-on experience! Try our Compose for Wear OS codelab, and check out the documentation and samples.

The new Wear OS quality requirements will come into effect on August 31, 2023, so consider them early when designing and developing your app.

We’re looking forward to seeing the experiences that you build!

Introducing the Android Design Hub: The ultimate resource for building exceptional user interfaces across all form factors

Posted by Adhithya Ramakumar, Design Lead, Android Developer Experience and Rebecca Gutteridge, Senior Developer Relations Engineer, Android

Cross posted from Android Medium

We’re introducing the Android Design Hub to make it easier to build compelling UI across form factors.

What is the Android UI design hub?

The design hub is a comprehensive resource with opinionated guidance designed to empower designers and developers like you to create stunning and user-friendly interfaces for Android apps. It's all about sharing takeaways, examples and do’s and don’ts, starter Figma kits, UI samples and inspiring galleries. This is the beginning of a journey we'd love for you to join us on.

Why the Android UI design hub?

A well-designed UI is crucial to the success of an app, and that's why we created a one-stop shop designed to help create outstanding user interfaces across all Android form factors:

The design hub goes into depth about what it means to design for Android, at the same time complementing and extending our Material Design open-source design system. As the Android ecosystem adds an increasing variety of devices, it's more important than ever to create seamless and adaptable experiences for your users. The Android UI Design Hub is your comprehensive resource for mastering the art of designing and implementing UI on a diverse range of devices, from smartphones and tablets to foldables to wearables and TV.

Here’s a sneak peek of what you'll find in the design hub:

Resources organized by form factors

Gain access to a library tailored for each form factor, including guidance, design templates, and sample projects. With these resources at your fingertips, you're well equipped to create exceptional UI experiences for every device category.

moving image of various form factors

Galleries with UI inspiration across form factors

Spark your creativity by exploring our collection of Android app designs for a variety of popular categories of apps such as Social, Productivity, Health & Fitness, Shopping, and more. Get inspired by thoughtfully curated examples and discover design patterns to tackle common design challenges across form factors for

moving image of various Android aopp designs

Design guidelines

Dive into the nitty gritty of Android design principles, learning how to implement high-quality Android designs and understanding app layout with Android system bars, navigation modes, theming, and more. Our comprehensive documentation helps ensure that your app's design adheres to the highest standard and directs you to the latest Material Design guidelines.

Guidelines:

Design tools and resources

image of a 6 x 3 grid of illustrated designs

Equip yourself with an extensive selection of design tools, templates, and resources, specially tailored for Android development. Streamline your workflow and bring your app ideas to life more efficiently:

Check out the code samples with their corresponding Figma Kits:

Whether you're an experienced designer or a developer looking to enhance your design skills, the Android Design Hub is here to support and guide you throughout your journey.

So what are you waiting for? Visit the Android UI design hub today, and start creating exceptional user interfaces that captivate your audience and leave a lasting impression. Happy designing! 🚀

Media transcoding and editing, transform and roll out!

Posted by Andrew Lewis - Software Engineer, Android Media Solutions

The creation of user-generated content is on the rise, and users are looking for more ways to personalize and add uniqueness to their creations. These creations are then shared to a vast network of devices, each with its own capabilities. The Jetpack Media3 1.0 release includes new functionality in the Transformer module for converting media files between formats, or transcoding, and applying editing operations. For example, you can trim a clip from a longer piece of media and apply effects to the video track to share over social media, or transcode media into a more efficient codec for upload to a server.

The overall goal of Transformer is to provide an easy to use, reliable and performant API for transcoding and editing media, including support for customizing functionality, following the same API design principles to ExoPlayer. The library is supported on devices running Android 5.0 Lollipop (API 21) onwards and includes device-specific optimizations, giving developers a strong foundation to build on. This post gives an introduction to the new functionality and describes some of the many features we're planning for upcoming releases!


Getting Started

Most operations with Transformer will follow the same general pattern:

  1. Configure a TransformationRequest with settings like your desired output format
  2. Create a Transformer and pass it your TransformationRequest
  3. Apply additional effects and edits
  4. Attach a listener to react to completion events
  5. Start the transformation

Of course, depending on your desired transformations, you may not need every step. Here's an example of transcoding an input video to the H.265/HEVC video format and removing the audio track.

// Create a TransformationRequest and set the output format to H.265 val transformationRequest = TransformationRequest.Builder().setVideoMimeType(MimeTypes.VIDEO_H265).build() // Create a Transformer val transformer = Transformer.Builder(context) .setTransformationRequest(transformationRequest) // Pass in TransformationRequest .setRemoveAudio(true) // Remove audio track .addListener(transformerListener) // transformerListener is an implementation of Transformer.Listener .build() // Start the transformation val inputMediaItem = MediaItem.fromUri("path_to_input_file") transformer.startTransformation(inputMediaItem, outputPath)

During transformation you can get progress updates with Transformer.getProgress. When the transformation completes the listener is notified in its onTransformationCompleted or onTransformationError callback, and you can process the output media as needed.

Check out our documentation to learn about further capabilities in the Transformer APIs. You can also find details about using Transformer to accurately convert 10-bit HDR content to 8-bit SDR in the "Dealing with color washout" blog post to ensure your video's colors remain as vibrant as possible in the case that your app or the device doesn't support HDR content.


Edits, effects, and extensions

Media3 includes a set of core video effects for simple edits, such as scaling, cropping, and color filters, which you can use with Transformer. For example, you can create a Presentation effect to scale the input to 480p resolution while maintaining the original aspect ratio, and apply it with setVideoEffects:

Transformer.Builder(context) .setVideoEffects(listOf(Presentation.createForHeight(480))) .build()

You can also chain multiple effects to create more complex results. This example converts the input video to grayscale and rotates it by 30 degrees:

Transformer.Builder(context) .setVideoEffects(listOf( RgbFilter.createGrayscaleFilter(), ScaleToFitTransformation.Builder() .setRotationDegrees(30f) .build())) .build()

It's also possible to extend Transformer’s functionality by implementing custom effects that build on existing ones. Here is an example of subclassing MatrixTransformation, where we start zoomed in by 2 times, then zoom out gradually as the frame presentation time increases:

val zoomOutEffect = MatrixTransformation { presentationTimeUs -> val transformationMatrix = Matrix() val scale = 2 - min(1f, presentationTimeUs / 1_000_000f) // Video will zoom from 2x to 1x in the first second transformationMatrix.postScale(/* sx= */ scale, /* sy= */ scale) transformationMatrix // The calculated transformations will be applied each frame in turn } Transformer.Builder(context) .setVideoEffects(listOf(zoomOutEffect)) .build()

Here's a screen recording that shows this effect being applied in the Transformer demo app:

moving image showing what subclassing matrix transformation looks like in the Transformer demo app

For even more advanced use cases, you can wrap your own OpenGL code or other processing libraries in a custom GL texture processor and plug those into Transformer as custom effects. See the demo app for some examples of custom effects. The README also has instructions for trying a demo of MediaPipe integration with Transformer.


Coming soon

Transformer is actively under development but ready to use, so please give it a try and share your feedback! The Media3 development branch includes a sneak peek into several new features building on the 1.0 release described here, including support for tone-mapping HDR videos to SDR using OpenGL, previewing video effects using ExoPlayer.setVideoEffects, and custom audio processing. We are also working on support for editing multiple videos in more flexible compositions, with export from Transformer and playback through ExoPlayer, making Media3 an end-to-end solution for transforming media.

We hope you'll find Transformer an easy-to-use and powerful tool for implementing fantastic media editing experiences on Android! You can send us feature requests and bug reports in the Media3 GitHub issue tracker, and follow this blog to get updates on new features. Stay tuned for our upcoming talk “High quality Android media experiences” at Google I/O.

Improving Consistency of Background Work on Android

Posted by Sanat Kamal Bahl, Product Manager, Android Frameworks

Since its inception, Android has been designed to be the world’s first open and innovative platform for mobile devices.

Today, Android powers a rich and open ecosystem of devices serving billions of users around the world. The openness of the Android platform enables innovation in new mobile form factors like foldable phones. This openness also enables smart features in cars, watches, and televisions. While this openness unlocks great opportunities, with so many unique devices, it can make your life harder as a developer. One such challenge we have heard from the community involves restrictions on foreground services and background work that make it harder for you to create apps that work across different device models.

Looking to solve these consistency challenges, we are announcing deeper partnerships with Android hardware manufacturers to help ensure APIs for background work are supported predictably and consistently across the ecosystem. We are excited to announce that Samsung, representing one of Android’s longest partnerships, is our first partner on this journey:

“To strengthen the Android platform, our collaboration with Google has resulted in a unified policy that we expect will create a more consistent and reliable user experience for Galaxy users. Since One UI 6.0, foreground services of apps targeting Android 14 will be guaranteed to work as intended so long as they are developed according to Android's new foreground service API policy.” - Samsung

As mentioned in the Android 14 Developer Preview 1 blog post, we have:

We believe our expanding partnerships with hardware manufacturers and these changes will make it easier for developers to create apps that work consistently across different Android devices.

We encourage you to try the new Android 14 APIs and let us know what you think using the Android 14 Issue Tracker. We welcome you to contribute to CTS-D tests to help catch consistency issues. Lastly, If you see behavior differences across Android devices, be sure to file a ticket using goo.gle/devicespecificissue to bring it to our attention.

How to optimize your Android app for large screens (And what NOT to do!)

Posted by the Android team

Large foldables, tablets, and desktop devices like Chromebooks – with more active large screen Android devices each year, it’s more important than ever for apps to ensure they provide their users with a seamless experience on large screens. For example, these devices offer more screen space, and users expect apps to do more with that space. We’ve seen that apps enjoy better business metrics on these devices when they do work to support them.

These devices can also be used in different places and in different ways than we might expect on a handset. For example foldables can be used in tabletop mode, users may sit further away from a desktop display, and many large screen devices may be used with a mouse and keyboard.

These differences introduce new things to consider. For example:
  • Can a user reach the most important controls when using your app with two hands on a tablet?
  • Does all of your app’s functionality work with a keyboard and mouse?
  • Does your app’s camera preview have the right orientation regardless of the position of the device?
image showing differentiated experiences across large sceen devices

Large Screen Design and Quality

Defining great design and quality on large screens can be difficult, because different apps will have different ways to excel. You know your product best, so it’s important to use your app on large screen devices and reflect on what will provide the best experience. If you don’t have access to a large screen device; try one of the foldable, desktop, or tablet virtual devices.

Google also provides resources thoughout the development process to help as you optimize your app. If you’re looking for design guidance, there are thoughtful design resources like the large screen Material Design guidance and ready to use compositions like the Canonical layouts. For inspiration, there are great examples of a variety of different apps in the large screens gallery. If you’re looking for a structured way to approach large screen quality, the Large screen app quality guidelines provide a straight-forward checklist and a set of tests to give you confidence that your app is ready for large screens.


Dos and Don’ts of Optimizing for Large Screens

Whether you already have beautiful large screen designs or not, we want to highlight some helpful tips and common mistakes to avoid when optimizing your app for large screens.

Don’t: assume exclusive access to resources

  • Don’t assume you have exclusive access to hardware resources like the camera. Large screens commonly have more than one app active at a time, and those other apps may try to access the same resources.
  • This means you should test your app side by side simultaneously with other apps, and never assume a resource is available at any given time.

Do: handle hardware access gracefully

  • Check for hardware resources like the camera before trying to use them. Remember that hardware peripherals can be added and removed at any time via USB.
  • Fail gracefully when access to a given resource isn’t available at runtime.
try { // Attempt to use the camera ... } catch (e: CameraAccessException) { e.message?.let { Log.e(TAG, it) } // Fail gracefully if the camera isn't currently available } }

Do: respond appropriately to lifecycle events:

  • Your app may still be visible during onPause(), especially when multiple apps on onscreen, so you need to keep media playing and your UI fresh until onStop() is called

Don’t: stop your app’s UI in onPause()

override fun onPause() { //DON'T clean up resources here. //Your app can still be visible. super.onPause() }

Don’t: rely on device-type booleans like “isTablet”

  • In the past, a common pattern for apps to use was to leverage screen width to create a boolean like “isTablet” to make changes based on the kind of device the app is running on, but this approach is fragile. The core problem with this approach is that it looks for a proxy to determine what the device type is, and those proxies are error-prone. For example, if you determine a device is a tablet because it has a large display when your app launches, your app can behave incorrectly when its window is resized to not take up the full screen. Even if your device-type boolean responds to configuration changes, unfolding a foldable would change your experience in a way that it couldn’t return to until another configuration change occurs, such as refolding the device.

Do: work to replace existing uses of device-type booleans with the right approach

Query for the information about the device that’s necessary for what you’re trying to accomplish. For example:

  • If you’re using device-type booleans to adapt your layout, use WindowSizeClasses instead. The library has support for both Views and for Jetpack Compose, and it makes it clear and easy to adapt your UI to pre-defined breakpoints.
// androidx.compose.material3.windowsizeclass.WindowSizeClass class MainActivity : ComponentActivity() { … setContent { val windowSizeClass = calculateWindowSizeClass(this) WindowSizeClassDisplay(windowSizeClass) } @Composable fun WindowSizeClassDisplay(windowSizeClass : WindowSizeClass) { when (windowSizeClass.widthSizeClass) { WindowWidthSizeClass.Compact -> { compactLayout() } WindowWidthSizeClass.Medium -> { mediumLayout() } WindowWidthSizeClass.Expanded -> { expandedLayout() } } }
  • If you’re using isTablet for changing user facing strings like “your tablet”, you might not need any more information. The solution can be as simple as using more general phrasing such as “your Android device”.
  • If you’re using a device-type boolean to predict the presence of a hardware feature or resource (e.g. - telephony, bluetooth, etc), check for the desired capabilities directly at runtime before trying to use them, and fail gracefully when they become unavailable. This feature-based approach ensures that your app can respond appropriately to peripheral devices that can be attached or removed. It also avoids cases where a feature is missing even though it could be supported by the device.
val packageManager: PackageManager = context.packageManager val hasTelephony = packageManager.hasSystemFeature(PackageManager.FEATURE_TELEPHONY)

Do: use Jetpack CameraX when possible

  • There can be a surprising amount of complexity in showing camera previews – orientation, aspect ratio, and more. When you use Jetpack CameraX, the library will handle as many of these details as possible for you.

Don’t: assume that your camera preview will align with device orientation

  • There are several kinds of orientation to consider when implementing a camera preview in your app - natural orientation, device orientation, and display orientation. Proper implementation of a camera preview requires accounting for the various kinds of orientation and adapting as the device’s conditions change.

Don’t: assume that aspect ratios are static

Do: declare hardware feature requirements correctly

  • When you’re declaring your app’s feature requirements, refer to the guidance in the Large Screens Cookbook. To ensure that you aren’t unnecessarily limiting your app’s reach, be sure to use the most inclusive manifest entries that work with your app.
<uses-feature android:name="android.hardware.camera.any" android:required="false" /> <uses-feature android:name="android.hardware.camera" android:required="false" /> <uses-feature android:name="android.hardware.camera.autofocus" android:required="false" /> <uses-feature android:name="android.hardware.camera.flash" android:required="false" />

Don’t: assume window insets are static

  • Large screens can change frequently, and that includes their WindowInsets. This means we can’t just check the insets when our app is launched and never change them.

Do: use the WindowInsetsListener APIs for Views

  • The WindowInsetsListener APIs notify your app when insets change
    ViewCompat.setOnApplyWindowInsetsListener(view) { view, windowInsets -> val insets = windowInsets.getInsets( WindowInsetsCompat.Type.systemBars()) view.updateLayoutParams<MarginLayoutParams>( leftMargin = insets.left, bottomMargin = insets.bottom, rightMargin = insets.right, ) WindowInsetsCompat.CONSUMED }

    Do: use the windowInsetsPadding Modifier for Jetpack Compose

    • The windowInsetsPadding Modifier will dynamically pad based on the given type of window insets. Additionally, multiple instances of the Modifier can communicate with each other to avoid adding duplicate padding, and they’re automatically animated.

    Don’t: assume the device has a touch screen

    Do: test your app on large screens

    • The most important thing you can do to ensure your app’s experience is great on large screens is to test it yourself. If you want a rigorous test plan that’s already prepared for you, try out the large screen compatibility tests.

    Do: leverage the large screen tools in Android Studio

    • Android Studio provides tools to use during development that make it much easier to optimize for large screens. For example, multipreview annotations allow you to visualize your app in many conditions at the same time. There’s also a wide variety of tablet, desktop, and foldable AVDs available in the Android Virtual Device Manager to help you test your app on large screens today.

    Stay tuned for Google I/O

    These tips are a great starting point as you optimize your app for large screens, and there are even more updates to come at Google I/O on May 10th. Tune in to watch the latest news and innovations from Google, with live streamed keynotes and helpful product updates on demand.