Create and edit images with Gemini 2.0 in preview

Gemini 2.0 Flash's image generation capabilities, now available in preview in Google AI Studio and Vertex AI, feature higher rate limits, enhanced visual quality, more precise text rendering, and more, allowing developers to create applications for product recontextualization, collaborative image editing, and dynamic SKU generation.

Beta Channel Update for ChromeOS / ChromeOS Flex

The Beta channel is being updated to OS version 16238.41.0 (Browser version 136.0.7103.96) for most ChromeOS devices.

If you find new issues, please let us know one of the following ways:



  1. File a bug

  2. Visit our ChromeOS communities

    1. General: Chromebook Help Community

    2. Beta Specific: ChromeOS Beta Help Community

  3. Report an issue or send feedback on Chrome

  4. Interested in switching channels? Find out how.



Andy Wu
Google ChromeOS

Building delightful Android camera and media experiences

Posted by Donovan McMurray, Mayuri Khinvasara Khabya, Mozart Louis, and Nevin Mital – Developer Relations Engineers

Hello Android Developers!

We are the Android Developer Relations Camera & Media team, and we’re excited to bring you something a little different today. Over the past several months, we’ve been hard at work writing sample code and building demos that showcase how to take advantage of all the great potential Android offers for building delightful user experiences.

Some of these efforts are available for you to explore now, and some you’ll see later throughout the year, but for this blog post we thought we’d share some of the learnings we gathered while going through this exercise.

Grab your favorite Android plush or rubber duck, and read on to see what we’ve been up to!

Future-proof your app with Jetpack

Nevin Mital

One of our focuses for the past several years has been improving the developer tools available for video editing on Android. This led to the creation of the Jetpack Media3 Transformer APIs, which offer solutions for both single-asset and multi-asset video editing preview and export. Today, I’d like to focus on the Composition demo app, a sample app that showcases some of the multi-asset editing experiences that Transformer enables.

I started by adding a custom video compositor to demonstrate how you can arrange input video sequences into different layouts for your final composition, such as a 2x2 grid or a picture-in-picture overlay. You can customize this by implementing a VideoCompositorSettings and overriding the getOverlaySettings method. This object can then be set when building your Composition with setVideoCompositorSettings.

Here is an example for the 2x2 grid layout:

object : VideoCompositorSettings {
  ...

  override fun getOverlaySettings(inputId: Int, presentationTimeUs: Long): OverlaySettings {
    return when (inputId) {
      0 -> { // First sequence is placed in the top left
        StaticOverlaySettings.Builder()
          .setScale(0.5f, 0.5f)
          .setOverlayFrameAnchor(0f, 0f) // Middle of overlay
          .setBackgroundFrameAnchor(-0.5f, 0.5f) // Top-left section of background
          .build()
      }

      1 -> { // Second sequence is placed in the top right
        StaticOverlaySettings.Builder()
          .setScale(0.5f, 0.5f)
          .setOverlayFrameAnchor(0f, 0f) // Middle of overlay
          .setBackgroundFrameAnchor(0.5f, 0.5f) // Top-right section of background
          .build()
      }

      2 -> { // Third sequence is placed in the bottom left
        StaticOverlaySettings.Builder()
          .setScale(0.5f, 0.5f)
          .setOverlayFrameAnchor(0f, 0f) // Middle of overlay
          .setBackgroundFrameAnchor(-0.5f, -0.5f) // Bottom-left section of background
          .build()
      }

      3 -> { // Fourth sequence is placed in the bottom right
        StaticOverlaySettings.Builder()
          .setScale(0.5f, 0.5f)
          .setOverlayFrameAnchor(0f, 0f) // Middle of overlay
          .setBackgroundFrameAnchor(0.5f, -0.5f) // Bottom-right section of background
          .build()
      }

      else -> {
        StaticOverlaySettings.Builder().build()
      }
    }
  }
}

Since getOverlaySettings also provides a presentation time, we can even animate the layout, such as in this picture-in-picture example:

moving image of picture in picture on a mobile device

Next, I spent some time migrating the Composition demo app to use Jetpack Compose. With complicated editing flows, it can help to take advantage of as much screen space as is available, so I decided to use the supporting pane adaptive layout. This way, the user can fine-tune their video creation on the preview screen, and export options are only shown at the same time on a larger display. Below, you can see how the UI dynamically adapts to the screen size on a foldable device, when switching from the outer screen to the inner screen and vice versa.

What’s great is that by using Jetpack Media3 and Jetpack Compose, these features also carry over seamlessly to other devices and form factors, such as the new Android XR platform. Right out-of-the-box, I was able to run the demo app in Home Space with the 2D UI I already had. And with some small updates, I was even able to adapt the UI specifically for XR with features such as multiple panels, and to take further advantage of the extra space, an Orbiter with playback controls for the editing preview.

moving image of suportive pane adaptive layout

What’s great is that by using Jetpack Media3 and Jetpack Compose, these features also carry over seamlessly to other devices and form factors, such as the new Android XR platform. Right out-of-the-box, I was able to run the demo app in Home Space with the 2D UI I already had. And with some small updates, I was even able to adapt the UI specifically for XR with features such as multiple panels, and to take further advantage of the extra space, an Orbiter with playback controls for the editing preview.

moving image of sequential composition preview in Android XR

Orbiter(
  position = OrbiterEdge.Bottom,
  offset = EdgeOffset.inner(offset = MaterialTheme.spacing.standard),
  alignment = Alignment.CenterHorizontally,
  shape = SpatialRoundedCornerShape(CornerSize(28.dp))
) {
  Row (horizontalArrangement = Arrangement.spacedBy(MaterialTheme.spacing.mini)) {
    // Playback control for rewinding by 10 seconds
    FilledTonalIconButton({ viewModel.seekBack(10_000L) }) {
      Icon(
        painter = painterResource(id = R.drawable.rewind_10),
        contentDescription = "Rewind by 10 seconds"
      )
    }
    // Playback control for play/pause
    FilledTonalIconButton({ viewModel.togglePlay() }) {
      Icon(
        painter = painterResource(id = R.drawable.rounded_play_pause_24),
        contentDescription = 
            if(viewModel.compositionPlayer.isPlaying) {
                "Pause preview playback"
            } else {
                "Resume preview playback"
            }
      )
    }
    // Playback control for forwarding by 10 seconds
    FilledTonalIconButton({ viewModel.seekForward(10_000L) }) {
      Icon(
        painter = painterResource(id = R.drawable.forward_10),
        contentDescription = "Forward by 10 seconds"
      )
    }
  }
}

Jetpack libraries unlock premium functionality incrementally

Donovan McMurray

Not only do our Jetpack libraries have you covered by working consistently across existing and future devices, but they also open the doors to advanced functionality and custom behaviors to support all types of app experiences. In a nutshell, our Jetpack libraries aim to make the common case very accessible and easy, and it has hooks for adding more custom features later.

We’ve worked with many apps who have switched to a Jetpack library, built the basics, added their critical custom features, and actually saved developer time over their estimates. Let’s take a look at CameraX and how this incremental development can supercharge your process.

// Set up CameraX app with preview and image capture.
// Note: setting the resolution selector is optional, and if not set,
// then a default 4:3 ratio will be used.
val aspectRatioStrategy = AspectRatioStrategy(
  AspectRatio.RATIO_16_9, AspectRatioStrategy.FALLBACK_RULE_NONE)
var resolutionSelector = ResolutionSelector.Builder()
  .setAspectRatioStrategy(aspectRatioStrategy)
  .build()

private val previewUseCase = Preview.Builder()
  .setResolutionSelector(resolutionSelector)
  .build()
private val imageCaptureUseCase = ImageCapture.Builder()
  .setResolutionSelector(resolutionSelector)
  .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
  .build()

val useCaseGroupBuilder = UseCaseGroup.Builder()
  .addUseCase(previewUseCase)
  .addUseCase(imageCaptureUseCase)

cameraProvider.unbindAll()

camera = cameraProvider.bindToLifecycle(
  this,  // lifecycleOwner
  CameraSelector.DEFAULT_BACK_CAMERA,
  useCaseGroupBuilder.build(),
)

After setting up the basic structure for CameraX, you can set up a simple UI with a camera preview and a shutter button. You can use the CameraX Viewfinder composable which displays a Preview stream from a CameraX SurfaceRequest.

// Create preview
Box(
  Modifier
    .background(Color.Black)
    .fillMaxSize(),
  contentAlignment = Alignment.Center,
) {
  surfaceRequest?.let {
    CameraXViewfinder(
      modifier = Modifier.fillMaxSize(),
      implementationMode = ImplementationMode.EXTERNAL,
      surfaceRequest = surfaceRequest,
     )
  }
  Button(
    onClick = onPhotoCapture,
    shape = CircleShape,
    colors = ButtonDefaults.buttonColors(containerColor = Color.White),
    modifier = Modifier
      .height(75.dp)
      .width(75.dp),
  )
}

fun onPhotoCapture() {
  // Not shown: defining the ImageCapture.OutputFileOptions for
  // your saved images
  imageCaptureUseCase.takePicture(
    outputOptions,
    ContextCompat.getMainExecutor(context),
    object : ImageCapture.OnImageSavedCallback {
      override fun onError(exc: ImageCaptureException) {
        val msg = "Photo capture failed."
        Toast.makeText(context, msg, Toast.LENGTH_SHORT).show()
      }

      override fun onImageSaved(output: ImageCapture.OutputFileResults) {
        val savedUri = output.savedUri
        if (savedUri != null) {
          // Do something with the savedUri if needed
        } else {
          val msg = "Photo capture failed."
          Toast.makeText(context, msg, Toast.LENGTH_SHORT).show()
        }
      }
    },
  )
}

You’re already on track for a solid camera experience, but what if you wanted to add some extra features for your users? Adding filters and effects are easy with CameraX’s Media3 effect integration, which is one of the new features introduced in CameraX 1.4.0.

Here’s how simple it is to add a black and white filter from Media3’s built-in effects.

val media3Effect = Media3Effect(
  application,
  PREVIEW or IMAGE_CAPTURE,
  ContextCompat.getMainExecutor(application),
  {},
)
media3Effect.setEffects(listOf(RgbFilter.createGrayscaleFilter()))
useCaseGroupBuilder.addEffect(media3Effect)

The Media3Effect object takes a Context, a bitwise representation of the use case constants for targeted UseCases, an Executor, and an error listener. Then you set the list of effects you want to apply. Finally, you add the effect to the useCaseGroupBuilder we defined earlier.

moving image of sequential composition preview in Android XR
(Left) Our camera app with no filter applied. 
 (Right) Our camera app after the createGrayscaleFilter was added.

There are many other built-in effects you can add, too! See the Media3 Effect documentation for more options, like brightness, color lookup tables (LUTs), contrast, blur, and many other effects.

To take your effects to yet another level, it’s also possible to define your own effects by implementing the GlEffect interface, which acts as a factory of GlShaderPrograms. You can implement a BaseGlShaderProgram’s drawFrame() method to implement a custom effect of your own. A minimal implementation should tell your graphics library to use its shader program, bind the shader program's vertex attributes and uniforms, and issue a drawing command.

Jetpack libraries meet you where you are and your app’s needs. Whether that be a simple, fast-to-implement, and reliable implementation, or custom functionality that helps the critical user journeys in your app stand out from the rest, Jetpack has you covered!

Jetpack offers a foundation for innovative AI Features

Mayuri Khinvasara Khabya

Just as Donovan demonstrated with CameraX for capture, Jetpack Media3 provides a reliable, customizable, and feature-rich solution for playback with ExoPlayer. The AI Samples app builds on this foundation to delight users with helpful and enriching AI-driven additions.

In today's rapidly evolving digital landscape, users expect more from their media applications. Simply playing videos is no longer enough. Developers are constantly seeking ways to enhance user experiences and provide deeper engagement. Leveraging the power of Artificial Intelligence (AI), particularly when built upon robust media frameworks like Media3, offers exciting opportunities. Let’s take a look at some of the ways we can transform the way users interact with video content:

    • Empowering Video Understanding: The core idea is to use AI, specifically multimodal models like the Gemini Flash and Pro models, to analyze video content and extract meaningful information. This goes beyond simply playing a video; it's about understanding what's in the video and making that information readily accessible to the user.
    • Actionable Insights: The goal is to transform raw video into summaries, insights, and interactive experiences. This allows users to quickly grasp the content of a video and find specific information they need or learn something new!
    • Accessibility and Engagement: AI helps make videos more accessible by providing features like summaries, translations, and descriptions. It also aims to increase user engagement through interactive features.

A Glimpse into AI-Powered Video Journeys

The following example demonstrates potential video journies enhanced by artificial intelligence. This sample integrates several components, such as ExoPlayer and Transformer from Media3; the Firebase SDK (leveraging Vertex AI on Android); and Jetpack Compose, ViewModel, and StateFlow. The code will be available soon on Github.

moving images of examples of AI-powered video journeys
(Left) Video summarization  
 (Right) Thumbnails timestamps and HDR frame extraction

There are two experiences in particular that I’d like to highlight:

    • HDR Thumbnails: AI can help identify key moments in the video that could make for good thumbnails. With those timestamps, you can use the new ExperimentalFrameExtractor API from Media3 to extract HDR thumbnails from videos, providing richer visual previews.
    • Text-to-Speech: AI can be used to convert textual information derived from the video into spoken audio, enhancing accessibility. On Android you can also choose to play audio in different languages and dialects thus enhancing personalization for a wider audience.

Using the right AI solution

Currently, only cloud models support video inputs, so we went ahead with a cloud-based solution.Iintegrating Firebase in our sample empowers the app to:

    • Generate real-time, concise video summaries automatically.
    • Produce comprehensive content metadata, including chapter markers and relevant hashtags.
    • Facilitate seamless multilingual content translation.

So how do you actually interact with a video and work with Gemini to process it? First, send your video as an input parameter to your prompt:

val promptData =
"Summarize this video in the form of top 3-4 takeaways only. Write in the form of bullet points. Don't assume if you don't know"

val generativeModel = Firebase.vertexAI.generativeModel("gemini-2.0-flash")
_outputText.value = OutputTextState.Loading

viewModelScope.launch(Dispatchers.IO) {
    try {
        val requestContent = content {
            fileData(videoSource.toString(), "video/mp4")
            text(prompt)
        }
        val outputStringBuilder = StringBuilder()

        generativeModel.generateContentStream(requestContent).collect { response ->
            outputStringBuilder.append(response.text)
            _outputText.value = OutputTextState.Success(outputStringBuilder.toString())
        }

        _outputText.value = OutputTextState.Success(outputStringBuilder.toString())

    } catch (error: Exception) {
        _outputText.value = error.localizedMessage?.let { OutputTextState.Error(it) }
    }
}

Notice there are two key components here:

    • FileData: This component integrates a video into the query.
    • Prompt: This asks the user what specific assistance they need from AI in relation to the provided video.

Of course, you can finetune your prompt as per your requirements and get the responses accordingly.

In conclusion, by harnessing the capabilities of Jetpack Media3 and integrating AI solutions like Gemini through Firebase, you can significantly elevate video experiences on Android. This combination enables advanced features like video summaries, enriched metadata, and seamless multilingual translations, ultimately enhancing accessibility and engagement for users. As these technologies continue to evolve, the potential for creating even more dynamic and intelligent video applications is vast.

Go above-and-beyond with specialized APIs

Mozart Louis

Android 16 introduces the new audio PCM Offload mode which can reduce the power consumption of audio playback in your app, leading to longer playback time and increased user engagement. Eliminating the power anxiety greatly enhances the user experience.

Oboe is Android’s premiere audio api that developers are able to use to create high performance, low latency audio apps. A new feature is being added to the Android NDK and Android 16 called Native PCM Offload playback.

Offload playback helps save battery life when playing audio. It works by sending a large chunk of audio to a special part of the device's hardware (a DSP). This allows the CPU of the device to go into a low-power state while the DSP handles playing the sound. This works with uncompressed audio (like PCM) and compressed audio (like MP3 or AAC), where the DSP also takes care of decoding.

This can result in significant power saving while playing back audio and is perfect for applications that play audio in the background or while the screen is off (think audiobooks, podcasts, music etc).

We created the sample app PowerPlay to demonstrate how to implement these features using the latest NDK version, C++ and Jetpack Compose.

Here are the most important parts!

First order of business is to assure the device supports audio offload of the file attributes you need. In the example below, we are checking if the device support audio offload of stereo, float PCM file with a sample rate of 48000Hz.

       val format = AudioFormat.Builder()
            .setEncoding(AudioFormat.ENCODING_PCM_FLOAT)
            .setSampleRate(48000)
            .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
            .build()

        val attributes =
            AudioAttributes.Builder()
                .setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)
                .setUsage(AudioAttributes.USAGE_MEDIA)
                .build()
       
        val isOffloadSupported = 
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
                AudioManager.isOffloadedPlaybackSupported(format, attributes)
            } else {
                false
            }

        if (isOffloadSupported) {
            player.initializeAudio(PerformanceMode::POWER_SAVING_OFFLOADED)
        }

Once we know the device supports audio offload, we can confidently set the Oboe audio streams’ performance mode to the new performance mode option, PerformanceMode::POWER_SAVING_OFFLOADED.

// Create an audio stream
        AudioStreamBuilder builder;
        builder.setChannelCount(mChannelCount);
        builder.setDataCallback(mDataCallback);
        builder.setFormat(AudioFormat::Float);
        builder.setSampleRate(48000);

        builder.setErrorCallback(mErrorCallback);
        builder.setPresentationCallback(mPresentationCallback);
        builder.setPerformanceMode(PerformanceMode::POWER_SAVING_OFFLOADED);
        builder.setFramesPerDataCallback(128);
        builder.setSharingMode(SharingMode::Exclusive);
           builder.setSampleRateConversionQuality(SampleRateConversionQuality::Medium);
        Result result = builder.openStream(mAudioStream);

Now when audio is played back, it will be offloading audio to the DSP, helping save power when playing back audio.

There is more to this feature that will be covered in a future blog post, fully detailing out all of the new available APIs that will help you optimize your audio playback experience!

What’s next

Of course, we were only able to share the tip of the iceberg with you here, so to dive deeper into the samples, check out the following links:

Hopefully these examples have inspired you to explore what new and fascinating experiences you can build on Android. Tune in to our session at Google I/O in a couple weeks to learn even more about use-cases supported by solutions like Jetpack CameraX and Jetpack Media3!

Deprecation of Structured Data Files v7

Today we’re announcing the deprecation of Structured Data Files (SDF) v7. This version will sunset on November 4, 2025.

Migrate to SDF v7.1 or higher before the sunset date to avoid any interruption of service. See the differences between SDF v7 and v7.1 in the v7.1 release notes.

After November 4, 2025, the following changes will apply to all users:

If you run into issues or need help with your migration, please contact us using our new Display & Video 360 API Technical support contact form.

Chrome Beta for Desktop Update

The Beta channel has been updated to 137.0.7151.15 for Windows, Mac and Linux.

A partial list of changes is available in the Git log. Interested in switching release channels? Find out how. If you find a new issue, please let us know by filing a bug. The community help forum is also a great place to reach out for help or learn about common issues.

Chrome Release Team
Google Chrome

Zoho Achieves 6x Faster Logins with Passkey and Credential Manager Integration

Posted by Niharika Arora – Senior Developer Relations Engineer, Joseph Lewis – Staff Technical Writer, and Kumareshwaran Sreedharan – Product Manager, Zoho.

As an Android developer, you're constantly looking for ways to enhance security, improve user experience, and streamline development. Zoho, a comprehensive cloud-based software suite focused on security and seamless experiences, achieved significant improvements by adopting passkeys in their OneAuth Android app.

Since integrating passkeys in 2024, Zoho achieved login speeds up to 6x faster than previous methods and a 31% month-over-month (MoM) growth in passkey adoption.

This case study examines Zoho's adoption of passkeys and Android's Credential Manager API to address authentication difficulties. It details the technical implementation process and highlights the impactful results.

Overcoming authentication challenges

Zoho utilizes a combination of authentication methods to protect user accounts. This included Zoho OneAuth, their own multi-factor authentication (MFA) solution, which supported both password-based and passwordless authentication using push notifications, QR codes, and time-based one-time passwords (TOTP). Zoho also supported federated logins, allowing authentication through Security Assertion Markup Language (SAML) and other third-party identity providers.

Challenges

Zoho, like many organizations, aimed to improve authentication security and user experience while reducing operational burdens. The primary challenges that led to the adoption of passkeys included:

    • Security vulnerabilities: Traditional password-based methods left users susceptible to phishing attacks and password breaches.
    • User friction: Password fatigue led to forgotten passwords, frustration, and increased reliance on cumbersome recovery processes.
    • Operational inefficiencies: Handling password resets and MFA issues generated significant support overhead.
    • Scalability concerns: A growing user base demanded a more secure and efficient authentication solution.

Why the shift to passkeys?

Passkeys were implemented in Zoho's apps to address authentication challenges by offering a passwordless approach that significantly improves security and user experience. This solution leverages phishing-resistant authentication, cloud-synchronized credentials for effortless cross-device access, and biometrics (such as a fingerprint or facial recognition), PIN, or pattern for secure logins, thereby reducing the vulnerabilities and inconveniences associated with traditional passwords.

By adopting passkeys with Credential Manager, Zoho cut login times by up to 6x, slashed password-related support costs, and saw strong user adoption – doubling passkey sign-ins in 4 months with 31% MoM growth. Zoho users now enjoy faster, easier logins and phishing-resistant security.

Quote card reads 'Cloud Lion now enjoys logins that are 30% faster and more secure using passkeys – allowing us to use our thumb instead of a password. With passkeys, we can also protect our critical business data against phishing and brute force attacks.' – Fabrice Venegas, Founder, Cloud Lion (a Zoho integration partner)

Implementation with Credential Manager on Android

So, how did Zoho achieve these results? They used Android's Credential Manager API, the recommended Jetpack library for implementing authentication on Android.

Credential Manager provides a unified API that simplifies handling of the various authentication methods. Instead of juggling different APIs for passwords, passkeys, and federated logins (like Sign in with Google), you use a single interface.

Implementing passkeys at Zoho required both client-side and server-side adjustments. Here's a detailed breakdown of the passkey creation, sign-in, and server-side implementation process.

Passkey creation

Passkey creation in OneAuth on a small screen mobile device

To create a passkey, the app first retrieves configuration details from Zoho's server. This process includes a unique verification, such as a fingerprint or facial recognition. This verification data, formatted as a requestJson string), is used by the app to build a CreatePublicKeyCredentialRequest. The app then calls the credentialManager.createCredential method, which prompts the user to authenticate using their device screen lock (biometrics, fingerprint, PIN, etc.).

Upon successful user confirmation, the app receives the new passkey credential data, sends it back to Zoho's server for verification, and the server then stores the passkey information linked to the user's account. Failures or user cancellations during the process are caught and handled by the app.

Sign-in

The Zoho Android app initiates the passkey sign-in process by requesting sign-in options, including a unique challenge, from Zoho's backend server. The app then uses this data to construct a GetCredentialRequest, indicating it will authenticate with a passkey. It then invokes the Android CredentialManager.getCredential() API with this request. This action triggers a standardized Android system interface, prompting the user to choose their Zoho account (if multiple passkeys exist) and authenticate using their device's configured screen lock (fingerprint, face scan, or PIN). After successful authentication, Credential Manager returns a signed assertion (proof of login) to the Zoho app. The app forwards this assertion to Zoho's server, which verifies the signature against the user's stored public key and validates the challenge, completing the secure sign-in process.

Server-side implementation

Zoho's transition to supporting passkeys benefited from their backend systems already being FIDO WebAuthn compliant, which streamlined the server-side implementation process. However, specific modifications were still necessary to fully integrate passkey functionality.

The most significant challenge involved adapting the credential storage system. Zoho's existing authentication methods, which primarily used passwords and FIDO security keys for multi-factor authentication, required different storage approaches than passkeys, which are based on cryptographic public keys. To address this, Zoho implemented a new database schema specifically designed to securely store passkey public keys and related data according to WebAuthn protocols. This new system was built alongside a lookup mechanism to validate and retrieve credentials based on user and device information, ensuring backward compatibility with older authentication methods.

Another server-side adjustment involved implementing the ability to handle requests from Android devices. Passkey requests originating from Android apps use a unique origin format (android:apk-key-hash:example) that is distinct from standard web origins that use a URI-based format (https://example.com/app). The server logic needed to be updated to correctly parse this format, extract the SHA-256 fingerprint hash of the app's signing certificate, and validate it against a pre-registered list. This verification step ensures that authentication requests genuinely originate from Zoho's Android app and protects against phishing attacks.

This code snippet demonstrates how the server checks for the Android-specific origin format and validates the certificate hash:

val origin: String = clientData.getString("origin")

if (origin.startsWith("android:apk-key-hash:")) { 
    val originSplit: List<String> = origin.split(":")
    if (originSplit.size > 3) {
               val androidOriginHashDecoded: ByteArray = Base64.getDecoder().decode(originSplit[3])

                if (!androidOriginHashDecoded.contentEquals(oneAuthSha256FingerPrint)) {
            throw IAMException(IAMErrorCode.WEBAUTH003)
        }
    } else {
        // Optional: Handle the case where the origin string is malformed    }
}

Error handling

Zoho implemented robust error handling mechanisms to manage both user-facing and developer-facing errors. A common error, CreateCredentialCancellationException, appeared when users manually canceled their passkey setup. Zoho tracked the frequency of this error to assess potential UX improvements. Based on Android's UX recommendations, Zoho took steps to better educate their users about passkeys, ensure users were aware of passkey availability, and promote passkey adoption during subsequent sign-in attempts.

This code example demonstrates Zoho's approach for how they handled their most common passkey creation errors:

private fun handleFailure(e: CreateCredentialException) {
    val msg = when (e) {
        is CreateCredentialCancellationException -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_SETUP_CANCELLED", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "The operation was canceled by the user."
        }
        is CreateCredentialInterruptedException -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_SETUP_INTERRUPTED", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "Passkey setup was interrupted. Please try again."
        }
        is CreateCredentialProviderConfigurationException -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_PROVIDER_MISCONFIGURED", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "Credential provider misconfigured. Contact support."
        }
        is CreateCredentialUnknownException -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_SETUP_UNKNOWN_ERROR", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "An unknown error occurred during Passkey setup."
        }
        is CreatePublicKeyCredentialDomException -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_WEB_AUTHN_ERROR", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "Passkey creation failed: ${e.domError}"
        }
        else -> {
            Analytics.addAnalyticsEvent(eventProtocol: "PASSKEY_SETUP_FAILED", GROUP_NAME)
            Analytics.addNonFatalException(e)
            "An unexpected error occurred. Please try again."
        }
    }
}

Testing passkeys in intranet environments

Zoho faced an initial challenge in testing passkeys within a closed intranet environment. The Google Password Manager verification process for passkeys requires public domain access to validate the relying party (RP) domain. However, Zoho's internal testing environment lacked this public Internet access, causing the verification process to fail and hindering successful passkey authentication testing. To overcome this, Zoho created a publicly accessible test environment, which included hosting a temporary server with an asset link file and domain validation.

This example from the assetlinks.json file used in Zoho's public test environment demonstrates how to associate the relying party domain with the specified Android app for passkey validation.

[
    {
        "relation": [
            "delegate_permission/common.handle_all_urls",
            "delegate_permission/common.get_login_creds"
        ],
        "target": {
            "namespace": "android_app",
            "package_name": "com.zoho.accounts.oneauth",
            "sha256_cert_fingerprints": [
                "SHA_HEX_VALUE" 
            ]
        }
    }
]

Integrate with an existing FIDO server

Android's passkey system utilizes the modern FIDO2 WebAuthn standard. This standard requires requests in a specific JSON format, which helps maintain consistency between native applications and web platforms. To enable Android passkey support, Zoho did minor compatibility and structural changes to correctly generate and process requests that adhere to the required FIDO2 JSON structure.

This server update involved several specific technical adjustments:

      1. Encoding conversion: The server converts the Base64 URL encoding (commonly used in WebAuthn for fields like credential IDs) to standard Base64 encoding before it stores the relevant data. The snippet below shows how a rawId might be encoded to standard Base64:

// Convert rawId bytes to a standard Base64 encoded string for storage
val base64RawId: String = Base64.getEncoder().encodeToString(rawId.toByteArray())

      2. Transport list format: To ensure consistent data processing, the server logic handles lists of transport mechanisms (such as USB, NFC, and Bluetooth, which specify how the authenticator communicated) as JSON arrays.

      3. Client data alignment: The Zoho team adjusted how the server encodes and decodes the clientDataJson field. This ensures the data structure aligns precisely with the expectations of Zoho’s existing internal APIs. The example below illustrates part of the conversion logic applied to client data before the server processes it:

private fun convertForServer(type: String): String {
    val clientDataBytes = BaseEncoding.base64().decode(type)
    val clientDataJson = JSONObject(String(clientDataBytes, StandardCharsets.UTF_8))
    val clientJson = JSONObject()
    val challengeFromJson = clientDataJson.getString("challenge")
    // 'challenge' is a technical identifier/token, not localizable text.
    clientJson.put("challenge", BaseEncoding.base64Url()
        .encode(challengeFromJson.toByteArray(StandardCharsets.UTF_8))) 

    clientJson.put("origin", clientDataJson.getString("origin"))
    clientJson.put("type", clientDataJson.getString("type"))
    clientJson.put("androidPackageName", clientDataJson.getString("androidPackageName"))
    return BaseEncoding.base64().encode(clientJson.toString().toByteArray())
}

User guidance and authentication preferences

A central part of Zoho's passkey strategy involved encouraging user adoption while providing flexibility to align with different organizational requirements. This was achieved through careful UI design and policy controls.

Zoho recognized that organizations have varying security needs. To accommodate this, Zoho implemented:

    • Admin enforcement: Through the Zoho Directory admin panel, administrators can designate passkeys as the mandatory, default authentication method for their entire organization. When this policy is enabled, employees are required to set up a passkey upon their next login and use it going forward.
    • User choice: If an organization does not enforce a specific policy, individual users maintain control. They can choose their preferred authentication method during login, selecting from passkeys or other configured options via their authentication settings.

To make adopting passkeys appealing and straightforward for end-users, Zoho implemented:

    • Easy setup: Zoho integrated passkey setup directly into the Zoho OneAuth mobile app (available for both Android and iOS). Users can conveniently configure their passkeys within the app at any time, smoothing the transition.
    • Consistent access: Passkey support was implemented across key user touchpoints, ensuring users can register and authenticate using passkeys via:
        • The Zoho OneAuth mobile app (Android & iOS);

This method ensured that the process of setting up and using passkeys was accessible and integrated into the platforms they already use, regardless of whether it was mandated by an admin or chosen by the user. You can learn more about how to create smooth user flows for passkey authentication by exploring our comprehensive passkeys user experience guide.

Impact on developer velocity and integration efficiency

Credential Manager, as a unified API, also helped improve developer productivity compared to older sign-in flows. It reduced the complexity of handling multiple authentication methods and APIs separately, leading to faster integration, from months to weeks, and fewer implementation errors. This collectively streamlined the sign-in process and improved overall reliability.

By implementing passkeys with Credential Manager, Zoho achieved significant, measurable improvements across the board:

    • Dramatic speed improvements
        • 2x faster login compared to traditional password authentication.
        • 4x faster login compared to username or mobile number with email or SMS OTP authentication.
        • 6x faster login compared to username, password, and SMS or authenticator OTP authentication.
    • Reduced support costs
        • Reduced password-related support requests, especially for forgotten passwords.
        • Lower costs associated with SMS-based 2FA, as existing users can onboard directly with passkeys.
    • Strong user adoption & enhanced security:
        • Passkey sign-ins doubled in just 4 months, showing high user acceptance.
        • Users migrating to passkeys are fully protected from common phishing and password breach threats.
        • With 31% MoM adoption growth, more users are benefiting daily from enhanced security against vulnerabilities like phishing and SIM swaps.

Recommendations and best practices

To successfully implement passkeys on Android, developers should consider the following best practices:

    • Leverage Android's Credential Manager API:
        • Credential Manager simplifies credential retrieval, reducing developer effort and ensuring a unified authentication experience.
        • Handles passwords, passkeys, and federated login flows in a single interface.
    • Ensure data encoding consistency while migrating from other FIDO authentication solutions:
        • Make sure you handle consistent formatting for all inputs/outputs while migrating from other FIDO authentication solutions such as FIDO security keys.
    • Optimize error handling and logging:
        • Implement robust error handling for a seamless user experience.
        • Provide localized error messages and use detailed logs to debug and resolve unexpected failures.
    • Educate users on passkey recovery options:
        • Prevent lockout scenarios by proactively guiding users on recovery options.
    • Monitor adoption metrics and user feedback:
        • Track user engagement, passkey adoption rates, and login success rates to keep optimizing user experience.
        • Conduct A/B testing on different authentication flows to improve conversion and retention.

Passkeys, combined with the Android Credential Manager API, offer a powerful, unified authentication solution that enhances security while simplifying user experience. Passkeys significantly reduce phishing risks, credential theft, and unauthorized access. We encourage developers to try out the experience in their app and bring the most secure authentication to their users.

Get started with passkeys and Credential Manager

Get hands on with passkeys and Credential Manager on Android using our public sample code.

If you have any questions or issues, you can share with us through the Android Credentials issues tracker.