Tag Archives: Google Assistant

Empathy and trust: How 3 women influence Google products

This Women’s History Month, we’re celebrating Googlers who contribute to products and features that people rely on each day. Last week we heard from women working on our crisis hotlines and Street View. Today, we hear from three more Googlers about how they build products — like Maps, Google Assistant and Google News — with safety, empathy and collaboration in mind.

Making the internet safer for women

What do you love most about building at Google?

Few companies in the world have the level of impact that Google does. When I tell people I work on Google Maps, they tell me how much they love Maps and how it’s a part of their everyday life. That’s the best part of doing this job!

What is the best career advice another woman has given you?

I had a lot of self-doubt and uncertainty about returning to work after my daughter was born. During that period, my mother encouraged me to think beyond the initial anxiety and adapt to my new way of life as a working mother. Her simple advice was to take a long-term view of my life and happiness and make decisions based on that.

Why is it important to moderate things like reviews on Google Maps?

Women-owned businesses often face threats of trolling, shaming and digital sabotage through fake reviews and other malicious practices. To make Google Maps universally safe and trusted — especially for women — we must constantly invest in fighting abuse and misinformation.

Bringing empathy to technology

What’s early career advice you’ve gotten from another woman?

When women don’t meet every qualification on a job description, they often assume they’re not qualified for the job. A friend gave me the advice to believe in my capabilities and tell the story of what made me qualified in the moment, instead of assuming that I needed to do more or be more. It’s a mindset shift that’s allowed me to stretch and grow in unexpected ways.

When you create experiences with this in mind, it keeps you focused on being of service to people as opposed to being of service to goals or ideas.

Your work focuses on making the Assistant more empathetic. Why is empathy an important part of the user experience?

Empathy is an important part of the human experience. We need each other, and we all want to feel understood and cared for. When you create experiences with this in mind, it keeps you focused on being of service to people as opposed to being of service to goals or ideas.

Other than Google Assistant, what’s your favorite Google product or feature?

I’d be lost without Maps.

Collaborating across perspectives

How have your experiences impacted the way you build products?

Before joining Google, I’ve felt at times that my voice wasn’t heard; this led me to approach my current role with empathy and curiosity. I have a strong desire to deeply understand and incorporate a wide array of perspectives — from considering the needs of people using Google News to collaborating with others internally to create different ways to deliver essential news through the app. I believe this helps us build more inclusive experiences for people with a wide range of circumstances and needs.

Making sure everyone has equal access to this essential information empowers everyone, including women, and fuels a well-functioning society.

Why is it important for women to access quality news and information?

Everyone needs to be able to access reliable and authoritative news to make informed decisions, engage in their local community, have conversations with friends and family, and stay safe. Making sure everyone has equal access to this essential information empowers everyone, including women, and fuels a well-functioning society.

Empathy and trust: How 3 women influence Google products

This Women’s History Month, we’re celebrating Googlers who contribute to products and features that people rely on each day. Last week we heard from women working on our crisis hotlines and Street View. Today, we hear from three more Googlers about how they build products — like Maps, Google Assistant and Google News — with safety, empathy and collaboration in mind.

Making the internet safer for women

What do you love most about building at Google?

Few companies in the world have the level of impact that Google does. When I tell people I work on Google Maps, they tell me how much they love Maps and how it’s a part of their everyday life. That’s the best part of doing this job!

What is the best career advice another woman has given you?

I had a lot of self-doubt and uncertainty about returning to work after my daughter was born. During that period, my mother encouraged me to think beyond the initial anxiety and adapt to my new way of life as a working mother. Her simple advice was to take a long-term view of my life and happiness and make decisions based on that.

Why is it important to moderate things like reviews on Google Maps?

Women-owned businesses often face threats of trolling, shaming and digital sabotage through fake reviews and other malicious practices. To make Google Maps universally safe and trusted — especially for women — we must constantly invest in fighting abuse and misinformation.

Bringing empathy to technology

What’s early career advice you’ve gotten from another woman?

When women don’t meet every qualification on a job description, they often assume they’re not qualified for the job. A friend gave me the advice to believe in my capabilities and tell the story of what made me qualified in the moment, instead of assuming that I needed to do more or be more. It’s a mindset shift that’s allowed me to stretch and grow in unexpected ways.

When you create experiences with this in mind, it keeps you focused on being of service to people as opposed to being of service to goals or ideas.

Your work focuses on making the Assistant more empathetic. Why is empathy an important part of the user experience?

Empathy is an important part of the human experience. We need each other, and we all want to feel understood and cared for. When you create experiences with this in mind, it keeps you focused on being of service to people as opposed to being of service to goals or ideas.

Other than Google Assistant, what’s your favorite Google product or feature?

I’d be lost without Maps.

Collaborating across perspectives

How have your experiences impacted the way you build products?

Before joining Google, I’ve felt at times that my voice wasn’t heard; this led me to approach my current role with empathy and curiosity. I have a strong desire to deeply understand and incorporate a wide array of perspectives — from considering the needs of people using Google News to collaborating with others internally to create different ways to deliver essential news through the app. I believe this helps us build more inclusive experiences for people with a wide range of circumstances and needs.

Making sure everyone has equal access to this essential information empowers everyone, including women, and fuels a well-functioning society.

Why is it important for women to access quality news and information?

Everyone needs to be able to access reliable and authoritative news to make informed decisions, engage in their local community, have conversations with friends and family, and stay safe. Making sure everyone has equal access to this essential information empowers everyone, including women, and fuels a well-functioning society.

New Android features for messaging, entertainment and more

The Android team has been busy working on a suite of updates that add new dimensions to the apps you already love. From huge updates to messaging (including more connection with your iPhone friends) to powerful photo editing tools and even smarter ways to pay for parking, these upgrades bring more helpful technology to messaging, entertainment and more.

Share more with your iPhone friends, and stay on top of your inbox with Messages

An illustration of an Android user seeing the text reactions from an iPhone user
10:25

Feel the ❤️. Or the ?. Or whichever emoji your friends are feeling in the moment. Reactions from iPhone users will now appear as emoji on text messages — just like when you’re messaging with someone using an Android device. And now everyone can enjoy your videos in the same resolution you do when you send them as Google Photos links right inside the conversation. Soon, you’ll be able to send your photos this way, too.

New birthday reminders in Messages means you'll never miss their special day.

That’s not all that’s coming to Messages. There are also updates that help you break through the clutter to focus on the conversations that matter most:

  • Automatically sort your messages into Personal and Business tabs with an organized inbox, so you can find what you need faster.
  • Reduce the clutter of one-time password messages by having them automatically deleted after 24 hours.
  • Get gentle nudges to reply to messages you may have missed or need to follow up on.
  • Never forget to wish your friends a happy birthday, with reminders to reach out to your friends on their special day.

Read more about the new messaging updates in our blog post, and explore them on your device as they roll out over the coming weeks.

Type what you really want to say with grammar correction

Animation of an Android phone on a messages conversation. As the user types, Gboard offers suggestions to improve the sentence.

Grammar correction on Gboard helps you type the way you want to be heard.

More than just a spell check, the new grammar correction feature on Gboard works entirely on your device to detect grammatical errors and offer suggestions to help you bring your thoughts to life.

When words aren’t enough to tell your story, Emoji Kitchen now has more than 2,000 new emoji mashups available as stickers via Gboard. You can make it rain ?, share your love for ? or make your favorite emoji sparkle ? in an endless array of messaging possibilities. And for Pixel users, Gboard will convert your words into colorful stickers built with your exact text when typing in messaging apps in English (U.S.).

Accessible conversations, even when you’re offline, with Live Transcribe

Animation of an Android phone demonstrating the Live Transcribe app. As the user speaks, their words are automatically converted into large text on the screen.

Live Transcribe now works wherever you go, even if you don't have Wi-Fi or data.

Developed in collaboration with the premier university for people who are deaf or hard-of-hearing, Gallaudet University, Live Transcribe provides real-time speech-to-text captions to enable everyday in-person conversations between people who are deaf, hard of hearing, and hearing. Preinstalled on Pixel and Samsung devices and available as a free download to all, the app now offers an offline mode for when Wi-Fi and data aren’t available, like on an airplane, in the subway, or in other areas without consistent internet access.

Turn more photos into portraits with Portrait Blur

Illustration of an Android phone Blurring the background of a pet photo.

Portrait Blur now can be used on photos of more subjects, like pets, food and plants, too. Even if the picture was taken years ago.

If you’re a Google One member or a Pixel user, you can already use Portrait Blur in Google Photos, a powerful photo editing tool that can intelligently blur the background on photos of people, post-snap. Coming soon to the Photos app on Android, these users will be able to use this effect on photos of more subjects, like pets, food and plants, too. Even if the picture was taken years ago. Even if portrait mode wasn’t on at the time. Whether it’s photos of your four-legged friends on vacation, an artistic shot of your prized houseplant or just what you had for lunch today, the only thing left to focus on is how you’re going to share them.

All the latest on all your favorites with Highlights on Google TV

Illustration of an Android user looking through Highlights on Google TV in front of his TV
10:25

The new Highlights tab on Google TV offers a personalized feed of entertainment news, reviews and more based on movies and shows you’re interested in. Catch up on articles about the entertainment you love, or track the next big sensation, all in one place.

Animation of an Android phone scrolling through a feed of articles and backstage videos from a selection of popular television shows.

Learn more about your current and soon-to-be favorite shows with Highlights on Google TV mobile.

And if you discover something new that you’re interested in, one tap will take you straight to the movie, TV series or video the article is based on, so you can play, rate or Watchlist it for later.

Pay for parking with Google Assistant

Animation of an Android phone, demonstrating a user following the steps to pay for parking using Google Assistant.

Pay for parking using just your voice, with a little help from Google Assistant.

Google Assistant and Google Pay can now help you pay for parking, check your parking status, and extend your parking using just your voice. Just say, “Hey Google, pay for parking” once you’ve parked, and follow Assistant prompts to pay from your phone. No more coins, no more confusion. And when you need to check your parking status or pay for more parking, just say “Hey Google, parking status” and “Hey Google, extend parking.” Available in ParkMobile street parking zones in over 400 cities in the US.

Fine-tune your digital habits with the screen time widget

Animation of an Android phone, demonstrating how the screen time widget can be resized on the Home screen. The app displays the users daily screen time, as well as their top three most used apps.

Customize your Home screen and fine-tune your digital habits with the screen time widget.

Staying on top of your screen time is all about balance. A new screen time widget helps get the ball rolling by offering a glimpse of the three apps you use the most each day. Tap the widget to open Digital Wellbeing, where you’ll find some helpful ways to fine-tune your digital habits, including:

  • Daily timers for your apps
  • Focus mode, which pauses distracting apps at set times
  • Bedtime mode, which silences your device and changes the screen to black and white at bedtime

Share with more people with Nearby Share

Animation of an Android phone using Nearby Share to send a scenic photo to multiple nearby Android friends.

Use Nearby Share to share files with more people around you.

Nearby Share already lets you easily share photos, videos, documents, links, audio files or entire folders between nearby devices. A new update makes it easy to share with multiple people rather than just one person at a time. Simply tap to add more recipients during a transfer. It’s rolling out now on all Android 6+ devices.

These new features make it easier than ever to share new ideas, enjoy your favorite entertainment, and explore the world around you — just another set of features as your Android device becomes more helpful all the time.

Treats, trends and touchdowns — it’s game time

We are just days away from the biggest night in football and Google is here to help you prep. Warm-up with some trending food and stats, get ready to watch the game with Google TV, tackle football-related queries with Assistant and re-watch all your favorite commercials on AdBlitz. Start to finish, we’ve got you covered.

Gameday stats

It isn’t a gameday without delicious food. This time last year, we saw searches for wings increased more than 130% compared to a typical Sunday in 2021 on Google Maps, and around this time every year, we see search interest in buffalo wings spike. This year, we decided to take a look at what the most-searched wing flavors were across the United States.

Map of the United States showing which the most-searched wing flavors were by state.

But Sunday isn't just about food: We also checked out regional search interest in this year’s teams and quarterbacks.

Map of the United States showing which quarterback was the most-searched by state.
Map of the United States showing which football team was the most-searched by state.

And the big game isn’t complete without an epic halftime show. Take a look at this year’s performers ranked by search interest over the past year.

Most-searched 2022 halftime artists over the past year in the U.S.

  1. Eminem
  2. Snoop Dogg
  3. Dr. Dre
  4. Kendrick Lamar
  5. Mary J. Blige

Say “Hey Google” to get your head in the game

Get into the game day spirit with Google Assistant, which is ready to answer all of your questions about football’s biggest game day. Start with some of these questions that will have you sounding like a pro:

  • “Hey Google, who’s going to win the big game?”
  • “Hey Google, help me talk like a football fan.”
  • "Hey Google, Touchdown!"
  • “Hey Google, give me facts about football.”

From endzone to AdBlitz

The commercials are (almost) just as important as the game, and you can find big game ads on the YouTube AdBlitz channel. There you can stream playlists featuring the most comedic, dramatic, action-packed and inspirational commercials and teasers. YouTube lets you decide which brands scored big with their game day ad, too — and the five ads with the most views by February 20 will be given the honor of entering the AdBlitz winners’ circle.

Catch the game with Google TV

Tune into the big game on Sunday and stream everything from kickoff to the winning play with your Google TV device, other Android TV OS device or in the Google TV app on Android devices in the U.S. Coverage begins at 12pm ET on NBC and Peacock, or on the NBC channel in your preferred live TV app with a YouTube TV or Sling TV subscription.

If you’ve recently bought a new Google TV or other Android TV OS device, you can get started by enjoying six months of Peacock Premium at no extra cost (terms apply). Just head to your Apps tab and scroll down to find your Peacock offer.

CES 2022: Making the most of your drive with Google

Editor's note: Google planned to attend CES 2022 but pulled out of having an in-person presence due to concerns over COVID-19.

Google’s vision of helpfulness on the road is to provide a safe and seamless connected experience. Today, we’re introducing several updates for cars compatible with Android Auto and cars with Google built-in.

Your Android phone as your car key

With digital car key, you can now use select Samsung Galaxy or Google Pixel phones to lock, unlock and start supported BMW vehicles from your phone. Starting later this year, phones with Ultra Wideband (UWB) technology will allow you to unlock your compatible car without taking your phone out at all, and key sharing will allow you to securely and remotely share your digital car key to trusted friends and family. We’re working to bring these features to even more Android phones and vehicles later this year.

Gif of a phone being placed on a car door handle to unlock it.

Unlock your car with your mobile phone and a simple tap.

A more helpful drive with Android Auto

Now available on nearly 150 million cars around the world, Android Auto brings the best of Google onto your car display via your Android phone. Most of the world's top car brands are bringing wireless Android Auto to their newest models, meaning you never need a cord to connect. And with just one tap, Fast Pair makes it easier than ever to connect your phone and wireless Android Auto. It’s available today on select BMW and Ford cars, with more to come. Also, with new accessories like the Motorola MA1 wireless adapter, you can now enjoy the wireless Android Auto experience even if your car doesn't have wireless support built-in. Connected via USB, the adapter stays in your car and wirelessly projects Android Auto from your phone to your car display as soon as you step inside.

You can already enjoy thousands of media, messaging, navigation, parking and charging apps that support Android Auto. And this summer, we’re bringing you more. We’re partnering with Lyft and Kakao Mobility to integrate their driver apps into Android Auto, so drivers will be able to view and accept rides right from their car display. And with access to points of interest apps like MochiMochi, Fuelio and Prezzi Benzina, you can explore more of the world, find and pay for all types of car services, and track your mileage and costs.

Image of car dash with Lyft app driver interface shown on car display.

Lyft Driver app for rideshare drivers will be projected onto their car display (UI for illustration purposes only and may not reflect final product).

New capabilities for cars with Google built-in

Car manufacturers like Polestar, Volvo Cars, and General Motors - with more to come - are already shipping select cars with Google technology built right into their infotainment systems, allowing you to get more use out of helpful features.

With Google Play built-in, you can soon download navigation apps like Sygic and Flitsmeister, charging apps like ChargePoint and PlugShare, parking apps like SpotHero and ParkWhiz and points of interest apps. In fact, later this year, you will be able to access video entertainment in the car while parked, starting with Volvo Cars with content from providers like YouTube. Suddenly, waiting for curbside pickup or for your car to charge isn't so boring anymore.

Image of car dashboard with Youtube video "Google Year in Search 202" playing.

Enjoy watching YouTube while parked on select Volvo Cars with Google built-in.

This deeper integration allows you to get help from Google beyond the car display. Coming this spring to Volvo Cars, you’ll be able to check on and adjust even more car settings by asking Google. From activating features like lane keeping assist to checking when your next service is due, your voice can replace looking around for the right button or scrolling through settings.

Smart home, meet smart car

With remote actions, you will soon be able to ask Google for help even before leaving the front door. Ask Google to warm up or cool down your car, lock and unlock it and check on the status of things like your battery charge, all from the comfort of your own home with the help of any Google Assistant-enabled device. This is coming first to select Volvo Cars, with more to follow.

Video showing woman using remote actions to pre-warm her car before setting off on a drive in the snow.
10:25

Whether you’re getting ready for a drive or already on the road, we’re working hard to make sure Google helps make every ride a safe, smart and seamless experience.

2021 Assistant Recap

Posted by Jessica Dene Earley-Cha, Mike Bifulco and Toni Klopfenstein, Developer Relations Engineers for Google Assistant

We've reached the end of the year - and what a year it's been! Between all of our live (virtual) events including I/O, developer summits, meetups and more, there are a lot of highlights for App Actions, Smart Home Actions and Conversational Actions. Let's dive in and take a look.

App Actions

App Actions allows developers to extend their Android App to Google Assistant. App Actions integrates more cleanly with Android using new Android platform features. With the introduction of the beta shortcuts.xml configuration resource, expanding existing Android features and our latest Google Assistant Plug App Actions is moving closer to the Android platform.

App Actions Benefits:

  • Display app information on Google surfaces. Provide Android widgets for Assistant to display, offering inline answers, simple confirmations and brief interactions to users without changing context.
  • Launch features from Assistant. Connect your app's capabilities to user queries that match predefined semantic patterns (BII).
  • Suggest voice shortcuts from Assistant. Use Assistant to proactively suggest tasks for users to discover or replay, in the right context.

Core Integration

Capabilities is a new Android framework API that allows you to declare the types of actions users can take to launch your app and jump directly to performing a specific task. Assistant provides the first available concrete implementation of the capabilities API. You can utilize capabilities by creating a shortcuts.xml resource and defining your capabilities. Capabilities specify two things: how it's triggered and what to do when it's triggered. To add a capability, you’ll need to select a Built-In intent (BII), which are pre-built language models that provide all the Natural Language Understanding to map the user's input to individual fields. When a BII is matched by the user’s request, your capability will trigger an Android Intent that delivers the understood BII fields to your app, so you can determine what to show in response.

To support a user query like “Hey Google, Find waterfall hikes on ExampleApp,” you can use the GET_THING BII. This BII supports queries that request an “item” and extracts the “item” from the user query as the parameter thing.name. The best use case for the GET_THING BII is to search for things in the app. Below is an example of a capability that uses the GET_THING BII: 

<!-- This is a sample shortcuts.xml -->
<shortcuts xmlns:android="http://schemas.android.com/apk/res/android">
  <capability android:name="actions.intent.GET_THING">
    <intent
      android:action="android.intent.action.VIEW"
      android:targetPackage="YOUR_UNIQUE_APPLICATION_ID"
      android:targetClass="YOUR_TARGET_CLASS">
      <!-- Eg. name = "waterfall hikes" -->
      <parameter
        android:name="thing.name"
        android:key="name"/>
    </intent>
  </capability>
</shortcuts>

This framework integration is in the Beta release stage, and will eventually replace the original implementation of App Actions that uses actions.xml. If your app provides both the new shortcuts.xml and old actions.xml, the latter will be disregarded.

Learn how to add your first capability with this codelab.

Voice shortcuts

Google Assistant suggests relevant shortcuts to users during contextually relevant times. Users can see what shortcuts they have by saying “Hey Google, shortcuts.”

Shortcut for Google Assistant

You can use the Google Shortcuts Integration library, currently in beta, to push an unlimited number of dynamic shortcuts to Google to make your shortcuts visible to users as voice shortcuts. Assistant can suggest relevant shortcuts to users to help make it more convenient for the user to interact with your Android app.

Learn how to push your dynamic shortcuts to Assistant with our dynamic shortcuts codelab.

Example of App using Dynamic Shortcuts CodeLab Tool

Simple Answers, Hands Free & Android Auto

During situations where users need a hand free experience, like on Android Auto, Assistant can display widgets to provide simple answers, brief confirmations and quick interactive experience as a response to a user’s inquiry. These widgets are displayed within the Assistant UI, and in order to implement a fully voice-forward interaction with your app, you can arrange for Assistant to speak a response with your widget, which is safe and natural for use in automobiles. A great re-engagement feature with widgets, is that a “Add this widget” chip can be included too!

Example of App using Dynamic Shortcuts CodeLab Tool

Re Engagement

Another re-engagement tool is In-App Promo SDK you can proactively suggest shortcuts in your app for actions that the user can repeat with a voice command to Assistant, in beta. The SDK allows you to check if the shortcut you want to suggest already exists for that user and prompt the user to create the suggested shortcut.

New Tooling

To support testing Capabilities, the Google Assistant plugin for Android Studio was launched. It contains an updated App Action Test Tool that creates a preview of your App Action, so you can test an integration before publishing it to the Play store.

New App Actions resources

Learn more with new or updated content:


Smart Home Actions

A big focus of this year's Smart Home launches were new and updated tools. At events like I/O, Works With: SiLabs, and the Google Smart Home Developer Summit, we shared these new resources to help you quickly build a high quality smart home integration.

New Resources

To make implementing new features even easier for developers, we released many new tools to help you get your Smart Home Action up and running.

To help consumers discover Google-compatible smart home devices and associated routines, we released the smart home directory, accessible on the web and through the Google Home app.

We heard your requests for more ways to localize your integrations, so we added sample utterances in English (en-US), German (de-DE), and French (fr-FR) to several device guides. Additionally, we also rolled out Chinese (zh-TW) as one of the supported languages for the overall platform. To make our documentation more accessible, we added a Japanese translation of our developer guides.

We also released several new device types and traits, along with new features to support your integrations, including proactive and follow-up responses, app discovery and deep linking.

Quality Improvements

For general onboarding, we've added three new codelabs to enable you to dive deeper into debugging and monitoring your projects. You can now walk through debugging smart home Actions, debugging local fulfillment Actions, and dig deeper into your log-based metrics for your Actions.

When you're actively developing your integration, the Google Home Playground can simulate a virtual home with configurable device types and traits. Here you can view the types and traits in Home Graph, modify device attributes, and share device configurations.

If you discover issues with your configuration, we've continued upgrading the monitoring and logging dashboards to show you detailed views of events with your integrations, as well as better guidance on how to handle errors and exceptions.

The WebRTC Validator Tool acts as a WebRTC peer to stream to or from, and generally emulates the WebRTC player on smart displays with Google Assistant. If you're specifically working with a smart camera, WebRTC is now supported on the CameraStream trait.

Local Home

In order to continue striving towards quality responses to user queries, we also added support to the Local Home SDK to support local queries and responses. Additionally, to help users onboard new devices in their homes quickly and use Google Nest devices as local hubs, we launched BLE Seamless Setup.

Matter

The new Google Home IDE enables you to improve your development process by enabling in-IDE access to Google Assistant Simulator, Cloud Logging, and more for each of your projects. This plugin is available for VSCode.

Finally, as we get closer to the official launch of the Matter protocol, we're working hard to unify all of our smart home ecosystem tools together under a single name - Google Home. The Google Home Developer Center will enable you to quickly find resources for integrating your Matter-compatible smart devices and platforms with Nest, Android, Google Home app, and Google Assistant.

Conversational Actions

Way back in January of 2021, we rolled up an updated Actions for Families program, which provides guidelines for teams building actions meant for kids. Conversational Actions which are approved for the Actions for Families program get a special badge in the Assistant Directory, which lets parents know that your Action is family-friendly.

During the What's New in Google Assistant keynote at Google I/O, Director of Product for the Google Assistant Developer Platform Rebecca Nathenson mentioned several coming updates and changes for Conversational Actions. This included the launch of a Developer Preview for a new client-side fulfillment model for Interactive Canvas. Client-side fulfillment changes the implementation strategy for Interactive Canvas apps, removing the need for a webhook relaying information between the Assistant NLU and their web application. This simplifies the infrastructure needed to deploy an action that uses Interactive Canvas. Since the release of this Developer Preview, we’ve been listening closely to developers to get feedback on client-side fulfillment.

Interactive Canvas Developer Tools

We also released Interactive Canvas Developer tools - a Chrome extension which can help dev teams mock and debug the web app side of Interactive Canvas apps and games. Best of all, it’s open source! You can install the dev tools from the Chrome Web Store, or compile them from source yourself on GitHub at actions-on-google/interactive-canvas-dev-tools.

Updates to SSML

Earlier this year we announced support for new SSML features in Conversational Actions. This expanded support lets you build more detailed and nuanced features using text to speech. We produced a short demonstration of SSML Features on YouTube, and you can find more in our docs on SSML if you’re ready to dive in and start building already

Updates to Transaction UX for Smart Displays

Also announced at I/O for Conversational Actions - we released an updated workflow for completing transactions on smart displays. The new transaction process lets users complete transactions from their smart screens, by confirming the CVC code from their chosen payment method, rather than using a phone to enter a CVC code. If you’d like to get an idea of what the new process looks like, check out our demo video showing new transaction features on smart devices.

Tips on Launching your Conversational Action

Driving a successful launch for Conversational Actions contains helpful information to help you think through some strategies for putting together a marketing team and go-to-market plan for releasing your Conversational Action.

Looking forward to 2022

We're looking forward to another exciting year in 2022. To stay connected, sign up for our new App Actions email series or Google Home newsletter, or for the general Assistant newsletter.

As always, you can also join us on Reddit or follow us on Twitter. Happy Holidays!

Winter is coming: 9 ways to enjoy it with Google

As a native Oregonian, I thought living in California would be an incredible break from the nine months of rain I’d endured growing up. What I didn’t realize was that 70-degree winters felt…wrong. Where were the mittens? The down jackets? The occasional snowy days? I’ve since moved back to the Pacific Northwest, and I’ve had a renewed appreciation for winter weather.

In fact, I enjoy the chilly months of the year so much, I’ve put together a few ways to make the most of the cold weather.

  1. I love snowshoeing, and I always want to find new trails. I use Google Maps to look for mountain biking and hiking trails that are covered in snow in the winter. (Just look for the hiking icons, or the light dash lines that indicate trails.) If I come across a good one, I label it on Maps so I know how to get back.
Animated GIF showing trails on Google Maps and how you can select and label them; this one is being saved to a list called “trails.”

2. I’m a year-round runner, but once the temperature dips below 50 Fahrenheit and the roads get wet or icy, I need new gear — all of which I can find in one place using Google Shopping. You can select the Sports & Outdoors tab to browse — and turn on the deals filter for discounts.

3. And when I’m returning from a chilly run, I can use the Google Home app to turn on my Nest Thermostat before I get home, so I know I’m not wasting energy while I’m out and the house will be toasty when I come in. I also use Home & Away Routines so that Nest knows when I’m out and can adjust my temperature automatically.

4. OK fine, there’s one downside of winter weather, and that’s how early it gets dark. I use Google Assistant to notify me an hour before sunset so I can get outside for some sunshine before the sun goes down.

5. We’ve started cutting down our own Christmas tree, which is actually pretty easy to do. A quick Google Search for cutting down a tree on federal land will help you find a map (and how you can purchase a permit). Then you can just use Google Maps to take you to the right area.

6. If I’m feeling really adventurous and ready to hit the slopes, I’ll check out the Explore tool on google.com/travel. I can set my home as the point of origin and then select “skiing” under the Interests filter and see what ski towns I can visit.

Animated GIF showing the United States on Google Maps. The arrow selects the “interests” tab and then “skiing” to surface ski towns in different parts of the country.

7. I love a good Google Alert to stay up to date on what’s going on locally. Once November rolls around, I set one for “Oregon winter festivals.”

8. Pixel cameras take incredible photos in dimly lit areas, so using Night Sight for shots of light displays or snowy nights is a no-brainer. And if you’ve already snagged a Pixel 6 or Pixel 6 Pro, those photos will look even better: The new Pixel camera lets in 2.5 times as much light as the Pixel 5, and you can try out the new Motion Mode setting to capture an artsy falling snow pic.

Read Article

Take holiday photos with Night Sight in Portrait Mode

Take beautiful holiday photos with this new feature for Pixel 4a 5G and Pixel 5.

Read Article

9. Most winter nights, I make a real fire — but when I don’t feel like hauling in wood, there’s always a YouTube version, complete with crackle.

Personalize user journeys by Pushing Dynamic Shortcuts to Assistant

Posted by Jessica Dene Earley-Cha, Developer Relations Engineer

Like many other people who use their smartphone to make their lives easier, I’m way more likely to use an app that adapts to my behavior and is customized to fit me. Android apps already can support some personalization like the ability to long touch an app and a list of common user journeys are listed. When I long press my Audible app (an online audiobook and podcast service), it gives me a shortcut to the book I’m currently listening to; right now that is Daring Greatly by Brené Brown.

Now, imagine if these shortcuts could also be triggered by a voice command – and, when relevant to the user, show up in Google Assistant for easy use.

Wouldn't that be lovely?

Dynamic shortcuts on a mobile device

Well, now you can do that with App Actions by pushing dynamic shortcuts to the Google Assistant. Let’s go over what Shortcuts are, what happens when you push dynamic shortcuts to Google Assistant, and how to do just that!

Android Shortcuts

As an Android developer, you're most likely familiar with shortcuts. Shortcuts give your users the ability to jump into a specific part of your app. For cases where the destination in your app is based on individual user behavior, you can use a dynamic shortcut to jump to a specific thing the user was previously working with. For example, let’s consider a ToDo app, where users can create and maintain their ToDo lists. Since each item in the ToDo list is unique to each user, you can use Dynamic Shortcuts so that users' shortcuts can be based on their items on their ToDo list.

Below is a snippet of an Android dynamic shortcut for the fictional ToDo app. 

val shortcut = = new ShortcutInfoCompat.Builder(context, task.id)
  .setShortLabel(task.title)
  .setLongLabel(task.title)
  .setIcon(Icon.createWithResource(context, R.drawable.icon_active_task))
  .setIntent(intent)
  .build()

ShortcutManagerCompat.pushDynamicShortcut(context, shortcut)

Dynamic Shortcuts for App Actions

If you're pushing dynamic shortcuts, it's a short hop to make those same shortcuts available for use by Google Assistant. You can do that by adding the Google Shortcuts Integration library and a few lines of code.

To extend a dynamic shortcut to Google Assistant through App Actions, two jetpack modules need to be added, and the dynamic shortcut needs to include .addCapabilityBinding. 

val shortcut = = new ShortcutInfoCompat.Builder(context, task.id)
  .setShortLabel(task.title)
  .setLongLabel(task.title)
  .setIcon(Icon.createWithResource(context, R.drawable.icon_active_task))
  .addCapabilityBinding("actions.intent.GET_THING", "thing.name", listOf(task.title))
  .setIntent(intent)
  .build()

ShortcutManagerCompat.pushDynamicShortcut(context, shortcut)

The addCapabilityBinding method binds the dynamic shortcut to a capability, which are declared ways a user can launch your app to the requested section. If you don’t already have App Actions implemented, you’ll need to add Capabilities to your shortcuts.xml file. Capabilities are an expression of the relevant feature of an app and contains a Built-In Intent (BII). BIIs are a language model for a voice command that Assistant already understands, and linking a BII to a shortcut allows Assistant to use the shortcut as the fulfillment for a matching command. In other words, by having capabilities, Assistant knows what to listen for, and how to launch the app.

In the example above, the addCapabilityBinding binds that dynamic shortcut to the actions.intent.GET_THING BII. When a user requests one of their items in their ToDo app, Assistant will process their request and it’ll trigger capability with the GET_THING BII that is listed in their shortcuts.xml. 

<shortcuts xmlns:android="http://schemas.android.com/apk/res/android">
   <capability android:name="actions.intent.GET_THING">
       <intent
           android:action="android.intent.action.VIEW"
           android:targetPackage="YOUR_UNIQUE_APPLICATION_ID"
           android:targetClass="YOUR_TARGET_CLASS">
           <!-- Eg. name = the ToDo item -->
           <parameter
               android:name="thing.name"
               android:key="name"/>
       </intent>
   </capability>
</shortcuts>

So in summary, the process to add dynamic shortcuts looks like this:

1. Configure App Actions by adding two jetpack modules ( ShortcutManagerCompat library and Google Shortcuts Integration Library). Then associate the shortcut with a Built-In Intent (BII) in your shortcuts.xml file. Finally push the dynamic shortcut from your app.

2. Two major things happen when you push your dynamic shortcuts to Assistant:

  1. Users can open dynamic shortcuts through Google Assistant, fast tracking users to your content
  2. During contextually relevant times, Assistant can proactively suggest your Android dynamic shortcuts to users, displaying it on Assistant enabled surfaces.

Not too bad. I don’t know about you, but I like to test out new functionality in a small app first. You're in luck! We recently launched a codelab that walks you through this whole process.

Dynamic Shortcuts Codelab

Looking for more resources to help improve your understanding of App Actions? We have a new learning pathway that walks you through the product, including the dynamic shortcuts that you just read about. Let us know what you think!

Thanks for reading! To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AppActions to share what you’re working on. Can’t wait to see what you build!

10 ways Google Assistant can help you during the holidays

As fun as the holidays can be, they’re also filled with lots of to-do lists, preparation and planning. Before the hustle and bustle of the season begins, we wanted to share a few ways you can use Google Assistant to stay on top of things and do what matters most — spending quality time with family and friends.

  1. Get together over a good meal made easy with hands-free help in the kitchen. Surprise your family and friends with a new dish or dessert or find inspiration by saying, “Hey Google, find me Thanksgiving recipes.”
  2. …And if you happen to come across a few new favorites, tap on that recipe and ask your Assistant to save it for you by saying “Hey Google, add to my cookbook.” Then when it comes time for a holiday feast, all your recent recipes will be waiting for you on your Smart Display and will show up when you say “Hey Google, show me my cookbook.” Once you've gathered your ingredients, select the recipe you want to cook and say “Hey Google, start cooking” to get step-by-step instructions on your Smart Display.
Illustration of a Smart Display with a recipe on the screen. There is also a photo of a warm drink with whipped cream on the screen.

3. When the food is prepared and the table is set, let everyone know dinner is ready withBroadcast. Just say, “Hey Google, broadcast ‘dinner is ready.’”

4. How early is too early for festive music? The limit does not exist! And even if you don’t have something queued up, you can just say“Hey Google, play Christmas music.”

5. Want to avoid scrolling endlessly for gifts? Android users can use Assistant to browse shopping apps like Walmart with just their voice. If you have the Walmart app installed on your Android phone, try saying“Hey Google, search Walmart for bicycles.”

6. Avoid spending hours waiting on hold when you call to adjust travel plans or return a gift. Pixel users can take advantage of Hold For Me, where Google Assistant will wait on the line for you and let you know when a real person is ready to take your call.

7. Connect and feel close from anywhere with video calling. Make a group call with Duo supporting up to 32 people on your Nest Hub Max — or send a “happy holidays!” message using one of the fun AR effects on mobile devices. To start a Duo call, just say, “Hey Google, make a video call.”

8. Keep your family’s busy holiday schedule on track with Family Bell from Google. Say “Hey Google, set up a Family Bell” to be reminded with delightful sounds on your speakers or smart displays when it’s time to tackle important moments of your day, like holiday meals or volunteering at the local gift drive. And for routines that require a little extra work — like getting the kids to bed after a get together — create a Family Bell checklist on your Smart Display with get ready bells that remind them of key tasks to complete, like brushing their teeth and putting on pajamas.

9. Have some fun and create new memories with a hands-free family game night. Put your game face on and say, “Hey Google, let’s play a game.”

10. Spark some holiday magic with a story from Google. We’ve added a new interactive story from Grabbit, a twist on the classic fairytale, “Hansel and Gretel.” Play the story from either the perspective of Hansel and Gretel or the Witch, and decide how the story unfolds. Just say “Hey Google, talk to Twisted Hansel and Gretel” and let the adventure begin! More interactive stories from Grabbit like “Jungle Book,” “Alice in Wonderland” and “Sherlock Holmes” will soon be available on your Google Nest smart display devices between now and the new year.

A communication tool for people with speech impairments

For millions of people, being able to speak and be understood can be difficult as a result of conditions that can impact speech, including stroke, ALS, Cerebral Palsy, traumatic brain injury or Parkinson's disease. Today, we’re inviting an initial group of people to test Project Relate, a new Android app that aims to help people with speech impairments communicate more easily with others and interact with the Google Assistant.

Project Relate is a continuation of years of research from both Google’s Speech and Research teams, made possible by over a million speech samples recorded by participants of our research effort. We are now looking for English-speaking testers in Australia, Canada, New Zealand and the United States to try out the app and provide feedback to help us improve it.

As an early tester of Project Relate, you will be asked to record a set of phrases. The app will use these phrases to automatically learn how to better understand your unique speech patterns, and give you access to the app's three main features: Listen, Repeat and Assistant.

Listen: Through the Listen feature, the Relate app transcribes your speech to text in real time, so you can copy-paste text into other apps, or let people read what you want to tell them.

Repeat: You can use the Repeat feature to restate what you’ve said using a clear, synthesized voice. We hope this can be especially helpful in face-to-face conversation or even when you want to speak a command to your home assistant device.

Assistant: Speak directly to your Google Assistant from within the Relate app, so you can take care of different tasks, such as turning on the lights or playing a song, with ease.

In creating the app, we worked closely with many people with speech impairments, including Aubrie Lee, a brand manager at Google, whose speech is affected by muscular dystrophy. “I’m used to the look on people’s faces when they can’t understand what I’ve said,” Aubrie shared with us. “Project Relate can make the difference between a look of confusion and a friendly laugh of recognition.” Since Aubrie works on the marketing team that names new products, she also helped us name the app!

If you have a condition that makes your speech difficult to understand, you may be able to help provide feedback on the Project Relate Android app as a trusted tester. To express interest, please fill out our interest form at g.co/ProjectRelate, and the team will get back to you in the coming months.

With your help, we hope to build a future in which people with disabilities can more easily communicate and be understood.