Tag Archives: actions on google

Join us for Google Assistant Developer Day on October 8

Posted by Baris Gultekin, Director, Product Management Google Assistant and
Payam Shodjai, Director, Product Management Google Assistant

More and more people turn to Google Assistant every day to help them get the most out of their phones and smart displays: From playing games to using their favorite app by voice, there are more opportunities than ever for developers to create new and engaging experiences for Google Assistant.

We welcome you to join us virtually at our Google Assistant Developer Day on Thursday, October 8, to learn more about new tools and features we’re building for developers to bring Google Assistant to mobile apps and Smart Displays and help drive discoverability and engagement via voice. This will also be a great chance to chat live with Google leaders and engineers on the team to get your questions answered.

You’ll hear from our product experts and partnership leads on best practices to integrate with Google Assistant to help users more easily engage with their favorite apps by voice. Other sessions will include in-depth conversations around native development on Google Assistant, with so much more.

We’ll also have guest speakers like: Garrett Gaudini, Head of Product at Postmates, Laurens Rutten, Founder & CEO of CoolGames, Corey Bozarth, VP of Product & Monetization at MyFitnessPal and many other, join us on stage to share their stories about how voice has transformed the way people interact with their apps and services.

Whether you build for mobile or smart home, these new tools will help make your content and services available to people who want to use their voice to get things done.

Registration is FREE! Head on over to the event website to register and check out the schedule.

Announcing New Smart Home SHED Types and Traits

Posted by Toni Klopfenstein, Developer Advocate

Back in April, we released the first set of Smart Home Entertainment Device (SHED) types, including TV, set-top box, and remote, as well as the traits AppSelector, InputSelector, MediaState, TransportControl, and Volume. We are excited to announce the release of new Smart Home Entertainment Device (SHED) types and traits. These new device types and traits compliment the original set we released earlier this year, and help build out a more complete solution for smart home media and gaming devices. By implementing these types and traits on your entertainment devices, you can enable users to fully access device and media controls from any Assistant surface.

SHED Types and Traits

To expand the SHED options, we've released the following new device types for Smart Home:

  • Audio-video receiver
  • Streaming box
  • Streaming stick
  • Soundbar
  • Streaming soundbar
  • Speaker

We've also released the following new trait:

  • Channel

To ensure a consistent, high-quality experience for all end users, each of these device types require your service to report activityState and playbackState to Google using the ReportState API. This requirement improves the portability between media devices and helps the Assistant better understand user intents for these devices. By implementing the complete set of recommended device traits, you can further improve the quality of your smart home Action and improve device targeting for media playback command fulfilment.

For more information on how to implement these new device features, check out the docs and samples. You can also join us at our "Hey Google" Smart Home Virtual Summit to learn more about these new features.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Announcing Enhanced Smart Home Analytics

Posted by Toni Klopfenstein, Developer Advocate

When creating scalable applications, consistent and reliable monitoring of resources is a valuable tool for any developer. Today we are releasing enhanced analytics and logging for Smart Home Actions. This feature enables you to more quickly identify and respond to errors or quality issues that may arise.

Request Latency Dashboard

You can now access the smart home dashboard with pre-populated metrics charts for your Actions on the Analytics tab in the Actions Console, or through Cloud Monitoring. These metrics help you quantify the health and usage of your Action, and gain insight into how users engage with your Action. You can view:

  • Execution types and device traits used
  • Daily users and request counts
  • User query response latency
  • Success rate for Smart Home engagements
  • Comparison of cloud and local fulfilment interactions

Successful Requests Dashboard

Cloud Logging provides detailed logs based on the events observed in Cloud Monitoring.

We've added additional features to the error logs to help you quickly debug why intents fail, which particular device commands malfunction, or if your local fulfilment falls back to cloud fulfilment.

New details added to the event logs include:

  • Cloud vs. local fulfilment
  • EXECUTE vs. QUERY intents
  • Locale of request
  • Device Type

You can additionally export these logs through Cloud Pub/Sub, and build log-based metrics and alerts for your development teams to gain insights into common issues.

For more guidance on accessing your Smart Home Action analytics and logs, check out the developer guide or watch the video.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Join the "Hey Google" Smart Home Virtual Summit

Posted by Toni Klopfenstein, Developer Relations

Over the past year, we've been focused on building new tools and features to support our smart home developer community. Though we weren't able to engage with you in person at Google I/O, we are pleased to announce the "Hey Google" Smart Home Virtual Summit on July 8th - an opportunity for us to come together and dive into the exciting new and upcoming features for smart home developers and users.

Join us in the keynote where Michele Turner, the Product Management director of the Smart Home Ecosystem, will share our recent smart home product initiatives and how developers can benefit from these capabilities. She will also introduce new tools that make it easier for you to develop with Google Assistant. We will also be hosting a partner panel, where you can hear from industry leaders on how they navigate the impact of COVID-19 and their thoughts on the state of the industry.

Registration is FREE! Head on over to the Summit website to register and check out the schedule. Events will be held during EMEA, APAC, and AMER friendly times. We hope to see you and your colleagues there!

Local Home SDK support on Nest WiFi

Posted by Toni Klopfenstein, Developer Advocate

Today, we're expanding the support of the Local Home SDK to the Google Nest Wifi routers with the latest firmware update to M81. The Local Home SDK we recently launched allows you to create a local fulfilment path for your smart home Action. Local fulfillment provides lower latency and higher reliability for your smart home Action.

By adding support for the Node.js runtime of the Nest WiFi routers, the Local Home platform is now compatible with the full Nest WiFi system. This update means your local execution application can run on a self-healing mesh wireless network, and your users gain the benefits of expanded reliable home automation coverage.

To support this additional runtime, we've updated the Actions Console to enable you to add the Node.js on-device testing URL. The Nest WiFi routers will receive the the node-targeted bundle.js files you've already uploaded during deployment of your Action automatically. Since Chrome DevTools have built-in Node.js support, your development flow doesn't require any additional tools for inspecting your Node.js app or debugging your smart home Action.

We have updated the developer guide and tools to help guide you through the various local fulfilment runtimes and features of these toolings. For additional guidance on enabling local fulfilment for your smart home Action, check out the Enable local fulfillment for smart home Actions codelab. The API reference and samples can also help you build your first local fulfilment app.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Announcing Actions Builder & Actions SDK: New tools optimized for the Google Assistant

Posted by the Assistant Developer Platform team

Since the launch of the Google Assistant, our developer ecosystem has been instrumental in delivering compelling voice experiences to more than 500 million active users. Today, we’re taking a major step forward in helping you build these custom voice apps and services by introducing a suite of new and improved developer tools: Actions Builder and Actions SDK. These tools make building Conversational Actions for the Assistant easier and more streamlined than ever.

Better design and development tools

Actions Builder is a web-based IDE that lets you develop, test, and deploy directly in the Actions console. The graphical interface lets you visualize the conversational flow, manage Natural Language Understanding (NLU) training data, and debug with advanced tools.

For those of you who prefer local IDEs, the updated Actions SDK provides a file based representation of your Actions project. This lets you author NLU training data and conversational flows locally as well as bulk import and export training data. We've also updated the CLI that accompanies Actions SDK, so you can build and manage Actions projects completely with code, using your favorite source control and continuous integration tools.

Together, Actions Builder and Actions SDK create a seamless, consolidated development experience. No matter what tool you start with, you can switch between them based on what works best for your workflow. For example, you can use Actions Builder to lay out conversational flows and provide NLU training data, Actions SDK to write fulfillment code, and the CLI to synchronize the two. These tools create an environment where all team members can contribute effectively and focus on what they do best: design and code.

New interaction model

A new, powerful interaction model lets you design conversations quickly and efficiently. Intents and scenes let you define robust NLU training data and behavior for specific conversational contexts. Using scenes as building blocks, you define active intents, declare context specific error handling, collect data through slot filling, and respond with prompts.

Scenes also separate conversational flow definitions from fulfillment logic, so you can reuse the same flows across multiple conversations. Transitions between scenes let you define when one conversational context switches to another. All your scenes and transitions describe a full conversational flow and all possible dialog turns.

You can express the entire interaction model with either the Actions Builder or Actions SDK. A typical way to develop is to use Actions Builder to view and edit your scenes and then use Actions SDK to sync changes to your local file system. This lets you version control your project, modify your project files, and build fulfillment in your favorite development environment.

Faster and smarter runtime engine

Under the hood, we also made a lot of improvements that your users will appreciate. We sped up the Assistant runtime engine, so users get faster responses and a smoother experience. We’ve also made the runtime engine smarter, so your Actions can understand users better with the same amount of training data.

Production ready platform

We've worked with Pretzel Labs and Galinha Pintadinha to test the capabilities of the new platform and to refine the interaction model and runtime engine improvements.

Pretzel Labs built Kids Court with Actions Builder, creating a full conversational flow with no code and added fulfilment for advanced functionality.

"Having the combination of a visual layout with webhook blocks for code helps us collaborate clearly and more efficiently. Something I liked very much about this was the separation between the designer and the developers' parts, making it very intuitive to make design changes without affecting backend logic."
-- Adva Levin, founder of Pretzel Labs

Galinha Pintadinha runs one of the biggest YouTube channels and built one the most popular Conversational Actions in their country. Their development team migrated to the new platform to optimize their workflow and simplify future Action development. Galinha Pintadinha’s Actions now contain half the number of intents and have a radically simplified conversation tree. Using features like contextual error handling, they were able to improve the user experience and quality with little to no cost.

"Actions Builder is a robust and well designed toolbox for developing conversational apps. The concept of scenes and transitions helped us define the flow of our Action in a much more streamlined way."
-- Mário Neto, engineer at Galinha Pintadinha

Get started

To learn more about Actions Builder and SDK and to start developing your next Actions, check out our new developer resources. Our codelabs will walk you through using the new tooling and interaction model. Samples for all major features are also available, so you can start playing with code immediately. See the full set of documentation to start building today.

Stay tuned for more platform updates and happy coding!

Local Home SDK Ready for Actions

Posted by Dave Smith, Developer Advocate

Last year we introduced the developer preview of the Local Home SDK, a suite of local technologies to enhance your smart home integration with Google Assistant by adding local fulfillment. Since then, we've been hard at work incorporating your feedback and getting the experience ready for production. Starting today, we're exiting developer preview and allowing you to submit local fulfillment apps along with your smart home Action through the Actions console using Local Home SDK v1.0.

Adding local fulfillment for your smart home Action.

As part of the Smart Home platform, local fulfillment extends your smart home Action and routes commands to devices through the local network, benefitting users with reduced latency and higher reliability. If a local path cannot be successfully established, commands fall back to your cloud fulfillment.

The Local Home SDK v1.0 supports discovery of local devices over Wi-Fi using the mDNS, UDP, or UPnP protocols. Once a local path is established, apps can send commands to devices using TCP, UDP, or HTTP. For more details on the API changes in SDK v1.0, check out the changelog.

Multi-scan configurations

Along with this release, we've also improved the scan configurations in the Actions console based on your feedback. You can now enter multiple scan configurations for a given project, enabling your local fulfillment app to handle multiple device families that may be using different discovery protocols.

New multi-scan configuration UI.

The new interface groups scan attributes by protocol and highlights required fields, making it clearer how to properly configure your project.

Submit your app

The Local Home SDK configuration page in the Actions console now accepts JavaScript bundles for your local fulfillment app. When you are ready to publish your app, upload your JavaScript files to the console and submit your Action. For more details on submitting your smart home Action for review, see the smart home launch guide.

Upload your local fulfillment app.

We've updated the test suite for smart home to support local fulfillment as well. Be sure to self-test your local fulfillment before submitting your updated smart home Action for review. You must provide updated test suite results with your certification request when you submit.

Get started

To learn more about enhancing your smart home Actions with local fulfillment, check out the Introduction to Local Home SDK and the developer guide. Build your first local fulfillment app with the codelab, and go deeper with the samples and API reference.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

New Analytics updates in Actions on Google Console

Posted by Mandy Chan, Developer Advocate

Have you built an Action for the Google Assistant and wondered how many people are using it? Or how many of your users are returning users? In this blog post, we will dive into 5 improvements that the Actions on Google Console team has made to give you more insight into how your Action is being used.

1. Multiple improvements for readability

We've updated three areas of the Actions Console for readability: Active Users Chart, Date Range Selection, and Filter Options. With these new updates, you can now better customize the data to analyze the usage of your Actions.

Active Users Chart

The labels at the top of the Active Users chart now read Daily, Weekly and Monthly, instead of the previous 1 Day, 7 Days and 28 Days labels. We also improved the readability of the individual date labels at the bottom of the chart to be more clear. You’ll also notice a quick insight at the bottom of the chart that shows the unique number of users during this time period.

Before:Active Users chartAfter:

Date Range Selection

Previously, the date range selectors applied globally to all the charts. These selectors are now local to each chart, allowing you more control over how you view your data.

The date selector provides the following ranges:

  • Daily (last 7 days, last 30 days, last 90 days)
  • Weekly (last 4 weeks, last 8 weeks, last 12 weeks, last 24 weeks)
  • Monthly (last 3 months, last 6 months, last 12 months)
Date Selector

Filter Options

Previously when you added a filter, it was applied to all the charts on the page. Now, the filters apply only to the chart you're viewing. We’ve also enhanced the filtering options available for the ‘Surface’ filter, such as mobile devices, smart speakers, and smart display.

Before:

Filter Options Before

After:

filter options after

The filter feature also lets you show data breakdowns over different dimensions. By default, the chart shows a single consolidated line, a result of all the filters applied. You can now select the ‘Show breakdown by’ option to see how the components of that data contribute to the totals based on the dimension you selected.

2. Introducing Retention metrics (New!)

A brand new addition to analytics is the introduction of a retention metrics chart to help you understand how well your action is retaining users. This chart shows you how many users you had in a week and how many returned each week for up to 5 weeks. The higher the percentage week after week, the better your retention.

When you hover over each cell in the chart, you can see the exact number of users who have returned for that week from the previous week.

Retention Metrics

3. Improvements to Conversation Metrics

Finally, we’ve consolidated the conversation metrics and brought them together into a single chart with separate tabs (‘Conversations’, ‘Messages’, ‘Avg Length’ and ‘Abort rate’) for easier comparison and visibility of trends over time. We’ve also updated the chart labels and tooltips for better interpretation.

Before:

Conversion Metrics Before

After:

Conversion Metrics After

Next steps

To learn more about what each metric means, you can check out our documentation.

Try out these new improvements to see how your Actions are performing with your users. You can also check out our documentation to learn more. Let us know if you have any feedback or suggestions in terms of metrics that you need to improve your Action. Thanks for reading! To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. Can’t wait to see what you build!

Announcing Dynamic Modes and Toggles

Posted by Dave Smith, Developer Advocate

Modes and toggles let you define the configurable attributes of your device that may exist outside the standard grammar for device control traits (such as On/Off or Start/Stop). This feature is often used to express device-specific settings, such as the "load size" for a clothes washer or the "cooking mode" for an oven.

When we initially introduced modes and toggles, we supported a whitelisted set of names and synonyms to ensure the most accurate responses and best user experience. Over time, we continued to add support based on the community's requests, but getting these requests approved has been a common pain point for many of you.

Starting today, you no longer have to get the names and synonyms provided in your SYNC response approved. The Google Assistant dynamically determines the necessary grammar for users to invoke these traits. If you're not already familiar with modes and toggles, here is an example using these traits to add support for custom cooking modes to an oven.

{
  availableModes: [{
    name: 'cook',
    name_values: [{
      name_synonym: ['cook setting'],
      lang: 'en'
    }],
    settings: [{
      setting_name: 'pizza',
      setting_values: [{
        setting_synonym: ['pizza'],
        lang: 'en'
      }]
    }, {
      setting_name: 'pasta',
      setting_values: [{
        setting_synonym: ['pasta'],
        lang: 'en'
      }]
    }]
  }],
}

Example modes in SYNC response

Controlling

Controlling a device using modes and toggles

We're excited to see what you build with these improved modes and toggles! For more details on using these features, see the updated guides for the Modes Trait and Toggles Trait. To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on.