Tag Archives: Smart Home

Policy changes and certification requirement updates for Smart Home Actions

Posted by Toni Klopfenstein, Developer Advocate

Illustration of 2 animated locks and phone with Actions on Google logo on screen

As more developers onboard to the Smart Home Actions platform, we have gathered feedback about the certification process for launching an Action. Today, we are pleased to announce we have updated our Actions policy to enable developers to more quickly develop their Actions, and to help streamline the certification and launch process for developers. These updates will also help to provide a consistent, cohesive experience for smart device users.

Device quality guidelines

Ensuring each device type meets quality benchmark metrics provides end users with reliable and timely responses from their smart devices.With these policy updates, minimum latency and reliability metrics have been added to each device type guide. To ensure consistent device control and timely updates to Home Graph, all cloud controlled smart devices need to maintain a persistent connection through a hub or the device itself, and cannot rely on mobile devices and tablets.

Along with these quality benchmarks, we have also updated our guides with required and recommended traits for each device. By implementing these within an Action, developers can ensure their end users can trigger devices in a consistent manner and access the full range of device capabilities. To assist you in ensuring your Action is compliant with the updated policy, the Test Suite testing tool will now more clearly flag any device type or trait issues.

Safety and security

Smart home users care deeply about the safety and security of the devices integrated into their homes, so we have also updated our requirements for secondary user verification. This verification step must be implemented for any Action that can set a device in an unprotected state, such as unlocking a door, regardless of whether you are building a Conversational Action or Smart Home Action. Once configured with a secondary verification method, developers can provide users a way to opt out of this flow. For any developer wishing to include an opt-out selection to their customers, we have provided a warning message template to ensure users understand the security implications for turning this feature off.

For devices that may pose heightened safety risks, such as cooking appliances, we require UL certificates or similar certification forms to be provided along with the Test Suite results before an Action can be released to production.

Works With 'Hey Google' badge

These policy updates also will affect the use of the Works With Hey Google badge. The badge will only be available for use on marketing materials for new Smart Home Direct Actions that have successfully integrated any device types referenced.

Any Conversational Actions currently using the badge will not be approved for use for any new marketing assets, including packaging/product refreshes. Any digital assets using the badge will need to be updated to remove the badge by the end of 2021.

Timeline

With the roll-out today, there will be a 1 month grace period for developers to update new integrations to match the new policy requirements. For Actions currently deployed to production, compliance will be evaluated when the Action is recertified. Once integrations have been certified and launched to production, Actions will need to be recertified annually, or any time new devices or device functionality is added to the Action. Notifications for recertification will be shared with the developer account associated with your Action in the console.

This policy grace-period ends April 12, 2021.

Please review the updated policy, as well as our updated docs for launching your Smart Home Action. You can also check out our policy video for more information.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Announcing New Smart Home App Discovery Features

Posted by Toni Klopfenstein, Developer Advocate

When a user connects a smart device to the Google Assistant via the Home app, the user must select the appropriate related Action from the list of all available Actions. The user then clicks through multiple screens to complete their device setup. Today, we're releasing two new features to improve this device discovery process and drive customer adoption of your Smart Home Action through the Google Home app. App Discovery and Deep Linking are two convenience features that help users find your Google-Assistant compatible smart devices quickly and onboard faster.

App Discovery enables users to quickly find your smart home Action thanks to suggestion chips within the Google Home app. You can implement this new feature through the Actions Console by creating a verified brand link between your Action, your website, and your mobile app. App Discovery doesn't require any coding work to implement, making this a development-light feature that provides great improvements to the user experience of device linking.

In addition to helping users discover your Action directly through suggestion chips, Deep Linking enables you to guide users to your account linking flow within the Google Home app in one step. These deep links are easily added to your mobile app or web content, guiding users to your smart home integration with a single tap.

Deep Linking and App Discovery can help you create a more streamlined onboarding experience for your users, driving increased engagement and user satisfaction, and can be implemented with minimal engineering work.

To implement App Discovery and Deep Linking for your Smart Home Action, check out the developer documents, or watch the video covering these new features.

You can also check out the smart home codelabs if you are just starting to build out your Action.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

New user features and developer tools to build the helpful home

Posted by Michele Turner, Director of Product and Smart Home Ecosystem for Google Nest

To create a helpful home experience, we have focused on foundational features necessary to make it easier for people to manage their smart devices. But as people spend more and more time at home during these challenging times, it’s important that we invest in additional ways to work with developers to build a more useful connected home.

Today, at the "Hey Google" Smart Home Virtual Summit, we gave updates on our latest smart home initiatives, talked more in-depth about the new smart home controls in Android 11, and previewed some platform tools that we're investing in to make devices easier to set up and work with Google Assistant.

Smart Home for Entertainment Device support with Google Assistant

As many of us continue to stay home, smart devices are being used a lot more. With the biggest growth coming from entertainment devices, we’re increasing our support in this area with our Smart Home API.

Last year, we launched Google Assistant support for Smart Home for Entertainment Device (SHED) device types and traits, including TVs, remotes, set-top boxes, speakers, soundbars, and even game consoles from top brands like Xbox, Roku, Dish, and LG. And now, we are making these APIs public for any Smart TV, set-top box or game developers to use. SHED gives users the ability to control their favorite entertainment devices from any Assistant-enabled smart display, smart speaker or mobile device.

Smart Home controls in Android 11

With the release of Android 11, coming out later this year, we are introducing a dedicated space for Smart Home controls that users can find quickly, and access any time. We’ve redesigned the power menu to make devices linked to Google Assistant just a button-press away.

Users with the Home App can choose all, or just their favorite controls to be in the space. For partners, you get this for free - there’s no new development work required. We’ll have sliders which will allow you to adjust specific settings, like the temperature of your thermostat in the morning, or how far to open the blinds. You can also customize what devices are visible from the control space and whether these devices can be accessible in your lockscreen.

Improved state reporting and reliability

With Android 11, we want to give users a quick and easy way to check or adjust the devices in their home. And as we continue to add new surfaces for device control, it becomes more critical to ensure we have accurate state. In the coming months, we’ll be introducing tools to measure your reliability and latency to help improve and debug state reporting. Once you hit key targets for reliability and latency, we will shift from a default of querying your state to using report state to render stateful controls. This will reduce query volume on your servers and improve the user experience with accurate device state across multiple surfaces.

In addition to state accuracy, the best user experience comes with strong reliability and low latency. To help achieve both, we launched local execution with the Local Home SDK back in April. As part of the Smart Home platform, local fulfillment can extend your Smart Home Action and routes commands to devices through the local network, benefitting users with reduced latency and higher reliability by removing an additional cloud hop.

To ease the development process, the Local Home platform supports both Chrome and Node.js runtime environments, as well as building and testing of apps on local development machines or personal servers. Once you've deployed your local fulfillment app, users will benefit immediately without having to upgrade hardware or manually update firmware. Nanoleaf and Yeelight have already enabled local execution for their devices. It’s available to all developers through the Actions on Google Console.

Improving linking

Implementing a high quality integration is important - it reduces churn and delights users. Yet, it’s still challenging to get users to discover these features, and we’re doing a couple of things on our end to increase the funnel of users linked to your action. We are excited to launch OAuth-based App Flip on the developer console today. With AppFlip, we streamline the standard account linking flow by flipping users from the Google Home App to the Partner app to gather consent without requiring the users to re-enter their credentials.

To increase awareness of your Action, you will soon be able to initiate the account linking flow within your app. There will also be more opportunities to increase awareness through feature promotion and in-app notification using your app, and we will have more details on discovery and linking opportunities later this year.

Robust monitoring, logging, analytics tools

We know that visibility into the behavior of your smart home integrations is critical, from debugging in early development to detailed analytics in production. To enhance developer productivity, we've integrated with the powerful monitoring and troubleshooting tools available in Google Cloud Platform to provide detailed event logs and usage metrics for smart home projects.

We’ve also recently launched new tools to help developers improve the reliability of their integrations and aid in debugging and resolving issues quickly. You can view aggregate metrics directly in the developer console, or build logs-based metrics to find trends and gain deeper insights into common problems. Google Cloud Platform also enables developers to build custom alerts to quickly identify production issues.

You can also find a new Smart Home Analytics Dashboard accessible from the developer console and pre-populated with charts for common metrics such as Daily Active Users and Request Breakdown — giving you an overall picture of your integration's behavior. This dashboard is powered by new usage and performance metrics in Google Cloud Monitoring, giving you the power to set alerts and be notified if your integration has an issue. Get started today by going to the “Analytics” tab in the Actions console or the Google Cloud console to check out these new logs, metrics, and alerting capabilities for your projects!

Updates to Device Access program

Last year, we announced that we’re moving from the Works with Nest program to Works with Google Assistant and build on a foundation of privacy and data security to ensure users have confidence in how Google and our partners are protecting the consumer’s home data.

As part of that effort, we created the Device Access program to provide a way for partners to integrate directly with Nest devices. To support the Device Access program, we will soon launch the Device Access Console, a self-serve console that guides commercial developers through the different project phases: development, certification and pilot testing, and finally production.

For a commercial developer the console allows them to manage your various projects and integrations. It also provides development guides and trait documentation for all supported Nest devices. Individuals who want to create their own automations with their Nest devices will be able to do so with this console, but only for the homes they are a member of.

Expanding routines

One of the most popular features with Nest users is the ability to automatically trigger routines based on whether users are Home or Away. Later this year, similar functionality will be available with Google Assistant through occupancy detection.

Sleep is also a critical part in maintaining our overall well-being as we stay more at home. Last year we launched the Gentle Sleep & Wake feature with Philips Hue, which slowly brightens or dims the lights at a specific time or can be tied to your morning alarms. Just say, “Turn on Gentle Wake up” to your bedroom speaker to ‘set it and forget it.’ The Light Effects trait is now public so all developers can integrate their native Sleep or Wake experiences - in fact LIFX has recently launched! We encourage you to build and integrate your own unique experiences. We’ll have a larger launch moment later this year when we launch emulated Sleep and Wake effects so that it’ll work out of the box for any smart light!

Another way partners will be able to innovate on our platform and provide more helpful experiences to users is by extending personal routines with custom routines designed by partners, available in the coming months. Developers will be able to create and suggest routines, not just for their devices, but that can work with other devices in a customer’s home. You’ll be able to create solutions for your customers that are based on your core business and bring value to your customers - whether it’s wellness, cleaning, or entertainment. Users will be able to browse and opt-in to approved routines and choose to have Nest and other devices react and participate in that routine.

Our Smart Home efforts have grown significantly over the past several years. We now have integrations with thousands of partners covering all the major connected product categories and devices, and will continue our ambitious goal to build deeper in-home integrations. Be sure to review our docs/samples/videos to learn about all the cool new stuff, and connect with us on our dev communities.

Announcing New Smart Home SHED Types and Traits

Posted by Toni Klopfenstein, Developer Advocate

Back in April, we released the first set of Smart Home Entertainment Device (SHED) types, including TV, set-top box, and remote, as well as the traits AppSelector, InputSelector, MediaState, TransportControl, and Volume. We are excited to announce the release of new Smart Home Entertainment Device (SHED) types and traits. These new device types and traits compliment the original set we released earlier this year, and help build out a more complete solution for smart home media and gaming devices. By implementing these types and traits on your entertainment devices, you can enable users to fully access device and media controls from any Assistant surface.

SHED Types and Traits

To expand the SHED options, we've released the following new device types for Smart Home:

  • Audio-video receiver
  • Streaming box
  • Streaming stick
  • Soundbar
  • Streaming soundbar
  • Speaker

We've also released the following new trait:

  • Channel

To ensure a consistent, high-quality experience for all end users, each of these device types require your service to report activityState and playbackState to Google using the ReportState API. This requirement improves the portability between media devices and helps the Assistant better understand user intents for these devices. By implementing the complete set of recommended device traits, you can further improve the quality of your smart home Action and improve device targeting for media playback command fulfilment.

For more information on how to implement these new device features, check out the docs and samples. You can also join us at our "Hey Google" Smart Home Virtual Summit to learn more about these new features.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Announcing Enhanced Smart Home Analytics

Posted by Toni Klopfenstein, Developer Advocate

When creating scalable applications, consistent and reliable monitoring of resources is a valuable tool for any developer. Today we are releasing enhanced analytics and logging for Smart Home Actions. This feature enables you to more quickly identify and respond to errors or quality issues that may arise.

Request Latency Dashboard

You can now access the smart home dashboard with pre-populated metrics charts for your Actions on the Analytics tab in the Actions Console, or through Cloud Monitoring. These metrics help you quantify the health and usage of your Action, and gain insight into how users engage with your Action. You can view:

  • Execution types and device traits used
  • Daily users and request counts
  • User query response latency
  • Success rate for Smart Home engagements
  • Comparison of cloud and local fulfilment interactions

Successful Requests Dashboard

Cloud Logging provides detailed logs based on the events observed in Cloud Monitoring.

We've added additional features to the error logs to help you quickly debug why intents fail, which particular device commands malfunction, or if your local fulfilment falls back to cloud fulfilment.

New details added to the event logs include:

  • Cloud vs. local fulfilment
  • EXECUTE vs. QUERY intents
  • Locale of request
  • Device Type

You can additionally export these logs through Cloud Pub/Sub, and build log-based metrics and alerts for your development teams to gain insights into common issues.

For more guidance on accessing your Smart Home Action analytics and logs, check out the developer guide or watch the video.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Join the "Hey Google" Smart Home Virtual Summit

Posted by Toni Klopfenstein, Developer Relations

Over the past year, we've been focused on building new tools and features to support our smart home developer community. Though we weren't able to engage with you in person at Google I/O, we are pleased to announce the "Hey Google" Smart Home Virtual Summit on July 8th - an opportunity for us to come together and dive into the exciting new and upcoming features for smart home developers and users.

Join us in the keynote where Michele Turner, the Product Management director of the Smart Home Ecosystem, will share our recent smart home product initiatives and how developers can benefit from these capabilities. She will also introduce new tools that make it easier for you to develop with Google Assistant. We will also be hosting a partner panel, where you can hear from industry leaders on how they navigate the impact of COVID-19 and their thoughts on the state of the industry.

Registration is FREE! Head on over to the Summit website to register and check out the schedule. Events will be held during EMEA, APAC, and AMER friendly times. We hope to see you and your colleagues there!

Local Home SDK support on Nest WiFi

Posted by Toni Klopfenstein, Developer Advocate

Today, we're expanding the support of the Local Home SDK to the Google Nest Wifi routers with the latest firmware update to M81. The Local Home SDK we recently launched allows you to create a local fulfilment path for your smart home Action. Local fulfillment provides lower latency and higher reliability for your smart home Action.

By adding support for the Node.js runtime of the Nest WiFi routers, the Local Home platform is now compatible with the full Nest WiFi system. This update means your local execution application can run on a self-healing mesh wireless network, and your users gain the benefits of expanded reliable home automation coverage.

To support this additional runtime, we've updated the Actions Console to enable you to add the Node.js on-device testing URL. The Nest WiFi routers will receive the the node-targeted bundle.js files you've already uploaded during deployment of your Action automatically. Since Chrome DevTools have built-in Node.js support, your development flow doesn't require any additional tools for inspecting your Node.js app or debugging your smart home Action.

We have updated the developer guide and tools to help guide you through the various local fulfilment runtimes and features of these toolings. For additional guidance on enabling local fulfilment for your smart home Action, check out the Enable local fulfillment for smart home Actions codelab. The API reference and samples can also help you build your first local fulfilment app.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Local Home SDK Ready for Actions

Posted by Dave Smith, Developer Advocate

Last year we introduced the developer preview of the Local Home SDK, a suite of local technologies to enhance your smart home integration with Google Assistant by adding local fulfillment. Since then, we've been hard at work incorporating your feedback and getting the experience ready for production. Starting today, we're exiting developer preview and allowing you to submit local fulfillment apps along with your smart home Action through the Actions console using Local Home SDK v1.0.

Adding local fulfillment for your smart home Action.

As part of the Smart Home platform, local fulfillment extends your smart home Action and routes commands to devices through the local network, benefitting users with reduced latency and higher reliability. If a local path cannot be successfully established, commands fall back to your cloud fulfillment.

The Local Home SDK v1.0 supports discovery of local devices over Wi-Fi using the mDNS, UDP, or UPnP protocols. Once a local path is established, apps can send commands to devices using TCP, UDP, or HTTP. For more details on the API changes in SDK v1.0, check out the changelog.

Multi-scan configurations

Along with this release, we've also improved the scan configurations in the Actions console based on your feedback. You can now enter multiple scan configurations for a given project, enabling your local fulfillment app to handle multiple device families that may be using different discovery protocols.

New multi-scan configuration UI.

The new interface groups scan attributes by protocol and highlights required fields, making it clearer how to properly configure your project.

Submit your app

The Local Home SDK configuration page in the Actions console now accepts JavaScript bundles for your local fulfillment app. When you are ready to publish your app, upload your JavaScript files to the console and submit your Action. For more details on submitting your smart home Action for review, see the smart home launch guide.

Upload your local fulfillment app.

We've updated the test suite for smart home to support local fulfillment as well. Be sure to self-test your local fulfillment before submitting your updated smart home Action for review. You must provide updated test suite results with your certification request when you submit.

Get started

To learn more about enhancing your smart home Actions with local fulfillment, check out the Introduction to Local Home SDK and the developer guide. Build your first local fulfillment app with the codelab, and go deeper with the samples and API reference.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Improving Smart Home Action Reviews

Posted by Dave Smith, Developer Advocate

Today we're announcing some improvements to the review process for smart home Actions, aimed at making the experience more transparent in the Actions console and reducing the time it takes for your Actions to get approved.

The smart home review process is slightly different from other Actions on Google projects because it has two separate phases: a policy review and a certification review. All Actions undergo a policy review, which verifies that your Action follows the policy guidelines for Actions on Google. Smart home Actions go through certification review for additional quality assurance validation. Certification reviewers verify the content you provide through the certification request form, including your test suite for smart home results.

Until now, developers did not have visibility into both phases of the review process. This was a common source of confusion as developers would receive notifications that their Action was approved during policy review, without realizing that certification was incomplete or still in process.

To address this concern, we've introduced a new view in the Release section of the Actions console to display detailed status for both policy and certification review. This provides a more complete picture of your submission and when it's ready to go live in production. When your Action encounters an issue during review that requires your attention, you are now able to see that status directly in the console user interface for your project.

Action submission requiring developer attention

These console updates reflect additional internal changes we've made to better synchronize the policy and certification review teams, enabling reviewers advance submissions through the process more quickly and reducing delays between Actions being approved and ready for production.

We're excited to bring these improvements to the Actions on Google developer community! Share your thoughts or questions with us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on.