Tag Archives: Smart Home

Announcing Enhanced Smart Home Analytics

Posted by Toni Klopfenstein, Developer Advocate

When creating scalable applications, consistent and reliable monitoring of resources is a valuable tool for any developer. Today we are releasing enhanced analytics and logging for Smart Home Actions. This feature enables you to more quickly identify and respond to errors or quality issues that may arise.

Request Latency Dashboard

You can now access the smart home dashboard with pre-populated metrics charts for your Actions on the Analytics tab in the Actions Console, or through Cloud Monitoring. These metrics help you quantify the health and usage of your Action, and gain insight into how users engage with your Action. You can view:

  • Execution types and device traits used
  • Daily users and request counts
  • User query response latency
  • Success rate for Smart Home engagements
  • Comparison of cloud and local fulfilment interactions

Successful Requests Dashboard

Cloud Logging provides detailed logs based on the events observed in Cloud Monitoring.

We've added additional features to the error logs to help you quickly debug why intents fail, which particular device commands malfunction, or if your local fulfilment falls back to cloud fulfilment.

New details added to the event logs include:

  • Cloud vs. local fulfilment
  • EXECUTE vs. QUERY intents
  • Locale of request
  • Device Type

You can additionally export these logs through Cloud Pub/Sub, and build log-based metrics and alerts for your development teams to gain insights into common issues.

For more guidance on accessing your Smart Home Action analytics and logs, check out the developer guide or watch the video.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Join the "Hey Google" Smart Home Virtual Summit

Posted by Toni Klopfenstein, Developer Relations

Over the past year, we've been focused on building new tools and features to support our smart home developer community. Though we weren't able to engage with you in person at Google I/O, we are pleased to announce the "Hey Google" Smart Home Virtual Summit on July 8th - an opportunity for us to come together and dive into the exciting new and upcoming features for smart home developers and users.

Join us in the keynote where Michele Turner, the Product Management director of the Smart Home Ecosystem, will share our recent smart home product initiatives and how developers can benefit from these capabilities. She will also introduce new tools that make it easier for you to develop with Google Assistant. We will also be hosting a partner panel, where you can hear from industry leaders on how they navigate the impact of COVID-19 and their thoughts on the state of the industry.

Registration is FREE! Head on over to the Summit website to register and check out the schedule. Events will be held during EMEA, APAC, and AMER friendly times. We hope to see you and your colleagues there!

Local Home SDK support on Nest WiFi

Posted by Toni Klopfenstein, Developer Advocate

Today, we're expanding the support of the Local Home SDK to the Google Nest Wifi routers with the latest firmware update to M81. The Local Home SDK we recently launched allows you to create a local fulfilment path for your smart home Action. Local fulfillment provides lower latency and higher reliability for your smart home Action.

By adding support for the Node.js runtime of the Nest WiFi routers, the Local Home platform is now compatible with the full Nest WiFi system. This update means your local execution application can run on a self-healing mesh wireless network, and your users gain the benefits of expanded reliable home automation coverage.

To support this additional runtime, we've updated the Actions Console to enable you to add the Node.js on-device testing URL. The Nest WiFi routers will receive the the node-targeted bundle.js files you've already uploaded during deployment of your Action automatically. Since Chrome DevTools have built-in Node.js support, your development flow doesn't require any additional tools for inspecting your Node.js app or debugging your smart home Action.

We have updated the developer guide and tools to help guide you through the various local fulfilment runtimes and features of these toolings. For additional guidance on enabling local fulfilment for your smart home Action, check out the Enable local fulfillment for smart home Actions codelab. The API reference and samples can also help you build your first local fulfilment app.

We want to hear from you! Continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Local Home SDK Ready for Actions

Posted by Dave Smith, Developer Advocate

Last year we introduced the developer preview of the Local Home SDK, a suite of local technologies to enhance your smart home integration with Google Assistant by adding local fulfillment. Since then, we've been hard at work incorporating your feedback and getting the experience ready for production. Starting today, we're exiting developer preview and allowing you to submit local fulfillment apps along with your smart home Action through the Actions console using Local Home SDK v1.0.

Adding local fulfillment for your smart home Action.

As part of the Smart Home platform, local fulfillment extends your smart home Action and routes commands to devices through the local network, benefitting users with reduced latency and higher reliability. If a local path cannot be successfully established, commands fall back to your cloud fulfillment.

The Local Home SDK v1.0 supports discovery of local devices over Wi-Fi using the mDNS, UDP, or UPnP protocols. Once a local path is established, apps can send commands to devices using TCP, UDP, or HTTP. For more details on the API changes in SDK v1.0, check out the changelog.

Multi-scan configurations

Along with this release, we've also improved the scan configurations in the Actions console based on your feedback. You can now enter multiple scan configurations for a given project, enabling your local fulfillment app to handle multiple device families that may be using different discovery protocols.

New multi-scan configuration UI.

The new interface groups scan attributes by protocol and highlights required fields, making it clearer how to properly configure your project.

Submit your app

The Local Home SDK configuration page in the Actions console now accepts JavaScript bundles for your local fulfillment app. When you are ready to publish your app, upload your JavaScript files to the console and submit your Action. For more details on submitting your smart home Action for review, see the smart home launch guide.

Upload your local fulfillment app.

We've updated the test suite for smart home to support local fulfillment as well. Be sure to self-test your local fulfillment before submitting your updated smart home Action for review. You must provide updated test suite results with your certification request when you submit.

Get started

To learn more about enhancing your smart home Actions with local fulfillment, check out the Introduction to Local Home SDK and the developer guide. Build your first local fulfillment app with the codelab, and go deeper with the samples and API reference.

We want to hear from you, so continue sharing your feedback with us through the issue tracker, and engage with other smart home developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!

Improving Smart Home Action Reviews

Posted by Dave Smith, Developer Advocate

Today we're announcing some improvements to the review process for smart home Actions, aimed at making the experience more transparent in the Actions console and reducing the time it takes for your Actions to get approved.

The smart home review process is slightly different from other Actions on Google projects because it has two separate phases: a policy review and a certification review. All Actions undergo a policy review, which verifies that your Action follows the policy guidelines for Actions on Google. Smart home Actions go through certification review for additional quality assurance validation. Certification reviewers verify the content you provide through the certification request form, including your test suite for smart home results.

Until now, developers did not have visibility into both phases of the review process. This was a common source of confusion as developers would receive notifications that their Action was approved during policy review, without realizing that certification was incomplete or still in process.

To address this concern, we've introduced a new view in the Release section of the Actions console to display detailed status for both policy and certification review. This provides a more complete picture of your submission and when it's ready to go live in production. When your Action encounters an issue during review that requires your attention, you are now able to see that status directly in the console user interface for your project.

Action submission requiring developer attention

These console updates reflect additional internal changes we've made to better synchronize the policy and certification review teams, enabling reviewers advance submissions through the process more quickly and reducing delays between Actions being approved and ready for production.

We're excited to bring these improvements to the Actions on Google developer community! Share your thoughts or questions with us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on.

Announcing Dynamic Modes and Toggles

Posted by Dave Smith, Developer Advocate

Modes and toggles let you define the configurable attributes of your device that may exist outside the standard grammar for device control traits (such as On/Off or Start/Stop). This feature is often used to express device-specific settings, such as the "load size" for a clothes washer or the "cooking mode" for an oven.

When we initially introduced modes and toggles, we supported a whitelisted set of names and synonyms to ensure the most accurate responses and best user experience. Over time, we continued to add support based on the community's requests, but getting these requests approved has been a common pain point for many of you.

Starting today, you no longer have to get the names and synonyms provided in your SYNC response approved. The Google Assistant dynamically determines the necessary grammar for users to invoke these traits. If you're not already familiar with modes and toggles, here is an example using these traits to add support for custom cooking modes to an oven.

{
  availableModes: [{
    name: 'cook',
    name_values: [{
      name_synonym: ['cook setting'],
      lang: 'en'
    }],
    settings: [{
      setting_name: 'pizza',
      setting_values: [{
        setting_synonym: ['pizza'],
        lang: 'en'
      }]
    }, {
      setting_name: 'pasta',
      setting_values: [{
        setting_synonym: ['pasta'],
        lang: 'en'
      }]
    }]
  }],
}

Example modes in SYNC response

Controlling

Controlling a device using modes and toggles

We're excited to see what you build with these improved modes and toggles! For more details on using these features, see the updated guides for the Modes Trait and Toggles Trait. To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on.

Developer Preview of Local Home SDK

Posted by Toni Klopfenstein

Recently at Google I/O, we gave you a sneak peek at our new Local Home SDK, a suite of local technologies to enhance your smart home integrations. Today, the SDK is live as a developer preview. We've been working hard testing the platform with our partners, including GE, LIFX, Philips Hue, TP-Link, and Wemo, and are excited to bring you these additional technologies for connecting smart devices to the Google Assistant.

Figure 1: The local execution path

This SDK enables developers to more deeply integrate their smart devices into the Assistant by building upon the existing Smart Home platform to create a local execution path via Google Home smart speakers and Nest smart displays. Developers can now run their business logic to control new and existing smart devices in JavaScript that executes on the smart speakers and displays, benefitting users with reduced latency and higher reliability.

How it works:

The SDK introduces two new intents, IDENTIFY and REACHABLE_DEVICES. The local home platform scans the user's home network via mDNS, UDP, or UPnP to discover any smart devices connected to the Assistant, and triggers IDENTIFY to verify that the device IDs match those returned from the familiar Smart Home API SYNC intent. If the detected device is a hub or bridge, REACHABLE_DEVICES is triggered and treats the hub as the proxy device for communicating locally. Once the local execution path from Google Home to a device is established, the device properties are updated in Home Graph.

Figure 2: The intents used for each execution path

When a user triggers a smart home Action that has a local execution path, the Assistant sends the EXECUTE intent to the Google Nest device rather than the developer's cloud fulfillment. The developer's JavaScript app is invoked, which then triggers the Local Home SDK to send control commands to the smart device over TCP, UDP socket, or HTTP/HTTPS requests. By defaulting to local execution rather than the cloud, users experience faster fulfillment of their requests. The execution requests can still be sent to the cloud path in case local execution fails. This redundancy minimizes the possibility of a failed request, and improves the overall user experience.

Additional features of the Local Home platform include:

  • Support for all Wi-Fi-enabled device types and device traits without two-factor authentication enabled.
  • No user action required to deploy Local Home benefits to all devices.
  • Easily configure discovery protocols and the hosted JavaScript app URL through the Actions console.

Figure 3: Local Home configuration tool in the Actions console

JavaScript apps can be tested on-device, allowing developers to employ familiar tools like Chrome Developer Console for debugging. Because the Local Home SDK works with the existing smart home framework, you can self-certify new apps through the Test suite for smart home as well.

Get started

To learn more about the Local Home platform, check out the API reference, and get started adding local execution with the developer guide and samples. For general information covering how you can connect smart devices to the Google Assistant, visit the Smart Home documentation, or check out the Local Technologies for the Smart Home talk from Google I/O this year.

You can send us any feedback you have through the bug tracker, or engage with the community at /r/GoogleAssistantDev. You can tag your posts with the flair local-home-sdk to help organize discussion.