Recently at Google I/O, we gave you a sneak peek at our new Local Home SDK, a suite of local technologies to enhance your smart home integrations. Today, the SDK is live as a developer preview. We've been working hard testing the platform with our partners, including GE, LIFX, Philips Hue, TP-Link, and Wemo, and are excited to bring you these additional technologies for connecting smart devices to the Google Assistant.
Figure 1: The local execution path
This SDK enables developers to more deeply integrate their smart devices into the Assistant by building upon the existing Smart Home platform to create a local execution path via Google Home smart speakers and Nest smart displays. Developers can now run their business logic to control new and existing smart devices in JavaScript that executes on the smart speakers and displays, benefitting users with reduced latency and higher reliability.
How it works:
The SDK introduces two new intents, IDENTIFY and REACHABLE_DEVICES. The local home platform scans the user's home network via mDNS, UDP, or UPnP to discover any smart devices connected to the Assistant, and triggers IDENTIFY to verify that the device IDs match those returned from the familiar Smart Home API SYNC intent. If the detected device is a hub or bridge, REACHABLE_DEVICES is triggered and treats the hub as the proxy device for communicating locally. Once the local execution path from Google Home to a device is established, the device properties are updated in Home Graph.
Figure 2: The intents used for each execution path
When a user triggers a smart home Action that has a local execution path, the Assistant sends the EXECUTE intent to the Google Nest device rather than the developer's cloud fulfillment. The developer's JavaScript app is invoked, which then triggers the Local Home SDK to send control commands to the smart device over TCP, UDP socket, or HTTP/HTTPS requests. By defaulting to local execution rather than the cloud, users experience faster fulfillment of their requests. The execution requests can still be sent to the cloud path in case local execution fails. This redundancy minimizes the possibility of a failed request, and improves the overall user experience.
Additional features of the Local Home platform include:
Support for all Wi-Fi-enabled device types and device traits without two-factor authentication enabled.
No user action required to deploy Local Home benefits to all devices.
Easily configure discovery protocols and the hosted JavaScript app URL through the Actions console.
Figure 3: Local Home configuration tool in the Actions console
JavaScript apps can be tested on-device, allowing developers to employ familiar tools like Chrome Developer Console for debugging. Because the Local Home SDK works with the existing smart home framework, you can self-certify new apps through the Test suite for smart home as well.
You can send us any feedback you have through the bug tracker, or engage with the community at /r/GoogleAssistantDev. You can tag your posts with the flair local-home-sdk to help organize discussion.
Today marks an important milestone for the Flutter framework, as we expand our focus from mobile to incorporate a broader set of devices and form factors. At I/O, we’re releasing our first technical preview of Flutter for web, announcing that Flutter is powering Google’s smart display platform including the Google Home Hub, and delivering our first steps towards supporting desktop-class apps with Chrome OS.
From Mobile to Multi-Platform
For a long time, the Flutter team mission has been to build the best framework for developing mobile apps for iOS and Android. We believe that mobile development is ripe for improvement, with developers today forced to choose between building the same app twice for two platforms, or making compromises to use cross-platform frameworks. Flutter hits the sweet spot of enabling a single codebase to deliver beautiful, fast, tailored experiences with high developer productivity for both platforms, and we’ve been excited to see how our early efforts have flourished into one of the most popular open source projects.
As we started to home in on our 1.0 release last year, we began experimenting with broadening the scope of Flutter to other platforms. This was triggered both by internal teams within Google who are increasingly relying on Flutter, as well as the latent potential of the Dart platform for delivering portable experiences. In particular, a small team who were already building a web framework for Dart for internal usage started an exploratory project (codename “Hummingbird”) to evaluate the technical merits of porting the Flutter engine to support the standards-based web.
The results of this project were startling, thanks in large part to the rapid progress in web browsers like Chrome, Firefox, and Safari, which have pervasively delivered hardware-accelerated graphics, animation, and text as well as fast JavaScript execution. Within a few months of beginning the project, we had the core Flutter framework primitives working, and soon after we had demos running on mobile and desktop browsers. Along with Dart’s long pedigree of compiling for the web, this proved that we could also bring the Flutter framework and apps to run on the web.
In parallel, the core Flutter project has been making progress to enable desktop-class apps, with input paradigms such as keyboard and mouse, window resizing, and tooling for Chrome OS app development. The exploratory work that we did for embedding Flutter into desktop-class apps running on Windows, Mac and Linux has also graduated into the core Flutter engine.
A Portable UI Framework for All Screens
It’s worth pausing for a moment to acknowledge the business potential of a high-performance, portable UI framework that can deliver beautiful, tailored experiences to such a broad variety of form factors from a single codebase.
For startups, the ability to reach users on mobile, web, or desktop through the same app lets them reach their full audience from day one, rather than having limits due to technical considerations. Especially for larger organizations, the ability to deliver the same experience to all users with one codebase reduces complexity and development cost, and lets them focus on improving the quality of that experience.
With support for mobile, desktop, and web apps, our mission expands: we want to build the best framework for developing beautiful experiences for any screen.
Flutter for Web
This week, we are releasing the first technical preview of Flutter for the web. While this technology is still in development, we are ready for early adopters to try it out and give us feedback. Our initial vision for Flutter on the web is not as a general purpose replacement for the document experiences that HTML is optimized for; instead we intend it as a great way to build highly interactive, graphically rich content, where the benefits of a sophisticated UI framework are keenly felt.
To showcase Flutter for the web, we worked with the New York Times to build a demo. In addition to world-class news coverage, the New York Times is famous for its crossword and other puzzle games. Since avid puzzlers want to play on whatever device they’re using at the time, their development team was attracted to Flutter as a potential solution for their needs. Discovering that they could reach the web with the same code was a huge boon. At Google I/O this week, you can get a sneak peek of their newly refreshed KENKEN puzzle game, which runs with the same code on Android, iOS, web, Mac, and Chrome OS.
Here’s what Eric von Coelln, Executive Director of Puzzles at the New York Times has to say about their experiences with Flutter:
"The New York Times Crossword has more than 400,000 stand-alone subscriptions and is a daily ritual for puzzle solvers. Along with the Crossword, we’ve grown our portfolio of digital puzzles that reaches more than two million solvers each month.
We were already beginning to explore Flutter as a potential solution to the challenge of quickly developing engaging, high-quality mobile experiences. Now the addition of being able to publish to web makes Flutter an even more appealing option to quickly deploy across all of our user platforms. This update of our old Flash-based KenKen game into a multi-platform playable experience is something we’re excited to bring to our solvers this year.”
There’s lots more to say about Flutter for web than we have space for here, so check out the dedicated article about Flutter for web on the Flutter blog.
At this early stage, we’re eager to get your feedback on how you’d like to use Flutter for web. We expect to rapidly evolve the code, with a particular focus on performance, and harmonizing the codebase with the rest of the Flutter project.
Flutter for Mobile Devices
The core Flutter framework also receives an upgrade this week, with the immediate availability of Flutter 1.5 in our stable channel. Flutter 1.5 includes hundreds of changes in response to developer feedback, including updates for new App Store iOS SDK requirements, updates to the iOS and Material widgets, engine support for new device types, and Dart 2.3 featuring new UI-as-code language features.
As the framework itself matures, we’re investing in building out the supporting ecosystem. The architectural model of Flutter has always prioritized a small core framework, supplemented by a rich package community. In the last few months, Google has contributed production-quality packages for web views, Google Maps, and Firebase ML Vision, and this week, we’re adding initial support for in-app payments. And with over 2,000 open source packages available for Flutter, there are options available for most scenarios.
One particularly exciting project that we’re announcing this week at I/O is the ML Kit Custom Image Classifier. Built using Flutter and Firebase, it offers an easy-to-use app-based workflow for creating custom image classification models. You can collect training data using the phone's camera, invite others to contribute to your datasets, trigger model training, and use trained models, all from the same app.
Flutter continues to grow in popularity and adoption. A growing roster of demanding customers including eBay, Sonos, Square, Capital One, Alibaba and Tencent are developing apps with Flutter. And they’re having fun! Here’s what Larry McKenzie, a senior developer at eBay had to say about Flutter:
“Flutter is fast! Features that once took us multiple days to implement can be finished in a single day. Many problems we used to spend a lot of time on, simply no longer occur. Our team can now focus on creating more polished user experiences and delivering functionality. Flutter is enabling us to exceed expectations!”
Another quickly growing Flutter platform is Chrome OS, with millions of Chromebooks being sold every year, particularly in education. Chrome OS is a perfect environment for Flutter, both for running Flutter apps, and as a developer platform, since it supports execution of both Android and Linux apps. With Chrome OS, you can use Visual Studio Code or Android Studio to develop a Flutter app that you can test and run locally on the same device without an emulator. You can also publish Flutter apps for Chrome OS to the Play Store, where millions of others can benefit from your creation.
Flutter for Embedded Devices
As the final example of Flutter’s portability, we offer Flutter embedded on other devices. We recently published samples that demonstrate Flutter running directly on smaller-scale devices like Raspberry Pi, and we offer an embedding API for Flutter that allows it to be used in scenarios including home, automotive and beyond.
Perhaps one of the most pervasive embedded platforms where Flutter is already running is on the smart display operating system that powers the likes of Google Home Hub.
Within Google, some Google-built features for the Smart Display platform are powered by Flutter today. And the Assistant team is excited to continue to expand the portfolio of features built with Flutter for the Smart Display in the coming months; the goal this year is to use Flutter to drive the overall system UI.
Other Resources
We often get asked by developers how they can get started with Flutter. We are pleased today to announce a comprehensive new training course for Flutter, built by The App Brewery, authors of the highest-rated iOS training course on Udemy. Their new course has over thirty hours of content for Flutter, including videos, demos and labs, and with Google’s sponsorship, they are announcing today a time-limited discount of this course from the retail price of $199 to just $10.
Many developers are creating inspiring apps with Flutter. In the run-up to Google I/O, we ran a contest called Flutter Create to encourage developers to see what they could build with Flutter in 5KB or less of Dart code. We had over 750 unique entries from around the world, with some amazing examples that pushed what we imagine would be possible in such a small size.
Today, we’re announcing the winners, which can be found on flutter.dev/create. Congratulations to the overall winner, Zebiao Hu, who wins a fully-loaded iMac Pro worth over $10,000!
Flutter is no longer a mobile framework, but a multi-platform framework that can help you reach your users wherever they are. We can’t wait to see what you’ll build with Flutter on the web, desktop, mobile, and beyond!
Posted by Chris Turkstra, Director, Actions on Google
People are using the Assistant every day to get things done more easily, creating lots of opportunities for developers on this quickly growing platform. And we’ve heard from many of you that want easier ways to connect your content across the Assistant.
At I/O, we’re announcing new solutions for Actions on Google that were built specifically with you in mind. Whether you build for web, mobile, or smart home, these new tools will help make your content and services available to people who want to use their voice to get things done.
Enhance your presence in Search and the Assistant
Help people with their “how to” questions
Every day, people turn to the internet to ask “how to” questions, like how to tie a tie, how to fix a faucet, or how to install a dog door. At I/O, we’re introducing support for How-to markup that lets you power richer and more helpful results in Search and the Assistant.
Adding How-to markup to your pages will enable the page to appear as a rich result on mobile Search and on Google Assistant Smart Displays. This is an incredibly lightweight way for web developers and creators to connect with millions of people, giving them helpful step-by-step instructions with video, images and text. You can start seeing How-to markup results on Search today, and your content will become available on the Smart Displays in the coming months.
Here’s an example where DIY Network added markup to their existing content on the web to provide a more helpful, interactive result on both Google Search and the Assistant:
For content creators that don’t maintain a website, we created a How-to Video Template where video creators can upload a simple spreadsheet with titles, text and timestamps for their YouTube video, and we’ll handle the rest. This is a simple way to transform your existing how-to videos into interactive, step-by-step tutorials across Google Assistant Smart Displays and Android phones.
Check out how REI is getting extra mileage out of their YouTube video:
How-to Video Templates are in developer preview so you can start building today, and your content will become available on Android phones and Smart Displays in the coming months.
Easier engagement with your apps
Help people quickly get things done with App Actions
If you’re an app developer, people are turning to your apps every day to get things done. And we see people turn to the Assistant every day for a natural way to ask for help via voice. This offers an opportunity to use intents to create voice-based entry points from the Assistant to the right spot in your app.
Last year, we previewed App Actions, a simple mechanism for Android developers that uses intents from the Assistant to deep link to exactly the right spot in your app. At I/O, we are announcing the release of built-in intents for four new App Action categories: Health & Fitness, Finance and Banking, Ridesharing, and Food Ordering. Using these intents, you can integrate with the Assistant in no time.
If I wanted to track my run with Nike Run Club, I could just say “Hey Google, start my run in Nike Run Club” and the app will automatically start tracking my run. Or, let’s say I just finished dinner with my friend Chad and we're splitting the check. I can say "Hey Google, send $15 to Chad on PayPal" and the Assistant takes me right into Paypal, I log in, and all of my information is filled in – all I need to do is hit send.
Each of these integrations were completed in less than a day with the addition of an Actions.xml file that handles the mapping of intents between your app and the Actions platform. You can start building with these new intents today and deploy to Assistant users on Android in the coming months. This is a huge opportunity to offer your fans an effortless way to engage more frequently with your apps.
Build for devices in the home
Take advantage of Smart Displays’ interactive screens
Last year, we saw the introduction of the Smart Display as a new device category. The interactive visual surface opens up many new possibilities for developers.
Today, we’re introducing a developer preview of Interactive Canvas which lets you create full-screen experiences that combine the power of voice, visuals and touch. Canvas works across Smart Displays and Android phones, and it uses open web technologies you’re likely already familiar with, like HTML, CSS and Javascript.
Here’s an example of what you can build when you can leverage the full screen of a Smart Display:
Interactive Canvas is available for building games starting today, and we’ll be adding more categories soon. Visit the Actions Console to be one of the first to try it out.
Enable smart home devices to communicate locally
There are now more than 30,000 connected devices that work with the Assistant across 3,500 brands, and today, we’re excited to announce a new suite of local technologies that are specifically designed to create an even better smart home.
Introducing a preview of the Local Home SDK which enables you to run your smart home code locally on Google Home Speakers and Nest Displays and use its radios to communicate locally with your smart devices. This reduces cloud hops and brings a new level of speed and reliability to the smart home. We’ve been working with some amazing partners including Philips, Wemo, TP-Link, and LIFX on testing this SDK and we’re excited to open it up for all developers next month.
Make setup more seamless
And, through the Local Home SDK, we’re improving the device setup experience by providing users with a seamless setup experience, something we launched in partnership with GE smart lights this past October. So far, people have loved the ability to set up their lights in less than a minute in the Google Home app. We’re now scaling this to more partners, so go here if you’re interested.
Make your devices smart with Assistant Connect
Also, at CES earlier this year we previewed Google Assistant Connect which leverages the Local Home SDK. Assistant Connect enables smart home and appliance developers to easily add Assistant functionality into their devices at low cost. It does this by offloading a lot of work onto the Assistant to complete Actions, display content and respond to commands. We've been hard at work developing the platform along with the first products built on it by Anker, Leviton and Tile. We can't wait to show you more about Assistant Connect later this year.
New device types and traits
For those of you creating Actions for the smart home, we’re also releasing 16 new device types and three new device traits including LockUnlock, ArmDisarm, and Timer. Head over to our developer documentation for the full list of 38 device types and 18 device traits, and check out our sample project on GitHub to start building.
Get started with our new tools for all types of developers
Whether you’re looking to extend the reach of your content, drive more usage in your apps, or build custom Assistant-powered experiences, you now have more tools to do so.
If you want to learn more about how you can start building with these tools, check out our website to get started and our schedule so you can tune in to all of our developer talks that we’ll be hosting throughout the week.
https://lh6.googleusercontent.com/oJQ6oer9k-oKSnaW3KR5MwCfRndlnGcAU-FN0yA9kkYKRtwSGa3fSclA27uJ7h5rPzT1yEXW2U7wCkdYfJKj4vMZM5IPVKYOhWAHcCZ5n1qevNxrtJTE_cnGHcXNL4zsOhY9ElbGRishi ChandraVP, Product ManagementGoogle Home
Bringing together the best of Google’s AI, software and hardware, now with a desi twist
Whether you’re getting the kids ready for school, doing a batch of laundry, or answering the doorbell for the morning vendors, Indian homes are busy ones. From catching that Bollywood blockbuster on your smart TV, to whipping up a quick Chole Bhature, to sinking into soulful Sufi tunes at the end of a tiring day, you can now get hands-free help.
Beginning today, Indian users can welcome in their lives Google Home -- our voice-activated speaker powered by the Google Assistant. With a simple “Ok, Google” or “Hey Google”, you can get answers, turn up the music, manage everyday tasks or even control smart devices around your home.
Google Home understands Indian accents, and will respond to you with uniquely Indian contexts. What’s great about the Google Assistant is that it’s the same across all your devices, so that it works seamlessly for you wherever you need a helping hand. You can for instance ask it for the quickest route to office, then tell it to push the directions to Google Maps on your smartphone, and you’re ready to navigate as you head out.
Designed to fit seamlessly into your home
We didn’t want Google Home to feel like a gadget, and took inspiration from consumer products that are commonly found in homes, like wine glasses, candles, and even donuts for Mini.
The top surface has LEDs that provide visual feedback when Google Home recognizes “Hey Google”, so you know when it is listening. In those rare moments when voice won’t do, the top surface is also a capacitive touch panel. You can simply use your finger to pause the music or adjust the volume.
Google Home was designed with two microphones to enable accurate far-field voice recognition. The microphone system uses a technique called neural beam forming. We’ve simulated hundreds of thousands of noisy environments and applied machine learning to recognize patterns that allow us to filter and separate speech from noise. This allows us to deliver best-in-class voice recognition and minimize error rates -- even from across the room. Home will be available in India in the Chalk color variant.
Google Home Mini is sleek and smooth, with no corners or edges. And it's small enough to easily place anywhere in your home. It’s almost entirely enclosed in custom fabric. We created this material from scratch, right down to the yarn. It’s durable and soft, but also transparent enough to let through both light and sound. And it is available in Chalk and Charcoal, with Coral coming soon. The four LED lights underneath the fabric that light up to show you when it hears you or when it’s thinking. Mini has far-field mics so it can hear you even when there’s music playing or loud noise in the background: its circular design it can project 360-degree sound, with just one speaker.
These devices join the Made by Google family of hardware products in India, and will be available for purchase online exclusively on Flipkart, and in over 750 retail stores across the country including Reliance Digital, Croma, Bajaj Electronics, Vijay Sales, Sangeetha, and Poorvika.
Tap into the power of Google with your Assistant
Need answers to a problem? Ask questions, translate phrases, run simple maths calculations and look up the meaning of a word. Too busy to stay on top of the news? Ask and you shall receive the latest stories from sources such as Times of India, NDTV, Dainik Bhaskar, India Today, Aaj Tak and more. Need a helping hand in the kitchen? Find ingredient substitutes, pull up nutritional information and unit conversions without having to wash your atta-covered fingers.
Google Home is truly ‘desi’
With a distinctly Indian voice, your Assistant on Google Home speaks and understands your language. Ask it “Hey Google, how desi are you?”, put its cricket knowledge to the test with “Hey Google, what is a silly point?”, tell it to “Play songs from the movie Satte Pe Satta”, or even get step-by-step cooking instructions in the kitchen with, “Hey Google, give me a recipe for Dum Biryani”.
Get personalised help for your everyday tasks
The Google Assistant on Google Home has been designed to help you get more stuff done when you have your hands full. With your permission, it will help with things like your commute, your daily schedule and more. And the best part? Up to six people can connect their account to one Google Home, so if you ask your Assistant to tell you about your day, it can distinguish your voice from other people in your family, and give you personalised answers. Just ask “Hey Google, tell me about my day” or say, “Hey Google, how long will it take to get to work?” and you’ll get up to speed on everything you need to know. It can wake you up in the morning (or let you snooze), set a timer while you’re baking, and so more.
Turn up those tunes
Find the right rhythm for every occasion, whether you’re getting into the zone with sunrise yoga, hosting a dinner party, or burning off calories dancing with your little ones. You can play songs, playlists, artists, and albums from your favourite music subscription services like Google Play Music (with a six-month subscription, on us), along with offers from Saavn and Gaana*. You can also pair Google Home or Home Mini with your favorite Bluetooth speaker and set it to be the default output for all your music.
Control your smart home
Google Home can help you keep track of everything going on in your home--you can control your lights, switches and more, using compatible smart devices from brands like Philips Hue, D-Link and TP-Link. Just ask your Google Home, and your Assistant will turn off the kitchen light. If you have a Chromecast, you can also use voice commands to play Netflix, or YouTube on your TV and binge watch your favorite shows. Enjoy multi-room by grouping Google Home devices together (with Chromecast Audio, Chromecast built-in and Bluetooth speakers) to listen to the same song in every room.
A speaker for any occasion
Whether you’re hosting a dinner or a solo dance party, Google Home delivers crystal-clear sound and creates an enjoyable listening experience. Plus, we designed Home to fit stylishly into any room. And you have the option to customize the base with different colors to reflect your home’s style.
With Google Home, we’re working with our partners to bring you many great launch offers: when buying Google Home or Google Home Mini on Flipkart you get a free JioFi router along with special offers on exchange and streaming music subscriptions; when buying a Google Home at Reliance Digital or MyJio stores you get a free JioFi router with 100GB of high-speed 4G data (worth Rs 2,499)**, and at select Philips Hue and Croma outlets you get a Philips Hue + Google Home Mini at a special bundled price. Also ACT Fibernet retail customers subscribing to 12-month advance rental plans of 90MBPS and above, will receive a Google Home Mini. And above all, users get 10 percent cashback when purchasing using HDFC Bank credit cards***.
Google Home and Google Home Mini will be priced at Rs 9,999, and Rs 4,499 respectively.
It’s just the beginning...
Your Assistant on Google Home will continue to get better over time as we add more features (look out for Hindi support coming later this year!) And Google Home is open to third-party apps for the Assistant, so expect even more of your favourite services and content.
Posted by Rishi Chandra, VP, Product Management, Google Home
Note:
*Both available from April 10 to October 31, 2018, for all Google Home and Home Mini users in India
Posted by Brad Abrams, Group Product Manager, & Chris Ramsdale, Product Manager
Though it's been just a few short weeks since we released a new set of features for Actions on Google, we're kicking off our presence at South by Southwest (SXSW) with a few more updates for you.
SXSW brings together creatives interested in fusing marketing and technology together, and what better way to start the festival than with new features that enable you to be more creative, and to build new type of Actions that help your users get more things done.
Support for media playback and better content carousels
This past year, we've heard from many developers who want to offer great media experiences as part of their Actions. While you can already make your podcasts discoverable to Assistant users, our new media response API allows you to develop deeper, more-engaging audio-focused conversational Actions that include, for example, clips from TV shows, interactive stories, meditation, relaxing sounds, and news briefings.
Your users can control this audio playback on voice-activated speakers like Google Home, Android phones, and more devices coming soon. On Android phones, they can even use the controls on their phone's notification area and lock screen.
Some developers who are already using our new media response API include The Daily Show, Calm, and CNBC.
To get started using our media response API, head over to our documentation to learn more.
And if your content is more visual than audio-based, we're also introducing a browse carousel for your Actions that allows you to show browsable content -- e.g., products, recipes, places -- with a visual experience that users can simply scroll through, left to right. See an example of how this would look to your users, below, then learn more about our browse carousel here.
Daily updates and push notifications on phones, now available to your users
While having a great user experience is important, we also want to ensure you have the right tools to re-engage your users so they keep coming back to the experience you've built. To that end, a few months ago, we introduced daily updates and push notifications as a developer preview.
Starting today, your users will have access to this feature. Esquire is already using it to send daily "wisdom tips", Forbes sends a quote of the day, and SpeedyBit sends daily updates of cryptocurrency prices to keep them in the know on market fluctuations.
As soon as you submit your Action for review with daily updates or push notifications enabled, and it's approved, your users will be able to opt into this re-engagement channel. Learn more in our docs.
Build connected experiences on Google Assistant for the paying users of your Android app
Actions for Google now allows you to access digital purchases (including paid app purchases, in-app purchases, and in-app subscriptions) that your users make from your Android app. By doing so, you can recognize when you're interacting with a user who's paid for a premium experience on your Android app, and similarly serve that experience in your Action, across devices.
And the best part? This is all done behind the scenes, so the user doesn't need to take any additional steps, like signing in, for you to provide this experience. Economist Espresso, for example, now knows when a user has already paid for a subscription with Google Play, and then offers an upgraded experience to the same user through their Action.
A new way to extend an embedded Google Assistant
In December of last year we announced the addition of Built-in Device Actions to the Google Assistant SDK for devices. This feature allows developers to extend any Google Assistant that is embedded in their device using traits and grammars that are maintained by Google and are largely focused on home automation. For example "turn on", "turn off" and "turn the temperature down".
Today we're announcing the addition of Custom Device Actions which are more flexible Device Actions, allowing developers to specify any grammar and command to be executed by their device. Once you build these Custom Device Actions, users will be able to activate specific capabilities through the Google Assistant. This leads to more natural ways in which users interact with their Assistant-enabled devices, including the ability to utilize more specific device capabilities.
Before:
"Ok Google, turn on the oven"
"Ok, turning on the oven"
After:
"Ok Google, set the oven to convection and preheat to 350 degrees"
"Ok, setting the oven to convection and preheating to 350 degrees"
To give you a sense of how this might work in the real world, check out a prototype, Talk to the Light from the talented Red Paper Heart team, that shows a zany use of this functionality. Then, check out our documentation to learn more about how you can start building these for your devices. We've provided a technical case study from Red Paper Heart and their code repository in case you want to see how they built it.
In addition to Custom Device Actions, we've also integrated device registration into the Actions on Google console, allowing developers to get up and running more quickly. To get started checkout the latest documentation and console.
A few creative explorations to inspire you
Similarly, we teamed up with a few cutting-edge teams to explore the creative potential of the Actions on Google platform. Following the Voice experiments the Google Creative Lab released a few months ago, these teams released four new experiments:
Safari Mixer, a fun game for everyone to discover the latest new animal species built by Rehab
The code for all of these Actions is open source and is accompanied by in-depth technical case studies from each team that shares their learnings when developing Actions.
Case studies of Actions, built with Dialogflow
Ready to build? Take a look at our three new case studies with KLM Royal Dutch Airlines, Domino's, and Ticketmaster. Learn about their development journey with Dialogflow and how the Actions they built help them stay ahead of the conversational technology curve, be where their customers are, and assist throughout the entire user journey:
We hope these updates get your creative juices flowing and inspire you to build even more Actions and embed the Google Assistant on more devices. Don't forget that once you publish your first Action you can join our community program* and receive your exclusive Google Assistant t-shirt and up to $200 of monthly Google Cloud credit. Thanks for being a part of our community, and as always, if you have ideas or requests that you'd like to share with our team, don't hesitate to join the conversation.
*Some countries are not eligible to participate in the developer community program, please review the terms and conditions
While Actions on the Google Assistant are available to users on more than 400 million devices, we're focused on expanding the availability of the developer platform even further. At Mobile World Congress, we're sharing some good news for our international developer community.
Starting today, you can build Actions for the Google Assistant in seven new languages:
Hindi
Thai
Indonesian
Danish
Norwegian
Swedish
Dutch
These new additions join English, French, German, Japanese, Korean, Spanish, Brazilian Portuguese, Italian and Russian. That brings our total count of supported languages to 16! You can develop for all of them using Dialogflow and its natural language processing capabilities, or directly with the Actions SDK. And we're not stopping here–expect more languages to be added later this year.
If you localize your apps in these new languages you won't just be among the first Actions available in the new locales, you'll also earn rewards while you do it! And if you're new to Actions on Google, check out our community program* to learn how you can snag an exclusive Google Assistant t-shirt and up to $200 of monthly Google Cloud credit by publishing your first Action. Already we've seen partners take advantage of other languages we've launched in the past like Bring!, which is now available in both English and German.
New updates to make it easier to build for global audiences
Besides supporting new languages, we're also making it easier to build your Action for global audiences. First, we recently added support for building with templates—creating an Action by filling in a Google Sheet without a single line of code—for French, German, and Japanese. For example, TF1 built Téléfoot, using templates in French to create an engaging World Cup-themed trivia game with famous commentators included as sound effects.
Additionally, we've made it a little easier for you to localize your Actions into different languages by enabling you to export your directory listing information as a file. With the file in hand, you can translate offline and upload the translations to your console, making localization quicker and more organized.
But before you run off and start building Actions in new languages, take a quick tour of some of the useful developer features rolling out this week…
Link to your Android app to help users get things done from their mobile devices
By the end of the year the Assistant will reach 95 percent of all eligible Android phones worldwide, and Actions are a great way for you to reach those users to help them get things done easily over voice. Sometimes, however, users may benefit from the versatility of your Android app for particularly complex or highly interactive tasks.
So today, we're introducing a new feature that lets you deep link from your Actions in the Google Assistant to a specific intent in your Android app. Here's an example of SpotHero linking from their Action to their Android app after a user purchased a parking reservation. The Android app allows the user to see more details about the reservation or redeem their spot.
As you integrate these links in your Action, you'll make it easier for your users to find what they're looking for and to move seamlessly to your Android app to complete their user journey. This new feature will roll out over the coming weeks, but you can check out our developer documentation for more information on how to get started.
A faster, easier way to help with location queries
We're also introducing askForPlace, a new conversation helper that integrates the Google Places API to enable developers to use the Google Assistant to understand location-based user queries mid-conversation.
Using the new helper, the Assistant leverages Google Maps' location and points of interest (POI) expertise to provide fast, accurate places for all your users' location queries. Once the location details have been sorted out with the user, the Assistant returns the conversation back to your Action so the user can finish the interaction.
So whether your business specializes in delivering a beautiful bouquet of flowers or a piping hot pepperoni pizza, you no longer need to spend time designing models for gathering users' location requests, instead you can focus on your Action's core experience.
Let's take a look at an example of how Uber uses the askForPlace helper to help their users book a ride:
We joined halfway through the interaction above, but it's worth pointing out that once the Uber action asked the user "Where would you like to go?" the developer triggered the askForPlace helper to handle location disambiguation. The user is still speaking with Uber, but the Assistant handled all user inputs on the back end until a drop-off location was resolved. From there, Uber was able to wrap up the interaction and dispatch a driver.
Head over to the askForPlace docs to learn how to create a better user experience for your customers.
Fewer introductions for returning users
And to wrap up our new feature announcements, today we're introducing an improved experience for users who use your app regularly—without any work required on your end. Specifically, if users consistently come back to your app, we'll cut back on the introductory lead-in to get users into your Actions as quickly as possible.
Today's updates are part of our commitment to improving the platform for developers, and making the Google Assistant and Actions on Google more widely available around the globe. If you have ideas or requests that you'd like to share with our team, don't hesitate to join the conversation.
*Some countries are not eligible to participate in the developer community program, please review the terms and conditions
Posted by Akansha Sharma, Product Marketing Manager
This past year we worked hard to make the Google Assistant better for users and developers like you, but we also wanted to find new ways to reward you for doing what you love – building great apps for the Google Assistant.
So at I/O 2017, we announced our first Actions on Google Developer Challenge encouraging you to build helpful, entertaining apps for the Assistant. Today, we're announcing the competition's winners, chosen from thousands of entries.
What it's all about: Credit Card Helper analyzes card features, customer reviews, reads the fine print and looks up Consumer Reports to help you find the best credit card offers.
What it's all about: Learn about the solar system with this fun and educational game.
In addition to the top three prize winners, we also selected winners among various categories including "best app by students," "best parenting app," "best life hack" and more. You can read up on all of the winners' apps here. Congratulations to our winners and to all those who submitted an app as part of the contest – we can't wait for users to check them out!
Happy holidays and happy New Year. We can't wait to see what the next year has in store.
Looking at a landmark and not sure what it is? Interested in learning more about a movie as you stroll by the poster? With Google Lens and your Google Assistant, you now have a helpful sidekick to tell you more about what’s around you, right on your Pixel.
When we introduced the new Pixel 2 last month, we talked about how Google Lens builds on Google’s advancements in computer vision and machine learning. When you combine that with the Google Assistant, which is built on many of the same technologies, you can get quick help with what you see. That means that you can learn more about what’s in front of you—in real time—by selecting the Google Lens icon and tapping on what you’re interested in.
Here are the key ways your Assistant and Google Lens can help you today:
Text: Save information from business cards, follow URLs, call phone numbers and navigate to addresses.
Landmarks: Explore a new city like a pro with your Assistant to help you recognize landmarks and learn about their history.
Art, books and movies: Learn more about a movie, from the trailer to reviews, right from the poster. Look up a book to see the rating and a short synopsis. Become a museum guru by quickly looking up an artist’s info and more. You can even add events, like the movie release date or gallery opening, to your calendar right from Google Lens.
Barcodes: Quickly look up products by barcode, or scan QR codes, all with your Assistant.
Google Lens in the Assistant will be rolling out to all Pixel phones set to English in the U.S., U.K., Australia, Canada, India and Singapore over the coming weeks. Once you get the update, go to your Google Assistant on your phone and tap the Google Lens icon in the bottom right corner.
We can’t wait to see how Google Lens helps you explore the world around you, with the help of your Google Assistant. And don’t forget, Google Lens is also available in Google Photos, so even after you take a picture, you can continue to explore and get more information about what’s in your photo.
Posted by Ibrahim Badr, Associate Product Manager, Google Assistant
Whether you’re searching for a great restaurant for biryani, mapping your travels or helping the kids with their homework, your Google Assistant is always ready to help. You can ask about your day or your commute, explore your favorite topics, and get answers to hundreds of small and big questions during your day. But to be truly successful, your Google Assistant should be able to connect you across the apps and services in your life. So starting today, developers and companies can build apps to engage with Indian users through Actions on Google,the developer platform for the Google Assistant. And as a user, you’ll soon be able to access more of your favorite services and content straight through your Google Assistant.
Now that we’re launching the ability for everyone to create apps on the Google Assistant, Indians will soon have easy and fast access to all types of apps. Once an app works with the Assistant, you can just tell your Assistant to connect you with the app of your choice with a simple voice command – whether it’s on eligible Android phones or iPhones. And the best thing? You won’t need to install anything extra, we’ll connect you straight with the app you’d like to interact with. For example, just say “Ok Google, talk to….” to fire up an app.
Stay tuned as local Indian apps rollout soon, and dive in today by trying one of the many apps already available.
We hope that this growing platform will give more Indians the help they need, at home or on-the-go – from the morning rush hour to the weekend unwind. Together with developers from Kashmir to Kanyakumari, we look forward to exploring and delivering these new possibilities for the Indian Google Assistant.
Posted by Brad Abrams, Group Product Manager, Google Assistant
Voice interaction has the potential to simplify the way we use technology. And with Dialogflow,Actions on Google, and Speech Synthesis API, it's becoming easier for any developer to create voice-based experiences. That's why we've created Voice Experiments, a site to showcase how developers are exploring voice interaction in all kinds of exciting new ways.
The site includes a few experiments that show how voice interaction can be used to explore music, gaming, storytelling, and more. MixLab makes it easier for anyone to create music, using simple voice commands. Mystery Animal puts a new spin on a classic game. And Story Speakerlets you create interactive, spoken stories by just writing in a Google Doc – no coding required.
You can try the experiments through the Google Assistant on your phone and on voice-activated speakers like the Google Home. Or you can try them on the web using a browser like Chrome.
It's still early days for voice interaction, and we're excited to see what you will make. Visit g.co/VoiceExperiments to play with the experiments or submit your own.
