ML Kit expands into NLP with Language Identification and Smart Reply

Posted by Christiaan Prins and Max Gubin

Today we are announcing the release of two new features to ML Kit: Language Identification and Smart Reply.

You might notice that both of these features are different from our existing APIs that were all focused on image/video processing. Our goal with ML Kit is to offer powerful but simple-to-use APIs to leverage the power of ML, independent of the domain. As such, we are excited to expand ML Kit with solutions for Natural Language Processing (NLP)!

NLP is a category of ML that deals with analyzing and generating text, speech, and other kinds of natural language data. We're excited to start out with two APIs: one that helps you identify the language of text, and one that generates reply suggestions in chat applications. Both of these features work fully on-device and are available on the latest version of the ML Kit SDK, on iOS (9.0 and higher) and Android (4.1 and higher).

Generate reply suggestions based on previous messages

A new feature popping up in messaging apps is to provide the user with a selection of suggested responses, either as actions on a notification or inside the app itself. This can really help a user to quickly respond when they are busy or a handy way to initiate a longer message.

With the new Smart Reply API you can now quickly achieve the same in your own apps. The API provides suggestions based on the last 10 messages in a conversation, although it still works if only one previous message is available. It is a stateless API that fully runs on-device, so we don't keep message history in memory nor send it to a server.

textPlus app providing response suggestions using Smart Reply

We have worked closely with partners like textPlus to ensure Smart Reply is ready for prime time and they have now implemented in-app response suggestions with the latest version of their app (screenshot above).

Adding Smart Reply to your own app is done with a simple function call (using Swift in this example):

let smartReply = NaturalLanguage.naturalLanguage().smartReply()
smartReply.suggestReplies(for: conversation) { result, error in
    guard error == nil, let result = result else {
        return
    }
    if (result.status == .success) {
        for suggestion in result.suggestions {
            print("Suggested reply: \(suggestion.text)")
        }
    }
}

After you initialize a Smart Reply instance, call suggestReplies with a list of recent messages. The callback provides the result which contains a list of suggestions.

For details on how to use the Smart Reply API, check out the documentation.

Tell me more ...

Although as a developer, you can just pick up this new API and easily get it integrated in your app, it may be interesting to reveal a bit on how it works under the hood. At the core of Smart Reply is a machine-learned model that is executed using TensorFlow Lite and has a state-of-the-art modern architecture based on SentencePiece text encoding[1] and Transformer[2].

However, as we realized when we started development of the API, the core suggestion model is not all that's needed to provide a solution that developers can use in their apps. For example, we added a model to detect sensitive topics, so that we avoid making suggestions in response to profanity or in cases of personal tragedy/hardship. Also, we included language identification, to ensure we do not provide suggestions for languages the core model is not trained on. The Smart Reply feature is launching with English support first.

Identify the language of a piece of text

The language of a given text string is a subtle but helpful piece of information. A lot of apps have functionality with a dependency on the language: you can think of features like spell checking, text translation or Smart Reply. Rather than asking a user to specify the language they use, you can use our new Language Identification API.

ML Kit recognizes text in 103 different languages and typically only requires a few words to make an accurate determination. It is fast as well, typically providing a response within 1 to 2 ms across iOS and Android phones.

Similar to the Smart Reply API, you can identify the language with a function call (using Swift in this example):

let languageId = NaturalLanguage.naturalLanguage().languageIdentification()
languageId.identifyLanguage(for: "¿Cómo estás?") { languageCode, error in
  guard error == nil, let languageCode = languageCode else {
    print("Failed to identify language with error: \(error!)")
    return
  }

  print("Identified Language: \(languageCode)")
}

The identifyLanguage functions takes a piece of a text and its callback provides a BCP-47 language code. If no language can be confidently recognized, ML Kit returns a code of und for undetermined. The Language Identification API can also provide a list of possible languages and their confidence values.

For details on how to use the Language Identification API, check out the documentation.

Get started today

We're really excited to expand ML Kit to include Natural Language APIs. Give the two new NLP APIs a spin today and let us know what you think! You can always reach us in our Firebase Talk Google Group.

As ML Kit grows we look forward to adding more APIs and categories that enables you to provide smarter experiences for your users. With that, please keep an eye out for some exciting ML Kit announcements at Google I/O.

New types of answers from your Google Assistant on Android

We’ve been working to make the Assistant even better on the device that’s with you wherever you go: your phone. Last fall, we launched a visual refresh to the Assistant on phones, and today, we’re bringing a new update to Android phones that will provide better visual responses and more complete information at a glance.


This update includes answers with cards that clearly present the key information you’re looking for. In addition to a new interface for categories like events, you’ll also get access to useful tools like the tip calculator, metronome music pacer and bubble level. Ask your Assistant for “Events in Mountain View” or “Why is grass green?” to check out some of these new responses.


For some questions, the most helpful response might be showing you links to a variety of sources from across the web to learn more. In these cases, you’ll see the full set of search results from the web. When relevant, these results may include the existing ads that you’d see on Search today.


Here are some examples of the rich answers you’ll see when you ask the Assistant a question on your Android phone:


This new experience is rolling out now on Android phones, making it easier to use the Assistant on the go to find the answers you’re looking for.

A library’s next chapter: digital skills with help from Google

Editor's note: It's National Library Week, and to recognize the impact of libraries of local communities, we'll hear from Robyn Jonston of the Memphis Public Libraries. To share how libraries have affected your life, use the #MyLibraryStory hashtag.

Local libraries are essential community hubs and one of the few places that are free and open to anyone. An important part of a library’s mission is providing free access to information and opportunities, a goal we have in common with Google. Our Memphis Public Libraries and libraries across the U.S. are partnering with Grow with Google to make digital skills even more accessible to more people. Technology has changed the way people live, and in response, libraries have changed the way we fulfill our mission. We’ve taken the lead in helping people learn the skills they need to be successful in finding jobs.

At the Memphis Public Library, it’s so important to us to help people learn skills and find jobs that we’ve taken our efforts outside of the library’s walls. In 2018, we launched JobLINC, a 38-foot bus with 10 computer stations for job seekers and a station for employers to come on board and recruit. The bus travels throughout the city, reaching people who don’t have access to technology or transportation. With help from librarians, people can work on their resumes, search for jobs and practice interviewing. We serve more than 6,000 people in Memphis on this bus each year.

The program helps us reach people like Wanda Gray, who worked as a letter carrier for 20 years before being laid off. She didn’t have access to technology or a smartphone, so she turned to JobLINC to build her digital skills, get help with her resume and learn about interviewing for jobs. Now, Wanda has a new job as a receptionist.

We’re not alone in this important work at the Memphis Public Library. 90 percent of libraries help members of their communities learn basic digital skills. And thousands of librarians across the country are dedicated to making free resources and training available to everyone. In the U.S., Grow with Google hosts in-person workshops that help people learn new skills, like creating resumes and growing their businesses online. Together, we’re helping people grow their skills and businesses, find new jobs and get ahead.

I’m humbled to share my library story and the stories of Googlers whose lives were impacted by their local libraries. Show your support for librarians during National Library Week by sharing a story about what your local library means to you, using the #MyLibraryMyStory hashtag.

An ultramarathoner running so others can “rise”

Editor’s note: Passion Projects is a new Keyword series highlighting Googlers with unexpected interests outside the office.

If you had asked Zanele Hlatshwayo several years ago if she’d ever go on a run for 10 days straight, she’d probably laugh. But these days, that’s exactly what she’s training for—and she changed her mind about running for a deeply personal reason.

Zanele, an ad sales specialist based in Google’s Johannesburg office, turned to running to cope after a tragedy in her family. In 2010, her father committed suicide, and she needed to find a way to deal with her grief. She already went to the gym to work out, but one day she decided to check out the running track there, and that changed everything. Even though she used to hate running, the sport became a crucial outlet for her. Pushing through the pain of a long run taught her she could overcome anything.

“I got tired of feeling sorry for myself and crying and trying to make sense of the reason why he actually committed suicide,” she says. “Running became my sacred space, so to speak, a space where I could really clear my mind.” She started to run races with a few former colleagues, and she was hooked.

The self-described “adrenaline junkie” wasn’t content with just some 5Ks, though. She tried half marathons, then tried full marathons, and then entered the Comrades ultramarathon, which was a whopping 90 kilometers (55.9 miles). At that point, she was running to test her own limits, but wanted to do more. “There’s no point for me in running all these races and just running for medals,” she says. “I wanted to actually run for a purpose.”

Inspired by her father’s legacy, and also by a friend who was going through depression, Zanele decided to start a campaign called Rise 18 last year. In 2018, she ran 18 races to raise money for a suicide prevention help line, the only one of its kind in South Africa.

Zanele's Rise 18 promotion video

Zanele says there’s a major stigma around depression and suicide, not just in South Africa but around the world. “It’s really a state of emergency at the moment, because there aren’t enough resources to assist people who may be struggling,” she says. “And people are too scared to speak out because they don’t want to be made to feel as if there’s something wrong with them.”

The longest, and final, race of Rise 18 was 100 miles long, and took 26 hours to complete. She showed up to the race injured from her previous long-distance runs, and never stopped to sleep the entire race, but she was still determined to finish. For her, the race was mental, not just physical.

Zanele running the Washie 100 Miler ultramarathon.

Zanele running the Washie 100 Miler ultramarathon.

“The sun rises while you’re still on the road, the sun sets while you’re still on the road, and that takes a lot of mental preparation,” she says. “For me, what really kept me going was the goal I had made to myself, and the commitment I made to myself. I don’t want somebody else to go through what my father did.”

She finished that race, and went above and beyond her campaign’s goal. Her initial aim was to raise R 180,000 ($12,716) to support the help line, but she exceeded that, ultimately raising R 210,000 ($14,575). When she donated the money to the charity, they told her that money would fund 11,000 calls to the hotline, which is entirely run by volunteers. “That’s 11,000 lives,” she says. “It’s truly, truly amazing.”

An experience like that makes you realize how powerful the human mind and the human body is.

Now that Rise 18 has completed, Zanele is setting her sights on even bigger goals. She’s working on building an app to connect South Africans to therapists, and plans to raise funds for the project through her next set of races, which will include an Ironman triathlon. (You can find out more on her campaign page.)

But her biggest challenge is still ahead of her: a 10-day run from Johannesburg to Cape Town this May. She’ll travel with a group of 12, who will take day or night shifts on the road, for the Ocal Global Journey for Change. And through it all, she’ll have her larger mission in mind: The group is raising funds to help provide wheelchairs to children with disabilities.

“An experience like that makes you realize how powerful the human mind and the human body is. We’re able to take so much pain,” she says. “And for me, when I’m running, the pain I go through really signifies the pain people go through when they have challenges in their lives. That small pain I feel does not amount to the challenges those people have to face on a daily basis.”

If you or someone you know needs help, you can contact the National Suicide Prevention Lifeline in the U.S. at 1-800-273-TALK(8255), or, in South Africa, the South African Depression and Anxiety Group’s Suicide Crisis Line at 0800 567 567.

Kansas City is buzzing: explore it with Google Arts & Culture

In January 17, 1920, under the terms of the Eighteenth Amendment, Prohibition began in America. As the rest of the U.S. started drying out, Kansas City, Missouri filled its glass and earned the nickname “Paris of the Plains.” Thanks to political boss Tom Pendergast, who laid down the law in KC, the booze continued to flow. When asked how he was able to justify ignoring Prohibition, Pendergast simply explained, “the people are thirsty.” Journalist Edward Morrow quipped at the time, “if you want to see some sin, forget about Paris and go to Kansas City.”  

If you’re parched for more historical information about this Midwestern metropolis, here are seven things you can now discover on Google Arts & Culture—no speakeasy password required for entry. Today, Kansas City is a place where BBQ smoke rings meet finer things, where contemporary creatives cross cultural icons and where architectural treasures are housed in vibrant neighborhoods.

1. Time travel back to the Roaring Twenties: Want to know how the city earned its reputation as a wide-open town? Read about the unique style of jazz that flourished and the rise and fall of one Pendergast’s power in online stories from the Kansas City Public Library. From there, you can keep getting your history fix by checking out early footage of KC from the Kansas City Museum. Get to know the Man from Missouri—President Harry S. Truman—in 360 degree imagery, from his personal home at the Harry S. Truman National Historic Site to a replica of his Oval Office at the Harry S. Truman Presidential Library & Museum.

2. Check out KC’s culinary chronicles: We’d be remiss to not brag about the place where burnt ends were born - get the inside scoop with Visit KC. Or go beyond BBQ and get Jenny Vergara’s take on the thriving local culinary scene.

3. See KC in 360 degrees: Tour of some of Kansas City’smost beloved neighborhoods, from West Bottoms to Westport.  Continue your virtual tour with sites ranging from a former storage building (now the National Archives at Kansas City) to a panoramic view of the skyline from the top of the Liberty Memorial Tower at National WWI Museum and Memorial.

4. Marvel at the masterpieces of Missouri: Zoom into ultra high resolution images of artworks from The Nelson-Atkins Museum of Art (including Missouri’s own George Caleb Bingham), get to know some of the renowned alumni of the Kansas City Art Institute and discover cutting edge contemporary artists from the Kemper Museum of Contemporary Art.

5. Admire the local architectural: While KC is said to have been built with more boulevards than Paris, more fountains than Rome, the city today stands full of architectural gems. Explore including the renowned Kauffman Center for Performing Arts and the City of Fountains on Google Earth.

6. Stand at the intersection of where the blues meets baseball: Get to know the 18th and Vine neighborhood with a historical overview from the Black Archives of Mid-America, experience neon signs of jazz clubs past in virtual reality at the American Jazz Museum. Take a swing at learning more about the team Jackie Robinson represented, the Kansas City Monarchs, thanks to the Negro Leagues Baseball Museum.

7. Find inspiration from KC’s creatives: The city has served as the inspiration for a broad range of creatives, from Bob Dylan to Norman Rockwell. Get to know a few of the inventions we have Kansas Citians to thank for, the park that inspired Walt Disney World with the Jackson County Historical Society and the songs, TV shows, and movies that portray the city.

The vivacious spirit that once flowed through KC during the Prohibition era is more present than ever today. And now, anyone, anywhere can experience Kansas City by visiting g.co/kansascity, downloading the Android or iOS app or visiting Google Arts & Culture.

Google delivers digital skills training to Far North Queensland

Far North Queenslanders had the opportunity to learn new digital skills today, as Grow with Google came to the region for the first time.

More than 300 small businesses, students, community organisations and individuals attended workshops at the Cairns Convention Centre today.

Queensland Member for Cairns, Michael Healy MP formally opened the event and spoke about the importance of digital skills to the jobs of the future in regional communities.

Businesses and individuals learned how to grow their presence online and find new customers, and individuals at all stages of the digital journey picked up new skills.

Grow with Google aims to give all Australians access to digital skills training, both online and in-person, to help them make the most of the web. It includes a new online learning hub accessible from anywhere, on any device, with hundreds of handy training modules.

Front row: Hannah Frank (Google Australia), Michael Healy MP, Richard Flanagan (Google Australia), Anna Fredericks (James Cook University), Debbie-Anne Bender (CEO, Cairns Chamber of Commerce), Nick Loukas (President, Cairns Chamber of Commerce) Thanks to the Tourism and Investment Queensland (TIQ) and the Cairns Chamber of Commerce for working with us to help deliver these important skills for regional communities.

Grow with Google aims to create opportunity for all Australians to grow their skills, careers, and businesses with free tools, training, and events. Find events near you at: g.co/GrowAustralia.

Posted by Richard Flanagan, Head of Grow with Google, Google Australia

Stable Channel Update for Desktop

The stable channel has been updated to 73.0.3683.103 for Windows, Mac, and Linux, which will roll out over the coming days/weeks.

A list of all changes is available in the log. Interested in switching release channels? Find out how. If you find a new issue, please let us know by filing a bug. The community help forum is also a great place to reach out for help or learn about common issues.

Abdul Syed
Google Chrome

Take a walk through Kakadu on Google Street View

In Australia’s Top End, you will find the country’s largest national park: Kakadu National Park. Covering almost 20,000 square kilometers (about half the size of Switzerland) and with terrain encompassing wetlands, rivers and sandstone escarpments, it’s home to the world’s oldest living culture with more than a dozen Indigenous groups. One-third of Australia’s bird species, an estimated 10,000 crocodiles and approximately 2,000 plant species can also be found in the Park.


Today, on its 40th anniversary, we're inviting people across the world to visit Kakadu National Park on Google Street View—to walk through ancient “stone country”, stare at spectacular waterfalls and discover ancient rock art.
Street View Trekker at Nawurlandja

Google Street View Trekker taking in the panoramas at Nawurlandja. 

Considered a living cultural landscape, Kakadu National Park’s geological history spans more than two billion years. The Park is a place that boasts extraordinary archaeological sites that record the skills and ways of life of the region’s Aboriginal people, whose culture stretches back more than 65,000 years. The Street View journey captures a glimpse of this world, uncovering rock art galleries and stunning vistas across eight sites.

Anja Toms from Kakadu National Park

Anja Toms from Kakadu National Park explains the significance of rock art at Burrungkuy (Nourlangie).

Viewers can journey to Ubirr for incredible 360-degree views, or to take a look at rock art galleries that record animal life in the region going back thousands of years. This includes a painting of a thylacine—or Tasmanian Tiger—depicted before they became extinct on the mainland around 2,000 years ago.

Kakadu National Park Rangers

Sean Nadji and Fred Hunter, Kakadu National Park Rangers, look out over Ubirr.

You can also meander through towering sandstone pillars at Bardedjilidji, travel to Nawurlandja for world-class panoramas across Anbangbang Billabong and the Arnhem Land escarpment, and toward Burrungkuy (Nourlangie) where you can view rock art galleries, before cooling off in the pristine plunge pools at Maguk or Gunlom (one of the most popular sites for travellers looking to take a refreshing dip), then diving into Kakadu’s big waterfalls: Twin Falls and Jim Jim Falls.

This journey through Kakadu National Park is a continuation of our work with Traditional Owners, Tourism Northern Territory and Parks Australia to record and share sacred sites, and instill greater value and respect for the land—which began in 2017 with Uluṟu-Kata Tjuṯa National Park.

Visit Kakadu National Park, dual-listed by UNESCO World Heritage for both its natural and cultural significance, on Google Street View to learn more about the world’s oldest living culture and their connection with the land.


Source: Google LatLong


Take a walk through Kakadu on Google Street View

In Australia’s Top End, you will find the country’s largest national park: Kakadu National Park. Covering almost 20,000 square kilometers (about half the size of Switzerland) and with terrain encompassing wetlands, rivers and sandstone escarpments, it’s home to the world’s oldest living culture with more than a dozen Indigenous groups. One-third of Australia’s bird species, an estimated 10,000 crocodiles and approximately 2,000 plant species can also be found in the Park.


Today, on its 40th anniversary, we're inviting people across the world to visit Kakadu National Park on Google Street View—to walk through ancient “stone country”, stare at spectacular waterfalls and discover ancient rock art.
Street View Trekker at Nawurlandja

Google Street View Trekker taking in the panoramas at Nawurlandja. 

Considered a living cultural landscape, Kakadu National Park’s geological history spans more than two billion years. The Park is a place that boasts extraordinary archaeological sites that record the skills and ways of life of the region’s Aboriginal people, whose culture stretches back more than 65,000 years. The Street View journey captures a glimpse of this world, uncovering rock art galleries and stunning vistas across eight sites.

Anja Toms from Kakadu National Park

Anja Toms from Kakadu National Park explains the significance of rock art at Burrungkuy (Nourlangie).

Viewers can journey to Ubirr for incredible 360-degree views, or to take a look at rock art galleries that record animal life in the region going back thousands of years. This includes a painting of a thylacine—or Tasmanian Tiger—depicted before they became extinct on the mainland around 2,000 years ago.

Kakadu National Park Rangers

Sean Nadji and Fred Hunter, Kakadu National Park Rangers, look out over Ubirr.

You can also meander through towering sandstone pillars at Bardedjilidji, travel to Nawurlandja for world-class panoramas across Anbangbang Billabong and the Arnhem Land escarpment, and toward Burrungkuy (Nourlangie) where you can view rock art galleries, before cooling off in the pristine plunge pools at Maguk or Gunlom (one of the most popular sites for travellers looking to take a refreshing dip), then diving into Kakadu’s big waterfalls: Twin Falls and Jim Jim Falls.

This journey through Kakadu National Park is a continuation of our work with Traditional Owners, Tourism Northern Territory and Parks Australia to record and share sacred sites, and instill greater value and respect for the land—which began in 2017 with Uluṟu-Kata Tjuṯa National Park.

Visit Kakadu National Park, dual-listed by UNESCO World Heritage for both its natural and cultural significance, on Google Street View to learn more about the world’s oldest living culture and their connection with the land.


Source: Google LatLong


Take a walk through Kakadu on Google Street View

In Australia’s Top End, you will find the country’s largest national park: Kakadu National Park. Covering almost 20,000 square kilometers (about half the size of Switzerland) and with terrain encompassing wetlands, rivers and sandstone escarpments, it’s home to the world’s oldest living culture with more than a dozen Indigenous groups. One-third of Australia’s bird species, an estimated 10,000 crocodiles and approximately 2,000 plant species can also be found in the Park.
Today, on its 40th anniversary, we're inviting people across the world to visit Kakadu National Park on Google Street View--to walk through ancient “stone country”, stare at spectacular waterfalls and discover ancient rock art.
Considered a living cultural landscape, Kakadu National Park’s geological history spans more than two billion years. The Park is a place that boasts extraordinary archaeological sites that record the skills and ways of life of the region’s Aboriginal people, whose culture stretches back more than 65,000 years. The Street View journey captures a glimpse of this world, uncovering rock art galleries and stunning vistas across eight sites.
Viewers can journey to Ubirr for incredible 360-degree views, or to take a look at rock art galleries that record animal life in the region going back thousands of years. This includes a painting of a thylacine—or Tasmanian Tiger—depicted before they became extinct on the mainland around 2,000 years ago.
You can also meander through towering sandstone pillars at Bardedjilidji, travel to Nawurlandja for world-class panoramas across Anbangbang Billabong and the Arnhem Land escarpment, and toward Burrungkuy (Nourlangie) where you can view rock art galleries, before cooling off in the pristine plunge pools at Maguk or Gunlom (one of the most popular sites for travellers looking to take a refreshing dip), then diving into Kakadu’s big waterfalls: Twin Falls and Jim Jim Falls.
This journey through Kakadu National Park is a continuation of our work with Traditional Owners, Tourism Northern Territory and Parks Australia to record and share sacred sites, and instill greater value and respect for the land—which began in 2017 with Uluṟu-Kata Tjuṯa National Park.
Visit Kakadu National Park, dual-listed by UNESCO World Heritage for both its natural and cultural significance, on Google Street View to learn more about the world’s oldest living culture and their connection with the land.
https://4.bp.blogspot.com/-29zxGYJxYsI/XM_UN-L79JI/AAAAAAAACCY/ZIdjeDYM318BODCopAVijh3cCynPIVI3ACLcBGAs/s400/Kakadu%2BStreet%2BView%2B43.jpg