Tag Archives: language

The Go language turns 10: A Look at Go’s Growth in the Enterprise

Posted by Steve Francia, Go TeamGo's gopher mascot

The Go gopher was created by renowned illustrator Renee French. This image is adapted from a drawing by Egon Elbre.

November 10 marked Go’s 10th anniversary—a milestone that we are lucky enough to celebrate with our global developer community.

The Gopher community will be celebrating Go’s 10th anniversary at conferences such as Gopherpalooza in Mountain View and KubeCon in San Diego, and dozens of meetups around the world.

In recognition of this milestone, we’re taking a moment to reflect on the tremendous growth and progress Go (also known as golang) has made: from its creation at Google and open sourcing, to many early adopters and enthusiasts, to the global enterprises that now rely on Go everyday for critical workloads.

New to Go?

Go is an open-source programming language designed to help developers build fast, reliable, and efficient software at scale. It was created at Google and is now supported by over 2100 contributors, primarily from the open-source community. Go is syntactically similar to C, but with the added benefits of memory safety, garbage collection, structural typing, and CSP-style concurrency.

Most importantly, Go was purposefully designed to improve productivity for multicore, networked machines and large codebases—allowing programmers to rapidly scale both software development and deployment.

Millions of Gophers!

Today, Go has more than a million users worldwide, ranging across industries, experience, and engineering disciplines. Go’s simple and expressive syntax, ease-of-use, formatting, and speed have helped it become one of the fastest growing languages—with a thriving open source community.

As Go’s use has grown, more and more foundational services have been built with it. Popular open source applications built on Go include Docker, Hugo, Kubernetes. Google’s hybrid cloud platform, Anthos, is also built with Go.

Go was first adopted to support large amounts of Google’s services and infrastructure. Today, Go is used by companies including, American Express, Dropbox, The New York Times, Salesforce, Target, Capital One, Monzo, Twitch, IBM, Uber, and Mercado Libre. For many enterprises, Go has become their language of choice for building on the cloud.

An Example of Go In the Enterprise

One exciting example of Go in action is at MercadoLibre, which uses Go to scale and modernize its ecommerce ecosystem, improve cost-efficiencies, and system response times.

MercadoLibre’s core API team builds and maintains the largest APIs at the center of the company’s microservices solutions. Historically, much of the company’s stack was based on Grails and Groovy backed by relational databases. However this big framework with multiple layers was soon found encountering scalability issues.

Converting that legacy architecture to Go as a new, very thin framework for building APIs streamlined those intermediate layers and yielded great performance benefits. For example, one large Go service is now able to run 70,000 requests per machine with just 20 MB of RAM.

“Go was just marvelous for us,” explains Eric Kohan, Software Engineering Manager at MercadoLibre. “It’s very powerful and very easy to learn, and with backend infrastructure has been great for us in terms of scalability.”

Using Go allowed MercadoLibre to cut the number of servers they use for this service to one-eighth the original number (from 32 servers down to four), plus each server can operate with less power (originally four CPU cores, now down to two CPU cores). With Go, the company obviated 88 percent of their servers and cut CPU on the remaining ones in half—producing a tremendous cost-savings.

With Go, MercadoLibre’s build times are three times (3x) faster and their test suite runs an amazing 24 times faster. This means the company’s developers can make a change, then build and test that change much faster than they could before.

Today, roughly half of Mercadolibre's traffic is handled by Go applications.

"We really see eye-to-eye with the larger philosophy of the language," Kohan explains. "We love Go's simplicity, and we find that having its very explicit error handling has been a gain for developers because it results in safer, more stable code in production."

Visit go.dev to Learn More

We’re thrilled by how the Go community continues to grow, through developer usage, enterprise adoption, package contribution, and in many other ways.

Building off of that growth, we’re excited to announce go.dev, a new hub for Go developers.

There you’ll find centralized information for Go packages and modules, a wealth of learning resources to get started with the language, and examples of critical use cases and case studies of companies using Go.

MercadoLibre’s recent experience is just one example of how Go is being used to build fast, reliable, and efficient software at scale.

You can read more about MercadoLibre’s success with Go in the full case study.

AdSense now understands Marathi

Today, we’re excited to announce the addition of Marathi, a language spoken by over 80 millions people in Maharashtra, India and many other countries around the world, to the family of AdSense supported languages.


The interest for Marathi language content has been growing steadily over the last few years. With this launch, AdSense provides an easy way for publishers to monetize the content they create in Marathi, and advertisers can connect to a Marathi speaking audience with relevant ads.


To start monetizing your Marathi content website with Google AdSense:

  1. Check the AdSense Program policies and make sure your site is compliant.
  2. Sign up for an AdSense account
  3. Add the AdSense code to start displaying relevant ads to your users.

Welcome to AdSense! Sign Up now!

Posted by AdSense Internationalization Team

Indigenous speakers in Canada share their languages on Google Earth



Of the 7,000 languages spoken around the globe, 2,680 Indigenous languages—more than one third of
the world's languages—are in danger of disappearing. The United Nations declared 2019 the
International Year of Indigenous Languages to raise awareness about these languages and their
contribution to global diversity. To help preserve them, our new Google Earth tour,
Celebrating Indigenous Languages, shares audio recordings from more than 50 Indigenous language
speakers. 


“It is a human right to be able to speak your own language,” says Tania Haerekiterā Tapueluelu
Wolfgramm, a Māori and Tongan person who works as an educator and activist in Aotearoa--the Māori
name for New Zealand--and other Pacific countries. “You don’t have a culture without the language.”


Tania is one of several dozen Indigenous language speakers, advocates and educators who helped
create the tour. Thanks to their contributions, people can click on locations meaningful to Indigenous
speakers and hear people offer traditional greetings, sing songs, or say common words and phrases in
their languages. 


“Hundreds of languages are a few days away from never being spoken or heard again,” says Tania.
“By putting Indigenous languages on the global stage, we reclaim our right to talk about our lives in our
own words. It means everything to us.”
Listen to more than 50 Indigenous language speakers globally in Google Earth


The healing power of speaking one’s own language

The people who recorded audio in their languages and connected Google with Indigenous speakers
each have their own story about why revitalizing Indigenous languages strikes a chord for them. 


For Arden Ogg, director of Canada’s Cree Literacy Network, and Dolores Greyeyes Sand, a Plains Cree
person and Cree language teacher, the focus is on providing resources for language learners. For Brian
Thom, a cultural anthropologist and professor at the University of Victoria in British Columbia, the
interest grew out of his work helping Indigenous communities map their traditional lands


Brian asked yutustanaat, a member of the Snuneymuxw First Nation and a language teacher in British
Columbia, to record the hul’q’umi’num’ language. “Our language is very healing,” says yutustanaat. “It
brings out caring in our people and helps our students be strong, because the language comes from
the heart.” In her recording, yutustanaat speaks the traditional hul’q’umi’num’ greeting:
‘i ch ‘o’ ‘uy’ ‘ul’ or “How are you?”


By using their languages—and sharing them with the rest of the world—Indigenous people create closer
connections to a culture that is often endangered or has outright disappeared. 


Wikuki Kingi, a Māori Master Carver, recorded traditional chants in Te Reo Māori, an Eastern Polynesian
language indigenous to New Zealand. He says, “Speaking Te Reo Māori connects me to my relatives, to
the land, rivers, and the ocean, and it can take me to another time and place.” 

On the right, yutustanaat, a Snuneymuxw First Nation member, records the hul’q’umi’num’ language
with student Beatrix Taylor. Listen to it in the Celebrating Indigenous Languages collection.
Photo credit: Brian Thom

Ensuring that generations to come will hear their languages


“I do this not for myself, but for my children and grandchildren, so that in the future, they’ll hear our
language,” says Dolores, who recorded audio in her native Plains Cree




To ensure that future generations hear and speak Indigenous languages, more needs to be done to
support their revitalization. Tania Wolfgramm suggests checking out how her nonprofit organization,
Global Reach Initiative & Development Pacific, uses technology to connect far-flung Indigenous people
to their traditional communities—like bringing Google Street View to the remote island of Tonga. Arden
Ogg directs people interested in Indigenous languages to the Cree Literacy Network, which publishes
books in Cree and English to facilitate language learning. And a video from the University of Victoria
suggests five ways to support Indigenous language revitalization, such as learning words and phrases
using smartphone apps, and learning the names of rivers, mountains and towns in the local Indigenous
language.



This initial collection of audio recordings in Google Earth only scratches the surface of the world’s
thousands of Indigenous languages. If you’d like to contribute your language to this collection in the
future, please share your interest.    

The latest Android App Bundle updates including the additional languages API

Posted by Wojtek Kaliciński, Developer Advocate, Android

Last year, we launched Android App Bundles and Google Play's Dynamic Delivery to introduce modular development, reduce app size and streamline the release process. Since then, we've seen developers quickly adopt this new app model in over 60,000 production apps. We've been excited to see developers experience significant app size savings and reductions in the time needed to manage each release, and have documented these benefits in case studies with Duolingo and redBus.

Thank you to everyone who took the time to give us feedback on our initial launch. We're always open to new ideas, and today, we're happy to announce some new improvements based on your suggestions:

  • A new additional languages install API, which supports in-app language pickers
  • A streamlined publishing process for instant-enabled app bundles
  • A new enrollment option for app signing by Google Play
  • The ability to permanently uninstall dynamic feature modules that are included in your app's initial install


Additional languages API

When you adopt the Android App Bundle as the publishing format for your app, Google Play is able to optimize the installation by delivering only the language resources that match the device's system locales. If a user changes the system locale after the app is installed, Play automatically downloads the required resources.

Some developers choose to decouple the app's display language from the system locale by adding an in-app language switcher. With the latest release of the Play Core library (version 1.4.0), we're introducing a new additional languages API that makes it possible to build in-app language pickers while retaining the full benefits of smaller installs provided by using app bundles.

With the additional languages API, apps can now request the Play Store to install resources for a new language configuration on demand and immediately start using it.

Get a list of installed languages

The app can get a list of languages that are already installed using the SplitInstallManager#getInstalledLanguages() method.

val splitInstallManager = SplitInstallManagerFactory.create(context)
val langs: Set<String> = splitInstallManager.installedLanguages

Requesting additional languages

Requesting an additional language is similar to requesting an on demand module. You can do this by specifying a language in the request through SplitInstallRequest.Builder#addLanguage(java.util.Locale).

val installRequestBuilder = SplitInstallRequest.newBuilder()
installRequestBuilder.addLanguage(Locale.forLanguageTag("pl"))
splitInstallManager.startInstall(installRequestBuilder.build())

The app can also monitor install success with callbacks and monitor the download state with a listener, just like when requesting an on demand module.

Remember to handle the SplitInstallSessionStatus.REQUIRES_USER_CONFIRMATION state. Please note that there was an API change in a recent Play Core release, which means you should use the new SplitInstallManager#startConfirmationDialogForResult() together with Activity#onActivityResult(). The previous method of using SplitInstallSessionState#resolutionIntent() with startIntentSender() has been deprecated.

Check out the updated Play Core Library documentation for more information on how to access the newly installed language resources in your activity.

We've also updated our dynamic features sample on GitHub with the additional languages API, including how to store the user's language preference and apply it to your activities at startup.

Please note that while the additional languages API is now available to all developers, on demand modules are in a closed beta for the time being. You can experiment with on demand modules in your internal, open, and closed test tracks, while we work with our partners to make sure this feature is ready for production apps.

Instant-enabled App Bundle

In Android Studio 3.3, we introduced a way to build app bundles that contain both the regular, installed version of your app as well as a Google Play Instant experience for modules marked with the dist:instant="true" attribute in their AndroidManifest.xml:

<manifest ... xmlns:dist="http://schemas.android.com/apk/distribution">
    <dist:module dist:instant="true" />
    ...
</manifest>

Even though you could use a single project to generate the installed and instant versions of your app, up until now, developers were still required to use product flavors in order to build two separate app bundles and upload both to Play.

We're happy to announce that we have now removed this restriction. It's now possible to upload a single, unified app bundle artifact, containing modules enabled for the instant experience. This functionality is now available for everyone.

After you build an instant-enabled app bundle, upload it to any track on the Play Console, and you'll be able to select it when creating a new instant app release. This also means that the installed and instant versions of your app no longer need different version codes, which will simplify the release workflow.

Opt in to app signing by Google Play

You need to enable app signing by Google Play to publish your app using an Android App Bundle and automatically benefit from Dynamic Delivery optimizations. It is also a more secure way to manage your signing key, which we recommend to everyone, even if you want to keep publishing regular APKs for now.

Based on your feedback, we've revamped the sign-up flow for new apps to make it easier to initialize the key you want to use for signing your app.

Now developers can explicitly choose to upload their existing key without needing to upload a self-signed artifact first. You can also choose to start with a key generated by Google Play, so that the key used to locally sign your app bundle can become your upload key.

Read more about the new flow.

Permanent uninstallation of install time modules

We have now added the ability to permanently uninstall dynamic feature modules that are included in your app's initial install.

This is a behavior change, which means you can now call the existing SplitInstallManager#deferredUninstall() API on modules that set onDemand="false". The module will be permanently uninstalled, even when the app is updated.

This opens up new possibilities for developers to further reduce the installed app size. For example, you can now uninstall a heavy sign-up module or any other onboarding content once the user completes it. If the user navigates to a section of your app that has been uninstalled, you can reinstall it using the standard on demand modules install API.

We hope you enjoy these improvements and test them out in your apps. Continue to share your feedback as we work to make these features even more useful for you!

How useful did you find this blog post?

Budou: Automatic Japanese line breaking tool

Today we are pleased to introduce Budou, an automatic line breaking tool for Japanese. What is a line breaking tool and why is it necessary? English uses spacing and hyphenation as cues to allow for beautiful, aka more legible, line breaks. Japanese, which has none of these, is notoriously more difficult. Breaks occur randomly, usually in the middle of a word.

This is a long standing issue in Japanese typography on the web, and results in degradation of readability. We can specify the place which line breaks can occur with CSS coding, but this is a non-trivial manual process which requires Japanese vocabulary and knowledge of grammar.


Budou automatically translates Japanese sentences into organized HTML code with meaningful chunks wrapped in non-breaking markup so as to semantically control line breaks. Budou uses Cloud Natural Language API to analyze the input sentence, and it concatenates proper words in order to produce meaningful chunks utilizing PoS (part-of-speech) tagging and syntactic information. Budou outputs HTML code by wrapping the chunks in a SPAN tag. By specifying their display property as inline-block in CSS, semantic units will no longer be split at the end of a line.

Budou is a simple Python script that runs each sentence through the Cloud Natural Language API. It can easily be extended as a custom filter for template engines, or as a task for runners such as Grunt and Gulp. The latest version also caches the response so no duplicate requests are sent. If you are using Budou for a static website, you can process your HTML code before deployment.

Budou is aimed to be used in relatively short sentences such as titles and headings. Screen readers may read a sentence by splitting the chunks wrapped by SPAN tag or split by WBR tag, so it is discouraged to use Budou for body paragraphs.

As of October 2016, the Cloud Natural Language API supports English, Spanish, and Japanese, and Budou currently only supports Japanese. Support for other Asian languages with line break issues, such as Chinese and Thai, will be added as the API adds support.

Any comments and suggestions are welcome. You can find us on GitHub.

By Shuhei Iitsuka, UX Engineer

Which languages convey the most information in the least space? Introducing the Unimorph dataset.

Several years ago a science journalist asked me which languages could pack the most information into a 140-character Tweet. Because Twitter defines a character roughly as a single Unicode code point, this turns out to be an easy question to answer. Chinese almost certainly rates as the most “compact” language from that point of view because a single Chinese character represents a whole morpheme (in linguist terminology, a minimal unit of meaning) whereas an English letter only represents a part of a morpheme. The Chinese equivalent of I don’t eat meat, which in English takes 16 characters including spaces is 我不吃肉, which takes just four.

But this question relates to a broader question that as a linguist I have often been asked: which languages are the most “efficient” at conveying information? Or, which languages can convey the same information in the smallest amount of space? Untethered by the idiosyncrasies of Twitter, this question becomes quite difficult to answer. What do you mean by “space”? Number of characters? Number of bytes? Number of syllables? Each of these has its own problems. And perhaps more crucially, what do you mean by “information”? The Shannon notion of information does not straightforwardly apply here.

A group of us at Google set out to answer this question, or at least to provide the form that an answer would have to take. We had the resources and experience needed to annotate data in multiple languages, and we were able to divert some of those resources to this task. The results were published in a paper presented at the 2014 International Conference on Language Resources and Evaluation in Reykjavík, Iceland.

We are now releasing the data on GitHub. The data consist of 85 sentences typical of the kinds of sentences generated by Google Now, translated into eight typologically diverse languages: English, French, Italian, German, Russian, Arabic, Korean, Chinese, which include some highly inflected and uninflected languages, and various types of morphology including inflectional and agglutinative. The data were annotated by one to three annotators depending on the language, with morphological information, counts of the marked features and other information. The main data file is in HTML, color coded by language, which makes it easy to browse but also easy to extract into other formats.

Since the basic information conveyed by each sentence can be assumed to be the same across languages, the main focus of the research was on the additional information that each language marks, and cannot avoid marking. For example, the English sentence:

Use my location for the search results and other services.

has the French translation:

Utilisez ma position pour les résultats de recherche et d'autres services.

The verb ending -ez, in boldface above marks “addressee respect”, a bit of information that is missing from the English original.  One could have used a different ending on the French verb, but then that would not avoid this bit of information—it would be choosing to mark lack of respect, or familiarity with the addressee.

In the paper we tried various ways of measuring the differing information content of the languages relative to various definitions of “space”. Considering all the factors together, we concluded that the languages that conveyed the most information in a given amount of space were highly inflected languages like Russian, with uninflected languages like Chinese actually being the “least efficient” at conveying information.

We don’t expect this to be the final answer, which is why we are releasing the data as open source in the hopes that others will find it useful and maybe can even extend it to more sentences or a wider variety of languages. Ultimately though, any answer to the question of which languages convey the most information in the smallest amount of space must seriously address what is meant by “information”, and must pay heed to the famous maxim by the Russian linguist Roman Jakobson (1959) that “languages differ essentially in what they must convey and not in what they may convey.”

By Richard Sproat, Research Scientist