Tag Archives: Announcements

Building scalable private services with Internal Load Balancing

Posted by Prajakta Joshi, Product Manager, Cloud Networking


Cloud load balancers are a key part of building resilient and highly elastic services, allowing you to think less about infrastructure and focus more on your applications. But the applications themselves are evolving: they are becoming highly distributed and made up of multiple tiers. Many of these are delivered as internal-only microservices. That’s why we’re excited to announce the general availability of Internal Load Balancing, a fully-managed flavor of Google Cloud Load Balancing that enables you to build scalable and highly available internal services for internal client instances without requiring the load balancers to be exposed to the Internet.

In the past year, we’ve described our Global Load Balancing and Network Load Balancing technologies in detail at the GCP NEXT and NSDI conferences. More recently, we revealed that GCP customer Niantic deployed HTTP(S) LB, alongside Google Container Engine and other GCP technologies, to scale its wildly popular game Pokémon GO. Today Internal Load Balancing joins our arsenal of cloud load balancers to deliver the scale, performance and availability your private services need.

Internal Load Balancing architecture

When we present Cloud Load Balancing, one of the questions we get is “Where is the load balancer?” and then “How many connections per second can it support?”

Similar to our HTTP(S) Load Balancer and Network Load Balancer, Internal Load Balancing is neither a hardware appliance nor an instance-based solution. It is software-defined load balancing delivered using Andromeda, Google’s network virtualization stack.
(click to enlarge)

As a result, your internal load balancer is “everywhere” you need it in your virtual network, but “nowhere” as a choke-point in this network. By virtue of this architecture, Internal Load Balancing can support as many connections per second as needed since there is no load balancer in the path between your client and backend instances.

Internal Load Balancing features

Internal Load Balancing enables you to distribute internal client traffic across backends running private services. In the example below, client instance (192.168.1.1) in Subnet 1 connects to the Internal Load Balancing IP (10.240.0.200) and gets load balanced to a backend instance (10.240.0.2) in Subnet 2.
(click to enlarge)

With Internal Load Balancing, you can:

  • Configure a private RFC1918 load-balancing IP from within your virtual network;
  • Load balance across instances in multiple availability zones within a region;
  • Configure session affinity to ensure that traffic from a client is load balanced to the same backend instance;
  • Configure high-fidelity TCP, SSL(TLS), HTTP or HTTPS health checks;
  • Get instant scaling for your backend instances with no pre-warming; and
  • Get all the benefits of a fully managed load balancing service. You no longer have to worry about load balancer availability or the load balancer being a choke point.

Configuring Internal Load Balancing

You can configure Internal Load Balancing using the REST API, the gcloud CLI or Google Cloud Console. Click here to learn more about configuring Internal Load Balancing.

The (use) case for Internal Load Balancing

Internal Load Balancing delivers private RFC 1918 load balancing within your virtual private networks in GCP. Let’s walk through three interesting ILB use cases:


1. Scaling your internal services

In a typical microservices architecture, you deliver availability and scale for each service by deploying multiple instances of it and using an internal load balancer to distribute traffic across these instances. Internal Load Balancing does this, and also autoscales instances to handle increases in traffic to your service.
(click to enlarge)

2. Building multi-tier applications on GCP

Internal Load Balancing is a critical component for building n-tier apps. For example, you can deploy HTTP(S) Load Balancing as the global web front-end load balancer across web-tier instances. You can then deploy the application server instances (represented as the Internal Tier below) behind a regional internal load balancer and send requests from your web tier instances to it.
(click to enlarge)

3. Delivering high availability and scale for virtual appliances

Traditionally, high availability (HA) for hardware appliances is modeled as an active-standby or active-active set-up where two (or sometimes more) such devices exchange heart beats and state information across a dedicated, physical synchronization link in a Layer 2 network.

This model no longer works for cloud-based virtual appliances such as firewalls, because you do not have access to the physical hardware. Layer 2 based high availability doesn’t work either because public cloud virtual networks are typically Layer 3 networks. More importantly, cloud apps store shared state outside of the application for durability, etc., so it is possible to eliminate traditional session state synchronization.
(click to enlarge)

Considering all of these factors, a high availability model that works well on Google Cloud Platform is deploying virtual appliance instances behind Internal Load Balancing. Internal Load Balancing performs health checks on your virtual appliance instances, distributes traffic across the healthy ones, and scales the number of instances up or down based on traffic.

What’s next for Internal Load Balancing

We have a number of exciting Internal Load Balancing features coming soon, including service discovery using DNS, load balancing traffic from on-prem clients across a VPN to backends behind an internal load balancer, and support for regional instance groups.

Until then, we hope you will give Internal Load Balancing a spin. Start with the tutorial, read the documentation and deploy it on GCP. We look forward to your feedback!

Google partners with Improbable to support next generation of video games

Nan Boden, Head of Global Technology Partners, Google Cloud
Google Cloud’s guiding philosophy is to enable what’s next, and gaming is one industry that’s constantly pushing what’s possible with technical innovation. At Google, we are no stranger to these advancements, from AlphaGo’s machine learning breakthrough to Pokemon GO’s achievements in scaling and mapping on GCP.

We are always seeking new partners who share our enthusiasm for innovation, and today we are announcing a partnership with Improbable, a company focused on building large-scale, complex online worlds through their distributed operating system, SpatialOS. As part of the partnership, Improbable is launching the SpatialOS Games Innovation Program, which provides game developers with credits to access Improbable’s technology powered by GCP and the freedom to get creative and experiment with what’s possible up until they launch the game. Today, game developers can join the SpatialOS open alpha, and start to prototype, test and deploy games to the cloud. The program will fully launch in Q1 2017, along with the SpatialOS beta.

SpatialOS allows game developers to create simulations of great scale (a single, highly detailed world can span hundreds of square miles), great complexity (millions of entities governed by realistic physics) and huge populations (thousands of players sharing the same world). These exciting new games are possible with SpatialOS plus the scalability, reliability and openness of GCP, including the use of Google Cloud Datastore’s fully managed NoSQL database and Google Compute Engine’s internal network, instance uptime, live migration and provisioning speed.



Bossa Studios is already using SpatialOS and GCP to build Worlds Adrift, a 3D massively multiplayer game set to launch in early 2017. In Worlds Adrift, thousands of players share a single world of floating islands that currently cover more than 1000km². Players form alliances, build sky-ships and become scavengers, explorers, heroes or pirates in an open, interactive world. They can steal ships and scavenge wrecks while the islands’ flora and fauna can flourish and decline over time.
A collision of two fully customized ships flying through the procedurally generated and persistent universe of Worlds Adrift. Read about the game’s origin story and technical details of its physics.

We see many opportunities for GCP to support developers building next-generation games and look forward to what game studios large and small will create out of our partnership with Improbable. To join the SpatialOS open alpha or learn more about the developer program visit SpatialOS.com.



Announcing new Google Cloud Client Libraries for four key services


Posted by Omar Ayoub, Product Manager
Google Cloud Platform offers a range of services and APIs supported by an impressive backend infrastructure. But to benefit from the power and capabilities of our APIs, you as a developer also need a great client-side experience: client libraries you’ll actually want to use, that are well documented, and that are easy to access.

That’s why we are announcing today the beta release of the new Google Cloud Client Libraries for four of our cloud services: BigQuery, Google Cloud Datastore, Stackdriver Logging, and Google Cloud Storage. These libraries are idiomatic, well-documented, open-source, and cover seven server-side languages: C#, Go, Java, Node.js, PHP, Python, and Ruby. Most importantly, this new family of libraries is for GCP specifically and provides a consistent experience as you use each of these four services.

Finding client libraries fast

We want to make it easy for you to discover client libraries on cloud.google.com, so we updated our product documentation pages with a prominent client library section for each of these four products. Here’s what you can see in the left-hand navigation bar of the BigQuery documentation APIs & Reference section:



Click on the Client Libraries link to see the new Client Libraries page and select the language of your choice to learn how to install the library:


Right underneath the installation section, there’s a sample that shows how to make an API call. Set up auth using a single command, copy-paste the sample code and replace your variables, and you’ll be up and running in no time.



Lower in the page, you can find the links to access the library’s GitHub repo, ask a question on StackOverflow, or navigate to the client library reference for your specific language:


Client libraries you’ll want to use

The new Google Cloud Client Libraries were built with usability in mind from day one. We strive to make the libraries idiomatic and include the usage patterns you expect from your programming language -- so you feel right at home when you code against them.

They should also include plenty of samples. Each client library reference now includes a code example for every language and every API method showing you how to work with the API and best practices. For instance, the Node.js client library reference for BigQuery displays the following code with the createDataset method:

Furthermore, the product documentation on cloud.google.com for each of the four APIs contains many how-to guides with targeted samples for all our supported languages. For example, here is the code for learning how to stream data into BigQuery:





Next steps

This is just the beginning for Google Cloud Client Libraries. Our first job is to make the libraries for these four APIs generally available. We’ll also add support for more APIs, improve our documentation across the board, and keep adding more samples.

We take developer experience seriously and want to hear from you. Feel free to file issues on GitHub in one of our client library repositories or ask questions on StackOverflow.

Happy Coding!

Making every (leap) second count with our new public NTP servers

Posted by Michael Shields, Technical Lead, Time Team

As if 2016 wasn’t long enough, this year, a leap second will cause the last day of December to be one second longer than normal. But don’t worry, we’ve built support for the leap second into the time servers that regulate all Google services.

Even better, our Network Time Protocol (NTP) servers are now publicly available to anyone who needs to keep local clocks in sync with VM instances running on Google Compute Engine, to match the time used by Google APIs, or for those who just need a reliable time service. As you would expect, our public NTP service is backed by Google’s load balancers and atomic clocks in data centers around the world.

Here’s how we plan to handle the leap second and keep things running smoothly here at Google. It’s based on what we learned during the leap seconds in 2008, 2012 and 2015.

Leap seconds compensate for small and unpredictable changes in the Earth's rotation, as determined by the International Earth Rotation and Reference Systems Service (IERS). The IERS typically announces them six months in advance but the need for leap seconds is very irregular. This year, the leap second will happen at 23:59:60 UTC on December 31, or 3:59:60 pm PST.

No commonly used operating system is able to handle a minute with 61 seconds, and trying to special-case the leap second has caused many problems in the past. Instead of adding a single extra second to the end of the day, we'll run the clocks 0.0014% slower across the ten hours before and ten hours after the leap second, and “smear” the extra second across these twenty hours. For timekeeping purposes, December 31 will seem like any other day.

All Google services, including all APIs, will be synchronized on smeared time, as described above. You’ll also get smeared time for virtual machines on Compute Engine if you follow our recommended settings. You can use non-Google NTP servers if you don’t want your instances to use the leap smear, but don’t mix smearing and non-smearing time servers.

If you need any assistance, please visit our Getting Help page.

Happy New Year, and let the good times roll.

It’s that time again: Google Code-in starts today!

Today marks the start of the 7th year of Google Code-in (GCI), our pre-university contest introducing students to open source development. GCI takes place entirely online and is open to students between the ages of 13 and 17 around the globe.
The concept is simple: complete bite-sized tasks (at your own pace) created by 17 participating open source organizations on topic areas you find interesting:

  • Coding
  • Documentation/Training
  • Outreach/Research
  • Quality Assurance
  • User Interface

Tasks take an average of 3-5 hours to complete and include the guidance of a mentor to help along the way. Complete one task? Get a digital certificate. Three tasks? Get a sweet Google t-shirt. Finalists get a hoodie. Grand Prize winners get a trip to Google headquarters in California.

Over the last 6 years, 3213 students from 99 countries have successfully completed tasks in GCI. Intrigued? Learn more about GCI by checking out our rules and FAQs. And please visit our contest site and read the Getting Started Guide.

Teachers, if you are interested in getting your students involved in Google Code-in you can find resources here to help you get started.

By Mary Radomile, Open Source Programs Office

Announcing the Google Code-in 2016 mentor organizations

We’re excited to introduce the 17 open source organizations that are participating as mentor organizations for Google Code-in 2016. The contest, now in its seventh year, gives 13-17 year old pre-university students the opportunity to learn under the guidance of mentors by using their skills on real world applications, that is, open source projects.

Google Code-in officially starts for students on November 28, but students are encouraged to learn about the mentor organizations ahead of time and can get started by clicking on the links below.


  • Apertium - rule-based machine translation platform
  • BRL-CAD - computer graphics, 2D and 3D geometry modeling, and computer-aided design (CAD)
  • CCExtractor - open source tools for subtitle generation
  • Copyleft Games - building game development platforms for tomorrow
  • Drupal - content management platform
  • FOSSASIA - developing communities across all ages and borders to form a better future with Open Technologies and ICT
  • Haiku - operating system specifically targeting personal computing
  • KDE - team that creates Free Software for desktop and portable computing
  • MetaBrainz - builds community maintained databases
  • Mifos Initiative - transforming the delivery of financial services to the poor and the unbanked
  • MovingBlocks - like an open source Minecraft
  • OpenMRS - open source medical records system for the world
  • SCoRe - research lab that seeks sustainable solutions for problems faced by developing countries
  • Sugar Labs - learning platform and activities for elementary education
  • Systers - community for women involved in the technical aspects of computing
  • Wikimedia - non-profit foundation dedicated to bringing free content to the world, operating Wikipedia
  • Zulip - powerful, threaded open source group chat with apps for every major platform
Mentor organizations are currently creating thousands of tasks for students covering code, documentation, user interface, quality assurance, outreach, research and training. The contest officially starts for students on Monday, November 28th at 9:00am PST.

You can learn more about Google Code-in on the contest site where you’ll find Contest Rules, Frequently Asked Questions and Important Dates. There you’ll also find flyers and other helpful information including the Getting Started Guide. Our discussion mailing list is a great way to talk with other students, mentors and organization administrators about the contest. For questions about eligibility or other general questions, you can contact us at [email protected].

By Josh Simmons, Open Source Programs Office

Google Code-in 2016 now accepting organization applications


Google Code-in is our global online contest that invites pre-university students ages 13-17 to learn by contributing to open source software. The contest begins its 7th year on November 28th, 2016. With the start date of the contest rapidly approaching, we are now accepting applications for open source projects interested in being a part of Google Code-in.

Working with young students is a special responsibility and each year we hear inspiring stories from mentors who participate. To ensure these new, young contributors have a great support system, we select organizations that have gained experience in mentoring students by previously taking part in Google Summer of Code.

There were 14 organizations in 2015 that collectively created thousands of bite-sized tasks for students to choose from. Tasks are created in 5 categories:

  • Code: writing or refactoring 
  • Documentation/Training: creating/editing documents and helping others learn more
  • Outreach/Research: community management, outreach/marketing, or studying problems and recommending solutions
  • Quality Assurance: testing and ensuring code is of high quality
  • User Interface: user experience research or user interface design and interaction

Once an organization is selected for Google Code-in 2016 they will define these tasks and recruit mentors who are interested in providing online support for students.

You can find a timeline, FAQ and other information about Google Code-in on our website. If you’re an educator interested in sharing Google Code-in with your students, you can find resources here.

By Josh Simmons, Open Source Programs Office

Behind the Scenes with Dylan Lorimer, Google Surveys Product Lead

Google Surveys, a new product from Google Analytics Solutions, launched on Wednesday. Google Surveys makes it easy to get fast, reliable opinions from consumers on mobile devices and across the internet. The goal, as always, is to help you make more informed business decisions, understand your marketing impact, and take the pulse of your brand.

Today we're excited to talk with Dylan Lorimer, Lead Product Manager for Google Surveys, about how this product fits in with the rest of Google Analytics Solutions and what the future looks like.

Q: What is Google Surveys?
Lorimer: Google Surveys is a online market research tool for making fast, informed, data-driven business decisions. It gives businesses access to conduct research against a representative sample of users on the web and on our mobile app, Google Opinion Rewards. It’s the evolution of Google Consumer Surveys, a product that’s been in the market since 2012 and fields 1M surveys a week.

Q: How does Google Surveys differ from Google Surveys 360?
Lorimer: Google Surveys 360 is our enterprise research product; it's a fully-supported research tool that is integrated into the Google Analytics 360 Suite. Users get advanced audience targeting, dedicated support and consolidated billing, all of which make it easy for big organizations to use. Google Surveys has fewer advanced targeting features, but it's a pay-as-you-go product so that anyone can use it. Google Surveys is currently available in U.S., Canada, United Kingdom, Germany, Mexico, Japan, Australia, Brazil, France, Spain, Italy, and the Netherlands. Google Surveys 360 is available in U.S. and Canada.

Q: How is Google Surveys different from other market research tools?
Lorimer: Traditional market research research often involves long, time-intensive surveys answered by individuals who have signed up to respond to surveys for compensation. The process is typically slow, costly, and out of reach for most businesses who want to make a quick and informed decision. Google Surveys aims to make high-quality research more accessible, and to bring the customer voice into the process of media performance measurement.

Q: Why is Google Surveys now part of Google Analytics Solutions?
Lorimer: Google Analytics Solutions is a set of powerful tools that helps marketers use data to understand their consumers and how they behave. Google Surveys bridges the gap between metrics-based performance measurement and the need many marketers have to get qualitative feedback from their customers on their cross-channel, cross-media marketing campaigns. Google Surveys has grown on its own over the past few years. When we saw the investment Google was making with Google Analytics Solutions to empower the marketer — something we’ve been focused on with Google Consumer Surveys for a few years now — it felt like the obvious choice to provide a missing tool in the portfolio.

Q: What are some examples of how customers are using Google Surveys?
Lorimer: YouVisit, a leading virtual reality company, just used us to find out how many of their clients were aware of virtual reality. They suspected a significant number of consumers were at least aware of the concept, and that some percentage would want to try it. But to convince clients to invest in virtual reality content, YouVisit needed hard facts. How many people wanted to try virtual reality? How many were even aware of it? YouVisit turned to Google Surveys to gather those numbers, and they were pleasantly surprised. A statistically significant sample from Google Surveys let them extrapolate that 23 million Americans had already tried virtual reality. With this data, YouVisit can confidently educate clients on virtual reality’s power to engage consumers.

 “Google Surveys empowers YouVisit to accomplish our mission by better understanding what the end customer wants in immersive experience — so we can create those experiences that emotionally engage, that are impactful, that drive a real-life action.” 
– Suzanne Sanders, Senior Marketing Communications Manager, YouVisit 


Q: What does the future look like for Google Surveys?
Lorimer: The future is bright! We see immense potential and we're excited about new ways we can help marketers and even developers reach their target audience, based on unique signals like location, behavior, and ad exposure. We're already serving surveys to over 10 million respondents, so in some ways pulling insights from that is a big data challenge for us — which is the kind of puzzle we like to solve best at Google. So there’s a lot we think we can do to be proactive about providing actionable data to our customers. Google Surveys as a research tool is just the beginning.

We hope you found this additional information useful. Ready to get started with Google Surveys? Learn more at g.co/surveys.

Happy surveying!

Posted by Marisa Currie-Rose, Senior Product Marketing Manager, Google Surveys

Spotlight on Google Tag Manager: Open and Secure Tag Management

With Google Tag Manager and Tag Manager 360, we make it easy for marketers to manage tags on their sites and apps for fast, flexible implementation of new marketing technologies. Last week at SMX East, we announced that we’ve added more than 20 new tag templates to Google Tag Manager to empower marketers to move faster and make decisions with confidence. This is one of a couple recent changes to make Tag Manager more open and more secure.

New third-party tag templates

Google Tag Manager already includes easy-to-use tag templates for many Google and non-Google tools. And, while we support implementing all other tags using a variety of custom tag types, we’re dedicated to expanding the number of third-party tag templates available to you. Tag templates make it easy to add and maintain tags on your sites and apps without code. This means that less technical users can be more involved, errors are less likely, and your sites are ultimately more secure.

To make it easier and safer to deploy third-party tags, we've developed an open format that allows vendors like Microsoft, Twitter, and Nielsen to implement and maintain their own tag templates in Google Tag Manager. For companies whose products depend on tags, having a tag template in Google Tag Manager is a great way to make implementation easier than ever.



"The seamless integration of Digital Content Ratings — a key component of Nielsen's Total Audience measurement framework — into Google Tag Manager will enable easy deployment and rapid adoption of Nielsen measurement with savvy digital clients."
– David Wong, SVP of Product Leadership at Nielsen



Our Vendor Tag Template Program allows tag providers to natively integrate with our tag templating system. Google performs a security review of each tag that is submitted through the program. The resulting tags are integrated into Tag Manager, and displayed with the company’s logo and an easy-to-use form to configure and deploy the tag.


"At Hotjar we're obsessed about giving our users fast and actionable insights about how their visitors are using their site. Our top most priority is making it easy and simple to set up Hotjar. For this reason, we're extremely excited to provide 'out-of-the-box' support for Google Tag Manager."
– David Darmanin, CEO of Hotjar



Some recent additions include:


Are you a vendor who would like your tag supported? Learn more about the Vendor Tag Template Program and enroll here.

Secure container loading

At Google, we constantly look to develop products with best in class security. Google Tag Manager already has a host of security features such as user access controls, 2-step verification, malware detection, and tag blacklists. We’ve now also made some changes to the Google Tag Manager container snippet — given out in our user interface and documentation — to improve security and performance even more.

Previously, the Tag Manager container snippet loaded containers in a protocol-relative manner. That is, on pages with an https:// URL, the container would be loaded securely using HTTPS. Our new snippets will always use the secure protocol (HTTPS) by default, regardless of the protocol of the page. This helps protect containers from malicious parties and snooping, and in many cases, will also improve performance.

You’ll also notice that we’ve made a change to how we recommend Tag Manager be implemented, now with the container snippet split into two parts:
  • A JavaScript snippet that should be implemented as high in the <head> of your page as possible to ensure the best tag performance. This change is especially important if you plan to use Tag Manager to deploy Google Optimize.
  • An iframe snippet that should be implemented just after the opening <body> tag. This snippet handles firing of image pixels for users with JavaScript disabled, and also enables Google Search Console verification.
To get the best performance, it’s recommended that you implement both container snippets following the latest installation instructions.

What if you already have Google Tag Manager implemented on your site? Not to worry: These changes are optional, and your existing implementations will continue to work without a problem. To get the highest level of security and performance, you can update your implementations at your earliest convenience.

Happy tagging!

Posted by Scott Herman, Product Manager, Google Tag Manager

Spotlight on Data Studio: Beautiful Reports and Collaboration Just Got Easier

We launched Google Data Studio (beta) in the U.S. earlier this year, and last week, we expanded availability to 21 additional countries.* We’re excited to offer Data Studio to companies across the globe so they can easily create and share reports and dashboards, and ultimately use insights and collaboration to make better business decisions.

Organizations today collect an increasingly large amount of data. In a world of web analytics, CRM systems, and third-party sources, data-driven decisions should be easier than ever before. However, all of this data has presented a significant challenge: making it easily accessible and useful.

Having multiple data sources that live in silos within your organization compromises your ability to spot critical business opportunities and issues when they matter most. This is a widespread challenge, with 84% of marketers saying they don’t believe their data sources are well-integrated, according to a 2016 Forrester study.

Data Studio solves this problem by allowing you to easily connect your data and create beautiful, informative reports that are easy to read, easy to share, and fully customizable. Create up to five custom reports with unlimited editing and sharing. All for free.

Get started today so you can:

  • Put all your data to work. Easily access the data sources you need to understand your business and make better decisions.
  • Transform your data. Quickly transform raw data into easy-to-follow reports and dashboards — no code or queries required.
  • Build engaging reports and dashboards. Data Studio gives you the ability to create meaningful, shareable charts and graphs that bring your data to life.
  • Leverage teamwork that works. Work together quickly, from anywhere, by sharing reports via Google Drive.

Not sure where to start? Good news: We’re releasing our first batch of templates today to remove that hurdle. Businesses can use a library of templates to get up and running in a matter of minutes.



Get started for free.

Happy reporting!
The Data Studio team

*Canada, Brazil, Germany, Netherlands, Spain, France, United Kingdom, Italy, Indonesia, India, Japan, South Korea, Australia, Hong Kong, Malaysia, New Zealand, Philippines, Singapore, Thailand, Taiwan, Vietnam

Posted by Kevin Fields, Product Marketing Manager, Data Studio