Tag Archives: networking

Introducing Network Service Tiers: Your cloud network, your way

By Prajakta Joshi, Product Manager, Cloud Networking

We're excited to announce Network Service Tiers Alpha. Google Cloud Platform (GCP) now offers a tiered cloud network. We let you optimize for performance by choosing Premium Tier, which uses Google’s global network with unparalleled quality of service, or optimize for cost, using the new Standard Tier, an attractively-priced network with performance comparable to that of other leading public clouds.

"Over the last 18 years, we built the world’s largest network, which by some accounts delivers 25-30% of all internet traffic” said Urs Hölzle, SVP Technical Infrastructure, Google. “You enjoy the same infrastructure with Premium Tier. But for some use cases, you may prefer a cheaper, lower-performance alternative. With Network Service Tiers, you can choose the network that’s right for you, for each application.”

Power of Premium Tier 

If you use Google Cloud today, then you already use the powerful Premium Tier.

Premium Tier delivers traffic over Google’s well-provisioned, low latency, highly reliable global network. This network consists of an extensive global private fiber network with over 100 points of presence (POPs) across the globe. By this measure, Google’s network is the largest of any public cloud provider.

In Premium Tier, inbound traffic from your end user to your application in Google Cloud enters Google’s private, high performance network at the POP closest to your end user, and GCP delivers this traffic to your application over its private network.
Outbound and Inbound traffic delivery
Similarly, GCP delivers outbound traffic from your application to end users on Google’s network and exits at the POP closest to them, wherever the end users are across the globe. Thus, most of this traffic reaches its destination with a single hop to the end user’s ISP, so it enjoys minimum congestion and maximum performance.

We architected the Google network to be highly redundant, to ensure high availability for your applications. There are at least three independent paths (N+2 redundancy) between any two locations on the Google network, helping ensure that traffic continues to flow between these two locations even in the event of a disruption. As a result, with Premium Tier, your traffic is unaffected by a single fiber cut. In many situations, traffic can flow to and from your application without interruption even with two simultaneous fiber cuts.

GCP customers use Global Load Balancing, another Premium Tier feature, extensively. You not only get the management simplicity of a single anycast IPv4 or IPv6 Virtual IP (VIP), but can also expand seamlessly across regions, and overflow or fail over to other regions.
With Premium Tier, you use the same network that delivers Google’s Search, Gmail, YouTube, and other services as well as the services of customers such as The Home Depot, Spotify and Evernote.
"75% of homedepot.com is now served out of Google Cloud. From the get-go, we wanted to run across multiple regions for high availability. Google's global network is one of the strongest features for choosing Google Cloud."   
— Ravi Yeddula, Senior Director Platform Architecture & Application Development, The Home Depot.


Introducing Standard Tier 


Our new Standard Tier delivers network quality comparable to that of other major public clouds, at a lower price than our Premium Tier.

Why is Standard Tier less expensive? Because we deliver your outbound traffic from GCP to the internet over transit (ISP) networks instead of Google’s network.

Outbound and Inbound traffic delivery
Similarly, we deliver your inbound traffic, from end user to GCP, on Google’s network only within the region where your GCP destination resides. If your user traffic originates from a different region, their traffic will first travel over transit (ISP) network(s) until it reaches the region of the GCP destination.

Standard Tier provides lower network performance and availability compared to Premium Tier. Since we deliver your outbound and inbound traffic on Google’s network only on the short hop between GCP and the POP closest to it, the performance, availability and redundancy characteristics of Standard Tier depend on the transit provider(s) carrying your traffic. Your traffic may experience congestion or outages more frequently relative to Premium Tier, but at a level comparable to other major public clouds.

We also provide only regional network services in Standard Tier, such as the new regional Cloud Load Balancing service. In this tier, your Load Balancing Virtual IP (VIP) is regional, similar to other public cloud offerings, and adds management complexity compared to Premium Tier Global Load Balancing, if you require multi-region deployment.

Compare performance of tiers 

We commissioned Cedexis, an internet performance monitoring and optimization tools company, to take preliminary performance measurements for both Network Service Tiers. As expected, Premium Tier delivers higher throughput and lower latency than Standard Tier. You can view the live dashboards at www.cedexis.com/google-reports/ under the "Network Tiers" section. Cedexis also details their testing methodology on their website.

Cedexis graph below shows throughput for Premium and Standard Tier for HTTP Load Balancing traffic at 50th percentile. Standard (blue line) throughput is 3,223 kbps while Premium (green line) is 5,401 kbps, making Premium throughput ~1.7x times that of Standard. See Cedexis graph below:


In general, Premium Tier displays considerably higher throughput, at every percentile, than Standard Tier.

Compare pricing for tiers 


We're introducing new pricing for Premium and Standard Tiers. You can review detailed pricing for both tiers here. This pricing will take effect when Network Service Tiers become Generally Available (GA). While in alpha and beta, existing internet egress pricing applies.

With the new Network Tiers pricing (effective at GA), outbound traffic (GCP to internet) is priced 24-33% lower in Standard Tier than in Premium Tier for North America and Europe. Standard Tier is less expensive than internet egress options offered by other major public cloud providers (based on typical published prices for July, 2017). Inbound traffic remains free for both Premium and Standard Tiers. We'll also change our current destination-based pricing for Premium Tier to be based on both source and destination of traffic since the cost of network traffic varies with the distance your traffic travels over Google’s network. In contrast, Standard Tier traffic will be source-based since it does not travel much over Google’s network.

Choose the right tier 


Here’s a decision tree to help you choose the tier that best fits your requirements.


Configure the tier for your application(s) 


One size does not fit all, and your applications in Google Cloud often have differing availability, performance, footprint and cost requirements. Configure the tier at the resource-level (per Instance, Instance template, Load balancer) if you want granular control or at the overarching project-level if you want to use the same tier across all resources.

Try Network Service Tiers today 

“Cloud customers want choices in service levels and cost. Matching the right service to the right business requirements provides the alignment needed by customers. Google is the first public cloud provider to recognize that in the alpha release of Network Service Tiers. Premium Tier caters to those who need assured quality, and Standard Tier to those who need lower costs or have limited need for global networking.”  
— Dan Conde, Analyst at ESG

Learn more by visiting Network Service Tiers website, and give Network Service Tiers a spin by signing up for alpha. We look forward to your feedback!

TCP BBR congestion control comes to GCP – your Internet just got faster

By Neal Cardwell, Senior Staff Software Engineer; Yuchung Cheng, Senior Staff Software Engineer; C. Stephen Gunn, Senior Staff Site Reliability Engineer; Soheil Hassas Yeganeh, Senior Software Engineer; Van Jacobson, Research Scientist and Amin Vahdat, Google Fellow

 We're excited to announce that Google Cloud Platform (GCP) now features a cutting-edge new congestion control algorithm, TCP BBR, which achieves higher bandwidths and lower latencies for internet traffic. This is the same BBR that powers TCP traffic from google.com and that improved YouTube network throughput by 4 percent on average globally — and by more than 14 percent in some countries.

"BBR allows the 500,000 WordPress sites on our digital experience platform to load at lightning speed. According to Google’s tests, BBR's throughput can reach as much as 2,700x higher than today's best loss-based congestion control; queueing delays can be 25x lower. Network innovations like BBR are just one of the many reasons we partner with GCP." — Jason Cohen, Founder and CTO, WP Engine

GCP customers, like WP Engine, automatically benefit from BBR in two ways:

  • From GCP services to cloud users: First, when GCP customers talk to GCP services like Cloud Bigtable, Cloud Spanner or Cloud Storage, the traffic from the GCP service to the application is sent using BBR. This means speedier access to your data.
  • From Google Cloud to internet users: When a GCP customer uses Google Cloud Load Balancing or Google Cloud CDN to serve and load balance traffic for their website, the content is sent to users' browsers using BBR. This means faster webpage downloads for users of your site.

At Google, our long-term goal is to make the internet faster. Over the years, we’ve made changes to make TCP faster, and developed the Chrome web browser and the QUIC protocol. BBR is the next step. Here's the paper describing the BBR algorithm at a high level, the Internet Drafts describing BBR in detail and the BBR code for Linux TCP and QUIC.

What is BBR?


BBR ("Bottleneck Bandwidth and Round-trip propagation time") is a new congestion control algorithm developed at Google. Congestion control algorithms — running inside every computer, phone or tablet connected to a network — that decide how fast to send data.

How does a congestion control algorithm make this decision? The internet has largely used loss-based congestion control since the late 1980s, relying only on indications of lost packets as the signal to slow down. This worked well for many years, because internet switches’ and routers’ small buffers were well-matched to the low bandwidth of internet links. As a result, buffers tended to fill up and drop excess packets right at the moment when senders had really begun sending data too fast.

But loss-based congestion control is problematic in today's diverse networks:

  • In shallow buffers, packet loss happens before congestion. With today's high-speed, long-haul links that use commodity switches with shallow buffers, loss-based congestion control can result in abysmal throughput because it overreacts, halving the sending rate upon packet loss, even if the packet loss comes from transient traffic bursts (this kind of packet loss can be quite frequent even when the link is mostly idle).
  • In deep buffers, congestion happens before packet loss. At the edge of today's internet, loss-based congestion control causes the infamous “bufferbloat” problem, by repeatedly filling the deep buffers in many last-mile links and causing seconds of needless queuing delay.

We need an algorithm that responds to actual congestion, rather than packet loss. BBR tackles this with a ground-up rewrite of congestion control. We started from scratch, using a completely new paradigm: to decide how fast to send data over the network, BBR considers how fast the network is delivering data. For a given network connection, it uses recent measurements of the network's delivery rate and round-trip time to build an explicit model that includes both the maximum recent bandwidth available to that connection, and its minimum recent round-trip delay. BBR then uses this model to control both how fast it sends data and the maximum amount of data it's willing to allow in the network at any time.

Benefits for Google Cloud customers


Deploying BBR has resulted in higher throughput, lower latency and better quality of experience across Google services, relative to the previous congestion control algorithm, CUBIC. Take, for example, YouTube’s experience with BBR. Here, BBR yielded 4 percent higher network throughput, because it more effectively discovers and utilizes the bandwidth offered by the network. BBR also keeps network queues shorter, reducing round-trip time by 33 percent; this means faster responses and lower delays for latency-sensitive applications like web browsing, chat and gaming. Moreover, by not overreacting to packet loss, BBR provides 11 percent higher mean-time-between-rebuffers. These represent substantial improvements for all large user populations around the world, across both desktop and mobile users.

These results are particularly impressive because YouTube is already highly optimized; improving the experience for users watching video has long been an obsession here at Google. Ongoing experiments provide evidence that even better results are possible with continued iteration and tuning.

The benefits of BBR translate beyond Google and YouTube, because they're fundamental. A few synthetic microbenchmarks illustrate the nature (though not necessarily the typical magnitude) of the advantages:

  • Higher throughput: BBR enables big throughput improvements on high-speed, long-haul links. Consider a typical server-class computer with a 10 Gigabit Ethernet link, sending over a path with a 100 ms round-trip time (say, Chicago to Berlin) with a packet loss rate of 1%. In such a case, BBR's throughput is 2700x higher than today's best loss-based congestion control, CUBIC (CUBIC gets about 3.3 Mbps, while BBR gets over 9,100 Mbps). Because of this loss resiliency, a single BBR connection can fully utilize a path with packet loss. This makes it a great match for HTTP/2, which uses a single connection, and means users no longer need to resort to workarounds like opening several TCP connections to reach full utilization. The end result is faster traffic on today's high-speed backbones, and significantly increased bandwidth and reduced download times for webpages, videos or other data.
  • Lower latency: BBR enables significant reductions in latency in last-mile networks that connect users to the internet. Consider a typical last-mile link with 10 Megabits of bandwidth, a 40 ms round-trip time, and a typical 1000-packet bottleneck buffer. In a scenario like this, BBR keeps queuing delay 25x lower than CUBIC (CUBIC has a median round-trip time of 1090 ms, versus just 43 ms for BBR). BBR reduces queues and thus queuing delays on last-mile links while watching videos or downloading software, for faster web surfing and more responsive video conferencing and gaming. Because of this ability to curb bufferbloat, one might say that BBR could also stand for BufferBloat Resilience, in addition to Bottleneck Bandwidth and Round-trip propagation time.

GCP is continually evolving, leveraging Google technologies like Espresso, Jupiter, Andromeda, gRPC, Maglev, Cloud Bigtable and Spanner. Open source TCP BBR is just the latest example of how Google innovations provide industry-leading performance.

If you're interested in participating in discussions, keeping up with the latest BBR-related news, watching videos of talks about BBR or pitching in on open source BBR development or testing, join the public bbr-dev e-mail group.

Reimagining virtual private clouds

By Zach Pohlman, Cloud Solutions Architect

At Cloud Next '17 this year, we announced our reimagining of Virtual Private Cloud (VPC), a product that used to be known as GCP Virtual Networks. Today, we thought we’d share a little more insight into what’s different about VPC and what it can do.

Virtual Private Cloud offers you a privately administered space within Google Cloud Platform (GCP), providing the flexibility to scale and control how workloads connect regionally and globally. This means global connectivity across locations and regions, and the elimination of silos across projects and teams. When you connect your on-premise or remote resources to GCP, you’ll have global access to your VPCs without needing to replicate connectivity or administrative policies per region.

Here’s a little more on what that means.
  • VPC is global. Unlike traditional VPCs that communicate across the public internet, requiring redundant, complex VPNs and interconnections to maintain security, a single Google Cloud VPC can span multiple regions. Single connection points to on-premise resources via VPN or Cloud Interconnect provide private access, reducing costs and configuration complexity.
VMs in VPC do not need VPNs to communicate between regions. Inter-region traffic is both encrypted and kept on Google's private network.
  • VPC is sharable. With a single VPC for an entire organization, you can build multi-tenant architectures and share single private network connectivity between teams and projects with a centralized security model. Your teams can use the network as plug-and-play, instead of stitching connectivity with VPNs. Shared VPC also allows teams to be isolated within projects, with separate billing and quotas, yet still maintain a shared IP space and access to commonly used services such as Interconnect or BigQuery.
A single network can be shared across teams and regions, all within the same administrative domain, preventing duplicate work.
  • VPC is expandable. Google Cloud VPCs let you increase the IP space of any subnets without any workload shutdown or downtime. This gives you flexibility and growth options to meet your needs. If you initially build on an IP space of /24s, for example, but need to grow this in one or multiple regions, you can do so quickly and easily without impacting your users.
In Google VPC, the expanded IP range is available in the new zone without rebooting the running VMs. In other VPCs this incurs downtime.
  • VPC is private. With Google VPC you get private access to Google services, such as storage, big data, analytics or machine learning, without having to give your service a public IP address. Configure your application’s front-end to receive internet requests and shield your back-end services from public endpoints, all while being able to access Google Cloud services.
Within Google Cloud, services are directly addressable across regions using private networks and IP addresses without crossing the best-effort public internet.
Global VPCs are divided into regional subnets that use Google’s private backbone to communicate as needed. This allows you to easily distribute different parts of your application across multiple regions to enhance uptime, reduce end-user latency or address data sovereignty needs.

With these enhancements, GCP is delivering alternatives for increasingly complex networks and workloads, and enhancing the abilities for organizations to create and manage spaces in the cloud that map closely to business requirements. You can learn more about Google Virtual Private Clouds at https://cloud.google.com/vpc/.

Getting started with Shared VPC

By Neha Pattan, Staff Software Engineer

Large organizations with multiple cloud projects value the ability to share physical resources, while maintaining logical separation between groups or departments. At Google Cloud Next '17, we announced Shared VPC, which allows you to configure and centrally manage one or more virtual networks across multiple projects in your Organization, the top level Cloud Identity Access Management (Cloud IAM) resource in the Google Cloud Platform (GCP) cloud resource hierarchy.

With Shared VPC, you can centrally manage the creation of routes, firewalls, subnet IP ranges, VPN connections, etc. for the entire organization, and at the same time allow developers to own billing, quotas, IAM permissions and autonomously operate their development projects. Shared VPC is now generally available, so let’s look at how it works and how best to configure it.

How does Shared VPC work?

We implemented Shared VPC entirely in the management control plane, transparent to the data plane of the virtual network. In the control plane, the centrally managed project is enabled as a host project, allowing it to contain one or more shared virtual networks. After configuring the necessary Cloud IAM permissions, you can then create virtual machines in shared virtual networks, by linking one or more service projects to the host project. The advantage of sharing virtual networks in this way is being able to control access to critical network resources such as firewalls and centrally manage them with less overhead.

Further, with shared virtual networks, virtual machines benefit from the same network throughput caps and VM-to-VM latency as when they're not on shared networks. This is also the case for VM-to-VPN and load balancer-to-VM communication.

To illustrate, consider a single externally facing web application server that uses services such as personalization, recommendation and analytics, all internally available, but built by different development teams.

Example topology of a Shared VPC setup.

Let’s look at the recommended patterns when designing such a virtual network in your organization.

Shared VPC administrator role

The network administrator of the shared host project should also have the XPN administrator role in the organization. This allows a single central group to configure new service projects that attach to the shared VPC host project, while also allowing them to set up individual subnetworks in the shared network and configure IP ranges, for use by administrators of specific service projects. Typically, these administrators would have the InstanceAdmin role on the service project.

Subnetworks USE permission

When connecting a service project to the shared network, we recommend you grant the service project administrators compute.subnetworks.use permission (through the NetworkUser role) on one (or more) subnetwork(s) per region, such that the subnetwork(s) are used by a single service project.

This will help ensure cleaner separation of usage of subnetworks by different teams in your organization. In the future, you may choose to associate specific network policies for each subnetwork based on which service project is using it.

Subnetwork IP ranges

When configuring subnetwork IP ranges in the same or different regions, allow sufficient IP space between subnetworks for future growth. GCP allows you to expand an existing subnetwork without affecting IP addresses owned by existing VMs in the virtual network and with zero downtime.

Shared VPC and folders

When using folders to manage projects created in your organization, place all host and service projects for a given shared VPC setup within the same folder. The parent folder of the host project should be in the parent hierarchy of the service projects, so that the parent folder of the host project contains all the projects in the shared VPC setup. When associating service projects with a host project, ensure that these projects will not move to other folders in the future, while still being linked to the host project.


Control external access

In order to control and restrict which VMs can have public IPs and thus access to the internet, you can now set up an organization policy that disables external IP access for VMs. Do this only for projects that should have only internal access, e.g. the personalization, recommendation and analytics services in the example above.

As you can see, Shared VPC is a powerful tool that can make GCP more flexible and manageable for your organization. To learn more about Shared VPC, check out the documentation.

Google Cloud CDN joins CDN Interconnect providers, delivering choice to users

By Prajakta Joshi, Product Manager, Cloud Networking

 With the general availability of Google Cloud CDN, we’re pleased to announce that the number of CDNs in Google Cloud’s CDN ecosystem has risen to eight, offering choice and speed to Google Cloud Platform (GCP) customers.
(click to enlarge)
In this blog post, we provide details and testimonials from the companies behind these CDNs, and explain the unique advantages of our own CDN offering.
(click to enlarge)

Google Cloud CDN highlights:
  • Google Cloud CDN is delivered on Google’s high performance global network and edge infrastructure. Google Cloud CDN uses Google's globally distributed 80+ locations of presence to accelerate content delivery for websites and applications served out of Google Compute Engine and Google Cloud Storage, reducing latency and serving costs.
  • Google Cloud CDN does not charge extra for HTTPS (TLS) traffic.
  • Google Cloud CDN is integrated into GCP. Once you've set up HTTP(S) Load Balancing, you can enable Cloud CDN with a single checkbox as shown below:
(click to enlarge)
  • We'll be launching a new user interface (UI) for Google Cloud CDN at Google Cloud Next 17 with a separate navigation bar for CDN configuration and reporting, in addition to keeping the existing checkbox for enabling it via the Load Balancing UI. Here's a preview of the new CDN UI:
(click to enlarge)


Google Cloud CDN performance

Cedexis, a well known web traffic optimization and analytics company, benchmarks and provides visibility into the latency, throughput and availability of leading CDN providers. Cedexis provided data on Google Cloud CDN performance with Google Cloud CDN demonstrating industry-leading performance. Cedexis testing methodology used for gathering this data can be viewed on its website.

CDN Interconnect


The CDN Interconnect program provides choice and depth to Google Cloud customers. It allows select CDN providers to establish direct interconnect links so that customers egressing network traffic from GCP through one of these links can benefit from the direct connectivity to the CDN providers as well as Interconnect pricing.

Below, some of our providers describe their integrations with Google Cloud and the value their CDN’s deliver to GCP customers:

Akamai
“As more and more enterprises come to rely on cloud-based computing and storage resources to drive their businesses, it’s critically important that performance is maximized and cost effectiveness is maintained. As the operator of the world’s largest distributed CDN platform, we’re collaborating with Google Cloud Platform to ensure that our joint customers can pass traffic directly from Google Cloud Platform to the Akamai CDN, empowering them to take full advantage of their cloud investments." — Bill Wheaton, executive vice president and general manager, Media Products Division, Akamai
Cloudflare
"We view our collaboration with Google Cloud Platform through their CDN Interconnect program as a crucial step in our mission to help build a better Internet. As a result of the high-speed interconnections between Google Cloud Platform and Cloudflare’s CDN, our joint customers will benefit from amplified performance, increased security and significant savings on their monthly bandwidth bill, making for an altogether better user experience.”— Michelle Zatlyn, Co-Founder & Head of User Experience at Cloudflare.
Fastly
"Fastly customers benefit from pairing our platform’s advanced edge compute, enforcement and application delivery solutions with Google Cloud Platform's origin services. Businesses like Firebase and Vimeo leverage the power of Fastly’s edge features for live log streaming to Google BigQuery and deep analytics, as well as the ability to extend applications as close to the end user as possible for faster, more secure and more reliable global performance.” — Lee Chen, Head of Strategic Partnerships at Fastly.
Level 3
“Our CDN is architected for high quality video delivery to support the growth in OTT video. Content creators can process their content in Google Cloud, then deploy seamlessly over our CDN to deliver the fast, high quality video experience end users demand." — Jon Alexander, Level 3 Senior Director of Product Management
Verizon
“Verizon Digital Media Services’ interconnections with Google Cloud Platform help customers delight their users with a lightning fast digital experience that can be scaled globally through our extensive presence and robust connectivity. Our industry-leading cache efficiency supports great user experience while minimizing transaction needs and egress volumes from cloud origins. Our cache efficiency has negated the need for us to charge our customers for origin storage transactions or egress, which makes us an ideal companion CDN to GCP.”  — Kyle Okamoto, Chief Network Officer, Verizon.
Limelight Networks
“Using Google Cloud Platform in combination with the Limelight Orchestrate Platform helps our customers make the most of their cloud investments by extending their performance and capability with Limelight's content management, distribution and delivery services. Combining the flexibility of Google Cloud Platform with the workload efficiency and reach of Limelight's global infrastructure and experience, provides a unique platform to help deliver customer satisfaction,”  — Steve Miller-Jones, Senior Director of Product Management at Limelight Networks.
Come to Google Cloud Next '17 to learn more about Google Cloud CDN and CDN Interconnect Providers.

Google Cloud Platform for data center professionals: what you need to know

Posted by Peter-Mark Verwoerd, Solutions Architect

At Google Cloud, we love seeing customers migrate to our platform. Companies move to us for a variety of reasons, from low costs to our machine learning offerings. Some of our customers, like Spotify and Evernote, have described the various reasons that motivated them to migrate to Google Cloud.

However, we recognize that a migration of any size can be a challenging project, so today we're happy to announce the first part of a new resource to help our customers as they migrate. Google Cloud Platform for Data Center Professionals is a guide for customers who are looking to move to Google Cloud Platform (GCP) and are coming from non-cloud environments. We cover the basics of running IT — Compute, Networking, Storage, and Management. We've tried to write this from the point of view of someone with minimal cloud experience, so we hope you find this guide a useful starting point.

This is the first part of an ongoing series. We'll add more content over time, to help describe the differences in various aspects of running your company's IT infrastructure.

We hope you find this useful in learning about GCP. Please tell us what you think and what else you would like to add, and be sure to follow along with our free trial when you sign up!

Building scalable private services with Internal Load Balancing

Posted by Prajakta Joshi, Product Manager, Cloud Networking


Cloud load balancers are a key part of building resilient and highly elastic services, allowing you to think less about infrastructure and focus more on your applications. But the applications themselves are evolving: they are becoming highly distributed and made up of multiple tiers. Many of these are delivered as internal-only microservices. That’s why we’re excited to announce the general availability of Internal Load Balancing, a fully-managed flavor of Google Cloud Load Balancing that enables you to build scalable and highly available internal services for internal client instances without requiring the load balancers to be exposed to the Internet.

In the past year, we’ve described our Global Load Balancing and Network Load Balancing technologies in detail at the GCP NEXT and NSDI conferences. More recently, we revealed that GCP customer Niantic deployed HTTP(S) LB, alongside Google Container Engine and other GCP technologies, to scale its wildly popular game Pokémon GO. Today Internal Load Balancing joins our arsenal of cloud load balancers to deliver the scale, performance and availability your private services need.

Internal Load Balancing architecture

When we present Cloud Load Balancing, one of the questions we get is “Where is the load balancer?” and then “How many connections per second can it support?”

Similar to our HTTP(S) Load Balancer and Network Load Balancer, Internal Load Balancing is neither a hardware appliance nor an instance-based solution. It is software-defined load balancing delivered using Andromeda, Google’s network virtualization stack.
(click to enlarge)

As a result, your internal load balancer is “everywhere” you need it in your virtual network, but “nowhere” as a choke-point in this network. By virtue of this architecture, Internal Load Balancing can support as many connections per second as needed since there is no load balancer in the path between your client and backend instances.

Internal Load Balancing features

Internal Load Balancing enables you to distribute internal client traffic across backends running private services. In the example below, client instance (192.168.1.1) in Subnet 1 connects to the Internal Load Balancing IP (10.240.0.200) and gets load balanced to a backend instance (10.240.0.2) in Subnet 2.
(click to enlarge)

With Internal Load Balancing, you can:

  • Configure a private RFC1918 load-balancing IP from within your virtual network;
  • Load balance across instances in multiple availability zones within a region;
  • Configure session affinity to ensure that traffic from a client is load balanced to the same backend instance;
  • Configure high-fidelity TCP, SSL(TLS), HTTP or HTTPS health checks;
  • Get instant scaling for your backend instances with no pre-warming; and
  • Get all the benefits of a fully managed load balancing service. You no longer have to worry about load balancer availability or the load balancer being a choke point.

Configuring Internal Load Balancing

You can configure Internal Load Balancing using the REST API, the gcloud CLI or Google Cloud Console. Click here to learn more about configuring Internal Load Balancing.

The (use) case for Internal Load Balancing

Internal Load Balancing delivers private RFC 1918 load balancing within your virtual private networks in GCP. Let’s walk through three interesting ILB use cases:


1. Scaling your internal services

In a typical microservices architecture, you deliver availability and scale for each service by deploying multiple instances of it and using an internal load balancer to distribute traffic across these instances. Internal Load Balancing does this, and also autoscales instances to handle increases in traffic to your service.
(click to enlarge)

2. Building multi-tier applications on GCP

Internal Load Balancing is a critical component for building n-tier apps. For example, you can deploy HTTP(S) Load Balancing as the global web front-end load balancer across web-tier instances. You can then deploy the application server instances (represented as the Internal Tier below) behind a regional internal load balancer and send requests from your web tier instances to it.
(click to enlarge)

3. Delivering high availability and scale for virtual appliances

Traditionally, high availability (HA) for hardware appliances is modeled as an active-standby or active-active set-up where two (or sometimes more) such devices exchange heart beats and state information across a dedicated, physical synchronization link in a Layer 2 network.

This model no longer works for cloud-based virtual appliances such as firewalls, because you do not have access to the physical hardware. Layer 2 based high availability doesn’t work either because public cloud virtual networks are typically Layer 3 networks. More importantly, cloud apps store shared state outside of the application for durability, etc., so it is possible to eliminate traditional session state synchronization.
(click to enlarge)

Considering all of these factors, a high availability model that works well on Google Cloud Platform is deploying virtual appliance instances behind Internal Load Balancing. Internal Load Balancing performs health checks on your virtual appliance instances, distributes traffic across the healthy ones, and scales the number of instances up or down based on traffic.

What’s next for Internal Load Balancing

We have a number of exciting Internal Load Balancing features coming soon, including service discovery using DNS, load balancing traffic from on-prem clients across a VPN to backends behind an internal load balancer, and support for regional instance groups.

Until then, we hope you will give Internal Load Balancing a spin. Start with the tutorial, read the documentation and deploy it on GCP. We look forward to your feedback!

Build highly available services with general availability of Regional Managed Instance Groups

Posted by Jerzy Foryciarz, Product Manager

Businesses choose to build applications on Google Cloud Platform (GCP) for our low-latency and reliable global network. As customers build applications that are increasingly business-critical, designing for high-availability is no longer optional. That’s why we’re pleased to announce the general availability of Regional Managed Instance Groups in Google Compute Engine.

With virtually no effort on the part of customers, this release offers a fully managed service for creating highly available applications: simply specify the region in which to run your application, and Compute Engine automatically balances your machines across independent zones within the region. Combined with load balancing and autoscaling of your machine instances, your applications scale up and down gracefully based on policies fully within your control.

Distributing your application instances across multiple zones is a best practice that protects against adverse events such as a bad application build, networking problems or a zonal outage. Together with overprovisioning the size of your managed instance group, these practices ensure high availability for your applications in the regions where you serve your users.

Customers have vetted regional managed instance groups during our alpha and beta periods, ranging from major consumer-facing brands like Snap Inc. and Waze, to popular services like BreezoMeter, The Carousel and InShorts.
It’s easy to get started with regional managed instance groups. Or let us know if we can assist with architecting your most important applications with the reliability users expect from today’s best cloud apps.


Google Summer of Code 2016 wrap-up: Linux XIA

We're sharing guest posts from students, mentors and organization administrators who participated in Google Summer of Code 2016. This is the fifth post in that series and there are more on the way.


Linux XIA is the native implementation of XIA, a meta network architecture that supports evolution of all of its components, which we call “principals,” and promotes interoperability between these principals. It is the second year that our organization, Boston University / XIA, has participated in Google Summer of Code (GSoC), and this year we received 31 proposals from 8 countries.

Our ideas list this year focused on upgrading key forwarding data structures to their best known versions. Our group chose the most deserving students for each of the following projects:

Accelerating the forwarding speed of the LPM principal with poptrie

Student André Ferreira Eleuterio and mentor Cody Doucette implemented the first version of the LPM principal in Linux XIA for GSoC 2015. The LPM principal enables Linux XIA to leverage routing tables derived from BGP, OSPF, IS-IS and any other IP routing protocol to forward XIA packets natively, that is, without encapsulation in IP. For GSoC 2016, student Vaibhav Raj Gupta from India partnered with mentor Cody Doucette to speed up the LPM principal by employing a state-of-the-art data structure to find the longest prefix matching using general purpose processors: poptrie.

Upgrading the FIB hash table of principals to the relativistic hash table

Principals that rely on routing flat names have used a resizable hash table that supports lockless readers since 2011. While this data structure was unique in 2011, in the same year, relativistic hash tables were published. The appeal to upgrade to relativistic hash tables was twofold: reduced memory footprint per hashed element, and the fact they were implemented in the Linux kernel in 2014. Student Sachin Paryani, also from India, worked with mentor Qiaobin Fu to replace our resizable hash table with the relativistic hash table.

Google Summer of Code nurtures a brighter future. Thanks to GSoC, our project has received important code contributions, and our community has been enlarged. It was rewarding to learn that two of our GSoC students have decided to pursue graduate school after their GSoC experience with us: Pranav Goswami (2015) and Sachin Paryani (2016). We hope these examples will motivate other students to do their best because the world is what we make of it.

By Michel Machado, Boston University / XIA organization administrator

Cilium: Networking and security for containers with BPF and XDP

This is a guest post by Daniel Borkmann who was recently recognized through the Google Open Source Peer Bonus program for his work on the Cilium project. We invited Daniel to share his project on our blog.

Our open source project, called Cilium, started as an experiment for Linux container networking tackling four requirements:

  • Scale: How can we scale in terms of addressing and with regards to network policy?
  • Extensibility: Can we be as extensible as user space networking in the Linux kernel itself?
  • Simplicity: What is an appropriate abstraction away from traditional networking?
  • Performance: Do we sacrifice performance in the process of implementing the aforementioned aspects?

We realize these goals in Cilium with the help of eBPF. eBPF is an efficient and generic in-kernel bytecode engine, that allows for full programmability. There are many subsystems in the Linux kernel that utilize eBPF, mainly in the areas of networking, tracing and security.

eBPF can be attached to key ingress and egress points of the kernel's networking data path for every network device. As input, eBPF operates on the kernel's network packet representation and can thus access and mangle various kinds of data, redirect the packet to other devices, perform encapsulations, etc.

This is a typical workflow: eBPF is programmed in a subset of C, compiled with LLVM which contains an eBPF back-end. LLVM then generates an ELF file containing program code, specification for maps and related relocation data. In eBPF, maps are efficient key/value stores in the kernel that can be shared between various eBPF programs, but also between user space. Given the ELF file, tools like tc (traffic control) can parse its content and load the program into the kernel. Before the program is executed, the kernel verifies the eBPF bytecode in order to make sure that it cannot affect the kernel's stability (e.g. crash the kernel and out of bounds access) and always terminates, which requires programs to be free of loops. Once it passed verification, the program is JIT (just-in-time) compiled.

Today, architectures such as x86_64, arm64, ppc64 and s390 have the ability to compile a native opcode image out of an eBPF program, so that instead of an execution through an in-kernel eBPF interpreter, the resulting image can run natively like any other kernel code. tc then installs the program into the kernel's networking data path, and with a capable NIC, the program can also be offloaded entirely into the hardware.


Cilium acts as a middle layer, plugs into container runtimes and orchestrators such as Kubernetes, Docker or CNI, and can generate and atomically update eBPF programs on the fly without requiring a container to restart. Thus, unlike connection proxies, an update of the datapath does not cause connections to be dropped. These programs are specifically tailored and optimized for each container, for example, a feature that a particular container does not need can just be compiled out and the majority of configuration becomes constant, allowing LLVM for further optimizations.

We have many implemented building blocks in Cilium using eBPF, such as NAT64, L3/L4 load balancing with direct server return, a connection tracker, port mapping, access control, NDisc and ARP responder and integration with various encapsulations like VXLAN, Geneve and GRE, just to name a few. Since all these building blocks run in the Linux kernel and have a stable API, there is of course no need to cross kernel/user space boundary, which makes eBPF a perfectly suited and flexible technology for container networking.

One step further in that direction is XDP, which was recently merged into the Linux kernel and allows for DPDK-like performance for the kernel itself. The basic idea is that XDP is tightly coupled with eBPF and hooks into a very early ingress path at the driver layer, where it operates with direct access to the packet's DMA buffer.

This is effectively as low-level as it can get to reach near-optimal performance, which mainly allows for tailoring high-performance load balancers or routers with commodity hardware. One advantage that comes with XDP is also that it reuses the kernel's security model for accessing the device as opposed to user space based mechanisms. It doesn't require any third party modules and works in concert with the Linux kernel. Both XDP and tc with eBPF are complementary to each other, and constitute a bigger piece of the puzzle for Cilium itself.

If you’re curious, check out the Cilium code or demos on GitHub.


By Daniel Borkmann, Cilium contributor