Tag Archives: Open source

What’s new from Firebase at Google I/O 2017

Originally posted on the Firebase Blog by Francis Ma, Firebase Group Product Manager

It's been an exciting year! Last May, we expanded Firebase into our unified app platform, building on the original backend-as-a-service and adding products to help developers grow their user base, as well as test and monetize their apps. Hearing from developers like Wattpad, who built an app using Firebase in only 3 weeks, makes all the hard work worthwhile.

We're thrilled by the initial response from the community, but we believe our journey is just getting started. Let's talk about some of the enhancements coming to Firebase today.

Integrating with Fabric

In January, we announced that we were welcoming the Fabric team to Firebase. Fabric initially grabbed our attention with their array of products, including the industry-leading crash reporting tool, Crashlytics. As we got to know the team better, we were even more impressed by how closely aligned our missions are: to help developers build better apps and grow successful businesses. Over the last several months, we've been working closely with the Fabric team to bring the best of our platforms together.

We plan to make Crashlytics the primary crash reporting product in Firebase. If you don't already use a crash reporting tool, we recommend you take a look at Crashlytics and see what it can do for you. You can get started by following the Fabric documentation.

Phone authentication comes to Firebase

Phone number authentication has been the biggest request for Firebase Authentication, so we're excited to announce that we've worked with the Fabric Digits team to bring phone auth to our platform. You can now let your users sign in with their phone numbers, in addition to traditional email/password or identity providers like Google or Facebook. This gives you a comprehensive authentication solution no matter who your users are or how they like to log in.

At the same time, the Fabric team will be retiring the Digits name and SDK. If you currently use Digits, over the next couple weeks we'll be rolling out the ability to link your existing Digits account with Firebase and swap in the Firebase SDK for the Digits SDK. Go to the Digits blog to learn more.

Introducing Firebase Performance Monitoring

We recognize that poor app performance and stability are the top reasons for users to leave bad ratings on your app and possibly churn altogether. As part of our effort to help you build better apps, we're pleased to announce the beta launch of Performance Monitoring.

Firebase Performance Monitoring is a new free tool that helps you understand when your user experience is being impacted by poorly performing code or challenging network conditions. You can learn more and get started with Performance Monitoring in the Firebase documentation.

More robust analytics

Analytics has been core to the Firebase platform since we launched last I/O. We know that understanding your users is the number one way to make your app successful, so we're continuing to invest in improving our analytics product.

First off, you may notice that you're starting to see the name "Google Analytics for Firebase" around our documentation. Our analytics solution was built in conjunction with the Google Analytics team, and the reports are available both in the Firebase console and the Google Analytics interface. So, we're renaming Firebase Analytics to Google Analytics for Firebase, to reflect that your app analytics data are shared across both.

For those of you who monetize your app with AdMob, we've started sharing data between the two platforms, helping you understand the true lifetime value (LTV) of your users, from both purchases and AdMob revenue. You'll see these new insights surfaced in the updated Analytics dashboard.

Many of you have also asked for analytics insights into custom events and parameters. Starting today, you can register up to 50 custom event parameters and see their details in your Analytics reports. Learn more about custom parameter reporting.

Firebase for all - iOS, games, and open source

Firebase's mission is to help all developers build better apps. In that spirit, today we're announcing expanded platform and vertical support for Firebase.

First of all, as Swift has become the preferred language for many iOS developers, we've updated our SDK to handle Swift language nuances, making Swift development a native experience on Firebase.

We've also improved Firebase Cloud Messaging by adding support for token-based authentication for APNs, and greatly simplifying the connection and registration logic in the client SDK.

Second, we've heard from our game developer community that one of the most important stats you monitor is frames per second (FPS). So, we've built Game Loop support & FPS monitoring into Test Lab for Android, allowing you to evaluate your game's frame rate before you deploy. Coupled with the addition of Unity plugins and a C++ SDK, which we announced at GDC this year, we think that Firebase is a great option for game developers. To see an example of a game built on top of Firebase, check out our Mecha Hamster app on Github.

Finally, we've taken a big first step towards open sourcing our SDKs. We believe in open source software, not only because transparency is an important goal, but also because we know that the greatest innovation happens when we all collaborate. You can view our new repos on our open sourceproject page and learn more about our decision in this blog post.

Dynamic Hosting with Cloud Functions for Firebase

In March, we launched Cloud Functions for Firebase, which lets you run custom backend code in response to events triggered by Firebase features and HTTP requests. This lets you do things like send a notification when a user signs up or automatically create thumbnails when an image is uploaded to Cloud Storage.

Today, in an effort to better serve our web developer community, we're expanding Firebase Hosting to integrate with Cloud Functions. This means that, in addition to serving static assets for your web app, you can now serve dynamic content, generated by Cloud Functions, through Firebase Hosting. For those of you building progressive web apps, Firebase Hosting + Cloud Functions allows you to go completely server-less. You can learn more by visiting our documentation.

Firebase Alpha program and what's next

Our goal is to build the best developer experience: easy-to-use products, great documentation, and intuitive APIs. And the best resource that we have for improving Firebase is you! Your questions and feedback continuously push us to make Firebase better.

In light of that, we're excited to announce a Firebase Alpha program, where you will have the opportunity to test the cutting edge of our products. Things might not be perfect (in fact, we can almost guarantee they won't be), but by participating in the alpha community, you'll help define the future of Firebase. If you want to get involved, please register your interest in the Firebase Alpha form.

Thank you for your support, enthusiasm, and, most importantly, feedback. The Firebase community is the reason that we've been able to grow and improve our platform at such an incredible pace over the last year. We're excited to continue working with you to build simple, intuitive products for developing apps and growing mobile businesses. To get started with Firebase today, visit our newly redesigned website. We're excited to see what you build!

Google Summer of Code 2017 statistics: Part one

Since 2005 Google Summer of Code (GSoC) has been bringing new developers into the open source community every year. GSoC 2017 is the largest to date with 1,318 students from 72 countries accepted into the program who are working with a record 201 open source organizations this summer.

Students are currently participating in the Community Bonding phase of the program where they become familiar with the open source communities they will be working with. They also spend time learning the codebase and the community’s best practices so they can start their 12 week coding projects on May 30th.

Each year we like to share program statistics as we see GSoC continue to expand all over the world. This year there are three students that are the first to be accepted into GSoC from their home countries: Qatar, Tajikistan and Zimbabwe. A complete list of accepted students and their countries is below:

Country Students Country Students Country Students
Argentina 3 Ghana 1 Qatar 1
Armenia 1 Greece 29 Romania 11
Australia 6 Hungary 6 Russian Federation 54
Austria 13 India 569 Saudi Arabia 1
Bangladesh 2 Indonesia 2 Serbia 3
Belarus 3 Ireland 5 Singapore 10
Belgium 6 Israel 2 Slovak Republic 6
Bosnia and Herzegovina 1 Italy 23 Slovenia 2
Brazil 21 Jamaica 1 South Africa 2
Bulgaria 4 Japan 13 South Korea 8
Cameroon 8 Kazakhstan 1 Spain 19
Canada 27 Kenya 1 Sri Lanka 54
China 49 Latvia 1 Sweden 8
Colombia 1 Lithuania 2 Switzerland 5
Costa Rica 1 Macedonia 1 Taiwan 1
Croatia 1 Mexico 1 Tajikistan 1
Czech Republic 6 Moldova 1 Turkey 11
Denmark 2 Netherlands 14 Ukraine 12
Ecuador 2 New Zealand 1 United Arab Emirates 1
Egypt 10 Nigeria 1 United Kingdom 16
Estonia 1 Pakistan 8 United States 126
Finland 4 Peru 1 Uruguay 1
France 20 Poland 19 Vietnam 4
Germany 55 Portugal 10 Zimbabwe 1

In our next GSoC statistics post we will delve deeper into the schools, gender breakdown, mentors and registration numbers for the 2017 program.

Stephanie Taylor, Google Open Source

Istio: a modern approach to developing and managing microservices


Today Google, IBM and Lyft announced the alpha release of Istio: a new open-source project that provides a uniform way to help connect, secure, manage and monitor microservices.

Istio encapsulates many of the best practices Google has been using to run massive-scale services in production for years. We're happy to contribute this to the community as an open solution that works with Kubernetes; on-premises or in any cloud, to help solve challenges in modern application development. Istio provides developers and devops fine-grained visibility and control over traffic without requiring any changes to application code and provides CIOs and CSOs the tools needed to help enforce security and compliance requirements across the enterprise.

"Based on years of practical experience running container-based systems and working with enterprise clients, I've found that as developers adopt microservice architectures, they need a consistent way to connect, secure and manage the applications they are building", said Jason McGee, IBM Fellow, VP and CTO, IBM Cloud Platform. “IBM is thrilled to be joining forces with Google to launch the Istio project and give cloud developers the tools they need to turn disparate microservices into an integrated service mesh.”

Moving from monolithic apps to microservices
As monolithic applications are decomposed into microservices, teams have to worry about the challenges inherent in integrating services in distributed systems: they must account for service discovery, load balancing, fault tolerance, end-to-end monitoring, dynamic routing for feature experimentation and, perhaps most important of all, compliance and security.

How Istio helps
Istio is a layer of infrastructure between a service and the network that gives operators the controls they need and frees developers from having to solve distributed system problems in their code. This uniform layer of infrastructure combined with service deployments is commonly referred to as a service mesh. Istio is designed to run in any environment on any cloud, but we're starting our journey on Kubernetes. It only takes a single command to install Istio on any Kubernetes cluster, creating a service mesh that enables:
  • Automatic load balancing for HTTP, gRPC, and TCP traffic
  • Fine-grained control of traffic behavior with rich routing rules
  • Traffic encryption, service-to-service authentication and strong identity assertions
  • Fleet-wide policy enforcement
  • In-depth telemetry and reporting
The service mesh empowers operators with policy control and decouples them from feature development and release processes, providing centralized management regardless of the scale and velocity of applications. Google has been realizing the benefits of a service mesh for over a decade, to offer global-scale reliable services like YouTube and Gmail, Cloud PubSub and Cloud BigTable.
“Google's experience is that having a uniform substrate for developing and operating microservices is critical to our ability to scale while maintaining both feature velocity and reliability”  Eric Brewer, Vice President, Google Cloud

An open community

To learn more about Istio and the problems it addresses, visit the Istio launch blog post. Istio is being developed in the open on GitHub, and we invite the community to join us in shaping the project as we work toward a 1.0 release later this year. We look forward to working with the community in making Istio production ready and working everywhere.

Google Cloud is committed to open-source, whether it’s bringing new technologies in the open like Kubernetes or gRPC; contributing to projects like Envoy; or supporting open-source tools on Google Cloud Platform. Istio is the latest instance of Google's continuing contribution to open-source as part of a collaborative community effort.

Beyond Istio

Istio is just one piece of a solution to help make microservices easier to build, deploy, consume and manage. In large enterprises with diverse environments and widespread use of third-party software, developers also want to discover, instantiate and consume services in a platform-agnostic way. Developers providing services need faster time-to-market, greater reach and a simple way to track usage and costs. Towards this end, we've been working with the open source community to contribute to the Open Service Broker, a unified API that simplifies service delivery and consumption. Through the Open Service Broker model CIOs can define a catalog of services which may be used within their enterprise and auditing tools to enforce compliance. All services powered by Istio will be able to seamlessly participate in the Service Broker ecosystem.

Looking ahead

Today, you can manually install and use Istio on Google Container Engine; in the future, we intend to provide a more automated and integrated experience.

We also intend to bring Istio capabilities to Cloud Endpoints and Apigee suite of products. This will provide common visibility and management for both APIs and microservices for organizations of any size. As we work with the community to harden Istio for production-readiness, we plan to provide deeper integration with the rest of Google Cloud.

Get started today

You can get started with Istio here. We also have a sample application composed of four separate microservices that can be easily deployed and used to demonstrate various features of the Istio service mesh. In case of issues you can reach out via the istio-users@googlegroups.com mailing-list or file an issue on GitHub. If you’d like to build an integration with Istio, please fill out this form. We're excited about the future of microservices and API development built on Istio and Google Cloud.

Open sourcing the Firebase SDKs

Today, at Google I/O 2017, we are pleased to announce that we are taking our first steps towards open sourcing our client libraries. By making our SDKs open, we’re aiming to show our commitment to greater transparency and to building a stronger developer community. To help further that goal, we’ll be using GitHub as a core part of our own toolchain to enable all of you to contribute as well. As you find issues in our code, from inconsistent style to bugs, you can file issues through the standard GitHub issue tracker. You can also find our project in the Google Open Source directory. We’re really looking forward to your pull requests!

What’s open?

We’re starting by open sourcing several products in our iOS, JavaScript, Java, Node.js and Python SDKs. We'll be looking at open sourcing our Android SDK as well. The SDKs are being licensed under Apache 2.0, the same flexible license as existing Firebase open source projects like FirebaseUI.

Let's take a look at each repo:

Firebase iOS SDK 4.0

https://github.com/firebase/firebase-ios-sdk

With the launch of the Firebase iOS 4.0 SDKs we have made several improvements to the developer experience, such as more idiomatic API names for our Swift users. By open sourcing our iOS SDKs we hope to provide an additional avenue for you to give us feedback on such features. For this first release we are open sourcing our Realtime Database, Auth, Cloud Storage and Cloud Messaging (FCM) SDKs, but going forward we intend to release more.

Because we aren't yet able to open source some of the Firebase components, the full product build process isn't available. While you can use this repo to build a FirebaseDev pod, our libraries distributed through CocoaPods will continue to be static frameworks for the time being. We are continually looking for ways to improve the developer experience for developers, however you integrate.

Our GitHub README provides more details on how you build, test and contribute to our iOS SDKs.

Firebase JavaScript SDK 4.0

https://github.com/firebase/firebase-js-sdk

We are excited to announce that we are open sourcing our Realtime Database, Cloud Storage and Cloud Messaging (FCM) SDKs for JavaScript. We’ll have a couple of improvements hot on the heels of this initial release, including open sourcing Firebase Authentication. We are also in the process of releasing the source maps for our components, which we expect would really improve the debuggability of your app.

Our GitHub repo includes instructions on how you can build, test and contribute.

Firebase Admin SDKs

Node.js: https://github.com/firebase/firebase-admin-node
Java: https://github.com/firebase/firebase-admin-java
Python: https://github.com/firebase/firebase-admin-python

We are happy to announce that all three of our Admin SDKs for accessing Firebase on privileged environments are now fully open source, including our recently-launched Python SDK. While we continue to explore supporting more languages, we encourage you to use our source as inspiration to enable Firebase for your environment (and if you do, we'd love to hear about it!)

We're really excited to see what you do with the updated SDKs - as always reach out to us with feedback or questions in the Firebase-Talk Google Group, on Stack Overflow, via the Firebase Support team, and now on GitHub for SDK issues and pull requests! And to read about the other improvements to Firebase that launched at Google I/O, head over to the Firebase blog.

By Salman Qadri, Firebase Product Manager

Open Source at Google I/O 2017

One of the best parts of Google I/O every year is the chance to meet with the developers and community organizers from all over the world. It's a unique opportunity to have candid one-on-one conversations about the products and technologies we all love.

This year, I/O features a Community Lounge for attendees to relax, hangout, and play with neat experiments and games. It also features several mini-meetups during which you can chat with Googlers on a variety of topics.

Chris DiBona and Will Norris from the Google Open Source Programs Office will be around Thursday and Friday to talk about anything and everything open source, including our student outreach programs and the new Google Open Source website. If you're at Google I/O this year, make sure to drop by and say hello. Find dates, times, and other details in the Community Lounge schedule.

By Josh Simmons, Google Open Source

Here comes Treble: A modular base for Android

Posted by Iliyan Malchev, Project Treble team lead

On the Android team, we view each dessert release as an opportunity to make Android better for our users and our ecosystem partners. One thing we've consistently heard from our device-maker partners is that updating existing devices to a new version of Android is incredibly time consuming and costly.

With Android O, we've been working very closely with device makers and silicon manufacturers to take steps toward solving this problem, and we're excited to give you a sneak peek at Project Treble, the biggest change to the low-level system architecture of Android to date.

Life of an Android Release

First, it's helpful to understand the "life of an Android release". There are several steps a new Android release goes through before getting into the hands of users:

  1. The Android team publishes the open-source code for the latest release to the world.
  2. Silicon manufacturers, the companies that make the chips that power Android devices, modify the new release for their specific hardware.
  3. Silicon manufacturers pass the modified new release to device makers -- the companies that design and manufacture Android devices. Device makers modify the new release again as needed for their devices.
  4. Device makers work with carriers to test and certify the new release.
  5. Device makers and carriers make the new release available to users.

    With Project Treble, we're re-architecting Android to make it easier, faster and less costly for manufacturers to update devices to a new version of Android.

    The Vendor Interface

    Android was unveiled in 2007 as a free, open-source mobile operating system. From the beginning, we intended Android to be scaled across a variety of manufacturers. We knew that consistency of API was important for developers, so we created a compatibility program for the Developer API specified by the Compatibility Definition Document (CDD) and its associated Compatibility Test Suite (CTS), now comprising over a million tests.

    The result today is that app developers can write a single app that works across over a billion devices running on different hardware from different manufacturers.

    Project Treble aims to do what CTS did for apps, for the Android OS framework. The core concept is to separate the vendor implementation - the device-specific, lower-level software written in large part by the silicon manufacturers - from the Android OS Framework. This is achieved by the introduction of a new vendor interface between the Android OS framework and the vendor implementation. The new vendor interface is validated by a Vendor Test Suite (VTS), analogous to the CTS, to ensure forward compatibility of the vendor implementation.

    Benefits of Project Treble

    Today, with no formal vendor interface, a lot of code across Android needs to be updated when a device moves to a newer version of Android:

    With a stable vendor interface providing access to the hardware-specific parts of Android, device makers can choose to deliver a new Android release to consumers by just updating the Android OS framework without any additional work required from the silicon manufacturers:

    Project Treble will be coming to all new devices launched with Android O and beyond. In fact, the new Project Treble architecture is already running on the Developer Preview of O for Pixel phones.

    In addition to the architectural changes, we're working with our silicon and device partners to take their code changes, such as features for a carrier network in a specific country, and move them into the common Android Open Source Project (AOSP) codebase. For example, Sony and Qualcomm contributed dozens of features and hundreds of bugfixes to Android O so they no longer need to rework these patches with each new release of Android.

    We plan to publish the full documentation for Project Treble on source.android.com with the launch of O later this summer.

OSS-Fuzz: Five months later, and rewarding projects

Five months ago, we announced OSS-Fuzz, Google’s effort to help make open source software more secure and stable. Since then, our robot army has been working hard at fuzzing, processing 10 trillion test inputs a day. Thanks to the efforts of the open source community who have integrated a total of 47 projects, we’ve found over 1,000 bugs (264 of which are potential security vulnerabilities).

Breakdown of the types of bugs we're finding.

Notable results

OSS-Fuzz has found numerous security vulnerabilities in several critical open source projects: 10 in FreeType2, 17 in FFmpeg, 33 in LibreOffice, 8 in SQLite 3, 10 in GnuTLS, 25 in PCRE2, 9 in gRPC, and 7 in Wireshark, etc. We’ve also had at least one bug collision with another independent security researcher (CVE-2017-2801). (Some of the bugs are still view restricted so links may show smaller numbers.)

Once a project is integrated into OSS-Fuzz, the continuous and automated nature of OSS-Fuzz means that we often catch these issues just hours after the regression is introduced into the upstream repository, before any users are affected.

Fuzzing not only finds memory safety related bugs, it can also find correctness or logic bugs. One example is a carry propagating bug in OpenSSL (CVE-2017-3732).

Finally, OSS-Fuzz has reported over 300 timeout and out-of-memory failures (~75% of which got fixed). Not every project treats these as bugs, but fixing them enables OSS-Fuzz to find more interesting bugs.

Announcing rewards for open source projects

We believe that user and internet security as a whole can benefit greatly if more open source projects include fuzzing in their development process. To this end, we’d like to encourage more projects to participate and adopt the ideal integration guidelines that we’ve established.

Combined with fixing all the issues that are found, this is often a significant amount of work for developers who may be working on an open source project in their spare time. To support these projects, we are expanding our existing Patch Rewards program to include rewards for the integration of fuzz targets into OSS-Fuzz.

To qualify for these rewards, a project needs to have a large user base and/or be critical to global IT infrastructure. Eligible projects will receive $1,000 for initial integration, and up to $20,000 for ideal integration (the final amount is at our discretion). You have the option of donating these rewards to charity instead, and Google will double the amount.

To qualify for the ideal integration reward, projects must show that:
  • Fuzz targets are checked into their upstream repository and integrated in the build system with sanitizer support (up to $5,000).
  • Fuzz targets are efficient and provide good code coverage (>80%) (up to $5,000). 
  • Fuzz targets are part of the official upstream development and regression testing process, i.e. they are maintained, run against old known crashers and the periodically updated corpora (up to $5,000).
  • The last $5,000 is a “l33t” bonus that we may reward at our discretion for projects that we feel have gone the extra mile or done something really awesome.
We’ve already started to contact the first round of projects that are eligible for the initial reward. If you are the maintainer or point of contact for one of these projects, you may also reach out to us in order to apply for our ideal integration rewards.

The future

We’d like to thank the existing contributors who integrated their projects and fixed countless bugs. We hope to see more projects integrated into OSS-Fuzz, and greater adoption of fuzzing as standard practice when developing software.

By Oliver Chang, Abhishek Arya (Security Engineers, Chrome Security), Kostya Serebryany (Software Engineer, Dynamic Tools), and Josh Armour (Security Program Manager)

Students, Start Your Engineerings!


It’s that time again! Our 201 mentoring organizations have selected 1,318 the students they look forward to working with during the 13th Google Summer of Code (GSoC). Congratulations to our 2017 students and a big thank you to everyone who applied!

The next step for participating students is the Community Bonding period which runs from May 4th through May 30th. During this time, students will get up to speed on the culture and toolset of their new community. They’ll also get acquainted with their mentor and learn more about the languages or tools they will need to complete their projects. Coding commences May 30th.

To the more than 4,200 students who were not chosen this year - don’t be discouraged! Many students apply at least once to GSoC before being accepted. You can improve your odds for next time by contributing to the open source project of your choice directly; organizations are always eager for new contributors! Look around GitHub and elsewhere on the internet for a project that interests you and get started.

Happy coding, everyone!

By Cat Allman, Google Open Source

Saddle up and meet us in Texas for OSCON 2017

Program chairs at OSCON 2016, left to right:
Kelsey Hightower, Scott Hanselman, Rachel Roumeliotis.
Photo used with permission from O'Reilly Media.
The Google Open Source team is getting ready to hit the road and join the open source panoply that is Open Source Convention (OSCON). This year the event runs May 8-11 in Austin, Texas and is preceded on May 6-7 by the free-to-attend Community Leadership Summit (CLS).

You’ll find our team and many other Googlers throughout the week on the program schedule and in the expo hall at booth #401. We’ve got a full rundown of our schedule below, but you can swing by the expo hall anytime to discuss Google Cloud Platform, our open source outreach programs, the projects we’ve open-sourced including Kubernetes, TensorFlow, gRPC, and even our recently released open source documentation.

Of course, you’ll also find our very own Kelsey Hightower everywhere since he is serving as one of three OSCON program chairs for the second year in a row.

Are you a student, educator, project maintainer, community leader, past or present participant in Google Summer of Code or Google Code-in? Join us for lunch at the Google Summer of Code table in the conference lunch area on Wednesday afternoon. We’ll discuss our outreach programs which help open source communities grow while providing students with real world software development experience. We’ll be updating this blog post and tweeting with details closer to the date.

Without further ado, here’s our schedule of events:

Monday, May 8th (Tutorials)

Tuesday, May 9th (Tutorials)

Wednesday, May 10th (Sessions)
12:30pm Google Summer of Code and Google Code-in lunch

Thursday, May 11th (Sessions)

We look forward to seeing you deep in the heart of Texas at OSCON 2017!

By Josh Simmons, Google Open Source

Guest post: Using Terraform to manage Google Cloud Platform infrastructure as code



Managing infrastructure usually involves a web interface or issuing commands in the terminal. These work great for individuals and small teams, but managing infrastructure in this way can be troublesome for larger teams with complex requirements. As more organizations migrate to the cloud, CIOs want hybrid and multi-cloud solutions. Infrastructure as code is one way to manage this complexity.

The open-source tool Terraform, in particular, can help you more safely and predictably create, change and upgrade infrastructure at scale. Created by HashiCorp, Terraform codifies APIs into declarative configuration files that can be shared amongst team members, edited, reviewed and versioned in the same way that software developers can with application code.

Here's a sample Terraform configuration for creating an instance on Google Cloud Platform (GCP):

resource "google_compute_instance" "blog" {
  name         = "default"
  machine_type = "n1-standard-1"
  zone         = "us-central1-a"

  disk {
    image = "debian-cloud/debian-8"
  }

  disk {
    type    = "local-ssd"
    scratch = true
  }

  network_interface {
    network = "default"
  }
}

Because this is a text file, it can be treated the same as application code and manipulated with the same techniques that developers have had for years, including linting, testing, continuous integration, continuous deployment, collaboration, code review, change requests, change tracking, automation and more. This is a big improvement over managing infrastructure with wikis and shell scripts!

Terraform separates the infrastructure planning phase from the execution phase. The terraform plan command performs a dry-run that shows you what will happen. The terraform apply command makes the changes to real infrastructure.

$ terraform plan
+ google_compute_instance.default
    can_ip_forward:                    "false"
    create_timeout:                    "4"
    disk.#:                            "2"
    disk.0.auto_delete:                "true"
    disk.0.disk_encryption_key_sha256: ""
    disk.0.image:                      "debian-cloud/debian-8"
    disk.1.auto_delete:                "true"
    disk.1.disk_encryption_key_sha256: ""
    disk.1.scratch:                    "true"
    disk.1.type:                       "local-ssd"
    machine_type:                      "n1-standard-1"
    metadata_fingerprint:              ""
    name:                              "default"
    self_link:                         ""
    tags_fingerprint:                  ""
    zone:                              "us-central1-a"


$ terraform apply
google_compute_instance.default: Creating...
  can_ip_forward:                    "" => "false"
  create_timeout:                    "" => "4"
  disk.#:                            "" => "2"
  disk.0.auto_delete:                "" => "true"
  disk.0.disk_encryption_key_sha256: "" => ""
  disk.0.image:                      "" => "debian-cloud/debian-8"
  disk.1.auto_delete:                "" => "true"
  disk.1.disk_encryption_key_sha256: "" => ""
  disk.1.scratch:                    "" => "true"
  disk.1.type:                       "" => "local-ssd"
  machine_type:                      "" => "n1-standard-1"
  metadata_fingerprint:              "" => ""
  name:                              "" => "default"
  network_interface.#:               "" => "1"
  network_interface.0.address:       "" => ""
  network_interface.0.name:          "" => ""
  network_interface.0.network:       "" => "default"
  self_link:                         "" => ""
  tags_fingerprint:                  "" => ""
  zone:                              "" => "us-central1-a"
google_compute_instance.default: Still creating... (10s elapsed)
google_compute_instance.default: Still creating... (20s elapsed)
google_compute_instance.default: Creation complete (ID: default)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

This instance is now running on Google Cloud:
(click to enlarge)

Terraform can manage more that just compute instances. At Google Cloud Next, we announced support for GCP APIs to manage projects and folders as well as billing. With these new APIs, Terraform can manage entire projects and many of their resources.

By adding just a few lines of code to the sample configuration above, we create a project tied to our organization and billing account, enable a configurable number of APIs and services on that project and launch the instance inside this newly-created project.

resource "google_project" "blog" {
  name            = "blog-demo"
  project_id      = "blog-demo-491834"
  billing_account = "${var.billing_id}"
  org_id          = "${var.org_id}"
}

resource "google_project_services" "blog" {
  project = "${google_project.blog.project_id}"

  services = [
    "iam.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "cloudapis.googleapis.com",
    "compute-component.googleapis.com",
  ]
}

resource "google_compute_instance" "blog" {
  # ... 

  project = "${google_project.blog.project_id}" # <-- ...="" code="" new="" option="">

Terraform also detects changes to the configuration and only applies the difference of the changes.

$ terraform apply
google_compute_instance.default: Refreshing state... (ID: default)
google_project.my_project: Creating...
  name:        "" => "blog-demo"
  number:      "" => ""
  org_id:      "" => "1012963984278"
  policy_data: "" => ""
  policy_etag: "" => ""
  project_id:  "" => "blog-demo-491834"
  skip_delete: "" => ""
google_project.my_project: Still creating... (10s elapsed)
google_project.my_project: Creation complete (ID: blog-demo-491835)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

We can verify the project is created with the proper APIs:
(click to enlarge)

And the instance exists inside this project.
This project + instance can be stamped out multiple times. Terraform can also create and export IAM credentials and service accounts for these projects.

By combining GCP’s new resource management and billing APIs and Terraform, you have more control over your organization's resources. With the isolation guaranteed by projects and the reproducibility provided by Terraform, it's possible to quickly stamp out entire environments. Terraform parallelizes as many operations as possible, so it's often possible to spin up a new environment in just a few minutes. And in larger organizations with rollup billing, IT teams can use Terraform to stamp out pre-configured environments tied to a single billing organization.

Use Cases

There are many challenges that can benefit from an infrastructure as code approach to managing resources. Here are a few that come to mind:

Ephemeral environments
Once you've codified an infrastructure in Terraform, it's easy to stamp out additional environments for development, QA, staging or testing. Many organizations pay thousands of dollars every month for a dedicated staging environment. Because Terraform parallelizes operations, you can curate a copy of production infrastructure in just one trip to the water cooler. Terraform enables developers to deploy their changes into identical copies of production, letting them catch bugs early.

Rapid project stamping
The new Terraform google_project APIs enable quick project stamping. Organizations can easily create identical projects for training, field demos, new hires, coding interviews or disaster recovery. In larger organizations with rollup billing, IT teams can use Terraform to stamp out pre-configured environments tied to a single billing organization.
On-demand continuous integration
You can use Terraform to create continuous integration or build environments on demand that are always in a clean state. These environments only run when needed, reducing costs and improving parity by using the same configurations each time.

Whatever your use case, the combination of Terraform and GCP’s new resource management APIs represents a powerful new way to manage cloud-based environments. For more information, please visit the Terraform website or review the code on GitHub.