Author Archives: Open Source Programs Office

Introducing Nomulus: an open source top-level domain name registry

Today, Google is proud to announce the release of Nomulus, a new open source cloud-based registry platform that powers Google’s top level domains (TLDs). We’re excited to make this piece of Internet infrastructure available to everyone.

TLDs are the top level of the Internet Domain Name System (DNS), and they collectively host every domain name on the Internet.  To manage a TLD, you need a domain name registry -- a behind-the-scenes system that stores registration details and DNS information for all domain names under that TLD. It handles WHOIS queries and requests to buy, check, transfer, and renew domain names. When you purchase a domain name on a TLD using a domain name registrar, such as Google Domains, the registrar is actually conducting business with that TLD’s registry on your behalf. That’s why you can transfer a domain from one registrar to another and have it remain active and 100% yours the entire time.

The project that became Nomulus began in 2011 when the Internet Corporation for Assigned Names and Numbers (ICANN) announced the biggest ever expansion of Internet namespace, aimed at improving choice and spurring innovation for Internet users. Google applied to operate a number of new generic TLDs, and built Nomulus to help run them.

We designed Nomulus to be a brand-new registry platform that takes advantage of the scalability and easy operation of Google Cloud Platform. Nomulus runs on Google App Engine and is backed by Google Cloud Datastore, a highly scalable NoSQL database. Nomulus can manage any number of TLDs in a single shared instance and supports the full range of TLD functionality required by ICANN, including the Extensible Provisioning Protocol (EPP), WHOIS, reporting, and trademark protection. It is written in Java and is released under the Apache 2.0 license.

We hope that by providing access to our implementation of core registry functions and up-and-coming services like Registration Data Access Protocol (RDAP), we can demonstrate advanced features of Google Cloud Platform and encourage interoperability and open standards in the domain name industry for registry operators like Donuts. With approximately 200 TLDs, Donuts has made early contributions to the Nomulus code base and has spun up an instance which they'll be sharing soon.

For more information, view Nomulus on GitHub.

By Ben McIlwain, Software Engineer

Google Summer of Code 2016 wrap-up: NRNB

This post is part of our series of Google Summer of Code wrap-ups, guest posts from students, mentors and organization admins reflecting on Google Summer of Code 2016. Don't miss our first post and follow along for more wrap-up posts and announcements.

We were so excited to be a part of Google Summer of Code (GSoC) again after a year off, we pulled together over 50 project ideas and dozens of eager mentors to develop open source code for network biology research. Organized as the National Resource for Network Biology (NRNB), we selected 15 proposals that brought together well-matched students, mentors and project ideas.

All 15 students passed their midterm and final evaluations, resulting in a wide range of (mostly) production-ready code, covering algorithm, UI, importer and converter development for both web and desktop for Cytoscape, cytoscape.js, SBML, SBGN, cBioPortal, Cell Designer, GraphSpace and more.

We are proud of the technical accomplishments and productivity of our students, and we are also proud of the many important aspects of diversity our students represent in the GSoC program, including geographical, gender and academic. Here are some numbers and facts about our 15 students compared to overall GSoC 2016 student stats in parentheses:
  • 9 different countries, including 1 (of 2) from Croatia, 1 (of 3) from Armenia and 2 (of 12) from Turkey
  • 20% female (compared to 12% overall)
  • 67% Computer Science (compared to 78% overall), including PhD students in Biological Oceanography and Medical Biochemistry & Biotechnology, an MS student in Bioinformatics, and a pre-med undergraduate.



Here are some quotes and blogs from our students this year. If you are considering applying as student (or mentor) next year, here is some inspiration:

“I had the opportunity to learn and practice JavaScript with a very interesting project and having a mentor available was great for getting help when needed. The program seemed extremely well run and I would strongly recommend it to anyone interested.”

“Working in an NRNB [GSoC] training program helped to strengthen my resume and introduced me to the idea of combining a career in medicine with computer-based research.”

“I love the friendly atmosphere and the way the team works together. From the very beginning I [felt] well integrated in the group. It was pure fun to work together on the same project and to see how it [has] grown over the time. I [would] recommend everybody try the NRNB training program.”

Some of our student blogs:
  • Hovakim Grabski – "Java support for Deviser, a code generation system for SBML libraries"
  • Kaito Ii – "Interconvertible Layout software for CellDesigner" 
  • Roman Schulte – "Offline SBML validation in the Java-based JSBML library"
  • Mridul Seth – "Import graphs in multiple formats and Cytoscape files into GraphSpace"

By Alex Pico and Kristina Hanspers, NRNB organization admins

Google Open Source Report Card

Open source software enables Google to build things quickly and efficiently without reinventing the wheel, allowing us to focus on solving new problems. We stand on the shoulders of giants and we know it. This is why we support open source and make it easy for Googlers to release the projects they’re working on internally as open source.

Today we’re sharing our first Open Source Report Card, highlighting our most popular projects, sharing a few statistics and detailing some of the projects we’ve released in 2016.

We’ve open sourced over 20 million lines of code to date and you can find a listing of some of our best known project releases on our website. Here are some of our most popular projects:
  • Android - a software stack for mobile devices that includes an operating system, middleware and key applications.
  • Chromium - a project encompassing Chromium, the software behind Google Chrome, and Chromium OS, the software behind Google Chrome OS devices.
  • Angular - a web application framework for JavaScript and Dart focused on developer productivity, speed and testability.
  • TensorFlow - a library for numerical computation using data flow graphics with support for scalable machine learning across platforms from data centers to embedded devices.
  • Go - a statically typed and compiled programming language that is expressive, concise, clean and efficient.
  • Kubernetes - a system for automating deployment, operations and scaling of containerized applications.
  • Polymer - a lightweight library built on top of Web Components APIs for building encapsulated re-usable elements in web applications.
  • Protobuf - an extensible, language-neutral and platform-neutral mechanism for serializing structured data.
  • Guava - a set of Java core libraries that includes new collection types (such as multimap and multiset), immutable collections, a graph library, functional types, an in-memory cache, and APIs/utilities for concurrency, I/O, hashing, primitives, reflection, string processing and much more.
  • Yeoman - a robust and opinionated set of scaffolding tools including libraries and a workflow that can help developers quickly build beautiful and compelling web applications.
While it’s difficult to measure the full scope of open source at Google, we can use the subset of projects that are on GitHub to gather some interesting data. Today our GitHub footprint includes over 84 organizations and 3,499 repositories, 773 of which were created this year.

Googlers use countless languages from Assembly to XSLT, but what are their favorites? GitHub flags the most heavily used language in a repository and we can use that to find out. A survey of GitHub repositories shows us these are some of the languages Googlers use most often:
  • JavaScript
  • Java
  • C/C++
  • Go
  • Python
  • TypeScript
  • Dart
  • PHP
  • Objective-C
  • C#
Many things can be gleaned using the open source GitHub dataset on BigQuery, like usage of tabs versus spaces and the most popular Go packages. What about how many times Googlers have committed to open source projects on GitHub? We can search for Google.com email addresses to get a baseline number of Googler commits. Here’s our query:


SELECT count(*) as n
FROM [bigquery-public-data:github_repos.commits]
WHERE committer.date > '2016-01-01 00:00'
AND REGEXP_EXTRACT(author.email, r'.*@(.*)') = 'google.com'


With this, we learn that Googlers have made 142,527 commits to open source projects on GitHub since the start of the year. This dataset goes back to 2011 and we can tweak this query to find out that Googlers have made 719,012 commits since then. Again, this is just a baseline number as it doesn’t count commits made with other email addresses.

Looking back at the projects we’ve open-sourced in 2016 there’s a lot to be excited about. We have released open source software, hardware and datasets. Let’s take a look at some of this year’s releases.

Seesaw
Seesaw is a Linux Virtual Server (LVS) based load balancing platform developed in Go by our Site Reliability Engineers. Seesaw, like many projects, was built to scratch our own itch.

From our blog post announcing its release: “We needed the ability to handle traffic for unicast and anycast VIPs, perform load balancing with NAT and DSR (also known as DR), and perform adequate health checks against the backends. Above all we wanted a platform that allowed for ease of management, including automated deployment of configuration changes.”

Vendor Security Assessment Questionnaire (VSAQ)
We assess the security of hundreds of vendors every year and have developed a process to automate much of the initial information gathering with VSAQ. Many vendors found our questionnaires intuitive and flexible, so we decided to shared them. The VSAQ Framework includes four extensible questionnaire templates covering web applications, privacy programs, infrastructure as well as physical and data center security. You can learn more about it in our announcement blog post.

OpenThread
OpenThread, released by Nest, is a complete implementation of the Thread protocol for connected devices in the home. This is especially important because of the fragmentation we’re seeing in this space. Development of OpenThread is supported by ARM, Microsoft, Qualcomm, Texas Instruments and other major vendors.

Magenta
Can we use machine learning to create compelling art and music? That’s the question that animates Magenta, a project from the Google Brain team based on TensorFlow. The aim is to advance the state of the art in machine intelligence for music and art generation and build a collaborative community of artists, coders and machine learning researchers. Read the release announcement for more information.

Omnitone
Virtual reality (VR) isn’t nearly as immersive without spatial audio and much of VR development is taking place on proprietary platforms. Omnitone is an open library built by members of the Chrome Team that brings spatial audio to the browser. Omnitone builds on standard Web Audio APIs to deliver an immersive experience and can be used alongside projects like WebVR. Find out more in our blog post announcing the project’s release.

Science Journal
Today’s smartphones are packed with sensors that can tell us interesting things about the world around us. We launched Science Journal to help educators, students and citizen scientists tap into those sensors. You can learn more about the project in our announcement blog post.

Cartographer
Cartographer is a library for real-time simultaneous localization and mapping (SLAM) in 2D and 3D with Robot Operating System (ROS) support. Combining data from a variety of sensors, this library computes positioning and maps surroundings. This is a key element of self-driving cars, UAVs and robotics as well as efforts to map the insides of famous buildings. More information on Cartographer can be found in our blog post announcing its release.

This is just a small sampling of what we’ve released this year. Follow the Google Open Source Blog to stay apprised of Google’s open source software, hardware and data releases.

By Josh Simmons, Open Source Programs Office

Google Summer of Code 2016 wrap-up: HUES Platform

Every year Google Summer of Code pairs university students with mentors to hone their skills while working on open source projects, and every year we like to post wrap-ups from the open source projects about their experience and what students accomplished. Stay tuned for more!



The Holistic Urban Energy Simulation (HUES) platform is an open source platform for facilitating the design and control of renewables-based distributed energy systems. The platform is an initiative of the Urban Energy Systems Laboratory at Empa in Switzerland, in collaboration with our research partners at ETH-Zurich, EPFL, the University of Geneva and the Lucerne University of Applied Sciences. As we push towards the second version of the HUES platform, we had help from three bright and enthusiastic students as part of the Google Summer of Code (GSoC).

Project 1: Real-time wind flow in cities
Air flow pattern around a building configuration (left); link to Rhinoceros/Grasshopper (middle & right)
People in cities are suffering more and more from scorching heat, caused by global warming and bad urban planning. This traps heat inside cities and has led to soaring air conditioning demand, making cities even hotter - a vicious circle!  Clever bioclimatic urban design can mitigate urban heat by facilitating the use of natural ventilation and guiding air streams. However, the simulation of wind flow is a computationally and technically demanding task. There is a need to provide urban planners and architects with a tool able to predict wind flow patterns in real-time to facilitate development of energy efficient and passive designs.

Lukas Bystricky, a student at Florida State University, developed a Fast Fluid Dynamics (FFD) library in C# exactly for this purpose. Lukas’s implementation is based on the  paper by Jos Stam (1999). In contrast to the original implementation, where a cell centred finite difference is used to discretize the equations, Lukas applies a staggered grid finite difference, which is the standard finite difference in Computational Fluid Dynamics (CFD). This is done to prevent spurious pressure oscillations near the boundary which can occur in cell centered finite difference for the Navier-Stokes equations. This does not change much in the algorithm or solvers, but makes enforcing the boundary conditions significantly more complicated. So far, Lukas uses a simple Jacobi solver as linear solver, as was the case in Stam's original implementation, but he plans to replace it with more efficient solvers in the future. Also, he is validating his library with typical benchmarks. 

We are now coupling Lukas’s library into our HUES platform, more specifically into the 3D CAD software Rhinoceros and its visual programming platform Grasshopper. The final goal is to have an intuitive real-time visual design tool of wind flow for urban planners and architects. Also, we will use it to couple it to whole year dynamic building energy simulation programs, to better capture microclimatic effects of the urban context in simulating building energy consumption of designs.

Project 2: Modular energy hub modeling framework
A connection between two bus objects in a CopyHub container
Distributed energy system components are modular in nature and interact across multiple scales. As such, there is a need for a modeling framework that can easily construct and configure systems of modular entities (energy demands, sources, converters, storages and network links) across scales. Frederik Banis, a student at the University of Applied Sciences Stuttgart, developed a modular approach to modeling distributed multi-energy systems (energy hubs) in Python, based on the Open Energy System Modelling Framework (Oemof) and Pyomo. 

In the developed framework, energy systems components are specified in a common format allowing for easy duplication and reconfiguring at larger scales. The platform enables easy manipulation of an energy hub grouping multiple components (demand, sources: electricity, natural gas; systems: photovoltaic panels, wind turbines, gas boils, combined heat and power engines, etc.), as well as copying it (from hub1 to hub2) to create a larger interlinked system (district) where multiple energy hubs are connected. This hierarchical nested structure can be repeated as needed, and detailed results about the energy supply of each technology or energy stream can be analyzed in the form of different plots for each system or sub-system.

Project 3: Open source energy simulation database

The HUES platform includes a growing array of datasets describing the technical and economic characteristics of distributed energy technologies.  Currently, this data is stored in separate modules using different data structures and file formats, making it difficult to explore holistically and query systematically. To address this, GSoC student Khushboo Mandlecha has developed an open source database to enable the linked exploration, querying and visualization of data in the platform. 

The first part of the project involved the development of server based scripts to automatically extract relevant data from the modules of the existing HUES platform, and write this data to a common database. A standard format for technology component data was developed, enabling users to upload technology data files to be stored in the new database.  The new database has been developed in MongoDB, enabling fast data retrieval and allowing everything to be retrieved in the form of JSON objects. The second part of the project involved the development of a web-based portal for querying, visualizing and downloading data. Once this portal is complete, it will be possible to visualize the contents of the database in different ways, enabling users to get a sense of the distribution of property values and facilitating the identification of outliers.  Ultimately, the database will help researchers and practitioners using the HUES platform to develop models and perform comprehensive analyses of distributed energy systems.

By L. Andrew Bollinger, Julien Marquant and Christoph Waibel; Urban Energy Systems Laboratory, Empa, Switzerland

Announcing Google Code-in 2016 and Google Summer of Code 2017

One of the goals of the Open Source Programs Office is to encourage more people to contribute to open source software. One way we achieve that goal is through our student programs, Google Summer of Code (for university students) and Google Code-in (for pre-university students).

Over 15,000 students from more than 100 countries have worked with 23,000 mentors and contributed to 560+ open source projects.

This is why we’re excited to announce the next round of both of our student programs!

Google Code-in begins for students November 28, 2016


For the seventh consecutive year, Google Code-in will give students (ages 13-17) a chance to explore open source. Students will find opportunities to learn and get hands on experience with tasks from a range of categories. This structure allows students to stretch themselves as they take on increasingly more challenging tasks.

Getting started is easy: once the contest begins, simply choose an interesting task from our participating organizations’ lists and complete it. Mentors from the organizations are available to help online.

Google Code-in is for students asking questions like:
  • What is open source?
  • What kinds of stuff do open source projects do?
  • How can I write real code when all I’ve done is a little classroom work?
  • Can I contribute even if I’m not really a programmer?
With tasks in five different categories, there’s something to fit almost any student’s skills:
  1. Code: writing or refactoring 
  2. Documentation/Training: creating/editing documents and helping others learn more
  3. Outreach/research: community management, outreach/marketing, or studying problems and recommending solutions
  4. Quality Assurance: testing and ensuring code is of high quality
  5. User Interface: user experience research or user interface design and interaction
Google Summer of Code student applications open on March 20, 2017


Google Summer of Code (GSoC) provides university students from around the world with an opportunity to take their skills and hone them by contributing to open source projects during their summer break from university.

Students gain invaluable experience working with mentors on these open source software projects, earning a stipend upon successful completion of their project.

We’re proud to keep this tradition going: we’ll be opening student applications for Google Summer of Code 2017 on March 20, 2017. Applications for interested open source organizations open on January 19, 2017.

Students, it’s never too early to start preparing or thinking about your proposal. You can learn about the organizations that participated in Google Summer of Code 2016 and the projects students worked on. We also encourage you to explore other resources like the student and mentor manuals and frequently asked questions.

You can learn more on the program website.

Share the news with your friends and stay tuned, more details are coming soon!

By Josh Simmons, Open Source Programs Office

An open source font system for everyone

Originally posted on the Google Developers Blog

A big challenge in sharing digital information around the world is “tofu”—the blank boxes that appear when a computer or website isn’t able to display text: ⯐. Tofu can create confusion, a breakdown in communication, and a poor user experience.

Five years ago we set out to address this problem via the Noto—aka “No more tofu”—font project. Today, Google’s open source Noto font family provides a beautiful and consistent digital type for every symbol in the Unicode standard, covering more than 800 languages and 110,000 characters.

A few samples of the 110,000+ characters covered by Noto fonts.
The Noto project started as a necessity for Google’s Android and ChromeOS operating systems. When we began, we did not realize the enormity of the challenge. It required design and technical testing in hundreds of languages, and expertise from specialists in specific scripts. In Arabic, for example, each character has four glyphs (i.e., shapes a character can take) that change depending on the text that comes after it. In Indic languages, glyphs may be reordered or even split into two depending on the surrounding text.

The key to achieving this milestone has been partnering with experts in the field of type and font design, including Monotype, Adobe, and an amazing network of volunteer reviewers. Beyond “no more tofu” in the common languages used every day, Noto will be used to preserve the history and culture of rare languages through digitization. As new characters are introduced into the Unicode standard, Google will add these into the Noto font family.

Google has a deep commitment to openness and the accessibility and innovation that come with it. The full Noto font family, design source files, and the font building pipeline are available for free at the links below. In the spirit of sharing and communication across borders and cultures, please use and enjoy! 
By Xiangye Xiao and Bob Jung, Internationalization

Introducing Cartographer

We are happy to announce the open source release of Cartographer, a real-time simultaneous localization and mapping (SLAM) library in 2D and 3D with ROS support.

SLAM algorithms combine data from various sensors (e.g. LIDAR, IMU and cameras) to simultaneously compute the position of the sensor and a map of the sensor’s surroundings. For example, consider this approach to drawing a floor plan of your living room:
  • Grab a laser rangefinder, stand in the middle of the room, and draw an X on a piece of paper.
  • Measure the distance from where you’re standing to any wall.
  • Draw a line on the paper where the wall is and write down the distance between the X (your position) and the wall.
  • Measure the distance from where you’re standing to another wall and add it to the drawing as well.
  • Now, move to another part of the room.
  • Since the walls (hopefully) haven’t moved, you can measure your distance to the same two walls to determine your new position.


SLAM is an essential component of autonomous platforms such as self driving cars, automated forklifts in warehouses, robotic vacuum cleaners, and UAVs.

Cartographer builds globally consistent maps in real-time across a broad range of sensor configurations common in academia and industry. The following video is a demonstration of Cartographer’s real-time loop closure:


A detailed description of Cartographer’s 2D algorithms can be found in our ICRA 2016 paper.

Thanks to ROS integration and support from external contributors, Cartographer is ready to use on several robot platforms with ROS support:
At Google, Cartographer has enabled a range of applications from mapping museums and transit hubs to enabling new visualizations of famous buildings.

We recognize the value of high quality datasets to the research community. That’s why, thanks to cooperation with the Deutsches Museum (the largest tech museum in the world), we are also releasing three years of LIDAR and IMU data collected using our 2D and 3D mapping backpack platforms during the development and testing of Cartographer.


Our focus is on advancing and democratizing SLAM as a technology. Currently, Cartographer is heavily focused on LIDAR SLAM. Through continued development and community contributions, we hope to add both support for more sensors and platforms as well as new features, such as lifelong mapping and localizing in a pre-existing map.

By Damon Kohler, Wolfgang Hess, and Holger Rapp, Google Engineering

Introducing the Open Images Dataset

Originally posted on the Google Research Blog

In the last few years, advances in machine learning have enabled Computer Vision to progress rapidly, allowing for systems that can automatically caption images to apps that can create natural language replies in response to shared photos. Much of this progress can be attributed to publicly available image datasets, such as ImageNet and COCO for supervised learning, and YFCC100M for unsupervised learning.

Today, we introduce Open Images, a dataset consisting of ~9 million URLs to images that have been annotated with labels spanning over 6000 categories. We tried to make the dataset as practical as possible: the labels cover more real-life entities than the 1000 ImageNet classes, there are enough images to train a deep neural network from scratch and the images are listed as having a Creative Commons Attribution license*.

The image-level annotations have been populated automatically with a vision model similar to Google Cloud Vision API. For the validation set, we had human raters verify these automated labels to find and remove false positives. On average, each image has about 8 labels assigned. Here are some examples:
Annotated images form the Open Images dataset. Left: Ghost Arches by Kevin Krejci. Right: Some Silverware by J B. Both images used under CC BY 2.0 license
We have trained an Inception v3 model based on Open Images annotations alone, and the model is good enough to be used for fine-tuning applications as well as for other things, like DeepDream or artistic style transfer which require a well developed hierarchy of filters. We hope to improve the quality of the annotations in Open Images the coming months, and therefore the quality of models which can be trained.

The dataset is a product of a collaboration between Google, CMU and Cornell universities, and there are a number of research papers built on top of the Open Images dataset in the works. It is our hope that datasets like Open Images and the recently released YouTube-8M will be useful tools for the machine learning community.

By Ivan Krasin and Tom Duerig, Software Engineers

* While we tried to identify images that are licensed under a Creative Commons Attribution license, we make no representations or warranties regarding the license status of each image and you should verify the license for each image yourself.

Another year of Haskell Hacking in the Google Zurich Office

For the fourth time, the Google Open Source Programs Office have co-sponsored a three-day hackathon for Haskell, an open source functional programming language. Gleb Peregud from Google’s Zurich office talks more about the event below.

On the weekend of July 22nd, 126 Haskell enthusiasts got together for another installment of ZuriHac, a yearly open source Haskell hackathon held in Zurich, Switzerland, and like the last two years it took place at Google Zurich.

Participants could either hack uninterrupted in the main room or listen to a number of presentations in the tech talk room. Each day was kicked off with a keynote — Bas van Dijk talked about the use of functional programming at LumiGuide (slides), Edward Kmett about monad homomorphisms, and Andres Löh about generic-sop, a new approach to generic programming. All three talks drew a full room of interested listeners.

We also prepared two codelabs for Haskell beginners, and rallied 28 dedicated volunteers to serve as mentors (thank you all!) so that there was always someone to ask for help.

Aside from keynotes, there were five other talks: an experience report on parallelizing and distributing scientific computations (slides), an overview of a language to program FPGAs called CλaSH, an interactive tour through low-level pieces of the GHC Haskell compiler, an introduction to web programming using Spock (slides) and a talk on revamping the build system of GHC (slides).

Spontaneous mini-lectures attract smaller crowds.
As is traditional, after a full Saturday of hacking, we went out to barbecue down by the Zurich lake.

We were lucky that weather forecast was mistaken about a looming thunderstorm!
We managed to beat last year's record, and welcomed 126 attendees. We hope to have even more participants next year - our goal is to host 150 hackers in 2017. See you in a year!

By Gleb Peregud, Site Reliability Engineer

.NET and PowerShell tooling for the Google Cloud Platform

Last month Google made an announcement unveiling support for Visual Studio, C#, PowerShell, Microsoft SQL Server and more on the Google Cloud Platform. With so many  new features, it is easy to gloss over some of the technical aspects of the announcement, especially the fact that all of the developer tooling and libraries are open source and available on GitHub.

This post will go into some of the details behind the new C# libraries, PowerShell cmdlets, and Visual Studio extension. All three products are open source, have an exciting roadmap for the future and are hungry for your feedback.

C# bindings for Google APIs

Source: https://github.com/googlecloudplatform/google-cloud-dotnet
Docs: https://cloud.google.com/dotnet/

For years, Google has had innovative technologies powering its data centers, unfortunately Google’s internal APIs and technology couldn’t directly benefit you and your software. That was, until the Google Cloud Platform started exposing public APIs for things like machine learning, storage, logging etc. With these APIs publicly available, you can add powerful capabilities to your apps without needing to manage complex infrastructure.

There have been C# bindings for Google APIs for years. In fact, Google receives hundreds of millions of API calls from C# clients every day. But newer APIs, especially those from the Google Cloud Platform, require more advanced features like bidirectional streaming. That’s why rather than using HTTP/REST many newer Google APIs are built on top of gRPC, a high performance, open source universal RPC framework.

But don’t worry, we have C# bindings for those gRPC-based APIs too; all of it open source and on GitHub.

In both cases, the client library is the result of a C# code generator. We take the API’s discovery document (analogous to a WSDL) and generate C# code. gRPC APIs require more careful design than other APIs, but the end product is the same. Once built, the API libraries are published to NuGet.

C# code generators for Google APIs isn’t the entire story.

Source code generated from tools can look foreign at times. So for libraries where the codegen isn’t good enough, we have hand-written wrappers to provide a better, more idiomatic experience. In some cases -- such as CRUD operations using the Datastore API -- the hand-written library cuts down on the required lines of code by half.

Finally, support for C# doesn’t just mean code. We are also working to ensure Google APIs are supported on different runtimes too. Most Google APIs work on the cross-platform .NET Core runtime and we are continuing to expand support.

PowerShell support

Source: https://github.com/googlecloudplatform/google-cloud-powershell
Docs: http://googlecloudplatform.github.io/google-cloud-powershell/

C# support is great when you are writing full applications, but for DevOps, scripting is more typical. The Cloud SDK provides command-line tools (gcloud, gsutil) for managing cloud resources, but when running on Windows, Windows PowerShell is a dramatically more productive environment. Google Cloud tools for PowerShell is a set of cmdlets so you can manage your Google cloud resources. They are strongly typed, and integrate seamlessly with other PowerShell tools. For example, to learn more about a cmdlet, just use Get-Help.

In designing the PowerShell cmdlets, the main goal was to be idiomatic. We wanted to follow the best practices and guidelines so PowerShell novices and pros alike could use our cmdlets. Of course, if we have anything wrong, please log an issue on the GitHub repository. Pull requests are also welcome.

Visual Studio

Source: https://github.com/googlecloudplatform/google-cloud-visualstudio
Docs: https://cloud.google.com/visual-studio/

The C# and PowerShell features should help developers using Google services. But the biggest impact on developer productivity comes from being inside the Visual Studio IDE.

From within Visual Studio you can search for new extensions and find the Google Cloud Platform Extension for Visual Studio. It provides tools for viewing/managing data stored in Google Cloud Storage and Google Cloud SQL. It also provides support for deploying ASP.NET 4.x applications to Google Compute Engine.

It is only the first release and we have some big plans for the future. You can see a lot of the short-term features we have planned by looking at the issues list in GitHub. Like making Google APIs light up for the new .NET Core runtime, being able to deploy ASP.NET Core applications to Google App Engine or Google Container Engine will be huge. Stay tuned for a future blog post about how to run C# on Google App Engine Flexible Environment, as well.

We’re just getting started

Hopefully you share my enthusiasm for Google’s ongoing development in .NET tooling. Not only is it exciting to be able to take advantage of Google Cloud Platform technologies, but also to see a future where .NET Core enables C# code to run cross-platform.

But to be successful we need your help.

If you have questions, be sure to ask on Stack Overflow (e.g. the google-cloud-visualstudio or google-cloud-powershell tags). If you have problems, please open issues on GitHub (libraries, VS, PowerShell). If you still have trouble, participate in the google-cloud-dev group.

The team here at Google is thrilled to be working with the .NET stack and your feedback is immensely helpful in prioritizing things.

By Chris Smith, Software Engineer