Category Archives: Open Source Blog

News about Google’s open source projects and programs

Welcoming 1,000+ Interns to Open Source at Google

One of the core tenets of open source is about finding ways for people to build great things by working together, regardless of location. This summer, through our intern program we’re gathering incredible talent from schools around the world, Googlers with a passion for open source, and project maintainers both inside and outside of Google to see what we can build together. 

Onboarding that many interns and turning them into new open source contributors was no easy task. So in partnership with the Intern Programs team and engineering teams across Google, we’ve grounded our planning by answering four key questions. 

How can we make our internship program a force for good in the open source ecosystem?

We knew that having more than a thousand interns contribute to open source projects could have a huge impact, however, many projects aren’t set up to onboard dozens of new contributors at one time and many maintainers can’t take on hundreds of new pull requests. Early on, we established best practices for intern placement and support. We committed to:
  • Aligning interns’ work with project priorities to advance the project while also allowing the interns to learn and grow their skills.
  • Proactively communicating with project maintainers and contributors, keeping them in the loop on timelines and logistics.
  • Looking beyond Google. While we prioritized projects that have full-time Google engineerings support. That includes Google-owned projects like Go, TensorFlow, and Chromium, as well as Google-created projects we invest heavily in, such as Kubernetes, Apache Beam, and Tekton. But Google also has full-time engineers working on outside projects we rely on, so our interns will also be working on projects like Envoy, Rust, and Apache Maven.

How can we introduce the interns to open source at Google?

We are determined to support and empower the interns as they become lifelong contributors to open source. Every Noogler in engineering learns about using and contributing to open source in a training run by our Open Source Programs Office. With an unprecedented number of interns working on open source projects, we are also providing additional resources; from offering a platform for questions, office hours, enrichment talks, and partnerships with external open source organizations.

How can we learn from our interns about the experience of contributing to open source at Google and beyond?

We see a huge opportunity to listen to our interns this summer. By meeting with interns and hosts—as well as surveying the entire class of interns at the end of the summer—we can look for ways to improve open source at Google and the contributor experience for projects they’re working on. We’re excited to learn from the internship program and from interns’ perspectives working in and contributing to open source.

How can we have an impact on these students that carries on throughout their careers?

One of my favorite questions to ask Googlers who are active in open source is how they were first introduced to open source. There’s a well-trodden path of a developer fixing an annoying bug, then a few more bugs, then adding small features, becoming a core contributor, and eventually a project maintainer. That process requires persistence and patience, and projects lose a lot of great developers along the way.

But... What if your first experience with open source is being welcomed into a large and thriving community of contributors? What if you get to contribute to open source full time, mentored by creators and maintainers of the project you’re working on, collaborating across organizations and across time zones? Our hope is that this kind of experience will leave a lasting impression on this summer’s interns and that they’ll continue to contribute to open source for a long time to come.

By Jen Phillips, Google Open Source

COVID-19: How Google is helping the open source community

COVID-19 has affected so much of the world around us, and open source is no exception. Project resilience is being challenged by COVID-19. Community members have even less time to contribute. Event cancellations are impacting networking, collaboration, and fundraising.


Google wants to do everything it can to help. This means that it’s even more important for the Google Open Source Programs Office to step up our commitment to citizenship. We’re taking several steps to support industry organizations and the projects that we participate in to help them operate during this time.

Virtual Events Support

  • Participating in talks internally and externally to Google to share knowledge and insight into open source projects and practices with the wider open source communities.
  • To support the shift from an offline to online events model, we created an online guide to share resources and event planning knowledge: Open Source Virtual Events Guide.

Talent

  • COVIDActNow is a multidisciplinary team working to provide disease intelligence and data analysis on COVID in the U.S. Google contributed to this project by improving their data pipeline allowing for county level data visualization, providing more localized insight for crisis planning.
  • Nextstrain is a platform for real-time tracking of pathogen evolution. Google contributed engineering, design, and translation resources to help scientists conduct research into real-time tracking of pathogen evolution.
  • Schema.org - Google led Schema.org rapid response designs for structured data markup to contribute to the COVID-19 global response, leading to the UK making similar announcements.
  • Google’s annual internship program was converted to a digital program where interns will focus on open source projects, allowing projects to gain new contributors in a non-traditional environment.
  • Google Summer of Code brings over 1100 university students from around the world together with open source communities, many of which are working on various humanitarian efforts related to COVID-19. The program is completely online so students can work with their mentors remotely, allowing all organizations to continue receiving the support they need.
The impact from COVID-19 will have long-term effects on many organizations and projects that may not be immediately apparent. In the coming months, we will monitor the needs of projects and organizations across open source. We understand the value of open source not just to the tech world, but the impact it has on bringing communities together; Google has a long standing history in open source and we will continue supporting our community to stay strong during and after the passing of COVID-19.

We encourage folks who have the time and ability to support open source communities to do so by getting involved and reaching out directly to organizations that interest you. This is a time for all of us to come together and lift up each other and open source.

By Megan Byrd-Sanicki and Radha Jhatakia, Google Open Source

Tsunami: An extensible network scanning engine for detecting high severity vulnerabilities with high confidence

We have released the Tsunami security scanning engine to the open source communities. We hope that the engine can help other organizations protect their users’ data. We also hope to foster collaboration, and encourage the security community to create and share new detectors on top of Tsunami.

When an attacker begins to exploit security vulnerabilities or security misconfigurations, such as weak passwords, an organization needs to react quickly in order to protect potentially vulnerable assets. With attackers increasingly investing in automation, the time window to react to a newly released, high severity vulnerability is usually measured in hours. This poses a significant challenge for large organizations with thousands or even millions of internet-connected systems. In such hyperscale environments, security vulnerabilities must be detected and, ideally, remediated in a fully automated fashion. To make this possible, information security teams need to be able to roll out detectors for novel security issues at scale in a very short amount of time. Furthermore, it is important that the detection quality is consistently very high. To handle these challenges, we created Tsunami: an extensible network scanning engine for detecting high severity vulnerabilities with high confidence.

Google leverages Google's Kubernetes Engine (GKE) to continuously scan and protect all of our externally facing systems with the Tsunami scanning engine. When scanning a system, Tsunami executes a two-step process:
  1. Reconnaissance: In the first step, Tsunami detects open ports; then subsequently identifies protocols, services, and other software running on the target host using a set of fingerprinting plugins. To avoid reinventing the wheel, Tsunami leverages existing tools such as nmap for some of these tasks.
  2. Vulnerability verification: Based on the information gathered through reconnaissance, Tsunami selects all vulnerability verification plugins matching the identified services. To confirm that a vulnerability indeed exists Tsunami executes a fully working, benign exploit.
In its initial version, Tsunami ships with detectors for the following security issues:
  • Exposed sensitive UIs: Applications such as Jenkins, Jupyter, and Hadoop Yarn ship with UIs that allow a user to schedule workloads or to execute system commands. If these systems are exposed to the internet without authentication, attackers can leverage the functionality of the application to execute malicious commands.
  • Weak credentials: Tsunami uses other open source tools such as ncrack to detect weak passwords used by protocols and tools including SSH, FTP, RDP, and MySQL.
In the coming months, we plan to release many more detectors for vulnerabilities similar to remote code execution (RCE). Furthermore, we are working on several other new features that will make the engine more powerful and easier to use and extend.

In order to make contributions easy, we split our codebase into two Github Repositories:
  1. A repository for the main scanning engine
  2. A repository for Tsunami scanning plugins
If you have any questions or if you would like to contribute, don't hesitate to reach out to us.

By Guoli Ma, Claudio Criscione & Sebastian Lekies, Vulnerability Management Team

Three opportunities to connect with Google Open Source in June

One of our biggest challenges this year has been finding opportunities to stay connected with the many open source communities that we collaborate with across projects. As we continue to develop new ways of creating convenings with our different stakeholders, here are three opportunities to connect with Google Open Source later this month.

24 hours of Google Cloud Talks by DevRel

When: June 23, 2020
What: This is a free, digital series, organized by Google Developer Relations team, offering practitioners an opportunity to connect with our technical experts and deepen their awareness and knowledge of a variety of Google Cloud solutions including ML/AI, Serverless, DevOps, and many more.

Talks by Google Open Source:

June 23

OpenJS World

When: June 23-24, 2020
What: Organized by The Linux Foundation, and sponsored by Google, this annual event brings together the JavaScript and web ecosystem including Node.js, Electron, AMP and more. In 2020, we’re going virtual to learn and engage with leaders deploying innovative applications at massive scale.

Talks by Google Open Source:

June 23
June 24

Open Source Summit North America

When: June 29 – July 2, 2020
What: Organized by The Linux Foundation, and sponsored by Google, this event connects the open source ecosystem under one roof, summoning over 2,000 participants across 15 conference rooms. It’s a unique environment for cross-collaboration between developers, sysadmins, devops, architects, program and product managers and others who are driving technology forward.

Talks by Google Open Source:

June 29
June 30
If you attend any of these talks, and plan to share, you can tag @GoogleOSS on Twitter. We hope to see and connect with many of you at these virtual events!

By María Cruz, Google Open Source

Google Summer of Code 2020 Statistics: Part 1

Since 2005, Google Summer of Code (GSoC) has been bringing new developers into the open source community every year. This year, we accepted 1,199 from 66 countries into the 2020 GSoC program to work with 199 open source organizations over the summer. Students began coding June 1st and will spend the next 12 weeks working closely under the guidance from mentors from their open source communities.

Each year we like to share program statistics about the GSoC program and the accepted students and mentors involved in the program. 6,626 students from 121 countries submitted 8,903 applications for this year’s program.

Accepted Students

  • 86.6% are participating in their first GSoC
  • 71.7% are first time applicants to GSoC

Degrees

  • 77.4% are undergraduates, 16.8% are masters students, and 5.8% are in PhD programs
  • 72.5% are Computer Science majors, 3.6% are Mathematics majors, 23.9% are other majors including many from engineering fields like Electrical, Mechanical, Aerospace, etc.
  • Students are studying in a variety of fields including Atmospheric Science, Finance, Neuroscience, Economics, Biophysics, Linguistics, Geology, Pharmacy and Real estate.

Proposals

There were a record number of students submitting proposals for the program this year:
  • 6,626 students (18.2% increase from last year)
  • 121 countries
  • 8,902 proposals submitted

Registrations

We had a record breaking 51,244 students from 178 countries(!) register for the program this year—that’s a 65% increase in registrations from last year’s record numbers!

In our next GSoC statistics post, we will do a deeper dive into the schools and mentors for the 2020 program.

By Stephanie Taylor, Google Open Source

Season of Docs Now Accepting Technical Writer Applications

The technical writer applications for Season of Docs are now open.

Technical writers can submit project proposals based on the project ideas of participating organizations, or propose their own ideas. Refer to the guidelines on the website for how to create a technical writer application. The technical writer application form is located here.

The deadline for technical writer applications is July 9, 2020 at 18:00 UTC.

What is Season of Docs?

Documentation is essential to the adoption of open source projects as well as to the success of their communities. Season of Docs brings together technical writers and open source projects to foster collaboration and improve documentation in the open source space. You can find out more about the program on the introduction page of the website.

During the program, technical writers spend a few months working closely with an open source community. They bring their technical writing expertise to the project's documentation and, at the same time, learn about the open source project and new technologies.

Mentors from open source projects work with the technical writers to improve the project's documentation and processes. Together, they may choose to build a new documentation set, redesign the existing docs, or improve and document the project's contribution procedures and onboarding experience.

How do I take part in Season of Docs as a technical writer?

First, take a look at the technical writer guide on the website, which includes information on eligibility and the application process.

Explore the list of participating organizations and their project ideas. When you find one or more projects that interest you, you should approach the relevant open source organization directly to discuss project ideas.

Then, read the information on creating a technical writing application and submit it via this form. The deadline for technical writer applications is July 9, 2020 at 18:00 UTC.

Is there a stipend for participating technical writers?

Yes. There is an optional stipend available to the accepted technical writers. The stipend amount is calculated based on the technical writer's home location. See the technical writer stipends page for more information.

What kind of mentor will I be working with?

Season of Docs mentors are not necessarily technical writers, and they may have little experience in technical communication. They're members of an open source organization who know the value of good documentation and who are experienced in open source processes and tools.

The relationship between you and your mentors is a collaboration. You bring documentation experience and skills to the open source organization. Your mentors contribute their knowledge of open source and code. Together, you can develop technical documentation and improve the open source project's processes.

What if I have a full time job and don't have many hours per week to devote to Season of Docs?

In the technical writer application, there is an option to apply for a long-running project, which allows technical writers to complete their project in five months instead of the standard three months. This must be agreed upon with the open source organization before work begins.

If you have any questions about the program, please email us at [email protected].

General timeline

June 9 – July 9Technical writers submit their proposals to Season of Docs
August 16Google announces the accepted technical writer projects
August 17 – September 13Community bonding: Technical writers get to know mentors and the open source community, and refine their projects in collaboration with their mentors
September 14 – December 5Technical writers work with open source mentors on the accepted projects, and submit their work at the end of the period
January 6, 2021Google publishes the list of successfully-completed projects
See the full timeline for details, including the provision for projects that run longer than three months.

Join us

Explore the Season of Docs website at g.co/seasonofdocs to learn more about participating in the program. Use our logo and other promotional resources to spread the word. Examine the timeline, check out the FAQ, and apply now!

By Kassandra Dhillon and Erin McKean, Google Open Source

Celebrating 10 years of WebM and WebRTC

Originally posted on the Chromium Blog

Ten years ago, Google planted the seeds for two foundational web media technologies, hoping they would provide the roots for a more vibrant internet. Two acquisitions, On2 Technologies and Global IP Solutions, led to a pair of open source projects: the WebM Project, a family of cutting edge video compression technologies (codecs) offered by Google royalty-free, and the WebRTC Project building APIs for real-time voice and video communication on the web.

These initiatives were major technical endeavors, essential infrastructure for enabling the promise of HTML5 with support for video conferencing and streaming. But this was also a philosophical evolution for media as Product Manager Mike Jazayeri noted in his blog post hailing the launch of the WebM Project:
“A key factor in the web’s success is that its core technologies such as HTML, HTTP, TCP/IP, etc. are open and freely implementable.”
As emerging first-class participants in the web experience, media and communication components also had to be free and open.

A decade later, these principles have ensured compression and communication technologies capable of keeping pace with a web ecosystem characterized by exponential growth of media consumption, devices, and demand. Starting from VP8 in 2010, the WebM Project has delivered up to 50% video bitrate savings with VP9 in 2013 and an additional 30% with AV1 in 2018—with adoption by YouTube, Facebook, Netflix, Twitch, and more. Equally importantly, the WebM team co-founded the Alliance for Open Media which has brought the IP of over 40 major tech companies in support of open and free codecs. With Chrome, Edge, Firefox and Safari supporting WebRTC, more than 85% of all installed browsers globally have become a client for real-time communications on the Internet. WebRTC has become a stable standard and it is now the default solution for video calling on the Web. These technologies have succeeded together, as today over 90% of encoded WebRTC video in Chrome uses VP8 or VP9.

The need for these technologies has been highlighted by COVID-19, as people across the globe have found new ways to work, educate, and connect with loved ones via video chat. The compression of open codecs has been essential to keeping services running on limited bandwidth, with over a billion hours of VP9 and AV1 content viewed every day. WebRTC has allowed for an ecosystem of interoperable communications apps to flourish: since the beginning of March 2020, we have seen in Chrome a 13X increase in received video streams via WebRTC.

These successes would not have been possible without all the supporters that make an open source community. Thank you to all the code contributors, testers, bug filers, and corporate partners who helped make this ecosystem a reality. A decade in, Google remains as committed as ever to open media on the web. We look forward to continuing that work with all of you in the next decade and beyond.

By Matt Frost, Product Director Chrome Media and Niklas Blum, Senior Product Manager WebRTC

Google Cardboard XR Plugin for Unity

Late in 2019, we decided to open source Google Cardboard. Since then, our developer community has had access to create a plethora of experiences on both iOS and Android platforms, while reaching millions of users around the world. While this release has been considered a success by our developer community, we also promised that we would release a plugin for Unity. Our users have long preferred developing Cardboard experiences in Unity, so we made it a priority to develop a Unity SDK. Today, we have fulfilled that promise, and the Google Cardboard open source plugin for Unity is now available via the Unity Asset Store. 

What's Included in the Cardboard Unity SDK

Today, we’re releasing the Cardboard Unity SDK to our users so that they can continue creating smartphone XR experiences using Unity. Unity is one of the most popular 3D and XR development platforms in the world, and our release of this SDK will give our content creators a smoother workflow with Unity when developing for Cardboard.

In addition to the Unity SDK, we are also providing a sample application for iOS/Android, which will be a great aid for developers trying to debug their own creations. This release not only fulfills a promise we made to our Cardboard community, but also shows our support, as we move away from smartphone VR and leave it in the more-than-capable hands of our development community.



If you’re interested in learning how to develop with the Cardboard open source project, please see our developer documentation or visit the Google VR GitHub repo to access source code, build the project, and download the latest release.

By Jonathan Goodlow, Product Manager, AR & VR

Knative elects new Technical Oversight Committee members

Towards the end of 2019, Knative project initiated a series of changes to its governance to ensure sustainability in the long term. Over the last week, the project reached a new milestone by successfully wrapping up its first Technical Oversight Committee (TOC) elections, bringing more vendor diversity to the technical stewardship of Knative.

Google has grown thousands of open source projects throughout the years, and it is this collective knowledge that informed the changes proposed to Knative governance. Over the last six months, we worked together with the other members of the Knative Steering Committee, and with the project’s contributors to create a clear set of rules for technical leadership and governance, describing the many ways in which contributors could engage with the project. This process was key to developing trust with Knative’s community, the project’s most valued stakeholder. In the exercise of this vote, the community was able to test the new election process, which proved to be solid: it will be repeated annually for this project, and can serve as a model for other projects as well.

The TOC election, which had a turnout of 70% of active contributors to the project, yielded a new technical stewardship for Knative, with members representing RedHat, VMWare and Google, as follows:

Nghia Tran (Google) - new member
Markus Thömmes (Red Hat) - new member
Grant Rodgers (Google) - new member
Matt Moore (VMWare) - existing member
Evan Anderson (VMWare) - existing member

Members of Knative TOC not only have the technical stewardship of the project in their hands for the next two years, they also model the community’s values: they have strong technical skills, they contribute to the project, and they are collegial, mentoring other contributors and helping the project to grow in a sustainable and healthy way.

We celebrated this important milestone for Knative at the last community meetup. Watch the video to meet the new TOC members, and check out the contribution guidelines to join the project.

By María Cruz, Google Open Source

SoD: experiences and tips for participation

Have you ever heard about a program that lets technical writers contribute to open source documentation? Towards this effort, Google launched its pilot program for Season of Docs (SoD) around April 2019. SoD immediately caught my attention because it is the first-ever program that seamlessly brings tech writing to the open source world. By carefully selecting open source organizations and their projects, SoD allows seasoned or aspiring technical writers around the world to work closely with an open source organization, and understand their open source product, processes, and tools.

Luckily, I discovered SoD 2019, submitted my application—and after being selected to work for Tor—started my journey into the world of open source. I successfully completed my project in March 2020, and my project report was called out on the Results page for SoD 2019 that lists all successful contributions.

Now in its second year, SoD has gained momentum and is the talk of most technical writing communities and student circles. For the benefit of SoD aspirants, I am recounting my participation experiences along with useful tips in this blog post. I have included answers to questions that I am frequently asked, along with some simple tips to help you successfully complete your project.
  • Firstly, all aspiring participants for SoD 2020 must go through the program and its timeline hosted on the Season of Docs website. Make sure you read the Technical writer guide to understand the different phases of the program.
  • It’s a common myth that only seasoned technical writers who have coding knowledge are at fit for this program as SoD 2019 saw a lot of students participating in the program. Experienced technical writers or even students, who can show their interest in tech documentation are free to participate. From the highly to moderately technical projects available, you can opt for a project depending on your expertise and knowledge.
  • During the application phase, try to shortlist organizations closest to your domain knowledge and work area, and submit project proposals for two or three organizations. This increases your chance of being accepted by an organization. I shortlisted Tor and Hydra because of my interest in anonymous communications and my experience of working with APIs.
  • Before submitting your proposal, it’s a good idea to establish interaction with the org mentors to understand if you’re going in the right direction with your project proposal and most importantly to break the ice for future communications. I found some interesting project ideas and reached out to the mentors to understand their expectations of the project before submitting my application. In hindsight, this proved to be extremely valuable, as I was able to fine-tune my proposal in collaboration with my mentor, and I had already built a rapport with them.
  • Build your project proposal based on whether you’re opting for a three- or a five-month program. Break down details by months and weeks, which will testify your commitment to the project. I had chosen a standard three-month long project and detailed my work by months. When scoping your project, always allow time for roadblocks and unforeseen situations.
  • After your proposal is selected by an organization, it’s community-bonding time. Take this phase seriously and use this time to get to know your peers in the organization, build a rapport with your mentors, set up a communication channel with them, and prepare your work environment. My project proposal for Tor was accepted, and my mentors reached out to me to quickly get me up-to-speed with their communication tools, meeting timings, and smoothly inducted me into the networking team that I was going to work with. Even before I started working on the project, I was attending their weekly meetings and learning more about the work they do.
  • During the doc development phase, try to accomplish everything you promised in your proposal. At the same time, don’t feel bogged down by any changes that arise due to the complexity of your proposal. I faced hiccups during this phase because some of my ideas were not possible to implement and I had to rescope my proposal. Thanks to the invaluable support from my mentors and my peers at Tor, I was able to overcome all the obstacles and move forward with my project. The key to overcoming hurdles during your project is to keep your mentors updated about your work with frequent communication.
  • Once the work’s done complete your project report which will serve as the final assessment. Ensure that this report clearly shows all the work you’ve done; nothing is too big or too small to highlight in this report. A well-written report is more important than your project proposal as this decides whether your project has been successful. Based on this report, your mentor gives you a pass or fail mark for your project.
  • If your project is successful, you receive a stipend at the end of the program if you opted for one. Choosing to opt out of the stipend does not increase your chances of being selected to the program. This depends solely on your project proposal and your efforts at bonding with the chosen communities before applying.
For any more questions or concerns that you may have at any point in the program, I’d suggest looking up the FAQ for technical writers. You can also give a shout out to the extremely helpful program admins at [email protected].

I hope I have inspired SoD 2020 applicants to make their participation successful. I wish each one the very best.

By Swati Thacker, guest writer from Oracle