Tag Archives: code

Code Search for Google open source projects

We are pleased to launch Code Search for Google open source projects. Code Search is one of Google’s most popular internal tools, and now we have a version (same binary, different flags) targeted to open source communities.

Googlers use Code Search every day to help understand the codebase: they search for half-remembered functions and usages; jump through the codebase to figure out what calls the function they are viewing; and try to identify when and why a particular line of code changed.

The Code Search tool gives a rich code browsing experience. For example, the blame button shows which user last changed each line and you can display history on the same page as the file contents. In addition, it supports a powerful search language and, for some repositories, cross-references.

Suggest-as-you-type in any search box annotates suggestions with the type of code object, the repository and the path, helping users find what they want faster.


The search language supports regular expressions and a number of helpful search atoms. For a user looking for a function foo in a Go file, instead of sifting through thousands of results containing foo, the user can search for lang:go function:foo to limit search results to Go files where foo is a function and not a struct or a word in a comment.

One example is finding a file using only part of the name. The query file:KytheURI.java goes directly to the file, since there is only one such file.

See the quick reference for more information.

In addition to text search, some of the open source repositories have cross-references powered by Kythe. Kythe is a Google open source project that includes tools to help understand code. Project owners instrument a build of their repository to output compilation information for Kythe. Kythe tools convert this data to a graph. This graph connects definitions to declarations and code references to the abstract objects they represent (described by a graph schema). Google then runs an internal pipeline that combines these graphs for the different languages, prunes unnecessary pieces, and optimizes it for serving cross-references. The whole process runs several times per day to keep the data fresh.

Open source communities use a broader set of build systems than Google. In order to support cross-references, Kythe added drop-in support for Bazel, CMake, Maven, and Go. Projects using other build systems can use Kythe-provided wrappers for clang and javac to instrument their builds; these are used by Chromium and Android AOSP to provide compilation information for Kythe.

Because Kythe is based on the build, Kythe cross-references include links to files generated as part of the build process, such as Java files generated for AutoValues (example here) or protos. For repositories where cross-references are enabled, clicking on a symbol will take you to a definition of that symbol.

Clicking on the definition of a symbol will open a cross-reference panel, showing all the places where that symbol is referenced. For example, clicking on toVName below, we can see the places that reference this method. One of the callers is parseVName, and clicking on that shows the callers of that method.



At this time, we only provide search on the repositories listed below, but we plan to add more over time:
  • Angular
  • Bazel (with cross-references)
  • Dart
  • ExoPlayer
  • Firebase SDK
  • Flutter
  • Go (with cross-references)
  • gVisor (with cross-references)
  • Kythe (with cross-references)
  • Nomulus (with cross-references)
  • Outline
  • Tensorflow (with cross-references)
We are also investing in making the application keyboard-navigable and usable with a screen reader.

We hope you find this tool useful!

By Kris Hildrum, Code Search Team

Unleashing Open Source Silicon

Open Source Silicon

We all know that open source software has changed the fundamental nature of the software industry and that Google generously adds fuel to this culture of openness and community through Google Summer of Code. What few people realize is that there is another major industry that is ripe for an open source overhaul—the silicon industry. And, this summer, a Google Summer of Code student helped open the floodgates.

If you search social media for “open source silicon,” you’ll find a few dozen names that pop up with some frequency. These folks are fanatically driving forward with open source circuit models and software for creating them. You’ll also find people clambering to jump aboard the RISC-V bandwagon. RISC-V, like x86, MIPS, and others before it, is a CPU “instruction set architecture,” and the mere fact that it is free of proprietary licenses has inspired countless open source implementations and an industry shake-up that has ARM quaking in its boots.

While this open source silicon community is a hotbed of enthusiasm, it is several decades behind the world of open source software. In this post, I’ll reveal the three reasons this movement has, thus far, not been able to take off like open source software, and I’ll explain why these three obstacles are all coming to a very sudden and dramatic end, that will unleash a tidal wave, catching the silicon industry by surprise. And you’ll see that Google Summer of Code, this year, played a pivotal role.

What’s Standing in the Way

So, why is coding and sharing circuit models any different from sharing software? Three reasons:
  1. Implementation Details: There’s more to worry about with hardware than software. Correct functionality is far from the only concern. Particular care must be given to physical implementation. And this detail must be modified for specific silicon technology and design constraints. As a result, leveraging open source logic can involve a substantial amount of rework.
  2. Access to software: While compilers for software tend to be open source, electronic design automation (EDA) tools for compiling hardware are traditionally proprietary and prohibitively expensive.
  3. Access to hardware: Unlike software, circuit models must be turned into silicon to be useful. Fabricating custom silicon is out of the question for a hobbyist, but field-programmable gate arrays (FPGAs) provide a more realistic option. These are chips that can be quickly reconfigured, or “programmed,” to implement any logic function. While FPGAs are within reach, they still cost money, and they are delivered by postal service, not a web browser. And, worst of all, it could take weeks to get an FPGA platform up and running and communicating with the open source logic.

Breaking Down the Barriers

Let’s look at what the open source community is doing to help:
  1. Implementation Details: There is a trend toward designing more abstractly, and leaving the details to tools. Open source tools can now compile C++ into silicon (with caveats). And several open source hardware description languages leverage modern software language innovations that make it easier to rework implementation details. The open source community has shown a greater willingness than industry to explore and adopt these languages. Though hardware remains fundamentally different from software, their differences are becoming less prominent.
  2. Access to software: Open source EDA software has marked some significant achievements in the past several years. Circuit designs have been implemented on FPGAs using 100% free and open source EDA tools. (Google Summer of Code has helped to fund a few open source EDA capabilities in projects under the Free and Open Source Silicon Foundation.) The US government has recognized the opportunity and is providing significant fuel to the fire through the Posh Open Source Hardware initiative. Being restricted to open source software can still be a bit limiting, but it is no longer prohibitive.
  3. Access to hardware: Hmmm. This is still a problem.
My personal contributions to this open source silicon movement stem from my startup, Redwood EDA. We directly target problem #1 by providing tools that support advanced (yet simpler) circuit modeling techniques. And, to address #2, we make all of our software freely available online for open source development. But neither open source EDA nor the efforts of my startup had been able to noticeably impact problem #3, access to hardware.

This is where bigger forces have stepped in. In the past few years, cloud providers have begun incorporating FPGAs into their datacenters. These are available to anyone with an internet connection and a credit card, bundled with industry-class EDA software, on a pay-per-use basis. Wow! This is the solution to hardware access! An open source developer can provide not only their hardware model but also the platform for which their model was designed. A user can download and go, just like they can with software! …in theory.

So here’s the rub. The learning curve for cloud FPGA platforms has been way too high for the open source community to latch on.

Our Project

With a bit of help from Politecnico di Milano’s NECST Lab and ThroughPuter Inc., I was able to get a project off the ground, and it attracted some attention for this year’s Google Summer of Code. I was happy to see an application from Ákos Hadnagy who had done some other ground-breaking work with me in the last Summer of Code, and he was accepted into the program. Together, this summer, we built infrastructure, automated flows, and wrote documentation (or more to the point, eliminated documentation), and now, instead of a month to ramp up, it is now possible to develop for this platform in a matter of minutes!
We dubbed our framework “1st CLaaS,” where we have coined the term “CLaaS” for custom logic as a service. Very simply, 1st CLaaS wraps a developer’s custom FPGA logic as a microservice. Standard web protocols can be used to stream bits to and from this logic, and platform details are hidden by the framework.

Implications and Wrap-up

So there is no longer anything standing in the way! Hobbyists can build and share hardware, and open source silicon can thrive. Just imagine the disruption this will have on the industry, which is currently driven by corporate giants. And with easy web integration, the opportunity and demand for hardware acceleration should rise, and we could start to see some interesting new capabilities on the web that were not imaginable until now.

Google certainly didn’t have this specific industry transformation in mind when starting Google Summer of Code, but I suspect the whole point of the program was to inspire and enable the unexpected. And it did!

If you’d like to contribute to 1st CLaaS or collaborate on some of the world’s first FPGA-accelerated web applications, we’d be more than happy to have you involved. I look forward to next year's applications.

By Steve Hoover, Redwood EDA, Google Summer of Code mentor

Google Code-in 2019 is Right Around the Corner!

This year is the 10th anniversary of the Google Code-in (GCI) contest! Students ages 13–17, globally, can learn about open source development by working on real projects, with mentorship from active developers. GCI begins on December 2, 2019 and runs for seven weeks, ending January 23, 2020.

Google Code-in is unique because students have the autonomy to select what they’re interested in working on from 2,500+ tasks created by open source organizations, all while having mentors available to answer questions as they work on tasks.

There are many questions that developers of any age ask themselves when they initially get involved in open source; from where to start to whether they have the expertise to truly support the organization. The beauty of GCI lies in the participating open source organizations who realize teens are often first time contributors, leading mentors who volunteer to come prepared with the patience and experience to help these newcomers join the open source community.
New contributors bring fresh perspectives, ideas, and enthusiasm into their open source communities, helping them thrive. Throughout the last 9 years, 58 GCI organizations helped 11,000 students from 108 countries make real contributions to open source projects; and to this day may of those students continue to participate in various open source communities and many have become mentors themselves! Some have even gone on to join Google Summer of Code (GSoC).

Contest participants work on a varied level of tasks that require anywhere from beginner to advanced skills in the following five categories:
  • Code: writing or refactoring
  • Documentation/Training: creating/editing documents and helping others learn more
  • Outreach/Research: community management, marketing, or studying problems and recommending solutions
  • Quality Assurance: testing and ensuring code is of high quality
  • Design: graphic design or user interface design
Organizations that are interested in mentoring students, can apply for Google Code-in beginning Thursday, October 10th. Google Code-in starts for students Monday, December 2nd!
Visit the contest site g.co/gci to learn more about the contest and find flyers, slide decks, timelines, and more.

By Radha Jhatakia, Google Open Source

Change is Made with Code

Cross-posted from the Official Google Blog

What would the world look like if only 20 percent of women knew how to write? How many fewer great books would there be? How many important stories would go unreported? How many innovations would we lose? How many brilliant women would be unable to fulfill their potential?

That’s not just a theoretical question. Today, only a small minority of women know how to write code. That limits their ability to participate in a growing part of our global economy. It limits their ability to affect change as entire industries are transformed by technology. And it limits their potential to impact millions of lives through the power of code.

To change this trajectory, we need to do all we can to inspire women and girls that learning to code is critical to creating a brighter future for everyone. That’s why I’m excited to share that, today, Google’s Made with Code, together with YouTube, is teaming up with the Global Citizen Festival and millions of teen girls to ignite a movement for young women to change the world through the power of code.

Over the last five years, millions of Global Citizens have influenced world leaders and decision makers, and contributed to shaping our world for the better. As we’ve seen this movement grow, we’ve learned about some incredible women who saw problems in their communities and realized that the biggest impact they could have was through computer science. They’ve used an interest in computer science and tech to help the homeless, stop sexual assault, and bridge the gender gap in technology - check out their stories here:

 

These women are doing big things, blazing a path for the next generation of girls, but they can’t do it alone. The vast potential around using code to improve the world cannot be realized if there are only a few voices influencing how it’s shaped. That’s why, today, we’re inviting teen girls everywhere to join the movement. Our new coding project gives young women a chance to make their voice heard by coding a statement about the change they want to see in the world.

This week, hundreds of thousands of girls from around the country have already used code to share their vision for a better, more inclusive, more equitable world:


These coded designs will be displayed onstage at the Global Citizen Festival, as symbols of the many different voices from teen girls, standing up for the change they want to see in the world.

Together with musicians, sisters, YouTube sensations and newly minted coders, Chloe x Halle, teen girls are getting their start in code

Our efforts go well beyond this project. Made with Code is joining forces with Iridescent and UN Women to support the launch of the Technovation Challenge 2017 which gives girls the opportunity to build their own apps that tackle the real-life issues they see around them.

Please tune into the Global Citizen Festival livestream at youtube.com/globalcitizen on September 24 to catch all the action. And, more importantly, join us and encourage the young women in your life to try out coding and contribute their ideas for how to make a better future.

Posted by Susan Wojcicki, CEO, YouTube

Source: YouTube Blog