Tag Archives: Open source

Showing Our Work: A Study In Understanding Open Source Contributors

In 2022, the research team within Google’s Open Source Programs Office launched an in-depth study to better understand open source developers, contributors, and maintainers. Since Alphabet is a large consumer of and contributor to open source, our primary goals were to investigate the evolving needs and motivations of open source contributors, and to learn how we can best support the communities we depend on. We also wanted to share our findings with the community in order to further research efforts and our collective understanding of open source work.

Key findings from this work suggest that community leaders should:

  • Value your time together and apart: Lack of time was cited as the leading reason ‘not to contribute’ as well as motivation to ‘leave a community’. This should encourage community leaders to adopt practices that ensure that they are making the most of the time they have together. One example: some projects have planned breaks, no-meeting weeks, or official slowdowns during holidays or popular conference weeks.
  • Invest in documentation: Contributors and maintainers expressed that task variety, delegation, and onboarding new maintainers could help to reduce burnout in open source. Documentation is one way to make individual knowledge accessible to the community. In addition to technical and procedural overviews, documentation can also be used to clarify roles, tasks, expectations, and a path to leadership.
  • Always communicate with care: Contributors prefer projects that have welcoming communities, clear onboarding paths, and a code of conduct. Communication is the primary way for community leaders to promote welcoming and inclusive communities and set norms around language and behavior (as documented in a Code of Conduct). Communication is also how we build relationships, trust, and respect for each other.

  • Create spaces for anonymous feedback: Variable answers between demographic subsets in our research suggest that while systematic approaches can be taken to reduce burnout, there is no one-size-fits-all approach. Feedback is a valuable tool for any project to adjust to the evolving needs of their contributor and user communities. When designed appropriately, surveys can serve as safe, anonymous, retaliation-free spaces for individuals to provide honest feedback.

How do contributors select projects?

We asked respondents to share their most important criteria when selecting an open source project to contribute to in their personal time. The top responses were: welcoming community, clear onboarding path, and code of conduct.
Base: 517 international OSS developers, contributors, maintainers and students who worked on open source in their personal time

Within Google’s Open Source Programs office, we are constantly looking for ways to improve support for contributors inside and outside of Google. Studies such as this one provide guidance to our programs and investments in the community. This work helps us to see we should continue to:

  • Invest in documentation competency: Google Season of Docs provides support for open source projects to improve their documentation and gives professional technical writers an opportunity to gain experience in open source.
  • Document roles and promote tactics that recognize work within communities: The ACROSS project continues to work with projects and communities to establish consistent language to define roles, responsibilities, and work done within open source projects.
  • Exercise and discuss ‘better’ practices within the community: While we continually seek to improve our engagement practices within communities, we will also continue to share these experiences with the broader community in hopes that we can all learn from our successes and challenges. For example, we’ve published documentation around our release process, including resources for the creation and management of a code of conduct.

This research, along with other articles authored by the OSPO research team is now available on our site.

By Sophia Vargas – Researcher, Google Open Source Programs Office

GSoC 2023: project results and feedback part 1

In 2023, Google Summer of Code brought 966 new contributors into open source software development to work with open source organizations on a 12+ week project. We had 168 participating open source organizations with mentors and contributors from over 75 countries this year.

For 19 years, Google Summer of Code has thrived due to the enthusiasm of our open source communities and the 19k+ volunteer mentors that spend from 50-150 hours mentoring each of our 20k contributors since 2005! This year, there are 168 mentoring organizations and over 1,950 mentors participating in the 2023 program. A sincere thank you to our mentors and organization administrators for guiding and supporting our contributors this year. We are also looking forward to hosting many of the 2023 GSoC Mentors on campus this fall for the annual Mentor Summit.

September 4th concluded the standard 12-week project timeline and we are pleased to announce that 628 contributors have successfully completed this year’s program as of today, September 5th, 2023. Congratulations to all the contributors and mentors that have wrapped up their summer coding projects!

2023 has shown us that GSoC continues to grow in popularity with students and developers 19 years after the program began. GSoC had a record high 5,679 contributor applicants from 106 countries submit their project proposals this year. We also had huge interest in the program with over 43,765 registrants from 160 countries applying to the program during the two week application period.

The final step of every GSoC program is to hear back from mentors and contributors on their experiences through evaluations. This helps GSoC Admins continuously improve the program and gives us a chance to see the impact the program has on so many individuals! Some notable results and comments from the standard 12-week project length evaluations are below:

  • 95.63% of contributors think that GSoC helped their programming skills
  • 99.06% of contributors would recommend their GSoC mentors
  • 97.81% of contributors will continue working with their GSoC organization
  • 99.84% of contributors plan to continue working on open source
  • 82.81% of contributors said they would consider being a mentor
  • 96.25% of contributors said they would apply to GSoC again

Here’s some of what our GSoC 2023 Contributors had to say about the program!

At the suggestion of last year’s contributors, we added multiple live talks throughout the coding period to bring contributors together, providing tips to help them make the most of their GSoC experience. Each of these talks were attended on average by 42% of the 2023 GSoC contributors.

Another request from our previous contributors was to hear more about the cool projects their colleagues did over the summer and the opportunity to talk about their own projects with others. Over the coming weeks we are hosting three lightning talk sessions where over 40 of the 2023 contributors will have the opportunity to present their project learnings to the other contributors and their mentors.

We’ll be back in a couple of months to give a final update on the GSoC projects that will conclude later this year. Almost 30% of contributors (286 contributors) are still completing their projects, so please stay tuned for their results in part two of this blog post later this year!

By Perry Burnham – Google Open Source

ChromeOS EC testing suite in Renode for consumer products

Besides main application cores that are directly exposed to the users, many industrial and consumer devices include embedded controllers, which, although fairly invisible to the user, perform critical system tasks such as power management, receiving and processing user input, or signals from sensors like thermal. Given their role in the system, those MCUs need to be rigorously tested in CI. This is why the ChromeOS team has collaborated with Antmicro to simulate the ChromeOS FPMCU (Fingerprint Firmware) module, based on the ChromeOS EC (Embedded Controller) firmware in Antmicro’s Renode open source simulation framework.

This enables automated testing of embedded controllers in CI at scale, in a deterministic manner, and with complete observability. It also streamlines the developer feedback loop for faster development of microcontroller firmware that ChromeOS uses to drive peripherals, such as fingerprint readers or touchpads. To make this possible, we have improved the simulation capabilities for two of the microcontrollers used in FPMCU modules, popular in consumer electronics like Chromebooks and wearables, but also in many industrial applications: STM32F412 and STM32H743.

Testing consumer-grade products with Renode
Testing consumer-grade products with Renode

Continuous testing for embedded systems

The project required implementing continuous testing of the FPMCU module against tens of binaries that test the controller in the most common operations and scenarios to ensure maximum reliability at all times. A traditional approach would require reflashing the physical microcontroller memory with each binary, which is time-consuming and error-prone. To scratch that itch we developed the CrOS EC Tester, which runs all EC tests in a Renode simulation and uses GitHub Actions to handle building and executing test binaries for a truly automated workflow—useful both in CI and in an interactive development environment.

Renode has broad support for architectures such as (but not limited to) RISC-V, ARM Cortex-M, (recently added) Cortex-R or Cortex-A, and runs binary-compatible software. Thus, it is not limited to testing embedded controllers but entire multi-CPU systems. You can easily add Renode to an existing workflow without any major changes for testing in real-life scenarios. By moving all testing efforts into an interactive and deterministic environment of Renode, you can implement a fully CI-driven testing approach in your projects and benefit from advanced debugging, tracing, and prototyping capabilities.

Comprehensive simulation of STM32 microcontrollers

The Renode models of the STM32F412 and STM32H743 microcontrollers give you access to a broad range of peripherals, allowing you to run various scenarios you’d typically test on hardware. As a result of our collaboration with Google, we have added or improved models of ST peripherals like UART, EXTI, GPIO, DMA, ADC, SPI, flash controllers, timers, watchdogs, and more.

The need for in-depth testing has led to the introduction of many enhancements to ARM Cortex-M support in general, such as the MPU (Memory Protection Unit), which allows you to protect certain memory areas from unauthorized modification or access or FPU interrupts. These features can now be used by other Cortex-M-based projects to further extend their test coverage with Renode.

Renode for rapid, interactive prototyping

One of the tests from our test suite used the microcontroller's MPU module to test address space security. When you run the test-rollback test case, you can see that the MPU simulated in Renode successfully protected the OS from unauthorized memory access:

Testing consumer-grade products with Renode
Testing consumer-grade products with Renode

Another Renode feature that allowed us to increase our test coverage of the EC ecosystem is support for dummy SPI and I2C devices. While Renode supports a recently added advanced framework for time-controlled feeding of sensor data, many scenarios require much simpler interaction with the external device. For this purpose, we developed a dummy SPI device that simply returns pre-programmed data to the controller, which allowed us to pass initialization tests for a sensor without modeling the sensor itself. From the functional point of view of the simulation, the dummy sensor data is identical to the real data, which is useful when the specific component is difficult to model or lacks documentation.

Build a CI-driven test workflow with Renode

Renode is a powerful tool for automating and simplifying the test workflow in the project at any stage of development, even pre-silicon. It helps you reduce the tedium typically associated with embedded software testing by providing a fully controllable environment that can lead to fewer bugs and vulnerabilities, which is naturally important for mass-market products such as Chromebooks.

By Michael Gielda – Antmicro

A vision for more efficient media management

Petit Press’ new open source, cloud-based DAM platform helps publishers get rich media content in front of their audience at pace and scale.

Picture the scene: You’re an investigative journalist that has just wrapped up a new piece of video content that offers incisive, timely commentary on a pressing issue of the day. Your editor wants to get the content in front of your audience as quickly as possible and you soon find yourself bogged down in a laborious, manual process of archiving and uploading files. A process that is subject to human error, and involves repeating the same tasks as you prepare the content for YouTube and embedding within an article.

With the development of a new open source digital asset management (DAM) system, Slovak publishing house, Petit Press, is hoping to help the wider publishing ecosystem overcome these types of challenges.

Striving towards a universal, open source solution

Like many publishers in today’s fast-paced, fast-changing news landscape, Petit Press was feeling the pressure to be more efficient and do more with less, while at the same time maximizing the amount of high-quality, rich media content its journalists could deliver. “We wanted to find a solution to two main asset delivery issues in particular,” says Ondrej Podstupka, deputy editor in chief of SME.sk. “Firstly, to reduce the volume of work involved in transferring files from our journalists to our admin teams to the various platforms and CMS we use. Secondly, to avoid the risk of misplacing archived files or losing them entirely in an archive built on legacy technologies.”

As a publisher of over 35 print and digital titles, including one of Slovakia’s most-visited news portal, SME.sk, Petit Press also had a first-hand understanding of how useful the solution might be if it could flex to the different publishing scales, schedules, and platforms found across the news industry. With encouragement and support from GNI, Petit Press challenged themselves to build an entirely open source, API-based DAM system that flexes beyond their own use cases and can be easily integrated with any CMS, which means that other publishers can adapt and add functionality with minimal development costs.

Getting out of the comfort zone to overcome complexity

For the publisher, creating an open source project requires collaboration, skill development, and a strong sense of purpose. GNI inspired our team members to work together in a positive, creative, and supportive environment. Crucial resources from GNI also enabled the team to broaden the scope of the project beyond Petit Press’ direct requirements to cover the edge use cases and automations that a truly open source piece of software requires.

“GNI has enabled our organization to make our code open source, helping to create a more collaborative and innovative environment in the media industry.” 
– Ondrej Podstupka, deputy editor in chief of SME.sk

Building and developing the tool was difficult at times with a team of software engineers, product managers, newsroom managers, UX designers, testers, and cloud engineers all coming together to see the project to completion. For a team not used to working on GitHub, the open source aspect of the project proved the primary challenge. The team, however, also worked to overcome everything from understanding the complexities of integrating a podcast feature, to creating an interface all users felt comfortable with, to ensuring compliance with YouTube’s security requirements.

Unburdening the newsroom and minimizing costs

The hard work paid off though, when the system initially launched in early 2023. Serving as a unified distribution platform, asset delivery service and long term archive, the single solution is already unburdening the newsroom. It also benefits the tech/admin teams, by addressing concerns about the long-term costs of various media storage services.

On Petit Press’ own platforms, the DAM system has already been successfully integrated into SME.sk’s user-generated content (UGC) blog. This integration allows for seamless content management and curation, enhancing the overall user experience. The system also makes regulatory compliance easier, thanks to its GDPR-compliant user deletion process.

In addition to the UGC Blog system, the DAM system has now launched for internal Petit Press users—specifically for managing video and podcast content, which has led to increased efficiency and organization within the team. By streamlining the video and podcast creation and distribution processes, Petit Press has already seen a 5-10% productivity boost. The new DAM system saves an estimated 15-20 minutes of admin time off every piece of video/podcast content Petit Press produces.

Working towards bigger-picture benefits

Zooming out, the DAM system is also playing a central part in Petit Press’ year-long, org-wide migration to the cloud. This transformation was set in motion to enhance infrastructure, streamline processes, and improve overall efficiency within the department.

Podstupka also illustrates how the system might benefit other publishers. “It could be used as an effective standalone, automated archive for videos and podcasts,” he says. For larger publishing houses, “if you use [the DAM system] to distribute videos to YouTube and archive podcasts, there is minimal traffic cost and very low storage cost. But you still have full control over the content in case you decide to switch to a new distribution platform or video hosting service.”

As the team at Petit Press continues to get to grips with the new system, there is a clear goal in mind: To have virtually zero administrative overhead related to audio or video.

Beyond the automation-powered efficiency savings, the team at Petit Press are also exploring the new monetisation opportunities that the DAM system presents. They are currently working on a way to automatically redistribute audio and image assets to their video hosting platform, to automatically create video from every podcast they produce. This video is then pushed to their CMS and optimized for monetisation on the site with very little additional development required.

Ultimately, though, the open source nature of the system makes the whole team excited to see where other publishers and developers might take the product. “It’s a futureproof way to leverage media content with new services, platforms and ideas that emerge in technology or media landscapes,” says Igor, Head Of Development & Infrastructure. A succinct, but undeniably compelling way of summing up the system’s wide-ranging potential.

A guest post by the Petit Press team

Kubeflow joins the CNCF family

We are thrilled to announce a major milestone in the journey of the Kubeflow project. After a comprehensive review process and several months of meticulous preparation, Kubeflow has been accepted by the Cloud Native Computing Foundation (CNCF) as an incubating project. This momentous step marks a new chapter in our collaborative and open approach to accelerating machine learning (ML) in the cloud native ecosystem.

The acceptance of Kubeflow into the incubation stage by the CNCF reflects not just the project's maturity, but also its widespread adoption and expanding user base. It underscores the tremendous value of the diverse suite of components that Kubeflow provides, including Notebooks, Pipelines, Training Operators, Katib, Central Dashboard, Manifests, and many more. These tools have been instrumental in creating a cohesive, end-to-end ML platform that streamlines the development and deployment of ML workflows.

Furthermore, the alignment of Kubeflow with the CNCF acknowledges the project's foundational reliance on several existing CNCF projects such as Argo, Cert-Manager, and Istio. The joining of Kubeflow with the CNCF will serve to strengthen these existing relationships and foster greater collaboration among cloud native projects, leading to even more robust and innovative solutions for users.

Looking ahead, Google and the Kubeflow community are eager to collaborate with the CNCF on the transition process. Rest assured, our commitment to Kubeflow's ongoing development remains unwavering during this transition. We will continue to support new feature development, plan and execute upcoming releases, and strive to deliver further improvements to the Kubeflow project.

We extend our heartfelt thanks to the CNCF Technical Oversight Committee and the wider CNCF community for their support and recognition of the Kubeflow project. We look forward to this exciting new phase in our shared journey towards advancing machine learning in the cloud native landscape.

As Kubeflow continues to evolve, we invite developers, data scientists, ML engineers, and all other interested individuals to join us in shaping the future of cloud native machine learning. Let's innovate together, with Kubeflow and the CNCF, to make machine learning workflows more accessible, manageable, and scalable than ever before!

By James Liu – GCP Cloud AI

Google Dev Library Letters: 21st Edition

Posted by Swathi Dharshna Subbaraj, Google Dev Library

In this newsletter, we highlight the best projects developed with Google technologies that have been contributed to the Google Dev Library platform. We hope this will spark some inspiration for your next project!

Highlights of the Month

In the past two months, we asked contributors to look back, revisit, and update their older Dev Library contributions as a best practice. Most contributors took the time to revise their content and incorporate recent releases. This campaign encourages developers to update their repositories with the latest Google technologies, which is advantageous to users and the broader developer community.

Here are some of the standout up-to-date projects:

  • Sheets Compose Dialogs by Maximilian Keppeler

See how an Android library that offers dialogs and views for various use cases - built with Jetpack Compose for Compose projects. All dialogs and views are easy and quick to implement. 


  • Round Corner Progress Bar by Somkiat Khitwongwattana

Progress Bar Animation
Use this extensive “Rounded Corner progress bar” library for your own Android projects. 

During the campaign, we noticed that some new projects were submitted. Here are some of the new projects from our contributors:

  • Android TV sample projects by Ademir Queiroga

Android TV Project
See some of the Android TV sample projects on the main topics around Android TV development, and the project follows Google's best practices with a few experience-based insights.  

  • Storage provisioning with Cloud SQL using Workload Identity by Fermin Blanco

Learn how to create a production ready GKE cluster in a matter of seconds. 


Using Android’s new Credential Manager API by Priya Sindkar
Dive into this blog on how Android's new Credential Manager API provides a seamless way for your app’s users to log in with one-click solutions.  

KStore by Isuru Rajapakse
Learn how the tiny Kotlin multiplatform library that assists in saving and restoring objects to and from disk using kotlinx.coroutines, kotlinx.serialisation and okio.  

DevBricksX by Nan YE
Discover how DevBricksX is a remarkable remake and extended version of DevBricks, this project covers various aspects of daily development, from low-level database tasks to user interface design, as it eliminates the need for repetitive work.  

Dose app by Waseef Akhtar
Learn how Dose, a reminder app for people to take their medications on time, was built using Kotlin and Jetpack Compose with MVVM + clean architecture.  

Compose_adaptive_scaffold by Thomas Künneth
Explore how to write Jetpack Compose apps that support large screens and foldables.  


Troubleshooting reachability with a Network Intelligence Center connectivity test by Gaurav Madan
Learn how network troubleshooting processes become crucial when time is of the essence, and how to do so efficiently.  

From data chaos to data insights with Google Cloud and GitLab CI: A cutting-edge solution by Gursimar Singh
Take a look at a streamlined, effective approach to acquire important insights from data and learn how to deal with the turmoil of manual data deployment and analysis easily.  

Machine Learning

Client-side in-decent content checking
Discover a JavaScript library to help you quickly identify unseemly images; all in the client's browser.  

YoloV7 in Tensorflow.js by Hugo Zanini
Learn object detection using Yolov7 in tensorflow.js, and how it’s trained on the MS COCO dataset to recognizes up to 80 different classes  


Exploring Inherited Widget: The powerful state management solution by Muhammad Salman
Take a deep dive into the backstory of state management in Flutter and explore one of the most important concepts in Flutter state management, the Inherited Widget.  

Control your Flutter app on the fly with Firebase Remote Config by Mangirdas Kazlauskas
Flutter Forward agenda app
Learn the overview of Firebase Remote Config and how to use it to enable real-time features in your Flutter application.  

The ultimate Flutter Navigator 2.0 series using AutoRoute by Cavin Macwan
Explore the differences between Navigator 1.0 and 2.0 and why you need Navigator 2.0. You’ll also learn how you can implement Navigator 2.0 using the Auto Route package in Flutter.  


Papanasi (UI library) by Quique Fdez Guerra
Learn to use this frontend UI library across frameworks.  

How to manage complex forms in Angular by Roland Tubongye Wabubindja
See how to save and modify data from a form containing several FormArray.  

Community Updates

🚀 Announcing Google Maps Platform added to Dev Library

Progress Bar AnimationGoogle Maps platform in Dev Library

Google Maps Platform has now been officially added to the Dev Library! With these resources, developers can create applications that enable them to visualize geospatial data and build projects ranging from hyperlocal logistics to location-driven app development, and have access to even more resources to take their projects to the next level.

Dev Library contributors will be better able to write and create innovative and useful applications that utilize Google’s mapping, places, and routing data and features.

Visit the Google Maps Platform product page in Dev Library

Browse Dev Library | Google Developers Online on Discord | Newsletter Archives

An open-source gymnasium for machine learning assisted computer architecture design

Computer Architecture research has a long history of developing simulators and tools to evaluate and shape the design of computer systems. For example, the SimpleScalar simulator was introduced in the late 1990s and allowed researchers to explore various microarchitectural ideas. Computer architecture simulators and tools, such as gem5, DRAMSys, and many more have played a significant role in advancing computer architecture research. Since then, these shared resources and infrastructure have benefited industry and academia and have enabled researchers to systematically build on each other's work, leading to significant advances in the field.

Nonetheless, computer architecture research is evolving, with industry and academia turning towards machine learning (ML) optimization to meet stringent domain-specific requirements, such as ML for computer architecture, ML for TinyML accelerationDNN accelerator datapath optimization, memory controllers, power consumption, security, and privacy. Although prior work has demonstrated the benefits of ML in design optimization, the lack of strong, reproducible baselines hinders fair and objective comparison across different methods and poses several challenges to their deployment. To ensure steady progress, it is imperative to understand and tackle these challenges collectively.

To alleviate these challenges, in “ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design”, accepted at ISCA 2023, we introduced ArchGym, which includes a variety of computer architecture simulators and ML algorithms. Enabled by ArchGym, our results indicate that with a sufficiently large number of samples, any of a diverse collection of ML algorithms are capable of finding the optimal set of architecture design parameters for each target problem; no one solution is necessarily better than another. These results further indicate that selecting the optimal hyperparameters for a given ML algorithm is essential for finding the optimal architecture design, but choosing them is non-trivial. We release the code and dataset across multiple computer architecture simulations and ML algorithms.

Challenges in ML-assisted architecture research

ML-assisted architecture research poses several challenges, including:

  1. For a specific ML-assisted computer architecture problem (e.g., finding an optimal solution for a DRAM controller) there is no systematic way to identify optimal ML algorithms or hyperparameters (e.g., learning rate, warm-up steps, etc.). There is a wider range of ML and heuristic methods, from random walk to reinforcement learning (RL), that can be employed for design space exploration (DSE). While these methods have shown noticeable performance improvement over their choice of baselines, it is not evident whether the improvements are because of the choice of optimization algorithms or hyperparameters.

    Thus, to ensure reproducibility and facilitate widespread adoption of ML-aided architecture DSE, it is necessary to outline a systematic benchmarking methodology.

  2. While computer architecture simulators have been the backbone of architectural innovations, there is an emerging need to address the trade-offs between accuracy, speed, and cost in architecture exploration. The accuracy and speed of performance estimation widely varies from one simulator to another, depending on the underlying modeling details (e.g., cycle-accurate vs. ML-based proxy models). While analytical or ML-based proxy models are nimble by virtue of discarding low-level details, they generally suffer from high prediction error. Also, due to commercial licensing, there can be strict limits on the number of runs collected from a simulator. Overall, these constraints exhibit distinct performance vs. sample efficiency trade-offs, affecting the choice of optimization algorithm for architecture exploration.

    It is challenging to delineate how to systematically compare the effectiveness of various ML algorithms under these constraints.

  3. Finally, the landscape of ML algorithms is rapidly evolving and some ML algorithms need data to be useful. Additionally, rendering the outcome of DSE into meaningful artifacts such as datasets is critical for drawing insights about the design space.

    In this rapidly evolving ecosystem, it is consequential to ensure how to amortize the overhead of search algorithms for architecture exploration. It is not apparent, nor systematically studied how to leverage exploration data while being agnostic to the underlying search algorithm.

ArchGym design

ArchGym addresses these challenges by providing a unified framework for evaluating different ML-based search algorithms fairly. It comprises two main components: 1) the ArchGym environment and 2) the ArchGym agent. The environment is an encapsulation of the architecture cost model — which includes latency, throughput, area, energy, etc., to determine the computational cost of running the workload, given a set of architectural parameters — paired with the target workload(s). The agent is an encapsulation of the ML algorithm used for the search and consists of hyperparameters and a guiding policy. The hyperparameters are intrinsic to the algorithm for which the model is to be optimized and can significantly influence performance. The policy, on the other hand, determines how the agent selects a parameter iteratively to optimize the target objective.

Notably, ArchGym also includes a standardized interface that connects these two components, while also saving the exploration data as the ArchGym Dataset. At its core, the interface entails three main signals: hardware state, hardware parameters, and metrics. These signals are the bare minimum to establish a meaningful communication channel between the environment and the agent. Using these signals, the agent observes the state of the hardware and suggests a set of hardware parameters to iteratively optimize a (user-defined) reward. The reward is a function of hardware performance metrics, such as performance, energy consumption, etc. 

ArchGym comprises two main components: the ArchGym environment and the ArchGym agent. The ArchGym environment encapsulates the cost model and the agent is an abstraction of a policy and hyperparameters. With a standardized interface that connects these two components, ArchGym provides a unified framework for evaluating different ML-based search algorithms fairly while also saving the exploration data as the ArchGym Dataset.

ML algorithms could be equally favorable to meet user-defined target specifications

Using ArchGym, we empirically demonstrate that across different optimization objectives and DSE problems, at least one set of hyperparameters exists that results in the same hardware performance as other ML algorithms. A poorly selected (random selection) hyperparameter for the ML algorithm or its baseline can lead to a misleading conclusion that a particular family of ML algorithms is better than another. We show that with sufficient hyperparameter tuning, different search algorithms, even random walk (RW), are able to identify the best possible reward. However, note that finding the right set of hyperparameters may require exhaustive search or even luck to make it competitive.

With a sufficient number of samples, there exists at least one set of hyperparameters that results in the same performance across a range of search algorithms. Here the dashed line represents the maximum normalized reward. Cloud-1, cloud-2, stream, and random indicate four different memory traces for DRAMSys (DRAM subsystem design space exploration framework).

Dataset construction and high-fidelity proxy model training

Creating a unified interface using ArchGym also enables the creation of datasets that can be used to design better data-driven ML-based proxy architecture cost models to improve the speed of architecture simulation. To evaluate the benefits of datasets in building an ML model to approximate architecture cost, we leverage ArchGym’s ability to log the data from each run from DRAMSys to create four dataset variants, each with a different number of data points. For each variant, we create two categories: (a) Diverse Dataset, which represents the data collected from different agents (ACO, GA, RW, and BO), and (b) ACO only, which shows the data collected exclusively from the ACO agent, both of which are released along with ArchGym. We train a proxy model on each dataset using random forest regression with the objective to predict the latency of designs for a DRAM simulator. Our results show that:

  1. As we increase the dataset size, the average normalized root mean squared error (RMSE) slightly decreases.
  2. However, as we introduce diversity in the dataset (e.g., collecting data from different agents), we observe 9× to 42× lower RMSE across different dataset sizes.

Diverse dataset collection across different agents using ArchGym interface.
The impact of a diverse dataset and dataset size on the normalized RMSE.

The need for a community-driven ecosystem for ML-assisted architecture research

While, ArchGym is an initial effort towards creating an open-source ecosystem that (1) connects a broad range of search algorithms to computer architecture simulators in an unified and easy-to-extend manner, (2) facilitates research in ML-assisted computer architecture, and (3) forms the scaffold to develop reproducible baselines, there are a lot of open challenges that need community-wide support. Below we outline some of the open challenges in ML-assisted architecture design. Addressing these challenges requires a well coordinated effort and a community driven ecosystem.

Key challenges in ML-assisted architecture design.

We call this ecosystem Architecture 2.0. We outline the key challenges and a vision for building an inclusive ecosystem of interdisciplinary researchers to tackle the long-standing open problems in applying ML for computer architecture research. If you are interested in helping shape this ecosystem, please fill out the interest survey.


ArchGym is an open source gymnasium for ML architecture DSE and enables an standardized interface that can be readily extended to suit different use cases. Additionally, ArchGym enables fair and reproducible comparison between different ML algorithms and helps to establish stronger baselines for computer architecture research problems.

We invite the computer architecture community as well as the ML community to actively participate in the development of ArchGym. We believe that the creation of a gymnasium-type environment for computer architecture research would be a significant step forward in the field and provide a platform for researchers to use ML to accelerate research and lead to new and innovative designs.


This blogpost is based on joint work with several co-authors at Google and Harvard University. We would like to acknowledge and highlight Srivatsan Krishnan (Harvard) who contributed several ideas to this project in collaboration with Shvetank Prakash (Harvard), Jason Jabbour (Harvard), Ikechukwu Uchendu (Harvard), Susobhan Ghosh (Harvard), Behzad Boroujerdian (Harvard), Daniel Richins (Harvard), Devashree Tripathy (Harvard), and Thierry Thambe (Harvard).  In addition, we would also like to thank James Laudon, Douglas Eck, Cliff Young, and Aleksandra Faust for their support, feedback, and motivation for this work. We would also like to thank John Guilyard for the animated figure used in this post. Amir Yazdanbakhsh is now a Research Scientist at Google DeepMind and Vijay Janapa Reddi is an Associate Professor at Harvard.

Source: Google AI Blog

Celebrating Google Dev Library’s Women Contributors in AI/ML

Posted by Swathi Dharshna Subbaraj, Google Dev Library

Women have made remarkable progress in advancing AI/ML technology through their contributions to open source projects. They have developed and maintained tools, algorithms, and frameworks that enable researchers, developers, and businesses to create and implement cutting edge AI/ML solutions.

To celebrate those achievements, Google Dev Library has featured outstanding contributions from developers worldwide. It has also provided an opportunity to showcase contributions from women developers who are working on AI/ML projects. Read on to learn their projects and insights.

Contributors in Spotlight

Suzen Fylke

Suzen is a machine learning engineer with a passion for helping mission-driven and socially-minded companies leverage AI and data to drive impactful outcomes. With 3 years of experience at Twitter, Suzen developed platform tools that streamlined model development and deployment processes, allowing for faster iteration and improved efficiency. Sue recently shared her blog post titled "How to Visualize Custom TFX Artifacts With InteractiveContext" with Dev Library. Let's speak with Sue and learn more about her experience.

Headshot of Suzen Fylke, smiling

1.    Tell us more about your recent Dev Library submission on inspecting TFX artifactswith InteractiveContext and why you consider it invaluable for debugging TFX pipelines?
    One of my favorite things about TFX is being able to run pipeline steps individually and interactively inspect their results with InteractiveContext. I used to think you could only display standard artifacts with built-in visualizations, but, as it turns out, you can also use InteractiveContext with custom artifacts. Since I hadn't found any examples or documentation explaining how to display custom artifacts, I wrote a tutorial.

    2.    Can you walk me through your process for creating technical documentation for your projects to help other developers?   

    When I create technical documentation for work or open source projects, I do my best to follow the community's best practices and style guides and to center the reader. I think a lot about what readers can hope to learn or be able to do after reading the docs. I followed a similar approach when writing the tutorial I submitted.

    Most of my personal projects are active learning exercises. When I write about such projects, I focus much more on the process of building them than on the outcome. So, in addition to showing how they work, I describe what inspired me to create them, the challenges I encountered, and what's next for the project. I also include lots of links to resources I found helpful for understanding the tools and concepts I learned about.

    3.    What advice do you have for other women interested in developing open source AL/ML projects, and how can they get started? 

    I recommend contributing to communities you care about and projects you use and want to help improve. Create things using the project. Ask questions when documentation needs to be clarified. Report bugs when you encounter them. If you build something cool, demo it or write about it. If you find a problem you can fix, volunteer to do so. And if you get stuck or don't understand something, ask for help. I also recommend reading GitHub's "How to Contribute to Open Source" guide (https://opensource.guide/how-to-contribute/). My favorite takeaway is that open source projects are more than code and that there are many different ways to contribute based on your interests.

    4.    Your Dev Library author profile bio states that you’re exploring how to “make learning languages fun and approachable.” Can you walk me through that process? 
    This is aspirational and mainly a hobby right now. I love learning languages and learning how to learn languages. Languages are my "thing I can talk about for hours without getting bored." I don't actually have a process for this. Instead, I do a lot of exploring and experimenting and let my curiosity guide me. Sometimes this involves reading linguistics textbooks, trying different language-learning apps, contributing to projects like Common Voice, or learning how to use libraries like spaCy.

    5.    How do you see the field of open source AI/ML development evolving in the coming years, and how are you preparing for these changes?
    I see the continued development of tools and platforms aimed at democratizing machine learning. I hope this will enable people to meaningfully engage with the models and AI-powered products they use and better understand how they work. I also hope this will lead to more grassroots participatory research communities like Masakhane and encourage people without ML or software engineering backgrounds to create and contribute to open source projects.

    Aqsa is a passionate machine learning engineer with a strong curiosity for technology and a desire to share ideas with others. She has practical experience in diverse projects, including footfall forecasting, cataract detection, augmented reality, object detection, and recommender systems. Aqsa shared her blog post titled "Callbacks in TensorFlow — Customize the Behavior of your training" with Dev Library. Let's speak with Aqsa and learn more about her experience.

    Photo of Aqsa Kausar holding a microphone
    1.    Being Pakistan’s first Google Developer Expert (GDE), how do you approach building inclusive and diverse communities around you?
      As a Google Developer Expert (GDE), my responsibility is to help improve the tech community through inclusive and diverse events, workshops, and mentorship. With support from Google, fellow GDEs, and Google Developer Groups, we aim to create accessible opportunities for everyone, regardless of their background or experience level. As a speaker, I share my knowledge in ML with diverse audiences and offer mentorship to underrepresented individuals in tech, including women, minorities, and individuals from different backgrounds. I provide guidance on educational and career opportunities and connect people with resources, catering to as many as I can through various means of communication.

      2.     How do you approach collaborating with other developers on open source AI/ML projects, and what are some best practices you follow to ensure success?

      In our GDE community, we have active open source contributors who collaborate in groups for tutorials, research papers, and more. Collaboration is encouraged, and Googlers sometimes lead open source projects with GDEs. When you express interest, developers are open to working together. To foster a positive culture, we emphasize value and respect, clear goals, manageable tasks, communication channels, open communication, constructive feedback, and celebrating milestones. Successful collaboration hinges on valuing each other's time and skills.

      3.    How do you balance the need for technical rigor with the need for usability and accessibility in your open source projects?

      Understanding your audience and their needs is crucial to strike the right balance between technical rigor and usability. Simplify technical concepts for non-technical audiences and focus on practical applications. In open source projects, you have more flexibility, but in workshops or training, choose tools and technologies suitable for your audience. For beginners, use simpler language and interactive demos. For intermediate or advanced audiences, go deeper into technical details with coding snippets and complex concepts.

      4.    Why do you think it is important for technical writers to revise your content or projects regularly? Do you think it’s important that every tech writer or open source maintainer follow this best practice?

      Technology is ever-changing, so technical writers need to revise content regularly to ensure accuracy. Feedback from the audience can help make it accessible and relevant. However, contributors may not always have time to update their work due to busy schedules. Nevertheless, tech blogs and projects still provide a valuable kickstart for new developers, who can contribute with updates or follow-up blogs.

      5.    Can you tell me about a project you've worked on that you're particularly proud of, and what impact it has had on the open source community?

      I have been part of impactful initiatives such as Google Women Developer Academy, where I was a mentor for their pilot. The program helps women in tech improve their communication skills and prepares them for showcasing their talents, boosting their confidence. I also collaborated with fellow Google Developer Experts (GDEs) during the COVID-19 pandemic to create an open source course called "ML for Rookies," which simplifies machine learning concepts. Currently, I am working on a Cloud AI project supported by GCP and have started an open source "Cloud Playground" repo to make cloud-ai learning more accessible.

      Margaret, an ML Google Developer Expert (GDE) since 2018, is an ML research engineer who applies AI/ML to real world applications ranging from climate change to art and design. With expertise in deep learning, computer vision, TensorFlow, and on-device ML, she often writes and speaks at conferences. Margaret has shared multiple projects in topics like TensorFlow Lite with Dev Library. Let's speak with Margaret and learn more about her experience.

      Photo of Margaret Maynard-Reid, smiling

      1.    Can you share the Google technologies you work with?  
      Some of the Google technologies I work with are TensorFlow, TensorFlow Lite, Keras, Android, MediaPipe, and ML Kit. 

      2.    How do you approach collaborating with other developers on open source projects, and what are some best practices you follow to ensure a successful collaboration? 

      I’ve collaborated with Googlers, ML GDEs, students and professionals in tech. Consistent communication and observing best practices, such as code check-in and code reviews, are helpful to ensure a successful collaboration. 

      3.    What is your development process like for creating and maintaining open source AI/ML projects, and how do you prioritize which projects to work on? 

      There is limited time so prioritization is super important. I like to showcase new technologies or areas where developers including myself may have challenges with. Aside from code and tutorials, I also like to share my knowledge with sketchnotes and visual illustrations. 

      4.    You have been sharing learning resources on TensorFlow Lite. What advice do you have for other women interested in developing open source projects, and how can they get started? 
      There are many ways to contribute to open source projects: provide feedback on documentation or product features; write a tutorial with sample code; help fix bugs or contribute to libraries etc. It’s best to start simple and easy first, and then progress to more challenging projects. 

      5.    How do you see the field of open source AI/ML development evolving in the coming years, and how are you preparing for these changes? 

      Open source is becoming increasingly important for AI/ML development, evident in the recent development of generative AI and on-device machine learning for example. There will be even more opportunities for open source projects. Keep contributing because open source projects are a great way to learn the latest while helping others.

      Are you actively contributing to the AI/ML community? Become a Google Dev Library Contributor!

      Google Dev Library is a platform for showcasing open source projects featuring Google technologies. Join our global community of developers to showcase your projects. Submit your content.

      Rust fact vs. fiction: 5 Insights from Google’s Rust journey in 2022

      Reaching version 1.0 in just 2015, Rust is a relatively new language with a lot to offer. Developers eyeing the performance and safety guarantees that Rust provides, have to wonder if it's possible to just use Rust in place of what they've been using previously. What would happen if large companies tried to use it in their existing environment? How long would it take for developers to learn the language? Once they do, would they be productive?

      In this post, we will analyze some data covering years of early adoption of Rust here at Google. At Google, we have been seeing increased Rust adoption, especially in our consumer applications and platforms. Pulling from the over 1,000 Google developers who have authored and committed Rust code as some part of their work in 2022, we’ll address some rumors head-on, both confirming some issues that could be improved and sharing some enlightening discoveries we have made along the way.

      We’d like to particularly thank one of our key training vendors, Ferrous Systems, as we started our Rust adoption here at Google. We also want to highlight some new freely available self-service training materials called Comprehensive Rust 🦀 that we and the community have worked on over the last few quarters.

      Rumor 1: Rust takes more than 6 months to learn – Debunked !

      All survey participants are professional software developers (or a related field), employed at Google. While some of them had prior Rust experience (about 13%), most of them are coming from C/C++, Python, Java, Go, or Dart.

      Based on our studies, more than 2/3 of respondents are confident in contributing to a Rust codebase within two months or less when learning Rust. Further, a third of respondents become as productive using Rust as other languages in two months or less. Within four months, that number increased to over 50%. Anecdotally, these ramp-up numbers are in line with the time we’ve seen for developers to adopt other languages, both inside and outside of Google.

      Overall, we’ve seen no data to indicate that there is any productivity penalty for Rust relative to any other language these developers previously used at Google. This is supported by the students who take the Comprehensive Rust 🦀 class: the questions asked on the second and third day show that experienced software developers can become comfortable with Rust in a very short time.

      Pie graph depicting time until confident writing Rust. Still ramping up = 8.6% (orange), 2-3 weeks = 27% (blue), 1-2 months = 39.8% (red), 3-4 months = 15.6% (yellow), More than 4 months = 9% (green)

      Rumor 2: The Rust compiler is not as fast as people would like – Confirmed !

      Slow build speeds were by far the #1 reported challenge that developers have when using Rust, with only a little more than 40% of respondents finding the speed acceptable.

      There is already a fantastic community-wide effort improving and tracking rustc performance. This is supported by both volunteers and several companies (including Google), and we’re delighted to see key developers working in this space but clearly continuing and potentially growing additional support here would be beneficial.

      Rumor 3: Unsafe code and interop are always the biggest challenges – Debunked !

      The top three challenging areas of Rust for current Google developers were:

      Writing unsafe code and handling C/C++ interop were cited as something Google developers had encountered but were not top challenges. These three other areas are places where the Rust Language Design Team has been investing in flattening the learning curve overall as well as continued evolution, and our internal survey results strongly agree with these as areas of investment.

      Rumor 4: Rust has amazing compiler error messages – Confirmed !

      Rust is commonly regarded as having some of the most helpful error messages in the compiler space, and that held up in this survey as well. Only 9% of respondents are not satisfied with the quality of diagnostic and debugging information in Rust. Feedback from Comprehensive Rust 🦀 participants shows the same: people are amazed by the compiler messages. At first this is a surprise – people are used to ignoring large compiler errors, but after getting used to it, people love it.

      The following are excerpts from an exercise some internal Googlers have been doing to practice Rust – solving Advent of Code 2021 in Rust.

      On Day 5 of the exercises, we need to perform a search for entries within a table. The error below not only detects that our pattern matching on the result was missing a case, but also makes a suggestion for a fix.

      Image of code snippet showing error detection message for pattern matching in Rust

      On Day 11, we need to check for whether an element is within the bounds of a grid. The Rust warning below detects that we have a redundant comparison due to the fact that the types are unsigned, and suggests code that could be removed.

      Image of code snippet showing error detection message for redundant comparison in Rust

      Rumor 5: Rust code is high quality – Confirmed!

      The respondents said that the quality of the Rust code is high — 77% of developers were satisfied with the quality of Rust code. In fact, when asked to compare whether they felt that Rust code was more correct than the code that they write in other languages, an overwhelming 85% of respondents are confident that their Rust code is correct.

      And, it’s not just correct—it’s also easy to review. More than half of respondents say that Rust code is incredibly easy to review. As an engineering manager, that result is in many ways at least as interesting to me as the code authoring results, since code reviewing is at least as large a part of the role of a professional software engineer as authoring.

      As both we at Google and others have noted, developer satisfaction and productivity are correlated with both code quality and how long it takes to get a code review. If Rust is not only better for writing quality code, but also better for getting that code landed, that’s a pretty compelling set of reasons beyond even performance and memory safety for companies to be evaluating and considering adopting it.

      Looking forward

      While over a thousand developers is a good sample of engineers, we look forward to further adoption and a future survey that includes many more use cases. In addition, while many of the developers surveyed joined teams without Rust experience, this population does have more excited early adopters than we would like from a broader survey. Stay tuned over the next year for another update!

      By Lars Bergstrom, PhD – Android Platform Programming Languages and Kathy Brennan, PhD - Low-level Operating Systems Sr. User Experience Researcher

      Optimizing gVisor filesystems with Directfs

      gVisor is a sandboxing technology that provides a secure environment for running untrusted code. In our previous blog post, we discussed how gVisor performance improves with a root filesystem overlay. In this post, we'll dive into another filesystem optimization that was recently launched: directfs. It gives gVisor’s application kernel (the Sentry) secure direct access to the container filesystem, avoiding expensive round trips to the filesystem gofer.

      Origins of the Gofer

      gVisor is used internally at Google to run a variety of services and workloads. One of the challenges we faced while building gVisor was providing remote filesystem access securely to the sandbox. gVisor’s strict security model and defense in depth approach assumes that the sandbox may get compromised because it shares the same execution context as the untrusted application. Hence the sandbox cannot be given sensitive keys and credentials to access Google-internal remote filesystems.

      To address this challenge, we added a trusted filesystem proxy called a "gofer". The gofer runs outside the sandbox, and provides a secure interface for untrusted containers to access such remote filesystems. For architectural simplicity, gofers were also used to serve local filesystems as well as remote.

      Gofer process intermediates filesystem operations

      Isolating the Container Filesystem in runsc

      When gVisor was open sourced as runsc, the same gofer model was copied over to maintain the same security guarantees. runsc was configured to start one gofer process per container which serves the container filesystem to the sandbox over a predetermined protocol (now LISAFS). However, a gofer adds a layer of indirection with significant overhead.

      This gofer model (built for remote filesystems) brings very few advantages for the runsc use-case, where all the filesystems served by the gofer (like rootfs and bind mounts) are mounted locally on the host. The gofer directly accesses them using filesystem syscalls.

      Linux provides some security primitives to effectively isolate local filesystems. These include, mount namespaces, pivot_root and detached bind mounts1. Directfs is a new filesystem access mode that uses these primitives to expose the container filesystem to the sandbox in a secure manner. The sandbox’s view of the filesystem tree is limited to just the container filesystem. The sandbox process is not given access to anything mounted on the broader host filesystem. Even if the sandbox gets compromised, these mechanisms provide additional barriers to prevent broader system compromise.


      In directfs mode, the gofer still exists as a cooperative process outside the sandbox. As usual, the gofer enters a new mount namespace, sets up appropriate bind mounts to create the container filesystem in a new directory and then pivot_root(2)s into that directory. Similarly, the sandbox process enters new user and mount namespaces and then pivot_root(2)s into an empty directory to ensure it cannot access anything via path traversal. But instead of making RPCs to the gofer to access the container filesystem, the sandbox requests the gofer to provide file descriptors to all the mount points via SCM_RIGHTS messages. The sandbox then directly makes file-descriptor-relative syscalls (e.g. fstatat(2), openat(2), mkdirat(2), etc) to perform filesystem operations.

      Sandbox directly accesses container filesystem with directfs

      Earlier when the gofer performed all filesystem operations, we could deny all these syscalls in the sandbox process using seccomp. But with directfs enabled, the sandbox process's seccomp filters need to allow the usage of these syscalls. Most notably, the sandbox can now make openat(2) syscalls (which allow path traversal), but with certain restrictions: O_NOFOLLOW is required, no access to procfs and no directory FDs from the host. We also had to give the sandbox the same privileges as the gofer (for example CAP_DAC_OVERRIDE and CAP_DAC_READ_SEARCH), so it can perform the same filesystem operations.

      It is noteworthy that only the trusted gofer provides FDs (of the container filesystem) to the sandbox. The sandbox cannot walk backwards (using ‘..’) or follow a malicious symlink to escape out of the container filesystem. In effect, we've decreased our dependence on the syscall filters to catch bad behavior, but correspondingly increased our dependence on Linux's filesystem isolation protections.


      Making RPCs to the gofer for every filesystem operation adds a lot of overhead to runsc. Hence, avoiding gofer round trips significantly improves performance. Let's find out what this means for some of our benchmarks. We will run the benchmarks using our newly released systrap platform on bind mounts (as opposed to rootfs). This would simulate more realistic use cases because bind mounts are extensively used while configuring filesystems in containers. Bind mounts also do not have an overlay (like the rootfs mount), so all operations go through goferfs / directfs mount.

      Let's first look at our stat micro-benchmark, which repeatedly calls stat(2) on a file.

      Stat benchmark improvement with directfs
      The stat(2) syscall is more than 2x faster! However, since this is not representative of real-world applications, we should not extrapolate these results. So let's look at some real-world benchmarks.
      Stat benchmark improvement with directfs
      We see a 12% reduction in the absolute time to run these workloads and 17% reduction in Ruby load time!


      The gofer model in runsc was overly restrictive for accessing host files. We were able to leverage existing filesystem isolation mechanisms in Linux to bypass the gofer without compromising security. Directfs significantly improves performance for certain workloads. This is part of our ongoing efforts to improve gVisor performance. You can learn more about gVisor at gvisor.dev. You can also use gVisor in GKE with GKE Sandbox. Happy sandboxing!

      1Detached bind mounts can be created by first creating a bind mount using mount(MS_BIND) and then detaching it from the filesystem tree using umount(MNT_DETACH).

      By Ayush Ranjan, Software Engineer – Google