At Google, we are committed to accessibility and are constantly looking for ways to improve our development process to discover, debug and fix accessibility issues. Today we are excited to announce a new open source project: Accessibility Scanner for iOS (or GSCXScanner as we lovingly call it). This is a developer tool that can assist in locating and fixing accessibility issues while an app is being developed.
App development can be a time consuming process, especially when it involves human testers. Sometimes, as in the case with accessibility testing, they are necessary. A developer can write automated tests to perform some accessibility checks, but GSCXScanner takes this one step further. When a new feature is being developed, often there are several iterations of code changes, building, launching and trying out the new feature. It is faster and easier to fix accessibility issues with the feature if they can be detected during this phase when the developer is working with the new feature.
GSCXScanner lives in your app process and can perform accessibility checks on the UI currently on the screen simply with the touch of a button. The scanner’s UI which is overlaid on the app can be moved around so you can use your app normally and trigger a scan only when you need it. Also, it uses GTXiLib, a library of iOS accessibility checks to scan your app, and you can author your own GTX checks and have them run along with scanner’s default checks.
Using the scanner does not eliminate the need for manual testing or automated tests, these are must haves for delivering quality products. But GCSXScanner can speed up the development process by showing issues in app during development.
We're excited to announce that Google is a founding member of the newly formed Continuous Delivery Foundation (CDF). Continuous delivery (CD) is a critical part of modern software development and DevOps practices, and we're excited to collaborate in a vendor-neutral foundation with other industry leaders.
We're also thrilled to announce the contribution of two projects as part of our membership: Tekton, and in collaboration with Netflix, Spinnaker. These donations will enter alongside Jenkins and Jenkins X, providing an exciting portfolio of projects for the CDF to expand upon.
Continuous Delivery Foundation
Currently, the continuous integration/continuous delivery (CI/CD) tool landscape is highly fragmented. As companies migrate to the cloud and modernize their infrastructure, tooling decisions become increasingly complicated and difficult. DevOps practitioners constantly seek guidance on software delivery best practices and how to secure their software supply chains but gathering this information can be difficult. Enter the CDF.
The CDF is about more than just code. Modern application development brings new challenges around security and compliance. This foundation will work to define the practices and guidelines that, together with tooling, will help application developers everywhere deliver better and more secure software at speed.
At a foundation level, the CDF will help make CI/CD tooling easier. And at a project level, Tekton helps address complexity problems at their core. We will team up with the open source community and industry leaders to design and build the critical pieces common to CI/CD systems.
Tekton is a set of shared, open source components for building CI/CD systems. It provides a flexible, extensible workflow that accommodates deployment to Kubernetes, VMs, bare metal, mobile or even emerging use cases.
The project’s goal is to provide industry specifications for pipelines, workflows, source code access and other primitives. It modernizes the continuous delivery control plane by leveraging all of the built-in scaling, reliability, and extensibility advantages of Kubernetes, and moves software deployment logic there. Tekton was initially built as a part of Knative, but given its stand-alone power, and ability to deploy to a variety of targets, we’ve decided to separate its functionality out into a new project.
Today, Tekton includes primitives for pipeline definition, source code access, artifact management, and test execution. The project roadmap includes adding support for results and event triggering in the coming months. We also plan to work with CI/CD vendors to build out an ecosystem of components that will allow you to use Tekton with existing tools like Jenkins X, Knative and others.
Spinnaker is an open source, multi-cloud continuous delivery platform originally created by Netflix and jointly led by Netflix and Google. It is typically used in organizations at scale, where DevOps teams support multiple development teams, and has been battle-tested in production by hundreds of teams and in millions of deployments.
Spinnaker is a multi-component system that conceptually aligns with Tekton, and that includes many features important to making continuous delivery reliable, including support for advanced deployment strategies, and Kayenta, an open source canary analysis service.
Given Google’s significant contributions to both Tekton and Spinnaker, we’re very pleased to see them become part of the same foundation. Spinnaker’s large user community has a great deal of experience in the continuous delivery domain, and joining the CDF provides a great opportunity to share that expertise with the broader community.
If you'd like to participate in the future of Tekton, Spinnaker, or the CDF, please join us in Barcelona, Spain, on May 20th at the Continuous Delivery Summit ahead of KubeCon/CloudNativeCon EU. If you can’t make it, don’t worry, as there will be many opportunities to get involved and become a part of the community.
We look forward to working with the continuous delivery community on shaping the next wave of CI/CD innovations, alignments, and improvements, no matter where your applications are delivered to.
By Dan Lorenc and Kim Lewandowski, DevOps at Google Cloud
Google Open Source is delighted to announce Season of Docs, a new program which fosters the open source contributions of technical writers.
Season of Docs brings technical writers and open source projects together for a few months to work on open source documentation. 2019 is the first time we’re running this exciting new program.
Join us in making a substantive contribution to open source software development around the world.
Fostering collaboration between open source projects and technical writers
The Open Source Survey showed that documentation is highly valued in open source communities, yet there’s little good documentation out there. Why? Because creating documentation is hard. But...
There are people who know how to do docs well. Technical writers know how to structure a documentation site so that people can find and understand the content. They know how to write docs that fit the needs of their audience. Technical writers can also help optimize a community’s processes for open source contribution and on-boarding new contributors.
During Season of Docs, technical writers will spend a few months working closely with open source communities. Each writer works with their chosen open source project. The writers bring their expertise to the projects’ documentation while at the same time learning about open source and new technologies.
Mentors from participating open source organizations share knowledge of their communities’ processes and tools. Together the technical writers and mentors build a new doc set, improve the structure of the existing docs, develop a much-needed tutorial, or improve contribution processes and guides. See more ideas for technical writing projects.
By working together in Season of Docs we raise awareness of open source, of docs, and of technical writing.
How does it work?
April 2-23: Open source organizations apply to take part in Season of Docs
April 30: Google publishes the list of accepted mentoring organizations, along with their ideas for documentation projects
April 30 - June 28: Technical writers choose the project they’d like to work on and submit their proposals to Season of Docs
July 30: Google announces the accepted technical writer projects
August 1 - September 1: Community bonding: Technical writers get to know mentors and the open source community, and refine their projects in collaboration with their mentors
September 2 - November 29: Technical writers work with open source mentors on the accepted projects, and submit their work at the end of the period
December 10: Google publishes the list of successfully-completed projects.
See the timeline for details, including the provision for projects that run longer than three months.
Explore the Season of Docs website at g.co/seasonofdocs to learn more about participating in the program. Use our logo and other promotional resources to spread the word. Examine the timeline, check out the FAQ, and get ready to apply!
By Sarah Maddox, Google Technical Writer and Andrew Chen, Google Open Source
We are excited to announce the open source projects and organizations that have been accepted for Google Summer of Code (GSoC) 2019, the 15th year of the program! As usual, we received more applications this year than we did last year, about twice as many as we are able to accept into the program.
After careful review, we have chosen 207 open source projects to be mentor organizations this year, 28 of which are new to the program. Please see the program website for a complete list of the accepted organizations.
Are you a student interested in participating? We begin accepting student applications on Monday, March 25, 2019 at 18:00 UTC and the deadline to apply is Tuesday, April 9, 2019 at 18:00 UTC.
The most successful applications come from students who start preparing now. You can start by watching the short video below, checking out the Student Guide, and reviewing the list of accepted organizations and reaching out to the 2 or 3 that interest you the most now - before the application period begins.
Open source is usually associated with code behind utilities and applications, though you can find it in many other places: such as the LaTeX source code that describes the PDFs of scientific papers.
As an example, the following source code:
Generates this PDF when compiled using pdflatex:
You can see a huge repository of such open source code at arXiv.org, an open access repository of scientific papers currently containing about 1.5 million entries (140,616 uploads in 2018). One can not only download all papers in PDF format, but also obtain the source code to regenerate them and freely reuse any of their parts.
Open sourcing LaTeX code, however, comes with its risks and challenges. We’ve built and released the code of arXiv LaTeX cleaner to remedy some of these.
Scrubbing the Code
The main risk one faces when sharing LaTeX code with the world is accidentally releasing private information, primarily through commented code left over in the file itself.
While authors put a lot of effort into polishing the final PDF, the code isn’t usually cleaned up and is left with many pieces of text that don’t actually appear in the PDF. Things like, “I do not see why the following statement should be correct,” or “Look, I’m citing you!,” make it into arXiv for everyone to see. This happens so often there’s even a Twitter bot that finds and publishes them!
Cleaning up this commented out code manually is laborious, so arXiv LaTeX cleaner automatically removes it for you.
Private information can also be found in the many auxiliary files that LaTeX generates when the code is compiled. Some of them are needed in arXiv (e.g., .bbl files), some of them are not: arXiv LaTeX cleaner will delete the unneeded ones and keep the rest automatically.
Cleaning and Autoscaling Images
Challenges also come our way when preparing the code to submit to arXiv: one needs to upload a ZIP file smaller than 10 MBytes. With high resolution pictures and figures, it’s easy to go beyond the limit.
Manually resizing images and deleting images that aren’t actually in the final version is time consuming and cumbersome, so arXiv LaTeX cleaner does that automatically, too. If there’s a very intricate figure you’d like to keep in high resolution, you can specify a list of images and their expected resolution.
We hope that, by making open sourcing research papers faster and safer, arXiv LaTeX cleaner will help even more researchers embrace open access and make their work freely available.
arXiv LaTeX cleaner itself is open source, so you can adapt it to your needs. If you think your adaptation would be useful for others, we’d love your contributions, too.
Posted by Danijar Hafner, Student Researcher, Google AI
Research into how artificial agents can improve their decisions over time is progressing rapidly via reinforcement learning (RL). For this technique, an agent observes a stream of sensory inputs (e.g. camera images) while choosing actions (e.g. motor commands), and sometimes receives a reward for achieving a specified goal. Model-free approaches to RL aim to directly predict good actions from the sensory observations, enabling DeepMind's DQN to play Atari and other agents to controlrobots. However, this blackbox approach often requires several weeks of simulated interaction to learn through trial and error, limiting its usefulness in practice.
Model-based RL, in contrast, attempts to have agents learn how the world behaves in general. Instead of directly mapping observations to actions, this allows an agent to explicitly plan ahead, to more carefully select actions by "imagining" their long-term outcomes. Model-based approaches have achieved substantial successes, including AlphaGo, which imagines taking sequences of moves on a fictitious board with the known rules of the game. However, to leverage planning in unknown environments (such as controlling a robot given only pixels as input), the agent must learn the rules or dynamics from experience. Because such dynamics models in principle allow for higher efficiency and natural multi-task learning, creating models that are accurate enough for successful planning is a long-standing goal of RL.
To spur progress on this research challenge and in collaboration with DeepMind, we present the Deep Planning Network (PlaNet) agent, which learns a world model from image inputs only and successfully leverages it for planning. PlaNet solves a variety of image-based control tasks, competing with advanced model-free agents in terms of final performance while being 5000% more data efficient on average. We are additionally releasing the source code for the research community to build upon.
The PlaNet agent learning to solve a variety of continuous control tasks from images in 2000 attempts. Previous agents that do not learn a model of the environment often require 50 times as many attempts to reach comparable performance.
How PlaNet Works In short, PlaNet learns a dynamics model given image inputs and efficiently plans with it to gather new experience. In contrast to previous methods that plan over images, we rely on a compact sequence of hidden or latent states. This is called a latent dynamics model: instead of directly predicting from one image to the next image, we predict the latent state forward. The image and reward at each step is then generated from the corresponding latent state. By compressing the images in this way, the agent can automatically learn more abstract representations, such as positions and velocities of objects, making it easier to predict forward without having to generate images along the way.
Learned Latent Dynamics Model: In a latent dynamics model, the information of the input images is integrated into the hidden states (green) using the encoder network (grey trapezoids). The hidden state is then projected forward in time to predict future images (blue trapezoids) and rewards (blue rectangle).
To learn an accurate latent dynamics model, we introduce:
A Recurrent State Space Model: A latent dynamics model with both deterministic and stochastic components, allowing to predict a variety of possible futures as needed for robust planning, while remembering information over many time steps. Our experiments indicate both components to be crucial for high planning performance.
A Latent Overshooting Objective: We generalize the standard training objective for latent dynamics models to train multi-step predictions, by enforcing consistency between one-step and multi-step predictions in latent space. This yields a fast and effective objective that improves long-term predictions and is compatible with any latent sequence model.
While predicting future images allows us teach the model, encoding and decoding images (trapezoids in the figure above) requires significant computation, which would slow down planning. However,planning in the compact latent state space is fast since we only need to predict future rewards, and not images, to evaluate an action sequence. For example, the agent can imagine how the position of a ball and its distance to the goal will change for certain actions, without having to visualize the scenario. This allows us to compare 10,000 imagined action sequences with a large batch size every time the agent chooses an action. We then execute the first action of the best sequence found and replan at the next step.
Planning in Latent Space: For planning, we encode past images (gray trapezoid) into the current hidden state (green). From there, we efficiently predict future rewards for multiple action sequences. Note how the expensive image decoder (blue trapezoid) from the previous figure is gone. We then execute the first action of the best sequence found (red box).
Compared to our preceding work on world models, PlaNet works without a policy network -- it chooses actions purely by planning, so it benefits from model improvements on the spot. For the technical details, check out our online research paper or the PDF version.
PlaNet vs. Model-Free Methods We evaluate PlaNet on continuous control tasks. The agent is only given image observations and rewards. We consider tasks that pose a variety of different challenges:
A cartpole swing-up task, with a fixed camera, so the cart can move out of sight. The agent thus must absorb and remember information over multiple frames.
A finger spin task that requires predicting two separate objects, as well as the interactions between them.
A cheetah running task that includes contacts with the ground that are difficult to predict precisely, calling for a model that can predict multiple possible futures.
A cup task, which only provides a sparse reward signal once a ball is caught. This demands accurate predictions far into the future to plan a precise sequence of actions.
A walker task, in which a simulated robot starts off by lying on the ground, and must first learn to stand up and then walk.
PlaNet agents trained on a variety of image-based control tasks. The animation shows the input images as the agent is solving the tasks. The tasks pose different challenges: partial observability, contacts with the ground, sparse rewards for catching a ball, and controlling a challenging bipedal robot.
Our work constitutes one of the first examples where planning with a learned model outperforms model-free methods on image-based tasks. The table below compares PlaNet to the well-known A3C agent and the D4PG agent, that combines recent advances in model-free RL. The numbers for these baselines are taken from the DeepMind Control Suite. PlaNet clearly outperforms A3C on all tasks and reaches final performance close to D4PG while, using 5000% less interaction with the environment on average.
One Agent for All Tasks Additionally, we train a single PlaNet agent to solve all six tasks. The agent is randomly placed into different environments without knowing the task, so it needs to infer the task from its image observations. Without changes to the hyper parameters, the multi-task agent achieves the same mean performance as individual agents. While learning slower on the cartpole tasks, it learns substantially faster and reaches a higher final performance on the challenging walker task that requires exploration.
Video predictions of the PlaNet agent trained on multiple tasks. Holdout episodes collected with the trained agent are shown above and open-loop agent hallucinations below. The agent observes the first 5 frames as context to infer the task and state and accurately predicts ahead for 50 steps given a sequence of actions.
Conclusion Our results showcase the promise of learning dynamics models for building autonomous RL agents. We advocate for further research that focuses on learning accurate dynamics models on tasks of even higher difficulty, such as 3D environments and real-world robotics tasks. A possible ingredient for scaling up is the processing power of TPUs. We are excited about the possibilities that model-based reinforcement learning opens up, including multi-task learning, hierarchical planning and active exploration using uncertainty estimates.
Acknowledgements This project is a collaboration with Timothy Lillicrap, Ian Fischer, Ruben Villegas, Honglak Lee, David Ha and James Davidson. We further thank everybody who commented on our paper draft and provided feedback at any point throughout the project.
Fuzzing is an automated method for detecting bugs in software that works by feeding unexpected inputs to a target program. It is effective at finding memory corruption bugs, which often have serioussecurityimplications. Manually finding these issues is both difficult and time consuming, and bugs often slip through despite rigorous code review practices. For software projects written in an unsafe language such as C or C++, fuzzing is a crucial part of ensuring their security and stability.
In order for fuzzing to be truly effective, it must be continuous, done at scale, and integrated into the development process of a software project. To provide these features for Chrome, we wrote ClusterFuzz, a fuzzing infrastructure running on over 25,000 cores. Two years ago, we began offering ClusterFuzz as a free service to open source projects through OSS-Fuzz.
Today, we’re announcing that ClusterFuzz is now open source and available for anyone to use.
We developed ClusterFuzz over eight years to fit seamlessly into developer workflows, and to make it dead simple to find bugs and get them fixed. ClusterFuzz provides end-to-end automation, from bug detection, to triage (accurate deduplication, bisection), to bug reporting, and finally to automatic closure of bug reports.
ClusterFuzz has found more than 16,000 bugs in Chrome and more than 11,000 bugs in over 160 open source projects integrated with OSS-Fuzz. It is an integral part of the development process of Chrome and many other open source projects. ClusterFuzz is often able to detect bugs hours after they are introduced and verify the fix within a day.
Check out our GitHub repository. You can try ClusterFuzz locally by following these instructions. In production, ClusterFuzz depends on some key Google Cloud Platform services, but you can use your own compute cluster. We welcome your contributions and look forward to any suggestions to help improve and extend this infrastructure. Through open sourcing ClusterFuzz, we hope to encourage all software developers to integrate fuzzing into their workflows.
By Abhishek Arya, Oliver Chang, Max Moroz, Martin Barbella and Jonathan Metzman, ClusterFuzz team
Reinforcement learning (RL) has become one of the most popular fields of machine learning, and has seen a number of great advances over the last few years. As a result, there is a growing need from both researchers and educators to have access to a clear and reliable framework for RL research and education.
One of the most common requests we have received is support for more environments. This confirms what we have seen internally, where simpler environments, such as those supported by OpenAI’s Gym, are incredibly useful when testing out new algorithms. We are happy to announce Dopamine 2.0, which includes support for discrete-domain gym environments (e.g. discrete states and actions). The core of the framework remains unchanged, we have simply generalized the interface with the environment. For backwards compatibility, users will still be able to download version 1.0.
We include default configurations for two classic control environments: CartPole and Acrobot; on these environments one can train a Dopamine agent in minutes. When compared with the training time for a standard Atari 2600 game (around 5 days on a standard GPU), these environments allow researchers to iterate much faster on research ideas before testing them out on larger Atari games. We also include a Colaboratory that illustrates how to train an agent on Cartpole and Acrobot. Finally, our GymPreprocessing class serves as an example for how to use Dopamine with other custom environments.
We are excited by the new opportunities enabled by Dopamine 2.0, and look forward to seeing what the research community creates with it!
Do you lead or represent a free or open source software organization? Are you seeking new contributors? (Who isn’t?) Do you enjoy the challenge and reward of mentoring new developers? Apply to be a mentor organization for Google Summer of Code 2019!
We are searching for open source projects and organizations to participate in the 15th annual Google Summer of Code (GSoC). GSoC is a global program that draws university student developers from around the world to contribute to open source. Each student spends three months working on a coding project, with the support of volunteer mentors, for participating open source organizations from late May to August.
Last year 1,264 students worked with 206 open source organizations. Organizations include individual smaller and medium sized open source projects as well as a number of umbrella organizations with many sub-projects under them (Python Software Foundation, CERN, Apache Software Foundation).
You can apply to be a mentoring organization for GSoC starting today. The deadline to apply is February 6 at 20:00 UTC. Organizations chosen for GSoC 2019 will be publicly announced on February 26.
Please visit the program site for more information on how to apply and review the detailed timeline of important deadlines. We also encourage you to check out the Mentor Guide and our short video on why open source projects choose to apply to be a part of the program.
Cross-posted on the Google Security Blog. Open source software (OSS) is extremely important to Google, and we rely on OSS in a variety of customer-facing and internal projects. We also understand the difficulty and importance of securing the open source ecosystem, and are continuously looking for ways to simplify it.
For the OSS community, we currently provide OSS-Fuzz, a free continuous fuzzing infrastructure hosted on the Google Cloud Platform. OSS-Fuzz uncovers security vulnerabilities and stability issues, and reports them directly to developers. Since launching in December 2016, OSS-Fuzz has reported over 9,000 bugs directly to open source developers.
In addition to OSS-Fuzz, Google's security team maintains several internal tools for identifying bugs in both Google internal and open source code. Until recently, these issues were manually reported to various public bug trackers by our security team and then monitored until they were resolved. Unresolved bugs were eligible for the Patch Rewards Program. While this reporting process had some success, it was overly complex. Now, by unifying and automating our fuzzing tools, we have been able to consolidate our processes into a single workflow, based on OSS-Fuzz. Projects integrated with OSS-Fuzz will benefit from being reviewed by both our internal and external fuzzing tools, thereby increasing code coverage and discovering bugs faster.
We are committed to helping open source projects benefit from integrating with our OSS-Fuzz fuzzing infrastructure. In the coming weeks, we will reach out via email to critical projects that we believe would be a good fit and support the community at large. Projects that integrate are eligible for rewards ranging from $1,000 (initial integration) up to $20,000 (ideal integration); more details are available here. These rewards are intended to help offset the cost and effort required to properly configure fuzzing for OSS projects. If you would like to integrate your project with OSS-Fuzz, please submit your project for review. Our goal is to admit as many OSS projects as possible and ensure that they are continuously fuzzed.
Once contacted, we might provide a sample fuzz target to you for easy integration. Many of these fuzz targets are generated with new technology that understands how library APIs are used appropriately. Watch this space for more details on how Google plans to further automate fuzz target creation, so that even more open source projects can benefit from continuous fuzzing.
Thank you for your continued contributions to the open source community. Let’s work together on a more secure and stable future for open source software.
By Matt Ruhstaller, TPM and Oliver Chang, Software Engineer, Google Security Team