Tag Archives: open source software

Announcing the Atheris Python Fuzzer

Fuzz testing is a well-known technique for uncovering programming errors. Many of these detectable errors have serious security implications. Google has found thousands of security vulnerabilities and other bugs using this technique. Fuzzing is traditionally used on native languages such as C or C++, but last year, we built a new Python fuzzing engine. Today, we’re releasing the Atheris fuzzing engine as open source.

What can Atheris do?

Atheris can be used to automatically find bugs in Python code and native extensions. Atheris is a “coverage-guided” fuzzer, which means that Atheris will repeatedly try various inputs to your program while watching how it executes, and try to find interesting paths.

One of the best uses for Atheris is for differential fuzzers. These are fuzzers that look for differences in behavior of two libraries that are intended to do the same thing. One of the example fuzzers packaged with Atheris does exactly this to compare the Python “idna” package to the C “libidn2” package. Both of these packages are intended to decode and resolve internationalized domain names. However, the example fuzzer idna_uts46_fuzzer.py shows that they don’t always produce the same results. If you ever decided to purchase a domain containing (Unicode codepoints [U+0130, U+1df9]), you’d discover that the idna and libidn2 libraries resolve that domain to two completely different websites.

In general, Atheris is useful on pure Python code whenever you have a way of expressing what the “correct” behavior is - or at least expressing what behaviors are definitely not correct. This could be as complex as custom code in the fuzzer that evaluates the correctness of a library’s output, or as simple as a check that no unexpected exceptions are raised. This last case is surprisingly useful. While the worst outcome from an unexpected exception is typically denial-of-service (by causing a program to crash), unexpected exceptions tend to reveal more serious bugs in libraries. As an example, the one YAML parsing library we tested Atheris on says that it will only raise YAMLErrors; however, yaml_fuzzer.py detects numerous other exceptions, such as ValueError from trying to interpret “-_” as an integer, or TypeError from trying to use a list as a key in a dict. (Bug report.) This indicates flaws in the parser.

Finally, Atheris supports fuzzing native Python extensions, using libFuzzer. libFuzzer is a fuzzing engine integrated into Clang, typically used for fuzzing C or C++. When using libFuzzer with Atheris, Atheris can still find all the bugs previously described, but can also find memory corruption bugs that only exist in native code. Atheris supports the Clang sanitizers Address Sanitizer and Undefined Behavior Sanitizer. These make it easy to detect corruption when it happens, rather than far later. In one case, the author of this document found an LLVM bug using an Atheris fuzzer (now fixed).

What does Atheris support?

Atheris supports fuzzing Python code and native extensions in Python 2.7 and Python 3.3+. When fuzzing Python code, using Python 3.8+ is strongly recommended, as it allows for much better coverage information. When fuzzing native extensions, Atheris can be used in combination with Address Sanitizer or Undefined Behavior Sanitizer.

OSS-Fuzz is a fuzzing service hosted by Google, where we execute fuzzers on open source code free of charge. OSS-Fuzz will soon support Atheris!

How can I get started?

Take a look at the repo, in particular the examples. For fuzzing pure Python, it’s as simple as:

pip3 install atheris

And then, just define a TestOneInput function that runs the code you want to fuzz:

import atheris
import sys


def TestOneInput(data):
    if data == b"bad":
        raise RuntimeError("Badness!")


atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()

That’s it! Atheris will repeatedly invoke TestOneInput and monitor the execution flow, until a crash or exception occurs.

For more details, including how to fuzz native code, see the README.


By Ian Eldred Pudney, Google Information Security

Security scorecards for open source projects

When developers or organizations introduce a new open source dependency into their production software, there’s no easy indication of how secure that package is.

Some organizations—including Google—have systems and processes in place that engineers must follow when introducing a new open source dependency, but that process can be tedious, manual, and error-prone. Furthermore, many of these projects and developers are resource constrained and security often ends up a low priority on the task list. This leads to critical projects not following good security best practices and becoming vulnerable to exploits. These issues are what inspired us to work on a new project called “Scorecards” announced last week by the Open Source Security Foundation (OpenSSF). 

Scorecards is one of the first projects being released under the OpenSSF since its inception in August, 2020. The goal of the Scorecards project is to auto-generate a “security score” for open source projects to help users as they decide the trust, risk, and security posture for their use case. Scorecards defines an initial evaluation criteria that will be used to generate a scorecard for an open source project in a fully automated way. Every scorecard check is actionable. Some of the evaluation metrics used include a well-defined security policy, code review process, and continuous test coverage with fuzzing and static code analysis tools. A boolean is returned as well as a confidence score for each security check. Over time, Google will be improving upon these metrics with community contributions through the OpenSSF.

Check out the Security Scorecards project on GitHub and provide feedback. This is just the first step of many, and we look forward to continuing to improve open source security with the community.

By Kim Lewandowski, Dan Lorenc, and Abhishek Arya, Google Security team


Security scorecards for open source projects

When developers or organizations introduce a new open source dependency into their production software, there’s no easy indication of how secure that package is.

Some organizations—including Google—have systems and processes in place that engineers must follow when introducing a new open source dependency, but that process can be tedious, manual, and error-prone. Furthermore, many of these projects and developers are resource constrained and security often ends up a low priority on the task list. This leads to critical projects not following good security best practices and becoming vulnerable to exploits. These issues are what inspired us to work on a new project called “Scorecards” announced last week by the Open Source Security Foundation (OpenSSF). 

Scorecards is one of the first projects being released under the OpenSSF since its inception in August, 2020. The goal of the Scorecards project is to auto-generate a “security score” for open source projects to help users as they decide the trust, risk, and security posture for their use case. Scorecards defines an initial evaluation criteria that will be used to generate a scorecard for an open source project in a fully automated way. Every scorecard check is actionable. Some of the evaluation metrics used include a well-defined security policy, code review process, and continuous test coverage with fuzzing and static code analysis tools. A boolean is returned as well as a confidence score for each security check. Over time, Google will be improving upon these metrics with community contributions through the OpenSSF.

Check out the Security Scorecards project on GitHub and provide feedback. This is just the first step of many, and we look forward to continuing to improve open source security with the community.

By Kim Lewandowski, Dan Lorenc, and Abhishek Arya, Google Security team


Google joins the Open Source Security Foundation

In modern software development, much of the code developers use originates outside their organization and is open source. While the cloud and internet ecosystem depends on an open source foundation, the sheer scale and dependency chain of the libraries and packages we all use makes it difficult to validate and verify the origin of the code you’re ingesting; that it’s up to date on recent patches, and coming from projects following security best practices. To continue deriving benefits from open source, we need to ensure that as a community we are building on the strongest possible foundation. 



At Google, security is always top of mind, and we have developed robust systems and security tools—including open source ones—to protect our internal systems and our customers. We believe the more we share what we’ve learned about open source security, and the more we work with those who face similar challenges, the more we can improve the state of open source security for everyone.

We’re happy to announce that Google is joining the Open Source Security Foundation (OpenSSF) to work alongside the broader industry on this journey of improving the state of security of open source projects we all depend on. Google has key areas in open source security we want to work on, and we’re excited to share our ideas with the OpenSSF community and work together. Some of our key areas are:

Shared schemas and metadata that enable automation for enforcing security best practices along the entire software supply chain.

Dependency management and risk assessments through tooling and data. We want to make it easy to map vulnerabilities back to specific versions of code that are affected and take action.
Verifiable builds through trusted build systems so that we know artifacts haven’t been tampered with. The Tekton project has been exploring this idea, and we’re excited to share some of these ideas with OpenSSF.

A developer identity system to help associate code changes back to their original author and help code reviewers have developer authentication as part of their commit and PR review process.

Securing critical OSS projects and helping projects respond to vulnerabilities. If you’re a maintainer who’s interested in getting help with vulnerability response or security engineering efforts, watch this space!

Security challenges are never going to disappear, and we must work together to maintain the security of the open source software we collectively depend on. If you're interested in getting involved in the OpenSSF initiatives, visit openssf.org or OpenSSF on GitHub.You can be a part of how the OpenSSF serves the open source community and the world!

By Kim Lewandowski, Product Security Team, and Dan Lorenc, Infrastructure Security Team, Google