Tag Archives: scanning

Updates on the Tsunami Security Scanning Engine

Several months ago, we open sourced the Tsunami security scanner: a false-positive-free infrastructure scanning engine focusing on high severity, actively exploited vulnerabilities. Today, we are releasing the first major update for Tsunami.

In the last few months, we have done a lot of work in the background to prepare Tsunami for the next step and focused on the following:
  • Vulnerability research: In order to keep Tsunami's detection capabilities up-to-date, we kicked-off various projects to research the exploitation of vulnerabilities in the wild. We will soon publish more information about our initiatives in this space—stay tuned.
  • New detection capabilities: Based on our research, we have added 15 new detector plugins to Tsunami for actively exploited vulnerabilities.
  • Continuous Integration pipeline for our open-source builds: We set up a CI/CD pipeline that automatically mirrors and tests changes between our internal version management system and the open source repository. This will enable us to easily merge internal and external contributions.
  • Test bed for end-to-end testing: This summer we hosted an intern (Yuxin Wu), who built and open-sourced a test bed for Tsunami. The test bed can automatically deploy arbitrary versions of off-the-shelf software based on docker images. We are using the test bed to automatically check whether a Tsunami detector is working for all vulnerable versions of a software and keeps functioning for future versions.
  • Web application fingerprinting: We added Web application fingerprinting capabilities to Tsunami. Tsunami, now detects popular off-the-shelve Web applications. This information can be used by Tsunami for more precise and less intrusive vulnerability verification. Furthermore, it enables security teams to build a software inventory based on Tsunami scans. We'll keep working on refining our fingerprinting approach and extending our fingerprinting database.

Today, we are releasing the new detectors and the fingerprinting capabilities. You can find the new detectors and the web fingerprinter in our plugin repository.

If you are adopting Tsunami within your organization and if you have questions or would like to contribute, feel free to contact us at any time at [email protected].

By Guoli Ma, Claudio Criscione & Sebastian Lekies, Vulnerability Management Team

Tsunami: An extensible network scanning engine for detecting high severity vulnerabilities with high confidence

We have released the Tsunami security scanning engine to the open source communities. We hope that the engine can help other organizations protect their users’ data. We also hope to foster collaboration, and encourage the security community to create and share new detectors on top of Tsunami.

When an attacker begins to exploit security vulnerabilities or security misconfigurations, such as weak passwords, an organization needs to react quickly in order to protect potentially vulnerable assets. With attackers increasingly investing in automation, the time window to react to a newly released, high severity vulnerability is usually measured in hours. This poses a significant challenge for large organizations with thousands or even millions of internet-connected systems. In such hyperscale environments, security vulnerabilities must be detected and, ideally, remediated in a fully automated fashion. To make this possible, information security teams need to be able to roll out detectors for novel security issues at scale in a very short amount of time. Furthermore, it is important that the detection quality is consistently very high. To handle these challenges, we created Tsunami: an extensible network scanning engine for detecting high severity vulnerabilities with high confidence.

Google leverages Google's Kubernetes Engine (GKE) to continuously scan and protect all of our externally facing systems with the Tsunami scanning engine. When scanning a system, Tsunami executes a two-step process:
  1. Reconnaissance: In the first step, Tsunami detects open ports; then subsequently identifies protocols, services, and other software running on the target host using a set of fingerprinting plugins. To avoid reinventing the wheel, Tsunami leverages existing tools such as nmap for some of these tasks.
  2. Vulnerability verification: Based on the information gathered through reconnaissance, Tsunami selects all vulnerability verification plugins matching the identified services. To confirm that a vulnerability indeed exists Tsunami executes a fully working, benign exploit.
In its initial version, Tsunami ships with detectors for the following security issues:
  • Exposed sensitive UIs: Applications such as Jenkins, Jupyter, and Hadoop Yarn ship with UIs that allow a user to schedule workloads or to execute system commands. If these systems are exposed to the internet without authentication, attackers can leverage the functionality of the application to execute malicious commands.
  • Weak credentials: Tsunami uses other open source tools such as ncrack to detect weak passwords used by protocols and tools including SSH, FTP, RDP, and MySQL.
In the coming months, we plan to release many more detectors for vulnerabilities similar to remote code execution (RCE). Furthermore, we are working on several other new features that will make the engine more powerful and easier to use and extend.

In order to make contributions easy, we split our codebase into two Github Repositories:
  1. A repository for the main scanning engine
  2. A repository for Tsunami scanning plugins
If you have any questions or if you would like to contribute, don't hesitate to reach out to us.

By Guoli Ma, Claudio Criscione & Sebastian Lekies, Vulnerability Management Team

Security Crawl Maze: An Open Source Tool to Test Web Security Crawlers

Scanning modern web applications for security vulnerabilities can be a difficult task, especially if they are built with Javascript frameworks, which is why crawlers have to use a multi-stage crawling approach to discover all the resources on modern websites.

Living in the times of dynamically changing specifications and the constant appearance of new frameworks, we often have to adjust our crawlers so that they are able to discover new ways in which developers can link resources from their applications. The issue we face in such situations is measuring if changes to crawling logic improve the effectiveness. While working on replacing a crawler for a web security scanner that has been in use for a number of years, we found we needed a universal test bed, both to test our current capabilities and to discover cases that are currently missed. Inspired by Firing Range, today we’re announcing the open-source release of Security Crawl Maze – a universal test bed for web security crawlers.

Security Crawl Maze is a simple Python application built with the Flask framework that contains a wide variety of cases for ways in which a web based application can link other resources on the Web. We also provide a Dockerfile which allows you to build a docker image and deploy it to an environment of your choice. While the initial release is covering the most important cases for HTTP crawling, it’s a subset of what we want to achieve in the near future. You’ll soon be able to test whether your crawler is able to discover known files (robots.txt, sitemap.xml, etc…) or crawl modern single page applications written with the most popular JS frameworks (Angular, Polymer, etc.).

Security crawlers are mostly interested in code coverage, not in content coverage, which means the deduplication logic has to be different. This is why we plan to add cases which allow for testing if your crawler deduplicates URLs correctly (e.g. blog posts, e-commerce). If you believe there is something else, feel free to add a test case for it – it’s super simple! Code is available on GitHub and through a public deployed version.

We hope that others will find it helpful in evaluating the capabilities of their crawlers, and we certainly welcome any contributions and feedback from the broader security research community.

By Maciej Trzos, Information Security Engineer