Category Archives: Online Security Blog

The latest news and insights from Google on security and safety on the Internet

USB-C Titan Security Keys – available tomorrow in the US




Securing access to online accounts is critical for safeguarding private, financial, and other sensitive data online. Phishing - where an attacker tries to trick you into giving them your username and password - is one of the most common causes of data breaches. To protect user accounts, we’ve long made it a priority to offer users many convenient forms of 2-Step Verification (2SV), also known as two-factor authentication (2FA), in addition to Google’s automatic protections. These measures help to ensure that users are not relying solely on passwords for account security.

For users at higher risk (e.g., IT administrators, executives, politicians, activists) who need more effective protection against targeted attacks, security keys provide the strongest form of 2FA. To make this phishing-resistant security accessible to more people and businesses, we recently built this capability into Android phones, expanded the availability of Titan Security Keys to more regions (Canada, France, Japan, the UK), and extended Google’s Advanced Protection Program to the enterprise.

Starting tomorrow, you will have an additional option: Google’s new USB-C Titan Security Key, compatible with your Android, Chrome OS, macOS, and Windows devices.



USB-C Titan Security Key


We partnered with Yubico to manufacture the USB-C Titan Security Key. We have had a long-standing working and customer relationship with Yubico that began in 2012 with the collaborative effort to create the FIDO Universal 2nd Factor (U2F) standard, the first open standard to enable phishing-resistant authentication. This is the same security technology that we use at Google to protect access to internal applications and systems.

USB-C Titan Security Keys are built with a hardware secure element chip that includes firmware engineered by Google to verify the key’s integrity. This is the same secure element chip and firmware that we use in our existing USB-A/NFC and Bluetooth/NFC/USB Titan Security Key models manufactured in partnership with Feitian Technologies.

USB-C Titan Security Keys will be available tomorrow individually for $40 on the Google Store in the United States. USB-A/NFC and Bluetooth/NFC/USB Titan Security Keys will also become available individually in addition to the existing bundle. Bulk orders are available for enterprise organizations in select countries.


We highly recommend all users at a higher risk of targeted attacks to get Titan Security Keys and enroll into the Advanced Protection Program (APP), which provides Google’s industry-leading security protections to defend against evolving methods that attackers use to gain access to your accounts and data. You can also use Titan Security Keys for any site where FIDO security keys are supported for 2FA, including your personal or work Google Account, 1Password, Coinbase, Dropbox, Facebook, GitHub, Salesforce, Stripe, Twitter, and more.

No More Mixed Messages About HTTPS


Today we’re announcing that Chrome will gradually start ensuring that https:// pages can only load secure https:// subresources. In a series of steps outlined below, we’ll start blocking mixed content (insecure http:// subresources on https:// pages) by default. This change will improve user privacy and security on the web, and present a clearer browser security UX to users.
In the past several years, the web has made great progress in transitioning to HTTPS: Chrome users now spend over 90% of their browsing time on HTTPS on all major platforms. We’re now turning our attention to making sure that HTTPS configurations across the web are secure and up-to-date.
HTTPS pages commonly suffer from a problem called mixed content, where subresources on the page are loaded insecurely over http://. Browsers block many types of mixed content by default, like scripts and iframes, but images, audio, and video are still allowed to load, which threatens users’ privacy and security. For example, an attacker could tamper with a mixed image of a stock chart to mislead investors, or inject a tracking cookie into a mixed resource load. Loading mixed content also leads to a confusing browser security UX, where the page is presented as neither secure nor insecure but somewhere in between.
In a series of steps starting in Chrome 79, Chrome will gradually move to blocking all mixed content by default. To minimize breakage, we will autoupgrade mixed resources to https://, so sites will continue to work if their subresources are already available over https://. Users will be able to enable a setting to opt out of mixed content blocking on particular websites, and below we’ll describe the resources available to developers to help them find and fix mixed content.
Timeline
Instead of blocking all mixed content all at once, we’ll be rolling out this change in a series of steps.
  • In Chrome 79, releasing to stable channel in December 2019, we’ll introduce a new setting to unblock mixed content on specific sites. This setting will apply to mixed scripts, iframes, and other types of content that Chrome currently blocks by default. Users can toggle this setting by clicking the lock icon on any https:// page and clicking Site Settings. This will replace the shield icon that shows up at the right side of the omnibox for unblocking mixed content in previous versions of desktop Chrome.


Accessing Site settings, from which users will be able to unblock mixed content loads in Chrome 79.
    • In Chrome 80, mixed audio and video resources will be autoupgraded to https://, and Chrome will block them by default if they fail to load over https://. Chrome 80 will be released to early release channels in January 2020. Users can unblock affected audio and video resources with the setting described above.
    • Also in Chrome 80, mixed images will still be allowed to load, but they will cause Chrome to show a “Not Secure” chip in the omnibox. We anticipate that this is a clearer security UI for users and that it will motivate websites to migrate their images to HTTPS. Developers can use the upgrade-insecure-requests or block-all-mixed-content Content Security Policy directives to avoid this warning. 

    Omnibox treatment for websites that load mixed images in Chrome 80. 

    • In Chrome 81, mixed images will be autoupgraded to https://, and Chrome will block them by default if they fail to load over https://. Chrome 81 will be released to early release channels in February 2020.
    Resources for developers
    Developers should migrate their mixed content to https:// immediately to avoid warnings and breakage. Here are some resources:
    • Use Content Security Policy and Lighthouse’s mixed content audit to discover and fix mixed content on your site.
    • See this guide for general advice on migrating servers to HTTPS.
    • Check with your CDN, web host, or content management system to see if they have special tools for debugging mixed content. For example, Cloudflare offers a tool to rewrite mixed content to https://, and WordPress plugins are available as well.

    Chrome UI for Deprecating Legacy TLS Versions


    [Cross-posted from the Chromium blog]

    Last October we announced our plans to remove support for TLS 1.0 and 1.1 in Chrome 81. In this post we’re announcing a pre-removal phase in which we’ll introduce a gentler warning UI, and previewing the UI that we’ll use to block TLS 1.0 and 1.1 in Chrome 81. Site administrators should immediately enable TLS 1.2 or later to avoid these UI treatments.
    While legacy TLS usage has decreased, we still see over 0.5% of page loads using these deprecated versions. To ease the transition to the final removal of support and to reduce user surprise when outdated configurations stop working, Chrome will discontinue support in two steps: first, showing new security indicators for sites using these deprecated versions; and second, blocking connections to these sites with a full page warning.
    Pre-removal warning
    Starting January 13, 2020, for Chrome 79 and higher, we will show a “Not Secure” indicator for sites using TLS 1.0 or 1.1 to alert users to the outdated configuration:
    The new security indicator and connection security information that will be shown to users who visit a site using TLS 1.0 or 1.1 starting in January 2020.

    When a site uses TLS 1.0 or 1.1, Chrome will downgrade the security indicator and show a more detailed warning message inside Page Info. This change will not block users from visiting or using the page, but will alert them to the downgraded security of the connection.
    Note that Chrome already shows warnings in DevTools to alert site owners that they are using a deprecated version of TLS.
    Removal UI
    In Chrome 81, which will be released to the Stable channel in March 2020, we will begin blocking connections to sites using TLS 1.0 or 1.1, showing a full page interstitial warning:
    The full screen interstitial warning that will be shown to users who visit a site using TLS 1.0 or 1.1 starting in Chrome 81. Final warning subject to change.

    Site administrators should immediately enable TLS 1.2 or later. Depending on server software (such as Apache or nginx), this may be a configuration change or a software update. Additionally, we encourage all sites to revisit their TLS configuration. In our original announcement, we outlined our current criteria for modern TLS.
    Enterprise deployments can preview the final removal of TLS 1.0 and 1.1 by setting the SSLVersionMin policy to “tls1.2”. This will prevent clients from connecting over these protocol versions. For enterprise deployments that need more time, this same policy can be used to re-enable TLS 1.0 or TLS 1.1 and disable the warning UIs until January 2021.

    How Google adopted BeyondCorp: Part 3 (tiered access)




    Intro 

    This is the third post in a series of four, in which we set out to revisit various BeyondCorp topics and share lessons that were learnt along the internal implementation path at Google.

    The first post in this series focused on providing necessary context for how Google adopted BeyondCorp, Google’s implementation of the zero trust security model. The second post focused on managing devices - how we decide whether or not a device should be trusted and why that distinction is necessary. This post introduces the concept of tiered access, its importance, how we implemented it, and how we addressed associated troubleshooting challenges.

    High level architecture for BeyondCorp

    What is Tiered Access?

    In a traditional client certificate system, certificates are only given to trusted devices. Google used this approach initially as it dramatically simplified device trust. With such a system, any device with a valid certificate can be trusted. At predefined intervals, clients prove they can be trusted and a new certificate is issued. It’s typically a lightweight process and many off-the-shelf products exist to implement flows that adhere to this principle.

    However, there are a number of challenges with this setup:
    • Not all devices need the same level of security hardening (e.g. non-standard issue devices, older platforms required for testing, BYOD, etc.).
    • These systems don’t easily allow for nuanced access based on shifting security posture.
    • These systems tend to evaluate a device based on a single set of criteria, regardless of whether devices require access to highly sensitive data (e.g. corporate financials) or far less sensitive data (e.g. a dashboard displayed in a public space).
    The next challenge introduced by traditional systems is the inherent requirement that a device must meet your security requirements before it can get a certificate. This sounds reasonable on paper, but it unfortunately means that existing certificate infrastructure can’t be used to aid device provisioning. This implies you must have an additional infrastructure to bootstrap a device into a trusted state.

    The most significant challenge is the large amount of time in between trust evaluations. If you only install a new certificate once a year, this means it might take an entire year before you are able to recertify a device. Therefore, any new requirements you wish to add to the fleet may take up to a year before they are fully in effect. On the other hand, if you require certificates to be installed monthly or daily, you have placed a significant burden on your users and/or support staff, as they are forced to go through the certification issuance process far more often, which can be time consuming and frustrating. Additionally, if a device is found to be out of compliance with security policy, the only option is to remove all access by revoking the certificate, rather than degrading access, which can create a frustrating all-or-nothing situation for the user.

    Tiered access attempts to address all these challenges, which is why we decided to adopt it. In this new model, certificates are simply used to provide the device’s identity, instead of acting as proof of trust. Trust decisions are then made by a separate system which can be modified without interfering with the certificate issuance process or validity. Moving the trust evaluation out-of-band from the certificate issuance allows us to circumvent the challenges identified above in the traditional system. Below are three ways in which tiered access helps address these concerns.

    Different access levels for different security states

    By separating trust from identity, we can define infinite levels of trust, if we so desired. At any point in time, we can define a new trust level, or adjust existing trust level requirements, and reevaluate a device's compliance. This is the heart of the tiered access system. It provides us the flexibility to define different device trust criteria for low sensitivity applications from those used for high trusted applications.

    Solving the bootstrapping challenge

    Multiple trust states enable us to use the system to initiate an OS installation. We can now allow access to bootstrapping (configuration and patch management) services based solely on whether we own the device. This enables provisioning to occur from untrusted networks allowing us to replace the traditional IP-based checks.



    Configurable frequency of trust evaluations


    The frequency of device trust evaluation is independent from certificate issuance in a tiered access setup. This means you can evaluate trust as often as you feel necessary. Changes to trust definitions can be immediately reflected across the entire fleet. Changes to device posture can similarly immediately impact trust.

    We should note that the system’s ability to quickly remove trust from devices can be a double edged sword. If there are bugs in the trust definitions or evaluations themselves, this can also quickly remove trust from ‘good’ devices. You must have the ability to adequately test policy changes to mitigate the blast radius from these types of bugs, and ideally canary changes to subsets of the fleet for a baking period. Constant monitoring is also critical. A bug in your trust evaluation system could cause it to start mis-evaluating trust. It’s wise to add alarms if the system starts dropping (or raising) the trust of too many machines at once. The troubleshooting section below provides additional techniques to help minimize the impact of misconfigured trust logic.

    How did we define access tiers?

    The basic concept of tiers is relatively straightforward: access to data increases as the device security hardening increases. These tiers are useful for coarse grain access control of client devices, which we have found to be sufficient in most cases. At Google, we allow the user to choose the device tier that allows them to weigh access needs with security requirements and policy. If a user needs access to more corporate data, they may have to accept more device configuration restrictions. If a user wants more control over their device and less restrictions but don’t need access to higher risk resources, they can choose a tier with less access to corporate data. For more information about the properties of a trusted platform you can measure, visit our paper about Maintaining a Healthy Fleet.

    We knew this model would work in principle, but we didn’t know how many access tiers we should define. As described above, the old model only had two tiers: Trusted and Untrusted. We knew we wanted more than that to enable trust build up at the very least, but we didn’t know the ideal number. More tiers allow access control lists to be specified with greater fidelity at the cost of confusion for service owners, security engineers, and the wider employee base alike.

    At Google, we initially supported four distinct tiers ranging from Untrusted to Highly-Privileged Access. The extremes are easy to understand: Untrusted devices should only access data that is already public while Highly-Privileged Access devices have greater privilege internally. The middle two tiers allowed system owners to design their systems with the tiered access model in mind. Certain sensitive actions required a Highly-Privileged Access device while less sensitive portions of the system could be reached with less trusted devices. This degraded access model sounded great to us security wonks. Unfortunately, employees were unable to determine what tier they should choose to ensure they could access all the systems they needed. In the end, we determined that the extra middle tier led to intense confusion without much benefit.

    In our current model, the vast majority of devices fit in one of three distinct tiers: Untrusted, Basic Access, and Highly-Privileged Access. In this model, system owners are required to choose the more trusted path if their system is more sensitive. This requirement does limit the finesse of the system but greatly reduces employee confusion and was key to a successful adoption.

    In addition to tiers, our system is able to provide additional context to access gateways and underlying applications and services. This additional information is useful to provide finer grained, device-based access control. Imposing additional device restrictions on highly sensitive systems, in addition to checking the coarse grain tier, is a reasonable way to balance security vs user expectations. Because highly sensitive systems are only used by a smaller subset of the employee population, based on role and need, these additional restrictions typically aren’t a source of user confusion. With that in mind, please note that this article only covers device-based controls and does not address fine-grained controls based on a user’s identity.

    At the other end of the spectrum, we have OS installation/remediation services. These systems are required in order to support bootstrapping a device which by design does not yet adhere to the Basic Access tier. As described earlier, we use our certificates as a device identity, not trust validation. In the OS installation case, no reported data exists, but we can make access decisions based on the inventory data associated with that device identity. This allows us to ensure our OS and security agents are only installed on devices we own and expect to be in use. Once the OS and security agents are up and running, we can use them to lock down the device and prove it is in a state worthy of more trust.

    How did we create rules to implement the tiers?

    Device-based data is the heart of BeyondCorp and tiered access. We evaluate trust tiers using data about each device at Google to determine its security integrity and tier level. To obtain this data, we built an inventory pipeline which aggregates data from various sources of authority within our enterprise to obtain a holistic, comprehensive view of a device's security posture. For example, we gather prescribed company asset inventory in one service and observed data reported by agents on the devices in other services. All of this data is used to determine which tier a device belongs in, and trust tiers are reevaluated every time corporate data is changed or new data is reported.

    Trust level evaluations are made via "rules", written by security and systems engineers. For example, for a device to have basic access, we have a rule that checks that it is running an approved operating system build and version. For that same device to have highly-privileged access, it would need to pass several additional rules, such as checking the device is encrypted and contains the latest security patches. Rules exist in a hierarchical structure, so several rules can combine to create a tier. Requirements for tiers across device platforms can be different, so there is a separate hierarchy for each. Security engineers work closely with systems engineers to determine the necessary information to protect devices, such as determining thresholds for required minimum version and security patch frequency.

    Rule Enforcement and User Experience

    To create a good user experience, rules are created and monitored before being enforced. For example, before requiring all users to upgrade their Chrome browser, we monitor how many users will drop trust if that rule was enforced. Dashboards track rule impact on Googlers over 30 day periods. This enables security and systems teams to evaluate rule change impact before they affect end users.

    To further protect employee experience, we have measures called grace periods and exceptions. Grace periods provide windows of a predefined duration where devices can violate rules but still maintain trust and access, providing a fallback in case of unexpected consequences. Furthermore, grace periods can be implemented quickly and easily across the fleet in case for disaster recovery purposes. The other mechanism is called exceptions. Exceptions allow rule authors to create rules for the majority while enabling security engineers to make nuanced decisions around individual riskier processes. For example, if we have a team of Android developers specializing on user experience for an older Android version, they may be granted an exception for the minimum version rule.

    How did we simplify troubleshooting?

    Troubleshooting access issues proves challenging in a system where many pieces of data interact to create trust. We tackle this issue in two ways. First, we have a system to provide succinct and actionable explanations to end users on how to resolve problems on their own. Second, we have the capability to notify users when their devices have lost trust or are about to lose trust. The combination of these efforts improves the user experience of the tiered access solution and reduces toil for those supporting it.

    We are able to provide self-service feedback to users by closely integrating the creation of rule policy with resolution steps for that policy. In other words, security engineers who write rule policies are also responsible for attaching steps on how to resolve the issue. To further aid users, the rule evaluation system provides details about the specific pieces of data causing the failure. All this information is fed into a centralized system that generates user-friendly explanations, guiding users to self-diagnose and fix problems without the need for IT support. Likewise, a tech may not be able to see pieces of PII about a user when helping fix the device. These cases are rare but necessary to protect the parties involved in these scenarios. Having one centralized debugging system helps deal with all these nuances, enabling us to provide detailed and safe explanations to end users in accordance with their needs.

    Remediation steps are communicated to users in several ways. Before a device loses trust, notification pop-ups appear to the user explaining that a loss of access is imminent. These pop-ups contain directions to the remediation system so the user can self-diagnose and fix the problem. This circumvents user pain by offering solutions before the problem impacts the user. Premeditated notifications work in conjunction with the aforementioned grace periods, as we provide a window in which users can fix their devices. If the issue is not fixed and the device goes out of compliance, there is still a clear path on what to do. For example, when a user attempts to access a resource for which they do not have permission, a link appears on the access denied page directing them to the relevant remediation steps. This provides fast, clear feedback on how to fix their device and reduces toil on the IT support teams.

    Next time

    In the next and final post in this series, we will discuss how we migrated services to be protected by the BeyondCorp architecture at Google.

    In the meantime, if you want to learn more, you can check out the BeyondCorp research papers. In addition, getting started with BeyondCorp is now easier using zero trust solutions from Google Cloud (context-aware access) and other enterprise providers.

    Thank you to the editors of the BeyondCorp blog post series, Puneet Goel (Product Manager), Lior Tishbi (Program Manager), and Justin McWilliams (Engineering Manager).


    Trust but verify attestation with revocation

    Posted by Rob Barnes & Shawn Willden, Android Security & Privacy Team
    [Cross-posted from the Android Developers Blog]

    Billions of people rely on their Android-powered devices to securely store their sensitive information. A vital component of the Android security stack is the key attestation system. Android devices since Android 7.0 are able to generate an attestation certificate that attests to the security properties of the device’s hardware and software. OEMs producing devices with Android 8.0 or higher must install a batch attestation key provided by Google on each device at the time of manufacturing.
    These keys might need to be revoked for a number of reasons including accidental disclosure, mishandling, or suspected extraction by an attacker. When this occurs, the affected keys must be immediately revoked to protect users. The security of any Public-Key Infrastructure system depends on the robustness of the key revocation process.
    All of the attestation keys issued so far include an extension that embeds a certificate revocation list (CRL) URL in the certificate. We found that the CRL (and online certificate status protocol) system was not flexible enough for our needs. So we set out to replace the revocation system for Android attestation keys with something that is flexible and simple to maintain and use.
    Our solution is a single TLS-secured URL (https://android.googleapis.com/attestation/status) that returns a list containing all revoked Android attestation keys. This list is encoded in JSON and follows a strict format defined by JSON schema. Only keys that have non-valid status appear in the list, so it is not an exhaustive list of all issued keys.
    This system allows us to express more nuance about the status of a key and the reason for the status. A key can have a status of REVOKED or SUSPENDED, where revoked is permanent and suspended is temporary. The reason for the status is described as either KEY_COMPROMISE, CA_COMPROMISE, SUPERSEDED, or SOFTWARE_FLAW. A complete, up-to-date list of statuses and reasons can be found in the developer documentation.
    The CRL URLs embedded in existing batch certificates will continue to operate. Going forward, attestation batch certificates will no longer contain a CRL extension. The status of these legacy certificates will also be included in the attestation status list, so developers can safely switch to using the attestation status list for both current and legacy certificates. An example of how to correctly verify Android attestation keys is included in the Key Attestation sample.

    Expanding bug bounties on Google Play

    Posted by Adam Bacchus, Sebastian Porst, and Patrick Mutchler — Android Security & Privacy

    [Cross-posted from the Android Developers Blog]

    We’re constantly looking for ways to further improve the security and privacy of our products, and the ecosystems they support. At Google, we understand the strength of open platforms and ecosystems, and that the best ideas don’t always come from within. It is for this reason that we offer a broad range of vulnerability reward programs, encouraging the community to help us improve security for everyone. Today, we’re expanding on those efforts with some big changes to Google Play Security Reward Program (GPSRP), as well as the launch of the new Developer Data Protection Reward Program (DDPRP).

    Google Play Security Reward Program Scope Increases

    We are increasing the scope of GPSRP to include all apps in Google Play with 100 million or more installs. These apps are now eligible for rewards, even if the app developers don’t have their own vulnerability disclosure or bug bounty program. In these scenarios, Google helps responsibly disclose identified vulnerabilities to the affected app developer. This opens the door for security researchers to help hundreds of organizations identify and fix vulnerabilities in their apps. If the developers already have their own programs, researchers can collect rewards directly from them on top of the rewards from Google. We encourage app developers to start their own vulnerability disclosure or bug bounty program to work directly with the security researcher community.

    Vulnerability data from GPSRP helps Google create automated checks that scan all apps available in Google Play for similar vulnerabilities. Affected app developers are notified through the Play Console as part of the App Security Improvement (ASI) program, which provides information on the vulnerability and how to fix it. Over its lifetime, ASI has helped more than 300,000 developers fix more than 1,000,000 apps on Google Play. In 2018 alone, the program helped over 30,000 developers fix over 75,000 apps. The downstream effect means that those 75,000 vulnerable apps are not distributed to users until the issue is fixed.

    To date, GPSRP has paid out over $265,000 in bounties. Recent scope and reward increases have resulted in $75,500 in rewards across July & August alone. With these changes, we anticipate even further engagement from the security research community to bolster the success of the program.

    Introducing the Developer Data Protection Reward Program

    Today, we are also launching the Developer Data Protection Reward Program. DDPRP is a bounty program, in collaboration with HackerOne, meant to identify and mitigate data abuse issues in Android apps, OAuth projects, and Chrome extensions. It recognizes the contributions of individuals who help report apps that are violating Google Play, Google API, or Google Chrome Web Store Extensions program policies.

    The program aims to reward anyone who can provide verifiably and unambiguous evidence of data abuse, in a similar model as Google’s other vulnerability reward programs. In particular, the program aims to identify situations where user data is being used or sold unexpectedly, or repurposed in an illegitimate way without user consent. If data abuse is identified related to an app or Chrome extension, that app or extension will accordingly be removed from Google Play or Google Chrome Web Store. In the case of an app developer abusing access to Gmail restricted scopes, their API access will be removed. While no reward table or maximum reward is listed at this time, depending on impact, a single report could net as large as a $50,000 bounty.

    As 2019 continues, we look forward to seeing what researchers find next. Thank you to the entire community for contributing to keeping our platforms and ecosystems safe. Happy bug hunting!

    Protecting Chrome users in Kazakhstan



    When making secure connections, Chrome trusts certificates that have been locally installed on a user's computer or mobile device. This allows users to run tools to inspect and debug connections during website development, or for corporate environments to intercept and monitor internal traffic. It is not appropriate for this mechanism to be used to intercept traffic on the public internet.

    In response to recent actions by the Kazakhstan government, Chrome, along with other browsers, has taken steps to protect users from the interception or modification of TLS connections made to websites.

    Chrome will be blocking the certificate the Kazakhstan government required users to install:

    Common Name
    Qaznet Trust Network
    SHA-256 Fingerprint
    00:30:9C:73:6D:D6:61:DA:6F:1E:B2:41:73:AA:84:99:44:C1:68:A4:3A:15:
    BF:FD:19:2E:EC:FD:B6:F8:DB:D2
    SHA-256 of Subject Public Key Info
    B5:BA:8D:D7:F8:95:64:C2:88:9D:3D:64:53:C8:49:98:C7:78:24:91:9B:64:
    EA:08:35:AA:62:98:65:91:BE:50


    The certificate has been added to CRLSet. No action is needed by users to be protected. In addition, the certificate will be added to a blocklist in the Chromium source code and thus should be included in other Chromium based browsers in due course.

    How Google adopted BeyondCorp: Part 2 (devices)




    Intro

    This is the second post in a series of four, in which we set out to revisit various BeyondCorp topics and share lessons that were learnt along the internal implementation path at Google.

    The first post in this series focused on providing necessary context for how Google adopted BeyondCorp. This post will focus on managing devices - how we decide whether or not a device should be trusted and why that distinction is necessary. Device management provides both the data and guarantees required for making access decisions by securing the endpoints and providing additional context about it.


    How do we manage devices?

    At Google, we use the following principles to run our device fleet securely and at scale:
    • Secure default settings at depth with central enforcement
    • Ensure a scalable process
    • Invest in fleet testing, monitoring, and phased rollouts
    • Ensure high quality data
    Secure default settings

    Defense in depth requires us to layer our security defenses such that an attacker would need to pass multiple controls in an attack. To uphold this defensive position at scale, we centrally manage and measure various qualities of our devices, covering all layers of the platform;

    • Hardware/firmware configuration
    • Operating system and software
    • User settings and modifications
    We use automated configuration management systems to continuously enforce our security and compliance policies. Independently, we observe the state of our hardware and software. This allows us to determine divergence from the expected state and verify whether it is an anomaly.

    Where possible, our platforms use native OS capabilities to protect against malicious software, and we extend those capabilities across our platforms with custom and commercial tooling.


    Scalable process

    Google manages a fleet of several hundred thousand client devices (workstations, laptops, mobile devices) for employees who are spread across the world. We scale the engineering teams who manage these devices by relying on reviewable, repeatable, and automated backend processes and minimizing GUI-based configuration tools. By using and developing open-source software and integrating it with internal solutions, we reach a level of flexibility that allows us to manage fleets at scale without sacrificing customizability for our users. The focus is on operating system agnostic server and client solutions, where possible, to avoid duplication of effort.

    Software for all platforms is provided by repositories which verify the integrity of software packages before making them available to users. The same system is used for distributing configuration settings and management tools, which enforce policies on client systems using the open-source configuration management system Puppet, running in standalone mode. In combination, this allows us to easily scale infrastructure and management horizontally as described in more detail and with examples in one of our BeyondCorp whitepapers, Fleet Management at Scale.

    All device management policies are stored in centralized systems which allow settings to be applied both at the fleet and the individual device level. This way policy owners and device owners can manage sensible defaults or per-device overrides in the same system, allowing audits of settings and exceptions. Depending on the type of exception, they may either be managed self-service by the user, require approval from appropriate parties, or affect the trust level of the affected device. This way, we aim to guarantee user satisfaction and security simultaneously.


    Fleet testing, monitoring, and phased rollouts

    Applying changes at scale to a large heterogeneous fleet can be challenging. At Google, we have automated test labs which allow us to test changes before we deploy them to the fleet. Rollouts to the client fleet usually follow multiple stages and random canarying, similar to common practices with service management. Furthermore, we monitor various status attributes of our fleet which allows us to detect issues before they spread widely.

    High quality data

    Device management depends on the quality of device data. Both configuration and trust decisions are keyed off of inventory information. At Google, we track all devices in centralized asset management systems. This allows us to not only observe the current (runtime) state of a device, but also whether it’s a legitimate Google device. These systems store hardware attributes as well as the assignment and status of devices, which lets us match and compare prescribed values to those which are observed.

    Prior to implementing BeyondCorp, we performed a fleet-wide audit to ensure the quality of inventory data, and we perform smaller audits regularly across the fleet. Automation is key to achieving this, both for entering data initially and for detecting divergence at later points. For example, instead of having a human enter data into the system manually, we use digital manifests and barcode scanners as much as possible.


    How do we figure out whether devices are trustworthy?

    After appropriate management systems have been put in place, and data quality goals have been met, the pertinent security information related to a device can be used to establish a "trust" decision as to whether a given action should be allowed to be performed from the device.



    High level architecture for BeyondCorp


    This decision can be most effectively made when an abundance of information about the device is readily available. At Google, we use an aggregated data pipeline to gather information from various sources, which each contain a limited subset of knowledge about a device and its history, and make this data available at the point when a trust decision is being made.

    Various systems and repositories are employed within Google to perform collection and storage of device data that is relevant to security. These include tools like asset management repositories, device management solutions, vulnerability scanners, and internal directory services, which contain information and state about the multitude of physical device types (e.g., desktops, laptops, phones, tablets), as well as virtual desktops, used by employees at the company.

    Having data from these various types of information systems available when making a trust decision for a given device can certainly be advantageous. However, challenges can present themselves when attempting to correlate records from a diverse set of systems which may not have a clear, consistent way to reference the identity of a given device. The challenge of implementation has been offset by the gains in security policy flexibility and improvements in securing our data.


    What lessons did we learn?
    As we rolled out BeyondCorp, we iteratively improved our fleet management and inventory processes as outlined above. These improvements are based on various lessons we learned around data quality challenges.

    Audit your data ahead of implementing BeyondCorp

    Data quality issues and inaccuracies are almost certain to be present in an asset management system of any substantial size, and these issues must be corrected before the data can be utilized in a manner which will have a significant impact on user experience. Having the means to compare values that have been manually entered into such systems against similar data that has been collected from devices via automation can allow for the correction of discrepancies, which may interrupt the intended behavior of the system.


    Prepare to encounter unforeseen data quality challenges

    Numerous data incorrectness scenarios and challenging issues are likely to present themselves as the reliance on accurate data increases. For example, be prepared to encounter issues with data ingestion processes that rely on transcribing device identifier information, which is physically labeled on devices or their packaging, and may incorrectly differ from identifier data that is digitally imprinted on the device.

    In addition, over reliance on the assumed uniqueness of certain device identifiers can sometimes be problematic in the rare cases where conventionally unique attributes, like serial numbers, can appear more than once in the device fleet (this can be especially exacerbated in the case of virtual desktops, where such identifiers may be chosen by a user without regard for such concerns).

    Lastly, routine maintenance and hardware replacements performed on employee devices can result in ambiguous situations with regards to the "identity" of a device. When internal device components, like network adapters or mainboards, are found to be defective and replaced, the device's identity can be changed into a state which no longer matches the known inventory data if care is not taken to correctly reflect such changes. 


    Implement controls to maintain high quality asset inventory

    After inventory data has been brought to an acceptable correctness level, mechanisms should be put into place to limit the ability for new inaccuracies to be introduced. For example, at Google, data correctness checks have been integrated into the provisioning process for new devices so that inventory records must be correct before a device can be successfully imaged with an operating system, ensuring that the device will meet required data accuracy standards before being delivered to an employee.

    Next time
    In the next post in this series, we will discuss a tiered access approach, how to create rule-based trust and the lessons we’ve learned through that process.

    In the meantime, if you want to learn more, you can check out the BeyondCorp research papers. In addition, getting started with BeyondCorp is now easier using zero trust solutions from Google Cloud (context-aware access) and other enterprise providers.

    Thank you to the editors of the BeyondCorp blog post series, Puneet Goel (Product Manager), Lior Tishbi (Program Manager), and Justin McWilliams (Engineering Manager).

    New Research: Lessons from Password Checkup in action



    Back in February, we announced the Password Checkup extension for Chrome to help keep all your online accounts safe from hijacking. The extension displays a warning whenever you sign in to a site using one of over 4 billion usernames and passwords that Google knows to be unsafe due to a third-party data breach. Since our launch, over 650,000 people have participated in our early experiment. In the first month alone, we scanned 21 million usernames and passwords and flagged over 316,000 as unsafe---1.5% of sign-ins scanned by the extension.
    Today, we are sharing our most recent lessons from the launch and announcing an updated set of features for the Password Checkup extension. Our full research study, available here, will be presented this week as part of the USENIX Security Symposium.

    Which accounts are most at risk?

    Hijackers routinely attempt to sign in to sites across the web with every credential exposed by a third-party breach. If you use strong, unique passwords for all your accounts, this risk disappears. Based on anonymous telemetry reported by the Password Checkup extension, we found that users reused breached, unsafe credentials for some of their most sensitive financial, government, and email accounts. This risk was even more prevalent on shopping sites (where users may save credit card details), news, and entertainment sites.

    In fact, outside the most popular web sites, users are 2.5X more likely to reuse vulnerable passwords, putting their account at risk of hijacking.
    Anonymous telemetry reported by Password Checkup extension shows that users most often reuse vulnerable passwords on shopping, news, and entertainment sites.


    Helping users re-secure their unsafe passwords

    Our research shows that users opt to reset 26% of the unsafe passwords flagged by the Password Checkup extension. Even better, 60% of new passwords are secure against guessing attacks—meaning it would take an attacker over a hundred million guesses before identifying the new password.
    Improving the Password Checkup extension

    Today, we are also releasing two new features for the Password Checkup extension. The first is a direct feedback mechanism where users can inform us about any issues that they are facing via a quick comment box. The second gives users even more control over their data. It allows users to opt-out of the anonymous telemetry that the extension reports, including the number of lookups that surface an unsafe credential, whether an alert leads to a password change, and the domain involved for improving site coverage. By design, the Password Checkup extension ensures that Google never learns your username or password, regardless of whether you enable telemetry, but we still want to provide this option if users would prefer not to share this information.


    We're continuing to improve the Password Checkup extension and exploring ways to implement its technology into Google products. For help keeping all your online accounts safe from hijacking, you can install the Password Checkup extension here today.

    Making authentication even easier with FIDO2-based local user verification for Google Accounts


    Passwords, combined with Google's automated protections, help secure billions of users around the world. But, new security technologies are surpassing passwords in terms of both strength and convenience. With this in mind, we are happy to announce that you can verify your identity by using your fingerprint or screen lock instead of a password when visiting certain Google services. The feature is available today on Pixel devices and coming to all Android 7+ devices over the next few days.



    Simpler authentication experience when viewing your saved password for a website on passwords.google.com


    These enhancements are built using the FIDO2 standards, W3C WebAuthn and FIDO CTAP, and are designed to provide simpler and more secure authentication experiences. They are a result of years of collaboration between Google and many other organizations in the FIDO Alliance and the W3C.

    An important benefit of using FIDO2 versus interacting with the native fingerprint APIs on Android is that these biometric capabilities are now, for the first time, available on the web, allowing the same credentials be used by both native apps and web services. This means that a user only has to register their fingerprint with a service once and then the fingerprint will work for both the native application and the web service.

    Note that your fingerprint is never sent to Google’s servers - it is securely stored on your device, and only a cryptographic proof that you’ve correctly scanned it is sent to Google’s servers. This is a fundamental part of the FIDO2 design.

    Here is how it works

    Google is using the FIDO2 capability on Android to register a platform-bound FIDO credential. We remember the credential for that specific Android device. Now, when the user visits a compatible service, such as passwords.google.com, we issue a WebAuthn “Get” call, passing in the credentialId that we got when creating the credential. The result is a valid FIDO2 signature.


    High-level architecture of using fingerprint or screen lock on Android devices to verify a user’s identity without a password

    Please follow the instructions below if you’d like to try it out.
    Prerequisites
    • Phone is running Android 7.0 (Nougat) or later
    • Your personal Google Account is added to your Android device
    • Valid screen lock is set up on your Android device
    To try it
    • Open the Chrome app on your Android device
    • Navigate to https://passwords.google.com
    • Choose a site to view or manage a saved password
    • Follow the instructions to confirm that it’s you trying signing in
    You can find more detailed instructions here.

    For additional security
    Remember, Google's automated defenses securely block the overwhelming majority of sign-in attempts even if an attacker has your username or password. Further, you can protect your accounts with two-step verification (2SV), including Titan Security Keys and Android phone’s built-in security key.

    Both security keys and local user verification based on biometrics use the FIDO2 standards. However, these two protections address different use cases. Security keys are used for bootstrapping a new device as a second factor as part of 2SV in order to make sure it’s the right owner of the account accessing it. Local user verification based on biometrics comes after bootstrapping a device and can be used for re-authentication during step-up flows to verify the identity of the already signed-in user.

    What’s next
    This new capability marks another step on our journey to making authentication safer and easier for everyone to use. As we continue to embrace the FIDO2 standard, you will start seeing more places where local alternatives to passwords are accepted as an authentication mechanism for Google and Google Cloud services. Check out this presentation to get an early glimpse of the use cases that we are working to enable next.