How Google adopted BeyondCorp: Part 3 (tiered access)




Intro 

This is the third post in a series of four, in which we set out to revisit various BeyondCorp topics and share lessons that were learnt along the internal implementation path at Google.

The first post in this series focused on providing necessary context for how Google adopted BeyondCorp, Google’s implementation of the zero trust security model. The second post focused on managing devices - how we decide whether or not a device should be trusted and why that distinction is necessary. This post introduces the concept of tiered access, its importance, how we implemented it, and how we addressed associated troubleshooting challenges.

High level architecture for BeyondCorp

What is Tiered Access?

In a traditional client certificate system, certificates are only given to trusted devices. Google used this approach initially as it dramatically simplified device trust. With such a system, any device with a valid certificate can be trusted. At predefined intervals, clients prove they can be trusted and a new certificate is issued. It’s typically a lightweight process and many off-the-shelf products exist to implement flows that adhere to this principle.

However, there are a number of challenges with this setup:
  • Not all devices need the same level of security hardening (e.g. non-standard issue devices, older platforms required for testing, BYOD, etc.).
  • These systems don’t easily allow for nuanced access based on shifting security posture.
  • These systems tend to evaluate a device based on a single set of criteria, regardless of whether devices require access to highly sensitive data (e.g. corporate financials) or far less sensitive data (e.g. a dashboard displayed in a public space).
The next challenge introduced by traditional systems is the inherent requirement that a device must meet your security requirements before it can get a certificate. This sounds reasonable on paper, but it unfortunately means that existing certificate infrastructure can’t be used to aid device provisioning. This implies you must have an additional infrastructure to bootstrap a device into a trusted state.

The most significant challenge is the large amount of time in between trust evaluations. If you only install a new certificate once a year, this means it might take an entire year before you are able to recertify a device. Therefore, any new requirements you wish to add to the fleet may take up to a year before they are fully in effect. On the other hand, if you require certificates to be installed monthly or daily, you have placed a significant burden on your users and/or support staff, as they are forced to go through the certification issuance process far more often, which can be time consuming and frustrating. Additionally, if a device is found to be out of compliance with security policy, the only option is to remove all access by revoking the certificate, rather than degrading access, which can create a frustrating all-or-nothing situation for the user.

Tiered access attempts to address all these challenges, which is why we decided to adopt it. In this new model, certificates are simply used to provide the device’s identity, instead of acting as proof of trust. Trust decisions are then made by a separate system which can be modified without interfering with the certificate issuance process or validity. Moving the trust evaluation out-of-band from the certificate issuance allows us to circumvent the challenges identified above in the traditional system. Below are three ways in which tiered access helps address these concerns.

Different access levels for different security states

By separating trust from identity, we can define infinite levels of trust, if we so desired. At any point in time, we can define a new trust level, or adjust existing trust level requirements, and reevaluate a device's compliance. This is the heart of the tiered access system. It provides us the flexibility to define different device trust criteria for low sensitivity applications from those used for high trusted applications.

Solving the bootstrapping challenge

Multiple trust states enable us to use the system to initiate an OS installation. We can now allow access to bootstrapping (configuration and patch management) services based solely on whether we own the device. This enables provisioning to occur from untrusted networks allowing us to replace the traditional IP-based checks.



Configurable frequency of trust evaluations


The frequency of device trust evaluation is independent from certificate issuance in a tiered access setup. This means you can evaluate trust as often as you feel necessary. Changes to trust definitions can be immediately reflected across the entire fleet. Changes to device posture can similarly immediately impact trust.

We should note that the system’s ability to quickly remove trust from devices can be a double edged sword. If there are bugs in the trust definitions or evaluations themselves, this can also quickly remove trust from ‘good’ devices. You must have the ability to adequately test policy changes to mitigate the blast radius from these types of bugs, and ideally canary changes to subsets of the fleet for a baking period. Constant monitoring is also critical. A bug in your trust evaluation system could cause it to start mis-evaluating trust. It’s wise to add alarms if the system starts dropping (or raising) the trust of too many machines at once. The troubleshooting section below provides additional techniques to help minimize the impact of misconfigured trust logic.

How did we define access tiers?

The basic concept of tiers is relatively straightforward: access to data increases as the device security hardening increases. These tiers are useful for coarse grain access control of client devices, which we have found to be sufficient in most cases. At Google, we allow the user to choose the device tier that allows them to weigh access needs with security requirements and policy. If a user needs access to more corporate data, they may have to accept more device configuration restrictions. If a user wants more control over their device and less restrictions but don’t need access to higher risk resources, they can choose a tier with less access to corporate data. For more information about the properties of a trusted platform you can measure, visit our paper about Maintaining a Healthy Fleet.

We knew this model would work in principle, but we didn’t know how many access tiers we should define. As described above, the old model only had two tiers: Trusted and Untrusted. We knew we wanted more than that to enable trust build up at the very least, but we didn’t know the ideal number. More tiers allow access control lists to be specified with greater fidelity at the cost of confusion for service owners, security engineers, and the wider employee base alike.

At Google, we initially supported four distinct tiers ranging from Untrusted to Highly-Privileged Access. The extremes are easy to understand: Untrusted devices should only access data that is already public while Highly-Privileged Access devices have greater privilege internally. The middle two tiers allowed system owners to design their systems with the tiered access model in mind. Certain sensitive actions required a Highly-Privileged Access device while less sensitive portions of the system could be reached with less trusted devices. This degraded access model sounded great to us security wonks. Unfortunately, employees were unable to determine what tier they should choose to ensure they could access all the systems they needed. In the end, we determined that the extra middle tier led to intense confusion without much benefit.

In our current model, the vast majority of devices fit in one of three distinct tiers: Untrusted, Basic Access, and Highly-Privileged Access. In this model, system owners are required to choose the more trusted path if their system is more sensitive. This requirement does limit the finesse of the system but greatly reduces employee confusion and was key to a successful adoption.

In addition to tiers, our system is able to provide additional context to access gateways and underlying applications and services. This additional information is useful to provide finer grained, device-based access control. Imposing additional device restrictions on highly sensitive systems, in addition to checking the coarse grain tier, is a reasonable way to balance security vs user expectations. Because highly sensitive systems are only used by a smaller subset of the employee population, based on role and need, these additional restrictions typically aren’t a source of user confusion. With that in mind, please note that this article only covers device-based controls and does not address fine-grained controls based on a user’s identity.

At the other end of the spectrum, we have OS installation/remediation services. These systems are required in order to support bootstrapping a device which by design does not yet adhere to the Basic Access tier. As described earlier, we use our certificates as a device identity, not trust validation. In the OS installation case, no reported data exists, but we can make access decisions based on the inventory data associated with that device identity. This allows us to ensure our OS and security agents are only installed on devices we own and expect to be in use. Once the OS and security agents are up and running, we can use them to lock down the device and prove it is in a state worthy of more trust.

How did we create rules to implement the tiers?

Device-based data is the heart of BeyondCorp and tiered access. We evaluate trust tiers using data about each device at Google to determine its security integrity and tier level. To obtain this data, we built an inventory pipeline which aggregates data from various sources of authority within our enterprise to obtain a holistic, comprehensive view of a device's security posture. For example, we gather prescribed company asset inventory in one service and observed data reported by agents on the devices in other services. All of this data is used to determine which tier a device belongs in, and trust tiers are reevaluated every time corporate data is changed or new data is reported.

Trust level evaluations are made via "rules", written by security and systems engineers. For example, for a device to have basic access, we have a rule that checks that it is running an approved operating system build and version. For that same device to have highly-privileged access, it would need to pass several additional rules, such as checking the device is encrypted and contains the latest security patches. Rules exist in a hierarchical structure, so several rules can combine to create a tier. Requirements for tiers across device platforms can be different, so there is a separate hierarchy for each. Security engineers work closely with systems engineers to determine the necessary information to protect devices, such as determining thresholds for required minimum version and security patch frequency.

Rule Enforcement and User Experience

To create a good user experience, rules are created and monitored before being enforced. For example, before requiring all users to upgrade their Chrome browser, we monitor how many users will drop trust if that rule was enforced. Dashboards track rule impact on Googlers over 30 day periods. This enables security and systems teams to evaluate rule change impact before they affect end users.

To further protect employee experience, we have measures called grace periods and exceptions. Grace periods provide windows of a predefined duration where devices can violate rules but still maintain trust and access, providing a fallback in case of unexpected consequences. Furthermore, grace periods can be implemented quickly and easily across the fleet in case for disaster recovery purposes. The other mechanism is called exceptions. Exceptions allow rule authors to create rules for the majority while enabling security engineers to make nuanced decisions around individual riskier processes. For example, if we have a team of Android developers specializing on user experience for an older Android version, they may be granted an exception for the minimum version rule.

How did we simplify troubleshooting?

Troubleshooting access issues proves challenging in a system where many pieces of data interact to create trust. We tackle this issue in two ways. First, we have a system to provide succinct and actionable explanations to end users on how to resolve problems on their own. Second, we have the capability to notify users when their devices have lost trust or are about to lose trust. The combination of these efforts improves the user experience of the tiered access solution and reduces toil for those supporting it.

We are able to provide self-service feedback to users by closely integrating the creation of rule policy with resolution steps for that policy. In other words, security engineers who write rule policies are also responsible for attaching steps on how to resolve the issue. To further aid users, the rule evaluation system provides details about the specific pieces of data causing the failure. All this information is fed into a centralized system that generates user-friendly explanations, guiding users to self-diagnose and fix problems without the need for IT support. Likewise, a tech may not be able to see pieces of PII about a user when helping fix the device. These cases are rare but necessary to protect the parties involved in these scenarios. Having one centralized debugging system helps deal with all these nuances, enabling us to provide detailed and safe explanations to end users in accordance with their needs.

Remediation steps are communicated to users in several ways. Before a device loses trust, notification pop-ups appear to the user explaining that a loss of access is imminent. These pop-ups contain directions to the remediation system so the user can self-diagnose and fix the problem. This circumvents user pain by offering solutions before the problem impacts the user. Premeditated notifications work in conjunction with the aforementioned grace periods, as we provide a window in which users can fix their devices. If the issue is not fixed and the device goes out of compliance, there is still a clear path on what to do. For example, when a user attempts to access a resource for which they do not have permission, a link appears on the access denied page directing them to the relevant remediation steps. This provides fast, clear feedback on how to fix their device and reduces toil on the IT support teams.

Next time

In the next and final post in this series, we will discuss how we migrated services to be protected by the BeyondCorp architecture at Google.

In the meantime, if you want to learn more, you can check out the BeyondCorp research papers. In addition, getting started with BeyondCorp is now easier using zero trust solutions from Google Cloud (context-aware access) and other enterprise providers.

Thank you to the editors of the BeyondCorp blog post series, Puneet Goel (Product Manager), Lior Tishbi (Program Manager), and Justin McWilliams (Engineering Manager).


AdMob’s new reporting delivers better insights

At AdMob, we’re focused on helping publishers make smarter decisions to grow their mobile app earnings and deliver the best experience to their users. Clear, comprehensive reporting is a big part of this, and we’ve recently released some updates to our reporting so that you can gain more actionable insights about your app users. 


More insights about how people use your app

We’ve enhanced our AdMob SDK to begin providing insights into how your users are interacting with your app and engaging with your rewarded ads.  To enable these new features, you need to do two things:


  1. Update your SDK to the latest version (Android SDK 18.1.0 or later, iOS SDK 7.44 or later)

  2. Log into AdMob and take a few simple steps to enable user metrics  (Help Center Article)

Once you’ve done both, available new features will include:

User engagement card

When you log into your AdMob account, we’ll now present you with important basic user engagement metrics on your App Overview dashboard.  This report helps you keep an eye on your app’s most important top-line user numbers so you always know how your apps are performing.

User engagement card

             Basic metrics in the user engagement card

Holistic revenue report including in-app purchases

We’ve updated our revenue reporting to look beyond AdMob earnings to also include in-app purchases and subscriptions. This will minimize time spent going back and forth between interfaces so you can spend more time working on your app. IAP reporting is currently only offered on iOS, but will be coming to Android soon.

Holistic revenue report

                             Holistic revenue report including in-app purchases

Rewarded ads report

You’ll now have access to the rewarded ads report, helping you understand how users are interacting with rewarded ads.  This report captures activity from the entire AdMob platform, including AdMob, Open Bidding, and 3rd-party inventory. We’re also working to provide additional reporting around each of your ad formats so you can make the most informed decision with your app business.

Rewarded ads report

                                                   Example rewarded ads report

Expanded functionality and ease of use

In addition to adding new reports, we’ve also launched several new features that make it easier for you to access, analyze and understand your data: 


AdMob API

Traditionally, developers have had to use the AdSense API to access AdMob stats.  This was a suboptimal solution, as AdSense uses different ad metric definitions than AdMob, resulting in inconsistent reporting.  To address this, we’re introducing a brand new AdMob API to help you retrieve your publisher reports programmatically. Metrics in the AdMob API will be consistent with those in the AdMob front-end interface and will be more accurate than those in the AdSense API.  This API is currently in beta, so reach out to your account manager or a mobile specialist if you’re interested in trying it out.


Easy comparison reporting

When looking at changes to a single AdMob metric, such as revenue, we know that context is key. So we’ve added the ability for you to compare two metrics within the same chart (see upper right in image below) so that you can more easily correlate trends within your data.


You can also now break out metrics, such as estimated earnings, by dimensions like app, ad unit, format, and country (see upper left in image below).  Our improved graph interactivity allows you to add or remove dimensions by simply clicking on the corresponding circle in the key, as shown below on the upper left.

Reporting improvements gif

                                          New comparison and usability chart features

Reporting is an integral part of optimizing any app ads business, and we’re continuously taking steps to make actionable insights easier to discover for publishers.  We hope these new features will help publishers better understand their users and ad performance so they can focus on driving more revenue and delighting their users.

Google on the ACCC Digital Platforms Inquiry

Technology has provided significant opportunities for Australian consumers and businesses. And the potential upside is huge - research suggests that Australia stands to gain $1.2 trillion in economic benefit between 2015 and 2030 if it can successfully drive investment in productivity-enhancing technology.

The Australian Competition and Consumer Commission’s Digital Platforms Inquiry (DPI) explored the pace of digital transformation within the media sector and across the Australian economy. Last week we provided a submission as part of the Treasury consultation process on the DPI Final Report.

This process provides an opportunity for Australia to consider its place in the global digital economy and to review the rules of engagement between Australian consumers, businesses, and technology providers.

In Australia, Alphabeta estimated that to date in 2019, Google Search, grants from Google to the non-profit sector, and Google advertising tools have helped connect more than 1.1 million businesses, website publishers, and non-profits to consumers globally. The benefits realised by businesses using Google’s platforms enabled them to support up to 116,200 jobs in Australia, two-thirds in small and medium-sized businesses.

Google’s technology helps Australians find information and create content. More than one hour of Australian content is uploaded to YouTube every minute and, on an average night, Google Search and other Google tools like YouTube help Australian students research answers to 25 million homework questions.

Our ability to create products that are useful and successful relies on carefully balancing the interests of our users, publishers, and advertisers.

For consumers, we need to ensure that we provide the most relevant results and also ensure we keep their information safe and private - over the last 10 years Google has developed innovative privacy tools like Google Account and Google Takeout, which give consumers greater transparency and control over their data.

For publishers, large and small, we provide the ability to monetise their content with advertising and send over 24 billion visits (or clicks) to news publishers every month globally - that adds up to 9,000 visits (clicks) a second.

For advertisers, again, large and small - we need to ensure that we deliver effective advertising solutions across Search, YouTube and the Google Display Network. For Google to succeed - all three stakeholders need to succeed.

Google supports the DPI’s objectives to promote public interest journalism and digital media literacy, foster a dynamic and competitive digital ecosystem, protect consumer privacy, and drive greater understanding of data collection, but notes these should be balanced with the interests of consumers and wider social and economic objectives.

Google is broadly supportive of many of the Final Report's 23 recommendations, but some require further analysis on the associated costs and benefits. Two recommendations are of particular concern, specifically changes to Android defaults and aspects of the proposed publisher code.

Firstly, the recommendation to directly intervene in the Android operating system does not take into account Australian market conditions and competition laws, and provides no justification for focusing on Android when Apple’s iOS is the most-used mobile operating system in Australia (as noted in the Final Report) and Microsoft’s Windows remains the most-used PC-based operating system.

Secondly, the proposal for regulator-sanctioned negotiation of revenue sharing between platforms and news publishers - as part of the code contemplated by Recommendation 7 - overlooks existing commercial arrangements between Google and Australian news publishers and the broader value that Google provides through referred web traffic and technology.

In total in 2018, Google sent more than 2 billion clicks to Australian news publishers from Australian users, and more than 1 billion additional clicks to Australian news publishers from users globally. Our Google News Initiative supports news publishers of all sizes to develop, test and implement innovative approaches to drive revenue for publishers and support greater media literacy among consumers. Recently we made ranking updates and published changes to our search rater guidelines to help better recognise original reporting and surface it more prominently in Search.

Google welcomes the opportunity for further consultation. We look forward to continuing to engage with all interested parties, including Government and industry, in the coming weeks.

Watch up to 3x more with Data Saver on Android TV

Smart TVs running Android TV are quickly gaining popularity in India: there are millions of monthly active Android TV users across the country. Unfortunately, not everyone has Wi-Fi at home, which can make it difficult to connect the TVs to the internet. Instead, many smart TV owners decide to connect their devices by using their mobile hotspot. But that presents problems: watching HD TV on a mobile data connection can quickly drain your daily data plan.


Today, we're proud to introduce a set of 4 new features for India that help users get the most out of their smart TV with limited mobile data (and it works great with Google’s Files app). 


  • Data Saver reduces your data usage on mobile connections, increasing watch time by up to 3x
  • Data Alerts help you monitor your data usage while watching TV
  • Hotspot Guide helps you set up your TV with your mobile hotspot


You can easily set up Data Saver (left) and see Data Alerts (right) while watching TV using your mobile hotspot


  • Cast in Files lets you view downloaded media from your phone on your TV without using data


    Files allows you to cast offline media to your TV without using data


    These improvements will make it easy to watch streaming video on TV without draining your data. We’ll be rolling out these features to Android TV devices in India over the coming weeks, starting with Xiaomi, TCL and MarQ by Flipkart, with a global rollout following after. Cast in Google’s Files app is available today in the Beta program and will roll out to all users in the coming weeks.

    By Joris van Mens, Product Manager, Next Billion Users

    Control session length for Google Cloud Console and gcloud CLI

    What’s changing 

    We’re opening a public beta so G Suite, Google Cloud Platform (GCP), and Cloud Identity admins can set a fixed session duration for specific apps and services. After the session expires, users will need to re-enter their login credentials to continue to access:
    Settings can be customized for specific organizational units.

    Note that this is designed to work on the web. However, the settings will apply to authentication on all platforms, including the web and mobile apps where they exist. As a result, affected mobile apps may not work properly when the feature is enabled.

    Who’s impacted 

    Admins only

    Why you’d use it 

    Many apps and services include sensitive data, and it’s important that only specific users can access that information. By requiring re-authentication, you can make it more difficult for the wrong people to obtain that data if they gain unauthorized access to a device.

    How to get started 

    • Admins: Find session length controls at Admin console > Security > Google Cloud session control (Beta). See our Help Center to learn more about how to set session length for Google Cloud services
    • End users: If a session ends, users will simply need to log in to their account again using the familiar Google login flow. 

    Additional details 

    Third-party SAML identity providers and session length controls 
    If your organization uses a third-party SAML-based identity provider, the cloud sessions will expire, but the user may be transparently reauthenticated (i.e. without actually being asked to present their credentials) if their session with the IdP is valid at that time. This is working as intended, as Google will redirect the user to the IdP and accept a valid assertion from the IdP. To ensure that the user is rechallenged for authentication, be sure to match the session timeout at the IdP with the session length you’d like to enforce.

    Provides fixed-time controls (not activity-based) 
    Note that the new session control is a fixed time limit—it does not look for session activity, or ‘idle time’. At this time, Google Cloud and G Suite do not support activity-based session expiry.

    Re-authentication options 
    When choosing a session length, admins will be able to choose:
    • Between a range of predefined session lengths, or set a custom session length. 
    • Whether users need regular login credentials (password and, if configured, 2-Step Verification), or require a security key to re-authenticate. 


    Helpful links 

    Help Center: Beta: Set session length for Google Cloud services 

    Availability 

    Rollout details 


    Editions 
    Available to all G Suite and Cloud Identity editions

    On/off by default? 
    This feature will be OFF by default and can be enabled at the OU level.

    Stay up to date with G Suite launches

    New Android management client for devices registered after September 16, 2019

    What’s changing 

    On September 16, 2019, we’ll begin gradually rolling out a new Android management system called “Android Management API.” Apps managed through the new system will also use a new on-device management client, Android Device Policy, which will replace the existing Google Apps Device Policy client.

    While the new client has mostly similar features, there are some differences in management and usage that will impact G Suite admins and end users. The changes will make it easier for admins and users to set up and manage Android devices for work.

    You will receive an email notification before it impacts your domain 
    The rollout will be conducted in stages, and could take several months to reach all domains. We will email your organization’s primary admin around 3 weeks before it will reach your domain with more specific dates for deployment.

    See below for more details about the changes.

    Who’s impacted 

    Admins and end users

    Why you’d use it 

    The new client will have closer ties to the Android infrastructure, allowing us to quickly add new features and more easily develop updates for increased security. It will also provide a more seamless experience for end users, with fewer apps to manage and more integrated functionality.

    How to get started 


    • Admins: Familiarize yourself with the changes outlined in this post, including the additional details section below. Let your users know about the changes they can expect. 
    • End users: No action needed. 


    Additional details 

    Devices that will use the new Android management client 
    Once this change has been deployed to your domain, newly registered devices that meet the following requirements will be automatically managed using the Android Management API:

    • The device is using Android M or above. 
    • The device supports a work profile and company-owned (fully managed) device mode. 
    • The user account is under advanced mobile device management. 

    This will not impact previously enrolled devices; they will still be managed through the legacy Android management client.

    How managing devices is different with the new client 
    In the Admin console, most of the functionality will remain the same. All devices will be displayed and managed through the same interface at Admin console > Device management.

    There will be some changes, however, for devices managed through the new client.

    The following features will not be supported:
    • Device admin mode
    • Option to disable Work Profile setup (If you don’t want to use Work Profiles in your organization, you can instruct your users to set up devices without enabling the feature) 
    • Wipe Account for company-owned devices or devices in fully managed device (device owner) mode (Wipe Device will still be available) 


    The following new features will be available:


    The following features will change:
    • If you manually choose to Wipe Device, you’ll have a new option to either retain the factory reset protection settings or clear them along with the complete wipe. 
    • The Auto account wipe setting will perform Wipe Device for devices in fully managed device (device owner) mode. In addition to being applied when devices fall out of sync, Auto account wipe will be triggered when devices fall out of some policies (for instance, when a more restrictive passcode policy has been enforced by the admin). In both cases, the user will be given a grace period and notifications to correct the issue prior to the wipe taking place. 
    • Device management rules will initiate a device wipe instead of an account wipe for devices in fully managed device (device owner) mode. 


    You can see which client is managing a device in the Admin console at Security details > User agent. Devices using the legacy client will have a version of Google Apps Device Policy, while devices using the new client will have a version of Android Device Policy. Use our Help Center to learn how to view mobile device details.

    How using a device is different with the new client 
    The main end-user visible changes include the following:

    • Users will have an updated enrollment experience. 
    • After the new version is deployed, users will no longer see a Device Policy app in their app drawer. The new management system and Android Device Policy app will be integrated with Android for a more seamless experience. 
    • Users won’t be able to use My Devices to manage their device (for the time being, they can use Find My Device). 
    • If your organization enforces a strong password, the password will require a symbol in addition to the letter and number previously required. 


    Users will experience a slightly different setup flow when registering new devices. 


    Migrating from the legacy system to the Android Management API 
    Once this change has been deployed to your domain, you can manually migrate users and devices to the new Android Management API in the following ways:

    • Factory reset and re-register any eligible device. 
    • Provide a new device and register it. 

    In the future, we’ll add options and tools to help you migrate existing devices to use the Android Management API. Check out the G Suite Updates blog for the latest changes and migration updates. 

    Availability 

    Rollout details 
    • All domains: Extended rollout (potentially longer than 15 days for feature visibility) starting on September 16, 2019. The rollout will be conducted in stages, and could take several months to reach all domains. 
    • Primary admins will be notified by email around 3 weeks before it will reach your domain. 

    G Suite editions 
    • Available to all G Suite editions 

    On/off by default? 
    • This feature will be ON by default for new devices that meet the requirements above.


    Stay up to date with G Suite launches

    Live captions in Hangouts Meet, now available on Android

    What's changing

    Support for live captions on Hangouts Meet was announced at Google Cloud Next ‘19 in San Francisco. We’re continuing to invest in this area with the addition of captions support on the Meet Android app.

    Additionally, you can now more easily turn on live captions on web and in rooms with a button that’s one click away.

    Who’s impacted

    End users

    Why you’d use it

    Live captions help make your meetings more accessible by reducing barriers to holding meetings between users of different hearing abilities, regardless of whether they are participating remotely or in person.

    How to get started


    • Admins: No action required.
    • End users:
      • On Android: Tap the closed captions button on the top right of the Meet app when you are in a meeting. (Note that this button is only shown here for English language users. It can be found in the triple-dot menu for all other languages.
      • On the web and on Chromebase for meetings touchscreen devices: Tap the captions button at the bottom right of your screen. (Note that this button is only shown here for English language users. It can be found in the triple-dot menu for all other languages.)
      • On devices with a Mimo touchscreen: Tap the captions option in the right-hand panel. (Note that this is only shown here for English language users. It can be found under Settings for all other languages.)

    Additional details

    Live captions are not yet available on the Meet iOS app but we are working on making it available soon.

    When you turn on captions, they will be visible on that particular device. In order for other participants in the meeting to see captions, they’ll have to turn it on for their devices as well.

    Currently, live captions is only available in English. Additionally, captions will not appear in a recording of a meeting.

    Helpful links



    Availability

    Rollout details

    • Rapid Release domains: Extended rollout (potentially longer than 15 days for feature visibility) starting on September 16, 2019
    • Scheduled Release domains: Extended rollout (potentially longer than 15 days for feature visibility) starting on September 16, 2019

    G Suite editions

    • Available to all G Suite editions

    On/off by default?

    • This feature will be ON by default.

    Stay up to date with G Suite launches

    Making Review Rich Results more helpful

    Search results that are enhanced by review rich results can be extremely helpful when searching for products or services (the scores and/or “stars” you sometimes see alongside search results).
    Review stars example in search results

    To make them more helpful and meaningful, we are now introducing algorithmic updates to reviews in rich results. This also addresses some of the invalid or misleading implementations webmasters have flagged to us.

    Focus on schema types that lend themselves to reviews

    While, technically, you can attach review markup to any schema type, for many types displaying star reviews does not add much value for the user. With this change, we’re limiting the pool of schema types that can potentially trigger review rich results in search. Specifically, we’ll only display reviews with those types (and their respective subtypes):

    Self-serving reviews aren't allowed

    Reviews that can be perceived as “self-serving” aren't in the best interest of users. We call reviews “self-serving” when a review about entity A is placed on the website of entity A - either directly in their markup or via an embedded 3rd party widget. That’s why, with this change, we’re not going to display review rich results anymore for the schema types localBusiness and Organization (and their subtypes) in cases when the entity being reviewed controls the reviews themselves.

    Add the name of the item that's being reviewed

    With this update, the name property is now required, so you'll want to make sure that you specify the name of the item that's being reviewed.
    This update will help deliver a much more meaningful review experience for users, while requiring little to no changes on the part of most webmasters. You can find all those updates documented in our developer documentation. If you have any questions, feel free to come to our webmaster forums!

    Google at Interspeech 2019



    This week, Graz, Austria hosts the 20th Annual Conference of the International Speech Communication Association (Interspeech 2019), one of the world‘s most extensive conferences on the research and engineering for spoken language processing. Over 2,000 experts in speech-related research fields gather to take part in oral presentations and poster sessions and to collaborate with streamed events across the globe.

    As a Gold Sponsor of Interspeech 2019, we are excited to present 30 research publications, and demonstrate some of the impact speech technology has made in our products, from accessible, automatic video captioning to a more robust, reliable Google Assistant. If you’re attending Interspeech 2019, we hope that you’ll stop by the Google booth to meet our researchers and discuss projects and opportunities at Google that go into solving interesting problems for billions of people. Our researchers will also be on hand to discuss Google Cloud Text-to-Speech and Speech-to-text, demo Parrotron, and more. You can also learn more about the Google research being presented at Interspeech 2019 below (Google affiliations in blue).

    Organizing Committee includes:
    Michiel Bacchiani

    Technical Program Committee includes:
    Tara Sainath

    Tutorials
    Neural Machine Translation
    Organizers include: Wolfgang Macherey, Yuan Cao

    Accepted Publications
    Building Large-Vocabulary ASR Systems for Languages Without Any Audio Training Data (link to appear soon)
    Manasa Prasad, Daan van Esch, Sandy Ritchie, Jonas Fromseier Mortensen

    Multi-Microphone Adaptive Noise Cancellation for Robust Hotword Detection (link to appear soon)
    Yiteng Huang, Turaj Shabestary, Alexander Gruenstein, Li Wan

    Direct Speech-to-Speech Translation with a Sequence-to-Sequence Model
    Ye Jia, Ron Weiss, Fadi Biadsy, Wolfgang Macherey, Melvin Johnson, Zhifeng Chen, Yonghui Wu

    Improving Keyword Spotting and Language Identification via Neural Architecture Search at Scale (link to appear soon)
    Hanna Mazzawi, Javier Gonzalvo, Aleks Kracun, Prashant Sridhar, Niranjan Subrahmanya, Ignacio Lopez Moreno, Hyun Jin Park, Patrick Violette

    Shallow-Fusion End-to-End Contextual Biasing (link to appear soon)
    Ding Zhao, Tara Sainath, David Rybach, Pat Rondon, Deepti Bhatia, Bo Li, Ruoming Pang

    VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
    Quan Wang, Hannah Muckenhirn, Kevin Wilson, Prashant Sridhar, Zelin Wu, John Hershey, Rif Saurous, Ron Weiss, Ye Jia, Ignacio Lopez Moreno

    SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
    Daniel Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin Dogus Cubuk, Quoc Le

    Two-Pass End-to-End Speech Recognition
    Ruoming Pang, Tara Sainath, David Rybach, Yanzhang He, Rohit Prabhavalkar, Mirko Visontai, Qiao Liang, Trevor Strohman, Yonghui Wu, Ian McGraw, Chung-Cheng Chiu

    On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition
    Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen

    Contextual Recovery of Out-of-Lattice Named Entities in Automatic Speech Recognition (link to appear soon)
    Jack Serrino, Leonid Velikovich, Petar Aleksic, Cyril Allauzen

    Joint Speech Recognition and Speaker Diarization via Sequence Transduction
    Laurent El Shafey, Hagen Soltau, Izhak Shafran

    Personalizing ASR for Dysarthric and Accented Speech with Limited Data
    Joel Shor, Dotan Emanuel, Oran Lang, Omry Tuval, Michael Brenner, Julie Cattiau, Fernando Vieira, Maeve McNally, Taylor Charbonneau, Melissa Nollstadt, Avinatan Hassidim, Yossi Matias

    An Investigation Into On-Device Personalization of End-to-End Automatic Speech Recognition Models (link to appear soon)
    Khe Chai Sim, Petr Zadrazil, Francoise Beaufays

    Salient Speech Representations Based on Cloned Networks
    Bastiaan Kleijn, Felicia Lim, Michael Chinen, Jan Skoglund

    Cross-Lingual Consistency of Phonological Features: An Empirical Study (link to appear soon)
    Cibu Johny, Alexander Gutkin, Martin Jansche

    LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
    Heiga Zen, Viet Dang, Robert Clark, Yu Zhang, Ron Weiss, Ye Jia, Zhifeng Chen, Yonghui Wu

    Improving Performance of End-to-End ASR on Numeric Sequences
    Cal Peyser, Hao Zhang, Tara Sainath, Zelin Wu

    Developing Pronunciation Models in New Languages Faster by Exploiting Common Grapheme-to-Phoneme Correspondences Across Languages (link to appear soon)
    Harry Bleyan, Sandy Ritchie, Jonas Fromseier Mortensen, Daan van Esch

    Phoneme-Based Contextualization for Cross-Lingual Speech Recognition in End-to-End Models
    Ke Hu, Antoine Bruguier, Tara Sainath, Rohit Prabhavalkar, Golan Pundak

    Fréchet Audio Distance: A Reference-free Metric for Evaluating Music Enhancement Algorithms
    Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, Matthew Sharifi

    Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning
    Yu Zhang, Ron Weiss, Heiga Zen, Yonghui Wu, Zhifeng Chen, RJ Skerry-Ryan, Ye Jia, Andrew Rosenberg, Bhuvana Ramabhadran

    Sampling from Stochastic Finite Automata with Applications to CTC Decoding
    Martin Jansche, Alexander Gutkin

    Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model (link to appear soon)
    Anjuli Kannan, Arindrima Datta, Tara Sainath, Eugene Weinstein, Bhuvana Ramabhadran, Yonghui Wu, Ankur Bapna, Zhifeng Chen, SeungJi Lee

    A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet
    Jean-Marc Valin, Jan Skoglund

    Low-Dimensional Bottleneck Features for On-Device Continuous Speech Recognition
    David Ramsay, Kevin Kilgour, Dominik Roblek, Matthew Sharif

    Unified Verbalization for Speech Recognition & Synthesis Across Languages (link to appear soon)
    Sandy Ritchie, Richard Sproat, Kyle Gorman, Daan van Esch, Christian Schallhart, Nikos Bampounis, Benoit Brard, Jonas Mortensen, Amelia Holt, Eoin Mahon

    Better Morphology Prediction for Better Speech Systems (link to appear soon)
    Dravyansh Sharma, Melissa Wilson, Antoine Bruguier

    Dual Encoder Classifier Models as Constraints in Neural Text Normalization
    Ajda Gokcen, Hao Zhang, Richard Sproat

    Large-Scale Visual Speech Recognition
    Brendan Shillingford, Yannis Assael, Matthew Hoffman, Thomas Paine, Cían Hughes, Utsav Prabhu, Hank Liao, Hasim Sak, Kanishka Rao, Lorrayne Bennett, Marie Mulville, Ben Coppin, Ben Laurie, Andrew Senior, Nando de Freitas

    Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation
    Fadi Biadsy, Ron Weiss, Pedro Moreno, Dimitri Kanevsky, Ye Jia




    Source: Google AI Blog


    YouTube Charts Launch in India Showcasing The Hottest Artists, Songs and Music Videos



    At YouTube, we understand the importance of showcasing and celebrating the hottest artists, songs and music videos from around the world. Today, we’re closing a critical gap by rolling out the Music Charts experience in India. India has long been one of the most unique and vibrant music markets in the world. Nowhere is that more visible than on YouTube. We look forward to celebrating the success of the artists, songs and music videos that are resonating with Indian music fans. This week, the India Chart is topped by Arijit Singh’s Pachtaoge, followed by Psycho Saiyaan and Jass Manak’s Lehanga.


    Over the last year, many Indian artists including Guru Randhawa, Badshah, Arijit Singh and Neha Kakkar have each reached the YouTube music country charts in the United Kingdom, Australia, New Zealand and Canada, along with the global chart. This week, Ranu Maria Mondal’s song ‘Teri Meri Kahani’ for Himesh Reshammiya’s upcoming film is topping the Global Charts on YouTube.com/charts, with over 138M views and counting. Ranu comes from a humble background and recently became a global sensation  for her singing video. 


    With more than 2 billion global users and 265 million users every month in India, YouTube charts is the go-to destination to see what’s popular, what’s rising and trending both locally in India and globally. The YouTube charts are a reflection of the success achieved by artists on the world's most expansive music platform. 


    The YouTube Music Charts include:


    Trending: What’s new and hot in music right now
    The new Trending chart is updated multiple times a day to provide a unique, real time view of the hottest new music fans are enjoying. The Trending chart is YouTube’s first dedicated external signal highlighting videos which were instantly popular upon release based on the total of organic views. 


    Top Songs: The most played songs on YouTube
    The Top Songs chart highlights the number of organic views of a song on YouTube. Calculated by combining all official versions of a song (including the official music video, official song used in user generated content and lyric videos), the Top Songs chart reflects how users are consuming music on the platform and the multiple ways artists are using YouTube to share their music. The Top Songs chart is updated weekly on Monday at 12.30 IST. 


    Top Artists: The biggest artists on YouTube
    The Top Artists chart highlights the most popular artists on YouTube, based on the total organic views of their entire discography - official music videos, official song used in user generated content, lyric videos, and more. Top Artists is updated weekly on Monday at 12.30 IST.


    Top Music Videos: The most viewed music videos
    The Top Music Videos chart celebrates the music video experience on YouTube, highlighting the most viewed official music videos on the platform. Top Videos is updated weekly on Monday at 12.30 IST based on the total of organic views. 


    YouTube Charts are also available within YouTube Music, including Top 100 most played songs and top 100 music videos globally and locally and top 30 Trending chart.


    Full details about how the YouTube charts are calculated can be found here. Check out what’s hot and trending in India this week on charts.youtube.com

    By Chris Clark, Product Manager, YouTube Music