Category Archives: Google Webmaster Central Blog

Official news on crawling and indexing sites for the Google index

Why & how to secure your website with the HTTPS protocol

A few days ago, Fatih Ozkosemen and I led an episode of the AdSense On Air series. This program consists of monthly videos which cover many topics of interest to online publishers (we recommend you sign up if you use Google AdSense). The November 2018 version was dedicated to HTTPS migrations.

You can find the whole session, about one hour long, in this video:





The video covers the following topics:
  • What HTTPS encryption is, and why it is important to protect your visitors and yourself,
  • How HTTPS enables a more modern web,
  • What are the usual complaints about HTTPS, and are they still true today?
    • “But HTTPS certificates cost so much money!”
    • “But switching to HTTPS will destroy my SEO!”
    • “But “mixed content” is such a headache!”
    • “But my ad revenue will get destroyed!”
    • “But HTTPS is sooooo sloooow!"
  • Some practical advice to run the migration. Those are an aggregation of:

We hope that this sort of content is useful. Don’t hesitate to let us know if you like it and if we should do more! You can reach out to us directly on Twitter (Vincent & Fatih). Let us know which topics are of interest to you by commenting here or on the YouTube page. If you have questions when you plan your own HTTPS migration, don’t hesitate to ask in our Webmaster Help Forums.

Posted by Vincent Courson, Search Outreach Specialist

Introducing the Indexing API and structured data for livestreams

Over the past few years, it's become easier than ever to stream live videos online, from celebrity updates to special events. But it's not always easy for people to determine which videos are live and know when to tune in.
Today, we're introducing new tools to help more people discover your livestreams in Search and Assistant. With livestream structured data and the Indexing API, you can let Google know when your video is live, so it will be eligible to appear with a red "live" badge:

Add livestream structured data to your page

If your website streams live videos, use the livestream developer documentation to flag your video as a live broadcast and mark the start and end times. In addition, VideoObject structured data is required to tell Google that there's a video on your page.

Update Google quickly with the Indexing API

The Indexing API now supports pages with livestream structured data. We encourage you to call the Indexing API to request that your site is crawled in time for the livestream. We recommend calling the Indexing API when your livestream begins and ends, and if the structured data changes.
For more information, visit our developer documentation. If you have any questions, ask us in the Webmaster Help Forum. We look forward to seeing your live videos on Google!

Rich Results expands for Question & Answer pages

People come to Google seeking information about all kinds of questions.
Frequently, the information they're looking for is on sites where users ask and answer each other's questions. Popular social news sites, expert forums, and help and support message boards are all examples of this pattern.

A screenshot of an example search result for a page titled “Why do touchscreens sometimes register a touch when ...” with a preview of the top answers from the page.
In order to help users better identify which search results may give the best information about their question, we have developed a new rich result type for question and answer sites. Search results for eligible Q&A pages display a preview of the top answers. This new presentation helps site owners reach the right users for their content and helps users get the relevant information about their questions faster.
A screenshot of an example search result for a page titled “Why do touchscreens sometimes register a touch when ...” with a preview of the top answers from the page.

To be eligible for this feature, add Q&A structured data to your pages with Q&A content. Be sure to use the Structured Data Testing Tool to see if your page is eligible and to preview the appearance in search results. You can also check out Search Console to see aggregate stats and markup error examples. The Performance report also tells you which queries show your Q&A Rich Result in Search results, and how these change over time.
If you have any questions, ask us in the Webmaster Help Forum or reach out on Twitter!

PageSpeed Insights, now powered by Lighthouse

At Google, we know that speed matters and we provide a variety of tools to help everyone understand the performance of a page or site. Historically, these tools have used different analysis engines. Unfortunately, this caused some confusion because the recommendations from each tool were different. Today, we're happy to announce that Pagespeed Insights (PSI) now uses Lighthouse as its analysis engine. This allows developers to get the same performance audits and recommendations everywhere: on the web, from the command line, and in Chrome DevTools. PSI also incorporates field data provided by the Chrome User Experience Report (CrUX). Version 5 of the PageSpeed Insights API will now provide CrUX data and all of the Lighthouse audits. Previous versions of the PSI API will be deprecated in six months.
Pagespeed Insights is now powered by Lighthouse

PageSpeed Insights provides the following information:
  • Lab Data. PSI fetches and analyzes the page using Lighthouse, which simulates how a mobile device loads a page. It computes a set of performance metrics for the page (such as First Contentful Paint and Time to Interactive) and summarizes these metrics with a performance score from 0-100. Scores are categorized into three levels; 90 and up is considered to be a good score.
  • Field Data. PSI also displays real-world performance metrics (First Contentful Paint and First Input Delay) for the page and its origin. (As a result, we've also deprecated the origin: query in PSI). Note that not all sites may have field data available for display. The data set relies on a version of the Chrome User Experience Report that is updated daily and is aggregated over the previous 28 days. Keep in mind that the metrics here may be different from the ones in the Lab Data section as they capture a wide spectrum of real-world network conditions and devices used by Chrome users.
  • Opportunities. PSI provides suggestions on how to improve the page's performance metrics. Each suggestion in this section estimates how much faster the page will load if the improvement is implemented.
  • Diagnostics. This section provides additional information about how a page adheres to best practices for web development.
The PSI v5 API now returns this new analysis together with CrUX data, and all Lighthouse category data (Performance, Progressive Web App, Accessibility, Best Practices, and SEO) for a given URL.
We have more information about the changes in our FAQ. If you have any questions, please use Stack Overflow and tag your question with the pagespeed-insights tag.

Posted by Rui Chen and Paul Irish, PageSpeed Insights and Lighthouse teams

Notifying users of unclear subscription pages

Every month, millions of Chrome users encounter pages with insufficient mobile subscription information. Surprising charges that come from unclear communication are a poor user experience. That’s why starting from Chrome 71 (December 2018), Chrome will show a warning before these pages, so that users can make informed decisions when signing up to mobile based subscription services. Users will be offered the choice to proceed to the page or go back if they were unaware that they were entering a billing page.

Unclear mobile subscriptions


Picture this: Andrea is browsing the web on a mobile connection to access a gaming page and they’re presented with a page that asks them for their mobile phone details.




They fill in the blanks with their mobile number and press Continue, and get access to the content.

The next month, the phone bill arrives and they see a charge they were not expecting. Was the subscription to the online gaming service really that expensive? Did they really agree to pay that specific price for the service? How much did they agree to be charged to access the content?

Clearer billing information for Chrome users


We want to make sure Chrome users understand when they are going through a billing flow and trust that they’ll be able to make informed decisions while browsing the web.

To adequately inform users, it’s important to provide a sufficient level of details within the billing page as outlined by our new mobile billing charges best practices. Pages that answer positively to the following questions generally provide sufficient information for users:
  • Is the billing information visible and obvious to users? For example, adding no subscription information on the subscription page or hiding the information is a bad start because users should have access to the information when agreeing to subscribe.
  • Can customers easily see the costs they’re going to incur before accepting the terms? For example, displaying the billing information in grey characters over a grey background, therefore making it less readable, is not considered a good user practice.
  • Is the fee structure easily understandable? For example, the formula presented to explain how the cost of the service will be determined should be as simple and straightforward as possible.

If Chrome detects pages that don’t provide sufficient billing information to users, the following warning will be displayed to the user on Chrome mobile, Chrome desktop and Android’s WebView:



The warning will be shown to users entering unclear billing pages.


When we identify such pages, we will notify the webmaster through Search Console where there will be an option to let us know about the changes they’ve made to clarify the billing process. For websites that aren’t verified on Search Console, we will do our best to get in touch with the webmasters affected and will be available to answer questions in our public support forum available in 15 languages. Once an appeal has been sent via Search Console, we will review the changes and remove the warning accordingly.

If your billing service takes users through a clearly visible and understandable billing process as described in our best practices, you don't need to make any changes. Also, the new warning in Chrome doesn’t impact your website’s ranking in Google Search.

If you have any questions, please come and have a chat with us in the Webmaster Help Forum.


Posted by Emily Schechter‎, Chrome Security, Giacomo Gnecchi Ruscone & Badr Salmi El Idrissi, Trust & Safety

Introducing reCAPTCHA v3: the new way to stop bots

Today, we’re excited to introduce reCAPTCHA v3, our newest API that helps you detect abusive traffic on your website without user interaction. Instead of showing a CAPTCHA challenge, reCAPTCHA v3 returns a score so you can choose the most appropriate action for your website.

A Frictionless User Experience

Over the last decade, reCAPTCHA has continuously evolved its technology. In reCAPTCHA v1, every user was asked to pass a challenge by reading distorted text and typing into a box. To improve both user experience and security, we introduced reCAPTCHA v2 and began to use many other signals to determine whether a request came from a human or bot. This enabled reCAPTCHA challenges to move from a dominant to a secondary role in detecting abuse, letting about half of users pass with a single click. Now with reCAPTCHA v3, we are fundamentally changing how sites can test for human vs. bot activities by returning a score to tell you how suspicious an interaction is and eliminating the need to interrupt users with challenges at all. reCAPTCHA v3 runs adaptive risk analysis in the background to alert you of suspicious traffic while letting your human users enjoy a frictionless experience on your site.

More Accurate Bot Detection with "Actions"

In reCAPTCHA v3, we are introducing a new concept called “Action”—a tag that you can use to define the key steps of your user journey and enable reCAPTCHA to run its risk analysis in context. Since reCAPTCHA v3 doesn't interrupt users, we recommend adding reCAPTCHA v3 to multiple pages. In this way, the reCAPTCHA adaptive risk analysis engine can identify the pattern of attackers more accurately by looking at the activities across different pages on your website. In the reCAPTCHA admin console, you can get a full overview of reCAPTCHA score distribution and a breakdown for the stats of the top 10 actions on your site, to help you identify which exact pages are being targeted by bots and how suspicious the traffic was on those pages.

Fighting Bots Your Way

Another big benefit that you’ll get from reCAPTCHA v3 is the flexibility to prevent spam and abuse in the way that best fits your website. Previously, the reCAPTCHA system mostly decided when and what CAPTCHAs to serve to users, leaving you with limited influence over your website’s user experience. Now, reCAPTCHA v3 will provide you with a score that tells you how suspicious an interaction is. There are three potential ways you can use the score. First, you can set a threshold that determines when a user is let through or when further verification needs to be done, for example, using two-factor authentication and phone verification. Second, you can combine the score with your own signals that reCAPTCHA can’t access—such as user profiles or transaction histories. Third, you can use the reCAPTCHA score as one of the signals to train your machine learning model to fight abuse. By providing you with these new ways to customize the actions that occur for different types of traffic, this new version lets you protect your site against bots and improve your user experience based on your website’s specific needs.
In short, reCAPTCHA v3 helps to protect your sites without user friction and gives you more power to decide what to do in risky situations. As always, we are working every day to stay ahead of attackers and keep the Internet easy and safe to use (except for bots).
Ready to get started with reCAPTCHA v3? Visit our developer site for more details.

Google is introducing its Product Experts Program!

Over 12 years ago, we started answering webmaster questions and listening to feedback on our webmaster forums (although at the time, it was a Google Group for questions about sitemaps - original announcement). From a small mailing list, these forums have evolved to cover 15 languages and over 50,000 threads per year. These days, we learn a lot from some of the cases surfaced on this platform, and constantly use it to gather feedback to pass on to our teams.

Google’s Top Contributors (tc) and Rising Stars (tc) are some of our most active and helpful members on these forums. With over 100 members globally just for the Webmaster Forums (1000 members if you count all product forums), this community of experts helps thousands of people every year by sharing their knowledge and helping others get the most out of Google products.

Some of the Webmaster forum participants


Today, we’re excited to announce that we’re rebranding and relaunching the Top Contributor program as Google’s Product Experts program! Same community of experts, shiny new brand.

Over the following days, we’ll be updating our badges in the forums so you can recognize who our most passionate and dedicated Product Experts are:

   Silver Product Expert: Newer members who are developing their product knowledge

   Gold Product Expert: Trusted members who are knowledgeable and active contributors

   Platinum Product Expert: Seasoned members who contribute beyond providing help through mentoring, creating content, and more

   Product Expert Alumni: Past members who are no longer active, but were previously recognized for their helpfulness

More information about the new badges and names.



Those Product Experts are users who are passionate about Google products and enjoy helping other users. They also help us by giving feedback on the tools we all use, like the Search Console, by surfacing questions they think Google should answer better, etc… Obtaining feedback from our users is one of Google’s core values, and Product Experts often have a great understanding of what affects a lot of our users. For example, here is a blog post detailing how Product Expert feedback about the Search Console was used to build the new version of the tool.

Visit the new Product Experts program website to get information on how to become a Product Expert yourself, and come and join us on our Webmaster forums, we’d love to hear from you!

Written by Vincent Courson, Search Outreach team

The new Search Console is graduating out of Beta 🎓

Today we mark an important milestone in Search Console’s history: we are graduating the new Search Console out of beta! With this graduation we are also launching the Manual Actions report and a “Test Live” capability to the recently launched URL inspection tool, which are joining a stream of reports and features we launched in the new Search Console over the past few months.

Our journey to the new Search Console

We launched the new Search Console at the beginning of the year. Since then we have been busy hearing and responding to your feedback, adding new features such as the URL Inspection Tool, and migrating key reports and features. Here's what the new Search Console gives you:

More data:

  • Get an accurate view of your website content using the Index Coverage report.
  • Review your Search Analytics data going back 16 months in the Performance report.
  • See information on links pointing to your site and within your site using the Links report.
  • Retrieve crawling, indexing, and serving information for any URL directly from the Google index using the URL Inspection Tool.

Better alerting and new "fixed it" flows:

  • Get automatic alerts and see a listing of pages affected by Crawling, Indexing, AMP, Mobile Usability, Recipes, or Job posting issues.
  • Reports now show the HTML code where we think a fix necessary (if applicable).
  • Share information quickly with the relevant people in your organization to drive the fix.
  • Notify Google when you've fixed an issue. We will review your pages, validate whether the issue is fixed, and return a detailed log of the validation findings.

Simplified sitemaps and account settings management:

Out of Beta

While the old Search Console still has some features that are not yet available in the new one, we believe that the most common use cases are supported, in an improved way, in the new Search Console. When an equivalent feature exists in both old and new Search Console, our messages will point users to the new version. We'll also add a reminder link in the old report. After a reasonable period, we will remove the old report.

Read more about how to migrate from old to the new Search Console, including a list of improved reports and how to perform common tasks, in our help center.

Manual Actions and Security Issues alerts

To ensure that you don't miss any critical alerts for your site, active manual actions and security issues will be shown directly on the Overview page in the new console. In addition, the Manual Actions report has gotten a fresher look in the new Search Console. From there, you can review the details for any pending Manual Action and, if needed, file a reconsideration request.

URL Inspection - Live mode and request indexing

The URL inspection tool that we launched a few months ago now enables you to run the inspection on the live version of the page. This is useful for debugging and fixing issues in a page or confirming whether a reported issue still exists in a page. If the issue is fixed on the live version of the page, you can ask Google to recrawl and index the page.

We're not finished yet!

Your feedback is important to us! As we evolve Search Console, your feedback helps us to tune our efforts. You can still switch between the old and new products easily, so any missing functionality you need is just a few clicks away. We will continue working on moving more reports and tools as well as adding exciting new capabilities to the new Search Console.


Collaboration and user management in the new Search Console

As part of our reinvention of Search Console, we have been rethinking the models of facilitating cooperation and accountability for our users. We decided to redesign the product around cooperative team usage and transparency of action history. The new Search Console will gradually provide better history tracking to show who performed which significant property-affecting modifications, such as changing a setting, validating an issue or submitting a new sitemap. In that spirit we also plan to enable all users to see critical site messages.

New features

  • User management is now an integral part of Search Console.
  • The new Search Console enables you to share a read-only view of many reports, including Index coverage, AMP, and Mobile Usability. Learn more.
  • A new user management interface that enables all users to see and (if appropriate), manage user roles for all property users.

New Role definition

  • In order to provide a simpler permission model, we are planning to limit the "restricted" user role to read-only status. While being able to see all information, read-only users will no longer be able to perform any state-changing actions, including starting a fix validation or sharing an issue.

Best practices

As a reminder, here are some best practices for managing user permissions in Search Console:

User feedback

As part of our Beta exploration, we released visibility of the user management interface to all user roles. Some users reached out to request more time to prepare for the updated user management model, including the ability of restricted and full users to easily see a list of other collaborators on the site. We’ve taken that feedback and will hold off on that part of the launch. Stay tuned for more updates relating to collaboration tools and changes on our permission models.

As always, we love to hear feedback from our users. Feel free to use the feedback form within Search Console, and we welcome your discussions in our help forums as well!


Links, Mobile Usability, and site management in the new Search Console

More features are coming to the new Search Console. This time we've focused on importing existing popular features from the old Search Console to the new product.

Links Report

Search Console users value the ability to see links to and within their site, as Google Search sees them. Today, we are rolling out the new Links report, which combines the functionality of the “Links to your site” and “Internal Links” reports on the old Search Console. We hope you find this useful!

Mobile Usability report

Mobile Usability is an important priority for all site owners. In order to help site owners with fixing mobile usability issues, we launched the Mobile Usability report on the new Search Console. Issue names are the same as in the old report but we now allow users to submit a validation and reindexing request when an issue is fixed, similar to other reports in the new Search Console.

Site and user management

To make the new Search Console feel more like home, we’ve added the ability to add and verify new sites, and manage your property's users and permissions, directly in new Search Console using our newly added settings page.

Keep sending feedback

As always, we would love to get your feedback through the tools directly and our help forums so please share and let us know how we're doing.