This year in Search Spam – Webspam report 2018

Google aims to provide the highest quality results for any search. As part of this, we take action to prevent what we call “webspam” from degrading the search experience, content and behaviors that violate our webmaster guidelines. Our efforts help ensure that well under 1 percent of results visited by users are for spammy pages. Here’s more about how we fought webspam in 2018.



Google webspam trends and how we fought webspam in 2018



Of the types of spam we fought in 2018, three continue to stand out:


Spam on hacked websites: We reported in 2017 that we had seen a substantial reduction of spam from hacked websites in search results. This trend continued in 2018, with faster discovery of hacked web pages before they affect search results or put someone in harm’s way.   While we reduced how spam on hacked sites affects search, hacked websites remain a major security problem affecting the safety of the web. Even though we can’t prevent a website hack from happening, we’re committed to helping webmasters whose websites have been compromised by offering resources to help them recover from a hacked website. 


User-generated spam: A particular type of spam known as User-generated spam has been a continued focus for us. User-generated spam includes spammy posts on forums, as well as spammy accounts on free blogs and platforms, none of which are meant to be consumed by human beings, and all of which disrupt conversations while adding no value to users. In 2018, we were able to reduce the impact on search users from this type of spam by more than 80%. While we can’t prevent websites from being exploited, we do want to make it easier for website owners to learn how to protect themselves, which is why we provide resources on how to prevent abuse of your site’s public areas.


Link spam: We continued to protect the value of authoritative and relevant links as an important ranking signal for Search. We continued to deal swiftly with egregious link spam, and made a number of bad linking practices less effective for manipulating ranking. Above all, we continued to engage with webmasters and SEOs to chip away at the many myths that have emerged over the years relating to linking practices. We continued to remind website owners that if you simply stay away from building links mainly as an attempt to rank better and focus on creating great content, you should not have to worry about any of the myths or realities. We think that one of the best ways of fighting spam of all types is by encouraging website owners to just create great quality content. Resources such as the SEO starter-guide highlight best practices and bust some common myths and misconceptions related to what it takes to appear well in Google Search results. Reporting link spam is also a great way to assist us in fighting against this type of abuse and to help preserve fairness in Search ranking.



Working with users, webmasters and developers for a better web

Everyday users continue to help us find spam, malware and other issues in Search that escape our filters and processes by reporting spam on search, reporting phishing or  reporting malware. We received over 180,000 search spam user reports and we were able to take action on 64% of the reports we processed. These reports truly make a difference and we’d like to thank all of you who submitted them. 


We think it’s important to let website owners known when we detect something wrong with their website. In 2018, we generated over 186 million messages to website owners calling out potential improvements, issues and problems that could affect their site’s appearance on Search results. We can only deliver these notifications to site owners that verified their sites in Search Console, and we successfully delivered 96 million of those messages. The rest of the messages will be kept linked with the website for as long as they are relevant, so they can be seen when a webmaster successfully registers their site in Search Console. The majority of these messages were welcoming new users to Search Console, and the second largest group was informing registered Search Console users when Mobile-First Indexing became available. Of all messages, slightly over 2%—about 4 million—were related to manual actions resulting from violations of our Webmaster Guidelines. 


High quality content keeps spam off of search results, and we continued to improve the tools and reports we offer for webmasters that create that content. The Google Search Console was completely rebuilt from the ground up to provide both new and improved reports (Performance, Index Coverage, Links, Mobile Usability report), as well as brand new features (URL Inspection Tool and Site and User management). This improved Search Console graduated out of beta in 2018 and is now available generally to all registered website owners.


We didn’t forget the front-end developers who make the modern web work, and focused on helping them make their sites great for users and also search-friendly regardless of whether they are on a CMS, roll their own CSS and JS, or build on top of a web framework. With the new SEO audit capability in Lighthouse, the open-source and automated auditing tool for improving the quality of web pages, developers and webmasters can now run actionable SEO health-checks on their pages and quickly identify areas for improvement.


We also engage directly with website owners to provide help with thorny issues. Our dedicated team members meet with webmasters around the world regularly, both online and in-person. We delivered more than 190 online office hours, online events and offline events in more than 76 cities, to audiences totaling over 170,000 including SEOs, developers and online marketers. We hosted four search events in Tokyo, Singapore, Zurich and Osaka as well as an 11-city Search Conference in India. In 2018, we started live office hours in Spanish on top of English, French, German, Hindi and Japanese, where Webmasters can find help, tips and useful discussion on our Google Webmaster YouTube channel. Product experts continued to help webmasters find solutions through our official support forums in over a dozen languages. 


We look forward to continuing our work to deliver a spam-free Search experience to all in 2019!


Posted by Juan Felipe Rincón, Webmaster Outreach, Dublin

Dev Channel Update for Chrome OS

The Dev channel has been updated to 74.0.3729.0 (Platform version: 11895.21.0) for most Chrome OS devices. This build contains a number of bug fixes, security updates and feature enhancements. A list of changes can be found here.


If you find new issues, please let us know by visiting our forum or filing a bug. Interested in switching channels? Find out how. You can submit feedback using ‘Report an issue...’ in the Chrome menu (3 vertical dots in the upper right corner of the browser).

Daniel Gagnon
Google Chrome

Managed Google Play earns key certifications for security and privacy


Posted by Mike Burr, Android Enterprise Platform Specialist

[Cross-posted from the Android Enterprise Keyword Blog]



With managed Google Play, organizations can build a customized and secure mobile application storefront for their teams, featuring public and private applications. Organizations' employees can take advantage of the familiarity of a mobile app store to browse and download company-approved apps.
As with any enterprise-grade platform, it's critical that the managed Google Play Store operates with the highest standards of privacy and security. Managed Google Play has been awarded three important industry designations that are marks of meeting the strict requirements for information security management practices.
Granted by the International Organization for Standardization, achieving ISO 27001 certification demonstrates that a company meets stringent privacy and security standards when operating an Information Security Management System (ISMS). Additionally, managed Google Play received SOC 2 and 3 reports, which are benchmarks of strict data management and privacy controls. These designations and auditing procedures are developed by the American Institute of Certified Public Accountants (AICPA).
Meeting a high bar of security management standards
To earn the ISO 27001 certification, auditors from Ernst and Young performed a thorough audit of managed Google Play based on established privacy principles. The entire methodology of documentation and procedures for managing other companies' data are reviewed during an audit, and must be made available for regular compliance review. Companies that use managed Google Play are assured their data is managed in compliance with this industry standard. Additionally, ISO 27001 certification is in line with GDPR compliance.
Secure data management
With SOC 2 and SOC 3 reports, the focus is on controls relevant to data security, availability, processing integrity, confidentiality and privacy, which are verified through auditing reports. In managed Google Play, the data and private applications that enter Google's systems are administered according to strict protocols, including determinations for who can view them and under what conditions. Enterprises require and receive the assurance that their information is handled with the utmost confidentiality and that the integrity of their data is preserved. For many companies, the presence of an SOC 2 and 3 report is a requirement when selecting a specific service. These reports prove that a service company has met and is abiding by best practices set forth by AICPA to ensure data security.
Our ongoing commitment to enterprise security
With managed Google Play, companies' private apps for internal use are protected with a set of verified information security management processes and policies to ensure intellectual property is secure. This framework includes managed Google Play accounts that are used by enterprise mobility management (EMM) partners to manage devices.
Our commitment is that Android will continue to be a leader in enterprise security. As your team works across devices and shares mission-critical data through applications hosted in managed Google Play, you have the assurance of a commitment to providing your enterprise the highest standards of security and privacy.

Build your next iOS and Android app with Flutter

Mobile development is full of compromises. When you’re building a new app, you’re often forced into a difficult choice: do you build the same app twice—once for iOS, once for Android—so that you have a high quality experience for both platforms? Or do you create one app from a shared codebase that works across both platforms, but doesn't have the performance or user experience you were hoping for? Flutter offers a third way: enabling high-quality user experiences with excellent performance, along with letting you express your designers’ intent and share a common codebase.

Whether you're an entrepreneur with a new app idea, a developer who’s frustrated by the edit-compile-debug cycle of building an app, or a designer who wants to be able to iterate on new design concepts, here are a few reasons why you should consider creating your next app with Flutter.

Beautifully-designed apps on each platform

Creating app designs can involve trade-offs between the creative intent of a designer and the reality of what actually goes into production. Mobile designers and developers often work in different worlds, separated by a hard boundary between the different tools that they use and challenges with iterating on a design during the development process. Sometimes the designer’s vision is compromised by limitations in the APIs or framework the developer uses; sometimes, visual polish gets deferred until “later” because of other development work (and “later” often means “never” in practice).

Two mobile phone screens showing the differences between what was designed, and what was sent into production.

With Flutter, you have control over every pixel on the screen from the beginning, including a full set of widgets that deliver pixel-perfect experiences on both iOS and Android. Designers are using Flutter to create attractive experiences like Reflectly, the number one journaling app on the Apple app store:

The user interface of Reflectly, a journaling app.

There are also some design tools made for Flutter—like 2Dimensions Flare, which you can use to build animations and incorporate them into any app with a single line of code. Here’s an example of a custom animation built with Flare that interactively follows your typing at login:

A mobile log-in screen displaying a bear who follows your cursor as it moves across the screen.

You can find more details about this example here.

And Supernova, a design-to-code tool, recently announced support for exporting Sketch designs directly to Flutter, allowing users of this popular design and wire-framing tool to turn their ideas directly into code.

Fast apps on each platform

Rather than introducing a layer of abstraction between your code and the underlying operating system, Flutter apps are native apps—meaning they compile directly to both iOS and Android devices.

Flutter’s programming language, Dart, is designed around the needs of apps that are created for global audiences. It’s easy to learn, contains a comprehensive set of libraries and packages that reduce the amount of code you have to write and is built for developer performance. When you’re ready to release your app, you can compile your code directly to the ARM machine code of your phone—meaning what you write is exactly what appears on the device—so you can harness the full power of your phone, rather than using a language like JavaScript that needs a separate engine to run.

A side-by-side comparison of two mobile screens showing Flutter rendering animations on the phone in real-time.

Flutter rendering animations on the phone in real-time.

Flutter isn’t a games engine, but it brings games-level performance to your application. Every pixel in Flutter is drawn with the Skia graphics engine: the same hardware-accelerated engine that powers Android and Chrome. This combination enables fast, glitch-free performance for apps—meaning that apps can operate on a phone's screen at 60 frames per second—which will never feel sluggish, even on a slower device.

Productive app creation on each platform

If you’re a mobile app developer, you might feel like you spend more time waiting than coding. When you make a change to your code, you have to recompile it, deploy it to a mobile phone and then bring the app back into the same state it was in before you made the change to see the results. By contrast, Flutter introduces a new capability called Stateful Hot Reload, which transforms this development cycle, letting you implement changes in milliseconds—not minutes. Stateful Hot Reload also allows better collaboration between developers and designers when they want to improve the app design and immediately see the effects.

Two side-by-side screens show how an app’s user interface and logic are updated in the app without the need for recompilation.

Changes to an app’s user interface and logic are updated in the app without the need for recompilation.

Teams using Stateful Hot Reload report major gains to their productivity when making apps. Combining the quick turnaround on changes with the ability to ship for both iOS and Android, we’ve seen apps from brands like Abbey Road Studios, Hamilton and Reflectly go from writing their first line of code to a published app in just weeks.

Get started today

We just launched version 1.2 of Flutter at Mobile World Congress 2019. We encourage you to try Flutter by visiting flutter.dev. You’ll find all the resources you need to get started including videos, codelabs, case studies, documentation and community links.

Managed Google Play earns key certifications for security and privacy

With managed Google Play, organizations can build a customized and secure mobile application storefront for their teams, featuring public and private applications. Organizations’ employees can take advantage of the familiarity of a mobile app store to browse and download company-approved apps.

As with any enterprise-grade platform, it’s critical that the managed Google Play Store operates with the highest standards of privacy and security. Managed Google Play has been awarded three important industry designations that are marks of meeting the strict requirements for information security management practices.

Granted by the International Organization for Standardization, achieving ISO 27001 certification demonstrates that a company meets stringent privacy and security standards when operating an Information Security Management System (ISMS). Additionally, managed Google Play received SOC 2 and 3 reports, which are benchmarks of strict data management and privacy controls. These designations and auditing procedures are developed by the American Institute of Certified Public Accountants (AICPA).

Meeting a high bar of security management standards

To earn the ISO 27001 certification, auditors from Ernst & Young performed a thorough audit of managed Google Play based on established privacy principles. The entire methodology of documentation and procedures for managing other companies’ data are reviewed during an audit, and must be made available for regular compliance review. Companies that use managed Google Play are assured their data is managed in compliance with this industry standard. Additionally, ISO 27001 certification is in line with GDPR compliance.

Secure data management

With SOC 2 and SOC 3 reports, the focus is on controls relevant to data security, availability, processing integrity, confidentiality and privacy, which are verified through auditing reports. In managed Google Play, the data and private applications that enter Google’s systems are administered according to strict protocols, including determinations for who can view them and under what conditions. Enterprises require and receive the assurance that their information is handled with the utmost confidentiality and that the integrity of their data is preserved. For many companies, the presence of an SOC 2 and 3 report is a requirement when selecting a specific service. These reports prove that a service company has met and is abiding by best practices set forth by AICPA to ensure data security.

Our ongoing commitment to enterprise security

With managed Google Play, companies’ private apps for internal use are protected with a set of verified information security management processes and policies to ensure intellectual property is secure. This framework includes managed Google Play accounts that are used by enterprise mobility management (EMM) partners to manage devices.

Our commitment is that Android will continue to be a leader in enterprise security. As your team works across devices and shares mission-critical data through applications hosted in managed Google Play, you have the assurance of a commitment to providing your enterprise the highest standards of security and privacy.

How El País used AI to make their comments section less toxic

At El País, our vision for the perfect comments section was a place where readers would provide input, insight and tips on an investigative story, add knowledge about niche topics, double check facts and elevate the conversation to a different level. While the internet has brought amazing benefits, it didn’t deliver the utopia we—and others—had hoped for within the comments section. Around 2015, trolls, toxic comments, spam, insults and even threats took over, causing publishers to re-evaluate investing in this section of the online paper. No one seemed able to fix this broken system and several several sites either limited the amount of articles opened to comment, or shut them down completely.

We also thought about closing down comments at El País, but ultimately let them be. That was until last year, when the Google News Initiative contacted us to talk about Perspective API,  a free tool developed by Jigsaw that uses a machine learning model trained by human-generated comments labeled as toxic by human moderators. At that point Perspective API was available in English, but the aim was to use the more than 300,000 comments our readers write every month to train the model in Spanish. Earlier this year, we  partnered with Jigsaw to analyze our vast trove of public comments to understand how to spot toxicity in Spanish. We worked closely with the Jigsaw team to test the models and provided feedback in order to improve overall accuracy of the tool.

Now, when someone tries to post a toxic comment on our site we’ll show them a message in real time suggesting they make changes or rewrite it so that it’ll pass our moderation system. Since we put this system in place, the average toxicity of the comments has gone down seven percent and the number of comments has gone up 19%—leading us to suspect that the comments section is a nicer place and one our readers want to engage in. We’ve also improved the moderation process by sending the more toxic comments to experienced moderators and the less toxic to the less experienced ones.

A message on El Pais' site popping up in response to a toxic comment.

When someone posts a comment that may be perceived as toxic on El País’ site, we show a message in real time suggesting they make changes or re-write it.

Additionally, we’re including the toxicity of the articles in our data warehouse (a system we use for data analysis).Alongside the sentiment of the articles, we can now check if certain authors are associated with toxic comments, if the sentiment of the articles influences toxicity, or if some commenters always have high toxicity across all their comments. With this data, we aim to improve the conversation by not running certain articles we know won’t generate positive comments, developing a new badges system, highlighting top comments of the week in new products like a newsletter and even having journalists engage in certain conversations to help raise the level of debate.

Perspective API has rekindled our faith in the comments section and demonstrated a real value to our publication and our readers. This initial Spanish version of Perspective is available to anyone today and will continue to be developed so that any Spanish publisher can use Perspective in Spanish, for free.


Structured Data Files v4.2 now available in the DoubleClick Bid Manager API

Today we're announcing the general availability of Structured Data Files (SDF) v4.2 in the DoubleClick Bid Manager API. Highlights of this release include:
  • Support for TrueView connected TV bid adjustments
  • Ability to inherit Insertion Order start and end dates at the Line Item level
  • Renaming changes to better align SDF with the Display & Video 360 UI
All SDF users are encouraged to begin requesting v4.2 files to take advantage of these new features. To do so, simply pass 4.2 as the value of version when calling Sdf.download. For users with workflows that are dependent on older SDF formats, details of the file format changes between versions can be found in the release notes.


Digital News Innovation Fund: three years in, and 662 total projects supported

In 2015, Google launched the DNI Fund, an initiative open to publishers of all sizes in Europe that financially supports high-quality journalism through technology and innovation. After three years and six successful rounds of funding, we’re taking a look at the impact the Fund has had on the European news scene. Today, we’re announcing the results of the last round: 103 projects selected in 23 countries and more than 25 million Euros offered. This brings the total to 662 projects supported in more than 30 countries across six funding rounds.

Graphic depicting DNI’s total funding across Europe since its launch.

DNI’s total funding across Europe since its launch.

Welcoming newcomers to the DNI Fund

As with the previous rounds, we were impressed by the diversity and the quality of the projects that have been submitted. 162 of the applications came from collaborations between publishers, startups and academics. It was also heartening to see that 52 percent of applicants of the applicants (351 total, about 52 percent of applicants) were newcomers to the DNI Fund.

A graphic depicting round 6 of the DNI Fund, with results broken down by country.

Round 6 projects, broken down by country.

Once again, we asked large (up to €1 million) and medium (up to €300,000) applicants to focus on one of the most pressing issues identified by the news ecosystem: the diversification of revenue streams. We were excited to see such a wide variety of approaches from some of the biggest names in the industry alongside relative newcomers. Artificial intelligence and machine learning projects continued to be a top technology focus, and Round 6 applications demonstrated clear interest in exploring opportunities around driving subscriptions, creating new payment models and finding ways to minimize churn.

Across the prototype track (up to €50,000 awards), applicants offered plenty of stimulating new thinking, with individuals, organizations and companies looking at everything from fact checking to augmented reality.

Here’s a small selection of the projects that were offered funding in this round:

Single Sign-On, SSO Geste, France

The largest collaboration of publishers ever seen in France has formed a consortium and will receive €750,000 to create a Single Sign-On platform for more than 22 major media groups.

The ThinkIn Organised Listening Platform, Tortoise, UK

This start-up company is developing a new type of participation tool for users of its “slow journalism” enterprise, which will be the digital equivalent of giving members a seat at the newsroom table. It was awarded €553,000.

Readers club, DuMont.next, Germany

Digital unit of German publisher Dumont Mediengruppe will receive €475,000 to launch a gamified local rewards program. It will increase engagement and e-commerce on its local Hamburger Morgenpost site by rewarding all registered users on their sites with a virtual currency that can be used for products on their other platforms.

SAC (Subscription Accelerator Content), Diario de Navarra, Spain

The SAC is a collaborative project between this regional Spanish publisher and Hiberius Media Lab to develop a tool using AI and machine learning. The tool will provide real-time data-driven insights on which pieces of their published content successfully converts readers into subscribers and which retains them as engaged readers. It will receive €206,150.

PressHub Market, Freedom House, Romania

This Romanian journalism platform will receive €49,700 to develop revenue opportunities for its small independent member publications by building an advertisement marketplace for them to access and profit from.

Launching the Google News Innovation Challenge in Europe to support local news

Stimulating innovation within the news industry takes time. We’re in it for the long haul. That’s why we launched the Google News Initiative one year ago, a $300 million commitment to help journalism thrive. As part of these efforts—and after a pilot in Asia—the GNI Innovation Challenge will roll out in Europe. For this first round, we’ll specifically look for ideas around “local” journalism, since we know that building a sustainable business model for local news is a challenge here in Europe, similar to other parts of the world. The application window for this first European GNI Challenge will open this spring. In the meantime, I invite all applicants to check our website for more information, specific dates and eligibility requirements, which will be announced soon. We look forward to receiving GNI Challenge applications, as well as continuing to learn from the news ecosystem and participating publishers.

Honoring J.S. Bach with our first AI-powered Doodle

Ever wondered what Johann Sebastian Bach would sound like if he rocked out? You can find out by exploring today’s AI-powered Google Doodle, which honors Bach’s birthday and legacy as one of the greatest composers of all time. A musician and composer during the Baroque period of the 18th century, Bach produced hundreds of compositions including cantatas, concertos, suites and chorales. In today’s Doodle, you can create your own melody, and through the magic of machine learning, the Doodle will harmonize your melody in Bach’s style. You can also explore inside the Doodle to see how the model Bach-ifys familiar tunes, or how your new collaboration might sound in a more modern rock style.

Today’s Doodle is the result of a collaboration between the Doodle, Magenta and PAIR teams at Google. The Magenta team aims to help people make music and art using machine learning, and PAIR produces tools or experiences to make machine learning enjoyable for everyone.

The first step in creating an AI-powered Doodle was building a machine learning model to power it. Machine learning is the process of teaching a computer to come up with its own answers by showing it a lot of examples, instead of giving it a set of rules to follow (as is done in traditional computer programming). Anna Huang, an AI Resident on Magenta, developed Coconet, a model that can be used in a wide range of musical tasks—such as harmonizing melodies, creating smooth transitions between disconnected fragments of music and composing from scratch (check out more of these technical details in today’s Magenta blog post).

Next, we personalized the model to match Bach’s musical style. To do this, we trained Coconet on 306 of Bach’s chorale harmonizations. His chorales always have four voices: each carries their own melodic line, creating a rich harmonic progression when played together. This concise structure makes the melodic lines good training data for a machine learning model. So when you create a melody of your own on the model in the Doodle, it harmonizes that melody in Bach's specific style.

Beyond the artistic and machine learning elements of the Doodle, we needed a lot of servers in order to make sure people around the world could use the Doodle. Historically, machine learning has been run on servers, which means that info is sent from a person’s computer to data centers, and then the results are sent back to the computer. Using this same approach for the Bach Doodle would have generated a lot of back-and-forth traffic.

To make this work, we used PAIR’s TensorFlow.js, which allows machine learning to happen entirely within an internet browser. However, for cases where someone’s computer or device might not be fast enough to run the Doodle using TensorFlow.js, the machine learning model is run on Google’s new Tensor Processing Units (TPUs), a way of quickly handling machine learning tasks in data centers. Today’s Doodle is the first one ever to use TPUs in this way.

Head over to today’s Doodle and find out what your collaboration with the famous composer sounds like!

Update on the Google Groups Settings API

Posted by Zerzar Bukhari, Product Manager, G Suite

In February 2019, we announced upcoming changes to the Google Groups Settings API. Based on your feedback, we're making improvements to the Groups API to make it easier for you to assess the impact and take action. For the full list of changes, see this help center article.

When will API changes take effect?

The new features will be available starting March 25, 2019. It may take up to 72 hours for the features to rollout to everyone

What's changing?

  • Property 'membersCanPostAsTheGroup' will not be merged into 'whoCanModerateContent'
  • Property 'messageModerationLevel' will continue to support MODERATE_NEW_MEMBERS (it will not be deprecated)
  • New property 'customRoleUsedInMergedSetting'
    • This will indicate if a group uses custom roles in one of the merged settings. If a group uses a custom role, review the permissions in the Groups interface. The Groups API doesn't support custom roles and may report incorrect values for permissions.
  • New properties representing all to-be-merged settings, as well as the new settings, will be added
  • New property 'whoCanDiscoverGroup' to indicate the upcoming behavior for 'showInGroupDirectory'

For complete detail on Groups Settings API behavior changes, please reference this table.