Google’s $53 billion impact in Australia

Today, I’m pleased to launch Google’s 2020 Economic Impact Report - a look at our role in Australia which shows we’re providing businesses and our users with a combined $53 billion in benefits every year. 


Key findings: 

  • 1.3 million businesses receive $39 billion in benefits through increased revenues, millions of connections with customers and greater efficiencies, saving time and money

  • Consumers receive $14 billion in benefits via productivity, convenience and access to information

  • Search saves users almost 5 days a year, drivers save 5.6 hours per year using Google Maps

  • Australian app developers earned $639 million through Google Play reaching 1 billion users globally

  • 60% go to small to medium businesses; 90% of benefits go to non-technology industries


Google’s economic impact in Australia


Google  started our first Australian office in a Sydney lounge room almost twenty years ago and in that time there’s been incredible change. Our team in 2002 - just one person at the time - has now grown to be 1,800 strong and by enabling business expansion, our digital services like Google Ads and Google Play support an additional 116,200 jobs across the country. And many of our products, which were simply ideas back then, have grown to become an important part of the everyday lives of millions of Australians.


In all of that time, there’s never been a year with so much change as we’ve just seen in 2020. The impacts of the pandemic and its effects on businesses has been overwhelming. 


But at the same time, it’s been inspiring to watch the way businesses across Australia have managed those challenges to cope. Our Economic Impact Report shows how businesses have increasingly moved online in this difficult year to provide vital services and succeed.


As our report demonstrates, there are now more than 1.3 million businesses in Australia using Google’s free tools and services-to reach new customers, advertise effectively where they couldn’t before and make use of new digital skills.


Helping businesses stay connected during the coronavirus pandemic


One of those companies using Google’s tools is Bliss Gifts and Homewares, based on the South Coast of New South Wales. Early in 2020, the business was impacted by the devastating bushfire crisis before the coronavirus pandemic compounded the situation. 


For a small business like Bliss located in a tourist town, the effects of the bushfires and COVID should have been devastating. But due to the fact they were already online and with the help of tools like Google Ads, owner Melissa Stone was remarkably able to not only grow the business but saw their revenue jump by 70% during COVID. Bliss’ online presence is now 90 per cent of the business with the help of Google’s tools. 


Video of Bliss Gifts & Homewares
10:25

All over Australia, businesses like Melissa’s shared a total of $39 billion in benefits through increased revenues, millions of connections with customers and greater efficiencies, saving time and money


90 per cent of those benefits went to industries outside of technology - like retail, construction and professional services. And 60 per cent were shared among small to medium businesses.


The report shows Australian app developers earned around $639 million through Google Play, reaching more than one billion users worldwide.


Helping our users save time and access important information 


Google’s products also provided  $14 billion in annual benefits to consumers through increased productivity, convenience and improved access to information.


On average, Australians using Google Search save almost five days a year thanks to access to instantaneous information, while Australian drivers using Google Maps saved 5.6 hours on roads each year by optimising trips through our technology. 


We’re humbled by these findings and have been inspired by the resilience and spirit of the people in businesses across the country. Australia has ambitions to be a leading digital economy and we look forward to continuing to support that ambition. 


You can read more about these benefits here. 


AG Paxton’s misleading attack on our ad tech business

In December, Texas Attorney General Paxton filed a complaint about our ad tech business and hired contingency-fee plaintiff lawyers to handle the case. We look forward to showing in court why AG Paxton’s allegations are wrong. But given some of the misleading claims that have been circulating—in particular, the inaccurate portrayal of our well-publicized “Open Bidding” agreement with Facebook—we wanted to set the record straight.  

About our ad services 

Ad tech helps websites and apps make money and fund high-quality content. It also helps our advertising partners—most of whom are small merchants—reach customers and grow their businesses.  

AG Paxton tries to paint Google’s involvement in this industry as nefarious. The opposite is true. Unlike some B2B companies in this space, a consumer internet company like Google has an incentive to maintain a positive user experience and a sustainable internet that works for all—consumers, advertisers and publishers.

For example, as we’ve built our ad tech products, we have given people granular controls over how their information is used to personalize ads and limited the sharing of personal data to safeguard people’s privacy. We’ve invested in detecting and blocking harmful ads that violate our policies. We also build tools that load content and ads faster; block scammy ad experiences like pop-ups; and reduce the number of intrusive, annoying ads through innovations like skippable ads. Those tools not only help people, but by building trust, promote the sustainability of the free and open internet. 

We’ve worked to be open and upfront with the industry about the improvements we make to our technologies. We try to do the right thing as we balance the concerns of publishers, advertisers, and the people who use our services. Our ad tech rivals and large partners may not always like every decision we make—we’re never going to be able to please everybody. But that’s hardly evidence of wrongdoing and certainly not a credible basis for an antitrust lawsuit.

Here are just a few of the things AG Paxton’s complaint gets wrong:

Myth: Google “dominates the online advertising landscape for image-based web display ads.”
Fact: The ad tech industry is incredibly crowded and competitive.

Competition in online advertising has made ads more affordable and relevant, reduced ad tech fees, and expanded options for publishers and advertisers.

The online advertising space is famously crowded. We compete with household names like Adobe, Amazon, AT&T, Comcast, Facebook, Oracle, Twitter and Verizon. Facebook, for example, is the largest seller of display ads and Amazon last month surpassed us as the preferred ad buying platform for advertisers. We compete fiercely with those companies and others such as Mediaocean, Amobee, MediaMath, Centro, Magnite, The Trade Desk, Index Exchange, OpenX, PubMatic and countless more. A growing number of retail brands such as Walmart, Walgreens, Best Buy, Kroger and Target are also offering their own ad tech.

Myth: Google “extracts a very high ... percent of the ad dollars otherwise flowing to online publishers.”
Fact: Our fees are actually lower than reported industry averages.

Our ad tech fees are lower than reported industry averages. Publishers keep about 70 percent of the revenue when using our products, and for some types of advertising, publishers keep even more—that’s more money in publishers’ pockets to fund their creation of high-quality content.

Myth: We created an alternative to header bidding that “secretly stacks the deck in Google’s favor.”
Fact: We created Open Bidding to address the drawbacks of header bidding.

Header bidding refers to running an auction among multiple ad exchanges for given ad space. You won’t read this in AG Paxton’s complaint, but the technology has real drawbacks: Header bidding auctions take place within the browser, on your computer or mobile phone, so they require the device to use more data in order to work. This can lead to problems like webpages taking longer to load and device batteries draining faster. And the multilayered complexity of header bidding can lead to fraud and other problems that can artificially increase prices for advertisers, as well as billing discrepancies that can hurt publisher revenue.

So we created an alternative to header bidding, called Open Bidding, which runs within the ad server instead of on your device. This solves many of the problems associated with header bidding. Open Bidding provides publishers access to demand from dozens of networks and exchanges. This helps increase demand for publisher inventory and competition for ad space, which enables publishers to drive more revenue. In fact, our data shows that publishers who decide to use Open Bidding on Ad Manager typically see double-digit revenue increases across our partners and exchange—and they can measure this for themselves. 

Additionally, our publisher platform has always integrated with header bidding, so publishers have the choice to use their preferred bidding solution. Publishers can and do bring bids from non-Google header bidding tools into our platform.

Since we launched Open Bidding, traditional header bidding has continued to grow. In fact, a recent survey shows about 90 percent of publishers currently use header bidding for desktop and 60 percent use header bidding for mobile in-app or in-stream video. Amazon also launched an entirely new competitive header bidding solution, which uses the same server-side approach that we do. Header bidding is an evolving and growing space—and now, as a result of our work, there are alternatives to header bidding that improve the user experience.

Myth: Our Open Bidding agreement with Facebook harms publishers.
Fact: Facebook is one of over 25 partners in Open Bidding, and their participation actually helps publishers.

AG Paxton also makes misleading claims about Facebook’s participation in our Open Bidding program.  Facebook Audience Network (FAN)’s involvement isn’t a secret. In fact, it was well-publicized and FAN is one of over 25 partners participating in Open Bidding. Our agreement with FAN simply enables them (and the advertisers they represent) to participate in Open Bidding. Of course we want FAN to participate because the whole goal of Open Bidding is to work with a range of ad networks and exchanges to increase demand for publishers’ ad space, which helps those publishers earn more revenue. FAN’s participation helps that. But to be clear, Open Bidding is still an extremely small part of our ad tech business, accounting for less than 4 percent of the display ads we place.

AG Paxton inaccurately claims that we manipulate the Open Bidding auction in FAN’s favor. We absolutely don’t. FAN must make the highest bid to win a given impression. If another eligible network or exchange bids higher, they win the auction. FAN’s participation in Open Bidding doesn't prevent Facebook from participating in header bidding or any other similar system. In fact, FAN participates in several similar auctions on rival platforms.

And AG Paxton’s claims about how much we charge other Open Bidding partners are mistaken—our standard revenue share for Open Bidding is 5-10 percent.

Myth: AMP was designed to hurt header bidding.
Fact: AMP was designed in partnership with publishers to improve the mobile web.

AG Paxton’s claims about AMP and header bidding are just false. Engineers at Google designed AMP in partnership with publishers and other tech companies to help webpages load faster and improve the user experience on mobile devices—not to harm header bidding.

AMP supports a range of monetization options, including header bidding. Publishers are free to use both AMP and header bidding technologies together if they choose. The use of header bidding doesn’t factor into publisher search rankings. 

Myth: We force partners to use Google tools.
Fact: Partners can readily use our tools and other technologies side by side. 

This claim isn’t accurate either. Publishers and advertisers often use multiple technologies simultaneously. In fact, surveys show the average large publisher uses six different platforms to sell ads on its site, and plans to use even more this year. And the top 100 advertisers use an average of four or more platforms to buy ads.

All of this is why we build our technologies to be interoperable with more than 700 rival platforms for advertisers and 80 rival platforms for publishers.

AG Paxton’s complaint talks about the idea that we offer tools for both advertisers and publishers as if that’s unusual or problematic. But that reflects a lack of knowledge of the online ads industry, where serving both advertisers and publishers is actually commonplace. Many firms with competing ad tech businesses, such as AT&T, Amazon, Twitter, Verizon, Comcast and others, offer ad platforms and tools like ours that cater to both advertisers and publishers. We don’t require either advertisers or publishers to use our whole “stack,” and many don’t. Ultimately, advertisers and publishers can choose what works best for their needs.

Myth: “Google uses privacy concerns to advantage itself.”
Fact: Consumers expect us to secure their data—and we do.

AG Paxton misrepresents our privacy initiatives. We're committed to operating our advertising business in a way that gives people transparency into and control over how their data is used. Consumers also increasingly expect, and data privacy laws require, strict controls over ad tracking tools like cookies and ad identifiers. So we’re focused on meeting those expectations and requirements. As we do so, we’ve created privacy-protecting solutions that enable other ad tech companies to continue to operate and introduced an open and collaborative industry initiative called the Privacy Sandbox, which is working on alternatives to cookies that preserve privacy while protecting free content. Other web browsers have likewise taken similar steps to limit the use of cookies and protect user privacy.

More information

There are many other things this complaint simply gets wrong. You can read more about our ad tech business by visiting our competition website.

We look forward to defending ourselves in court. In the meantime, we’ll continue our work to help publishers and advertisers grow with digital ads and create a sustainable advertising industry that supports free content for everyone.

Ad Policy Error Management is evolving in Google Ads API

On March 1st, 2021, all versions of the Google Ads API will replace policy violations with policy findings for all remaining ad types.

The impact is limited to the creation and update of ads that trigger ad policy errors for the following types:
  • CALL_ONLY_AD
  • EXPANDED_DYNAMIC_SEARCH_AD
  • GMAIL_AD
  • HTML5_UPLOAD_AD
  • IMAGE_AD
  • LEGACY_APP_INSTALL_AD
  • LOCAL_AD
  • RESPONSIVE_DISPLAY_AD
  • RESPONSIVE_SEARCH_AD
  • VIDEO_RESPONSIVE_AD
If your application is impacted by this change and not upgraded before March 1st, 2021, then the ad policy errors will no longer be recognized and the requested exemptions will not be applied.

What’s Changing
Both AdGroupAdService.MutateAdGroupAds and AdService.MutateAds methods will behave differently: What’s Not Changing What to Do
Before March 1st, 2021, make sure to add the support of policy findings in your management of ad policy errors. To get started, you can refer to our guide and code example that are both dedicated to ad policy error management. We recommend testing with the ad types that already use policy findings: EXPANDED_TEXT_AD and RESPONSIVE_SEARCH_AD.

If you have any questions or need additional help, contact us through the forum or at [email protected].
Pierrick Voulet, Google Ads API Team

Ad Policy Error Management is evolving in Google Ads API

On March 1st, 2021, all versions of the Google Ads API will replace policy violations with policy findings for all remaining ad types.

The impact is limited to the creation and update of ads that trigger ad policy errors for the following types:
  • CALL_ONLY_AD
  • EXPANDED_DYNAMIC_SEARCH_AD
  • GMAIL_AD
  • HTML5_UPLOAD_AD
  • IMAGE_AD
  • LEGACY_APP_INSTALL_AD
  • LOCAL_AD
  • RESPONSIVE_DISPLAY_AD
  • RESPONSIVE_SEARCH_AD
  • VIDEO_RESPONSIVE_AD
If your application is impacted by this change and not upgraded before March 1st, 2021, then the ad policy errors will no longer be recognized and the requested exemptions will not be applied.

What’s Changing
Both AdGroupAdService.MutateAdGroupAds and AdService.MutateAds methods will behave differently: What’s Not Changing What to Do
Before March 1st, 2021, make sure to add the support of policy findings in your management of ad policy errors. To get started, you can refer to our guide and code example that are both dedicated to ad policy error management. We recommend testing with the ad types that already use policy findings: EXPANDED_TEXT_AD and RESPONSIVE_SEARCH_AD.

If you have any questions or need additional help, contact us through the forum or at [email protected].
Pierrick Voulet, Google Ads API Team

Google Workspace Updates Weekly Recap – January 15, 2021

New updates

Unless otherwise indicated, the features below are fully launched or in the process of rolling out (rollouts should take no more than 15 business days to complete), launching to both Rapid and Scheduled Release at the same time (if not, each stage of rollout should take no more than 15 business days to complete), and available to all Google Workspace and G Suite customers.

Smart Compose in Google Docs available in French and Portuguese
Smart Compose helps you write high-quality content in Google Docs faster and more easily—helping you cut back on spelling and grammatical errors and repetitive writing. It's rolling out now in French and Portuguese. | Learn more.


Previous announcements

The announcements below were published on the Workspace Updates blog earlier this week. Please refer to the original blog posts for complete details.

Deploy and manage Google Credential Provider for Windows via the Admin console
You can now deploy and manage Google Credential Provider for Windows (GCPW) in the Admin console. | Learn more.

Expanding the Gmail delegate limit
We’re expanding the number of allowed Gmail delegates from 25. In addition, to help with delegate management, delegation for Contacts is now available via the Contacts API. | Learn more.

New tools to troubleshoot network and performance issues in Google Meet
We’ve added tools in Google Meet that make it easier for end users to understand how their local desktop and network environments affect meeting quality. | Learn more.

New option to block devices with basic management from accessing your organization’s data
We’re adding the ability for admins to manually block or unblock mobile apps from accessing their organization’s Google Workspace data on Android and iOS devices with basic mobile management. | Learn more.

Improved mobile device management rules experience in the Admin console
We’re making improvements to how you manage rules related to mobile device management (MDM) in the Admin console. | Available to Google Workspace Enterprise Standard and Enterprise Plus, as well as G Suite Enterprise for Education and Cloud Identity Premium customers only. | Learn more.

ToTTo: A Controlled Table-to-Text Generation Dataset

Posted by Ankur Parikh and Xuezhi Wang, Research Scientists, Google Research

In the last few years, research in natural language generation, used for tasks like text summarization, has made tremendous progress. Yet, despite achieving high levels of fluency, neural systems can still be prone to hallucination (i.e.generating text that is understandable, but not faithful to the source), which can prohibit these systems from being used in many applications that require high degrees of accuracy. Consider an example from the Wikibio dataset, where the neural baseline model tasked with summarizing a Wikipedia infobox entry for Belgian football player Constant Vanden Stock summarizes incorrectly that he is an American figure skater.

While the process of assessing the faithfulness of generated text to the source content can be challenging, it is often easier when the source content is structured (e.g., in tabular format). Moreover, structured data can also test a model’s ability for reasoning and numerical inference. However, existing large scale structured datasets are often noisy (i.e., the reference sentence cannot be fully inferred from the tabular data), making them unreliable for the measurement of hallucination in model development.

In “ToTTo: A Controlled Table-To-Text Generation Dataset”, we present an open domain table-to-text generation dataset created using a novel annotation process (via sentence revision) along with a controlled text generation task that can be used to assess model hallucination. ToTTo (shorthand for “Table-To-Text”) consists of 121,000 training examples, along with 7,500 examples each for development and test. Due to the accuracy of annotations, this dataset is suitable as a challenging benchmark for research in high precision text generation. The dataset and code are open-sourced on our GitHub repo.

Table-to-Text Generation
ToTTo introduces a controlled generation task in which a given Wikipedia table with a set of selected cells is used as the source material for the task of producing a single sentence description that summarizes the cell contents in the context of the table. The example below demonstrates some of the many challenges posed by the task, such as numerical reasoning, a large open-domain vocabulary, and varied table structure.

Example in the ToTTo dataset, where given the source table and set of highlighted cells (left), the goal is to generate a one sentence description, such as the “target sentence” (right). Note that generating the target sentence would require numerical inference (eleven NFL seasons) and understanding of the NFL domain.

Annotation Process
Designing an annotation process to obtain natural but also clean target sentences from tabular data is a significant challenge. Many datasets like Wikibio and RotoWire pair naturally occurring text heuristically with tables, a noisy process that makes it difficult to disentangle whether hallucination is primarily caused by data noise or model shortcomings. On the other hand, one can elicit annotators to write sentence targets from scratch, which are faithful to the table, but the resulting targets often lack variety in terms of structure and style.

In contrast, ToTTo is constructed using a novel data annotation strategy in which annotators revise existing Wikipedia sentences in stages. This results in target sentences that are clean, as well as natural, containing interesting and varied linguistic properties. The data collection and annotation process begins by collecting tables from Wikipedia, where a given table is paired with a summary sentence collected from the supporting page context according to heuristics, such as word overlap between the page text and the table and hyperlinks referencing tabular data. This summary sentence may contain information not supported by the table and may contain pronouns with antecedents found in the table only, not the sentence itself.

The annotator then highlights the cells in the table that support the sentence and deletes phrases in the sentence that are not supported by the table. They also decontextualize the sentence so that it is standalone (e.g., with correct pronoun resolution) and correct grammar, where necessary.

We show that annotators obtain high agreement on the above task: 0.856 Fleiss Kappa for cell highlighting, and 67.0 BLEU for the final target sentence.

Dataset Analysis
We conducted a topic analysis on the ToTTo dataset over 44 categories and found that the Sports and Countries topics, each of which consists of a range of fine-grained topics, e.g., football/olympics for sports and population/buildings for countries, together comprise 56.4% of the dataset. The other 44% is composed of a much more broad set of topics, including Performing Arts, Transportation, and Entertainment.

Furthermore, we conducted a manual analysis of the different types of linguistic phenomena in the dataset over 100 randomly chosen examples. The table below summarizes the fraction of examples that require reference to the page and section titles, as well as some of the linguistic phenomena in the dataset that potentially pose new challenges to current systems.

Linguistic Phenomena Percentage
Require reference to page title 82%
Require reference to section title 19%
Require reference to table description 3%
Reasoning (logical, numerical, temporal etc.) 21%
Comparison across rows/columns/cells 13%
Require background information 12%

Baseline Results
We present some baseline results of three state-of-the-art models from the literature (BERT-to-BERT, Pointer Generator, and the Puduppully 2019 model) on two evaluation metrics, BLEU and PARENT. In addition to reporting the score on the overall test set, we also evaluate each model on a more challenging subset consisting of out-of-domain examples. As the table below shows, the BERT-to-BERT model performs best in terms of both BLEU and PARENT. Moreover, all models achieve considerably lower performance on the challenge set indicating the challenge of out-of-domain generalization.

  BLEU PARENT BLEU PARENT
Model (overall) (overall) (challenge) (challenge)
BERT-to-BERT 43.9 52.6 34.8 46.7
Pointer Generator 41.6 51.6 32.2 45.2
Puduppully et al. 2019 19.2 29.2 13.9 25.8

While automatic metrics can give some indication of performance, they are not currently sufficient for evaluating hallucination in text generation systems. To better understand hallucination, we manually evaluate the top performing baseline, to determine how faithful it is to the content in the source table, under the assumption that discrepancies indicate hallucination. To compute the “Expert” performance, for each example in our multi-reference test set, we held out one reference and asked annotators to compare it with the other references for faithfulness. As the results show, the top performing baseline appears to hallucinate information ~20% of the time.

  Faithfulness Faithfulness
Model (overall) (challenge)
Expert 93.6 91.4
BERT-to-BERT  76.2 74.2

Model Errors and Challenges
In the table below, we present a selection of the observed model errors to highlight some of the more challenging aspects of the ToTTo dataset. We find that state-of-the-art models struggle with hallucination, numerical reasoning, and rare topics, even when using cleaned references (errors in red). The last example shows that even when the model output is correct it is sometimes not as informative as the original reference which contains more reasoning about the table (shown in blue).

Reference Model Prediction
in the 1939 currie cup, western province lost to transvaal by 17–6 in cape town. the first currie cup was played in 1939 in transvaal1 at new- lands, with western province winning 17–6.
a second generation of micro- drive was announced by ibm in 2000 with increased capacities at 512 mb and 1 gb. there were 512 microdrive models in 2000: 1 gigabyte.
the 1956 grand prix motorcy- cle racing season consisted of six grand prix races in five classes: 500cc, 350cc, 250cc, 125cc and sidecars 500cc. the 1956 grand prix motorcycle racing season consisted of eight grand prix races in five classes: 500cc, 350cc, 250cc, 125cc and sidecars 500cc.
in travis kelce’s last collegiate season, he set personal career highs in receptions (45), re- ceiving yards (722), yards per receptions (16.0) and receiving touchdowns (8). travis kelce finished the 2012 season with 45 receptions for 722 yards (16.0 avg.) and eight touchdowns.

Conclusion
In this work, we presented ToTTo, a large, English table-to-text dataset that presents both a controlled generation task and a data annotation process based on iterative sentence revision. We also provided several state-of-the-art baselines, and demonstrated ToTTo could be a useful dataset for modeling research as well as for developing evaluation metrics that can better detect model improvements.

In addition to the proposed task, we hope our dataset can also be helpful for other tasks such as table understanding and sentence revision. ToTTo is available at our GitHub repo.

Acknowledgements
The authors wish to thank Ming-Wei Chang, Jonathan H. Clark, Kenton Lee, and Jennimaria Palomaki for their insightful discussions and support. Many thanks also to Ashwin Kakarla and his team for help with the annotations.

Source: Google AI Blog


MAD Skills Kotlin and Jetpack: wrap-up

Posted by Florina Muntenescu, Developer Relations Engineer

Kotlin and Jetpack image

We just wrapped up another series of MAD Skills videos and articles - this time on Kotlin and Jetpack. We covered different ways in which we made Android code more expressive and concise, safer, and easy to run asynchronous code with Kotlin.

Check out the episodes below to level up your Kotlin and Jetpack knowledge! Each episode covers a specific set of APIs, talking both about how to use the APIs but also showing how APIs work under the hood. All the episodes have accompanying blog posts and most of them link to either a sample or a codelab to make it easier to follow and dig deeper into the content. We also had a live Q&A featuring Jetpack and Kotlin engineers.

Episode 1 - Using KTX libraries

In this episode we looked at how you can make your Android and Jetpack coding easy, pleasant and Kotlin-idiomatic with Jetpack KTX extensions. Currently, more than 20 libraries have a KTX version. This episode covers some of the most important ones: core-ktx that provides idiomatic Kotlin functionality for APIs coming from the Android platform, plus a few Jetpack KTX libraries that allow us to have a better user experience when working with APIs like LiveData and ViewModel.

Check out the video or the article:

Episode 2 - Simplifying APIs with coroutines and Flow

Episode 2, covers how to simplify APIs using coroutines and Flow as well as how to build your own adapter using suspendCancellableCoroutine and callbackFlow APIs. To get hands-on with this topic, check out the Building a Kotlin extensions library codelab.

Watch the video or read the article:

Episode 3 - Using and testing Room Kotlin APIs

This episode opens the door to Room, peeking in to see how to create Room tables and databases in Kotlin and how to implement one-shot suspend operations like insert, and observable queries using Flow. When using coroutines and Flow, Room moves all the database operations onto the background thread for you. Check out the video or blog post to find out how to implement and test Room queries. For more hands-on work - check out the Room with a view codelab.

Episode 4 - Using WorkManager Kotlin APIs

Episode 4 makes your job easier with WorkManager, for scheduling asynchronous tasks for immediate or deferred execution that are expected to run even if the app is closed or the device restarts. In this episode we go over the basics of WorkManager and look a bit more in depth at the Kotlin APIs, like CoroutineWorker.

Find the video here and the article here, but nothing compares to practical experience so go through the WorkManager codelab.

Episode 5 - Community tip

Episode 5 is by Magda Miu - a Google Developer Expert on Android who shared her experience of leveraging foundational Kotlin APIs with CameraX. Check it out here:

Episode 6 - Live Q&A

In the final episode we launched into a live Q&A, hosted by Chet Haase, with guests Yigit Boyar - Architecture Components tech lead, David Winer - Kotlin product manager, and developer relations engineers Manuel Vivo and myself. We answered questions from you on YouTube, Twitter and elsewhere.

Chrome Beta for Android Update

Hi everyone! We've just released Chrome Beta 88 (88.0.4324.89) for Android: it's now available on Google Play.

You can see a partial list of the changes in the Git log. For details on new features, check out the Chromium blog, and for details on web platform updates, check here.

If you find a new issue, please let us know by filing a bug.

Krishna Govind
Google Chrome

Changes to conversion columns in AdWords API and Scripts

What's changing?
We are introducing restrictions on certain combinations of conversion columns in AdWords API and Google Ads scripts reports. If your reporting queries include these column combinations, you need to fix your queries before Feb 15, 2021.

Technical details
Starting the week of Feb 15, 2021, you will receive a ReportDefinitionError.INVALID_FIELD_NAME_FOR_REPORT error if your AdWords API report request contains columns from both of the restricted column sets listed below. Similarly, calls to the AdsApp.report method in Google Ads scripts will fail for queries with these restricted column combinations.

Restricted conversion columns:
  • ConversionAdjustment
  • ConversionAdjustmentLagBucket
  • ConversionAttributionEventType
  • ConversionCategoryName
  • ConversionLagBucket
  • ConversionTrackerId
  • ConversionTypeName
Metrics columns:
  • AllConversionRate
  • ConversionRate
  • CostPerAllConversion
  • CostPerConversion
  • CostPerCurrentModelAttributedConversion
The ReportDefinitionService.getReportFields method will reflect these restrictions in the exclusiveFields list of each impacted column.

What should you do?
Before Feb 15, 2021, review and modify the reporting queries in your AdWords API and Google Ads scripts applications to stop using the prohibited column combinations.

Why is this changing?
These column combinations are currently disallowed by the Google Ads UI, Google Ads Editor and the Google Ads API. This change makes the AdWords API and Google Ads scripts behaviour consistent with the rest of the Google Ads platform.

If you have any questions or need help, please contact us via the forum.
- Anash P. Oommen, Google Ads API Team

Recognizing Pose Similarity in Images and Videos

Posted by Jennifer J. Sun, Student Researcher and Ting Liu, Senior Software Engineer, Google Research

Everyday actions, such as jogging, reading a book, pouring water, or playing sports, can be viewed as a sequence of poses, consisting of the position and orientation of a person’s body. An understanding of poses from images and videos is a crucial step for enabling a range of applications, including augmented reality display, full-body gesture control, and physical exercise quantification. However, a 3-dimensional pose captured in two dimensions in images and videos appears different depending on the viewpoint of the camera. The ability to recognize similarity in 3D pose using only 2D information will help vision systems better understand the world.

In “View-Invariant Probabilistic Embedding for Human Pose” (Pr-VIPE), a spotlight paper at ECCV 2020, we present a new algorithm for human pose perception that recognizes similarity in human body poses across different camera views by mapping 2D body pose keypoints to a view-invariant embedding space. This ability enables tasks, such as pose retrieval, action recognition, action video synchronization, and more. Compared to existing models that directly map 2D pose keypoints to 3D pose keypoints, the Pr-VIPE embedding space is (1) view-invariant, (2) probabilistic in order to capture 2D input ambiguity, and (3) does not require camera parameters during training or inference. Trained with in-lab setting data, the model works on in-the-wild images out of the box, given a reasonably good 2D pose estimator (e.g., PersonLab, BlazePose, among others). The model is simple, results in compact embeddings, and can be trained (in ~1 day) using 15 CPUs. We have released the code on our GitHub repo.

Pr-VIPE can be directly applied to align videos from different views.

Pr-VIPE
The input to Pr-VIPE is a set of 2D keypoints, from any 2D pose estimator that produces a minimum of 13 body keypoints, and the output is the mean and variance of the pose embedding. The distances between embeddings of 2D poses correlate to their similarities in absolute 3D pose space. Our approach is based on two observations:

  • The same 3D pose may appear very different in 2D as the viewpoint changes.
  • The same 2D pose can be projected from different 3D poses.

The first observation motivates the need for view-invariance. To accomplish this, we define the matching probability, i.e., the likelihood that different 2D poses were projected from the same, or similar 3D poses. The matching probability predicted by Pr-VIPE for matching pose pairs should be higher than for non-matching pairs.

To address the second observation, Pr-VIPE utilizes a probabilistic embedding formulation. Because many 3D poses can project to the same or similar 2D poses, the model input exhibits an inherent ambiguity that is difficult to capture through deterministic mapping point-to-point in embedding space. Therefore, we map a 2D pose through a probabilistic mapping to an embedding distribution, of which we use the variance to represent the uncertainty of the input 2D pose. As an example, in the figure below the third 2D view of the 3D pose on the left is similar to the first 2D view of a different 3D pose on the right, so we map them into a similar location in the embedding space with large variances.

Pr-VIPE enables vision systems to recognize 2D poses across views. We embed 2D poses using Pr-VIPE such that the embeddings are (1) view-invariant (2D projections of similar 3D poses are embedded close together) and (2) probabilistic. By embedding detected 2D poses, Pr-VIPE enables direct retrieval of pose images from different views, and can also be applied to action recognition and video alignment.
View-Invariance
During training, we use 2D poses from two sources: multi-view images and projections of groundtruth 3D poses. Triplets of 2D poses (anchor, positive, and negative) are selected from a batch, where the anchor and positive are two different projections of the same 3D pose, and the negative is a projection of a non-matching 3D pose. Pr-VIPE then estimates the matching probability of 2D pose pairs from their embeddings.
During training, we push the matching probability of positive pairs to be close to 1 with a positive pairwise loss in which we minimize the embedding distance between positive pairs, and the matching probability of negative pairs to be small by maximizing the ratio of the matching probabilities between positive and negative pairs with a triplet ratio loss.
Overview of the Pr-VIPE model. During training, we apply three losses (triplet ratio loss, positive pairwise loss, and a prior loss that applies a unit Gaussian prior to our embeddings). During inference, the model maps an input 2D pose to a probabilistic, view-invariant embedding.
Probabilistic Embedding
Pr-VIPE maps a 2D pose to a probabilistic embedding as a multivariate Gaussian distribution using a sampling-based approach for similarity score computation between two distributions. During training, we use a Gaussian prior loss to regularize the predicted distribution.

Evaluation
We propose a new cross-view pose retrieval benchmark to evaluate the view-invariance property of the embedding. Given a monocular pose image, cross-view retrieval aims to retrieve the same pose from different views without using camera parameters. The results demonstrate that Pr-VIPE retrieves poses more accurately across views compared to baseline methods in both evaluated datasets (Human3.6M, MPI-INF-3DHP).

Pr-VIPE retrieves poses across different views more accurately relative to the baseline method (3D pose estimation).

Common 3D pose estimation methods (such as the simple baseline used for comparison above, SemGCN, and EpipolarPose, amongst many others), predict 3D poses in camera coordinates, which are not directly view-invariant. Thus, rigid alignment between every query-index pair is required for retrieval using estimated 3D poses, which is computationally expensive due to the need for singular value decomposition (SVD). In contrast, Pr-VIPE embeddings can be directly used for distance computation in Euclidean space, without any post-processing.

Applications
View-invariant pose embedding can be applied to many image and video related tasks. Below, we show Pr-VIPE applied to cross-view retrieval on in-the-wild images without using camera parameters.


We can retrieve in-the-wild images from different views without using camera parameters by embedding the detected 2D pose using Pr-VIPE. Using the query image (top row), we search for a matching pose from a different camera view and we show the nearest neighbor retrieval (bottom row). This enables us to search for matching poses across camera views more easily.

The same Pr-VIPE model can also be used for video alignment. To do so, we stack Pr-VIPE embeddings within a small time window, and use the dynamic time warping (DTW) algorithm to align video pairs.

Manual video alignment is difficult and time-consuming. Here, Pr-VIPE is applied to automatically align videos of the same action repeated from different views.

The video alignment distance calculated via DTW can then be used for action recognition by classifying videos using nearest neighbor search. We evaluate the Pr-VIPE embedding using the Penn Action dataset and demonstrate that using the Pr-VIPE embedding without fine-tuning on the target dataset, yields highly competitive recognition accuracy. In addition, we show that Pr-VIPE even achieves relatively accurate results using only videos from a single view in the index set.

Pr-VIPE recognizes action across views using pose inputs only, and is comparable to or better than methods using pose only or with additional context information (such as Iqbal et al., Liu and Yuan, Luvizon et al., and Du et al.). When action labels are only available for videos from a single view, Pr-VIPE (1-view only) can still achieve relatively accurate results.

Conclusion
We introduce the Pr-VIPE model for mapping 2D human poses to a view-invariant probabilistic embedding space, and show that the learned embeddings can be directly used for pose retrieval, action recognition, and video alignment. Our cross-view retrieval benchmark can be used to test the view-invariant property of other embeddings. We look forward to hearing about what you can do with pose embeddings!

Acknowledgments
Special thanks to Jiaping Zhao, Liang-Chieh Chen, Long Zhao (Rutgers University), Liangzhe Yuan, Yuxiao Wang, Florian Schroff, Hartwig Adam, and the Mobile Vision team for the wonderful collaboration and support.

Source: Google AI Blog