Object Detection and Tracking using MediaPipe

Posted by Ming Guang Yong, Product Manager for MediaPipe

MediaPipe in 2019

MediaPipe is a framework for building cross platform multimodal applied ML pipelines that consist of fast ML inference, classic computer vision, and media processing (e.g. video decoding). MediaPipe was open sourced at CVPR in June 2019 as v0.5.0. Since our first open source version, we have released various ML pipeline examples like

In this blog, we will introduce another MediaPipe example: Object Detection and Tracking. We first describe our newly released box tracking solution, then we explain how it can be connected with Object Detection to provide an Object Detection and Tracking system.

Box Tracking in MediaPipe

In MediaPipe v0.6.7.1, we are excited to release a box tracking solution, that has been powering real-time tracking in Motion Stills, YouTube’s privacy blur, and Google Lens for several years and that is leveraging classic computer vision approaches. Pairing tracking with ML inference results in valuable and efficient pipelines. In this blog, we pair box tracking with object detection to create an object detection and tracking pipeline. With tracking, this pipeline offers several advantages over running detection per frame:

  • It provides instance based tracking, i.e. the object ID is maintained across frames.
  • Detection does not have to run every frame. This enables running heavier detection models that are more accurate while keeping the pipeline lightweight and real-time on mobile devices.
  • Object localization is temporally consistent with the help of tracking, meaning less jitter is observable across frames.

Our general box tracking solution consumes image frames from a video or camera stream, and starting box positions with timestamps, indicating 2D regions of interest to track, and computes the tracked box positions for each frame. In this specific use case, the starting box positions come from object detection, but the starting position can also be provided manually by the user or another system. Our solution consists of three main components: a motion analysis component, a flow packager component, and a box tracking component. Each component is encapsulated as a MediaPipe calculator, and the box tracking solution as a whole is represented as a MediaPipe subgraph shown below.

Visualization of Tracking State for Each Box

MediaPipe Box Tracking Subgraph

The MotionAnalysis calculator extracts features (e.g. high-gradient corners) across the image, tracks those features over time, classifies them into foreground and background features, and estimates both local motion vectors and the global motion model. The FlowPackager calculator packs the estimated motion metadata into an efficient format. The BoxTracker calculator takes this motion metadata from the FlowPackager calculator and the position of starting boxes, and tracks the boxes over time. Using solely the motion data (without the need for the RGB frames) produced by the MotionAnalysis calculator, the BoxTracker calculator tracks individual objects or regions while discriminating from others. To track an input region, we first use the motion data corresponding to this region and employ iteratively reweighted least squares (IRLS) fitting a parametric model to the region’s weighted motion vectors. Each region has a tracking state including its prior, mean velocity, set of inlier and outlier feature IDs, and the region centroid. See the figure below for a visualization of the tracking state, with green arrows indicating motion vectors of inliers, and red arrows indicating motion vectors of outliers. Note that by only relying on feature IDs we implicitly capture the region’s appearance, since each feature’s patch intensity stays roughly constant over time. Additionally, by decomposing a region’s motion into that of the camera motion and the individual object motion, we can even track featureless regions.

Visualization of Tracking State for Each Box

An advantage of our architecture is that by separating motion analysis into a dedicated MediaPipe calculator and tracking features over the whole image, we enable great flexibility and constant computation independent of the number of regions tracked! By not having to rely on the RGB frames during tracking, our tracking solution provides the flexibility to cache the metadata across a batch of frame. Caching enables tracking of regions both backwards and forwards in time; or even sync directly to a specified timestamp for tracking with random access.

Object Detection and Tracking

A MediaPipe example graph for object detection and tracking is shown below. It consists of 4 compute nodes: a PacketResampler calculator, an ObjectDetection subgraph released previously in the MediaPipe object detection example, an ObjectTracking subgraph that wraps around the BoxTracking subgraph discussed above, and a Renderer subgraph that draws the visualization.

MediaPipe Example Graph for Object Detection and Tracking. Boxes in purple are subgraphs.

In general, the ObjectDetection subgraph (which performs ML model inference internally) runs only upon request, e.g. at an arbitrary frame rate or triggered by specific signals. More specifically, in this example PacketResampler temporally subsamples the incoming video frames to 0.5 fps before they are passed into ObjectDetection. This frame rate can be configured differently as an option in PacketResampler.

The ObjectTracking subgraph runs in real-time on every incoming frame to track the detected objects. It expands the BoxTracking subgraph described above with additional functionality: when new detections arrive it uses IoU (Intersection over Union) to associate the current tracked objects/boxes with new detections to remove obsolete or duplicated boxes.

A sample result of this object detection and tracking example can be found below. The left image is the result of running object detection per frame. The right image is the result of running object detection and tracking. Note that the result with tracking is much more stable with less temporal jitter. It also maintains object IDs across frames.

Comparison Between Object Detection Per Frame and Object Detection and Tracking

Follow MediaPipe

This is our first Google Developer blog post for MediaPipe. We look forward to publishing new blog posts related to new MediaPipe ML pipeline examples and features. Please follow the MediaPipe tag on the Google Developer blog and Google Developer twitter account (@googledevs)

Acknowledgements

We would like to thank Fan Zhang, Genzhi Ye, Jiuqiang Tang, Jianing Wei, Chuo-Ling Chang, Ming Guang Yong, and Matthias Grundman for building the object detection and tracking solution in MediaPipe and contributing to this blog post.

Stable Channel Update for Desktop

The Chrome team is delighted to announce the promotion of Chrome 79 to the stable channel for Windows, Mac and Linux. This will roll out over the coming days/weeks.

Chrome 79.0.3945.79 contains a number of fixes and improvements -- a list of changes is available in the log. Watch out for upcoming Chrome and Chromium blog posts about new features and big efforts delivered in 79.


Security Fixes and Rewards

Note: Access to bug details and links may be kept restricted until a majority of users are updated with a fix. We will also retain restrictions if the bug exists in a third party library that other projects similarly depend on, but haven’t yet fixed.

This update includes 51 security fixes. Below, we highlight fixes that were contributed by external researchers. Please see the Chrome Security Page for more information.

[$20000][1025067] Critical CVE-2019-13725: Use after free in Bluetooth. Reported by Gengming Liu, Jianyu Chen at Tencent Keen Security Lab on 2019-11-15
[$TBD][1027152] Critical CVE-2019-13726: Heap buffer overflow in password manager. Reported by Sergei Glazunov of Google Project Zero on 2019-11-21
[$10000][944619] High CVE-2019-13727: Insufficient policy enforcement in WebSockets. Reported by @piochu on 2019-03-21
[$7500][1024758] High CVE-2019-13728: Out of bounds write in V8. Reported by Rong Jian and Guang Gong of Alpha Lab, Qihoo 360 on 2019-11-14
[$5000][1025489] High CVE-2019-13729: Use after free in WebSockets. Reported by Zhe Jin（金哲），Luyao Liu(刘路遥) from Chengdu Security Response Center of Qihoo 360 Technology Co. Ltd on 2019-11-16
[$5000][1028862] High CVE-2019-13730: Type Confusion in V8. Reported by Wen Xu of SSLab, Georgia Tech on 2019-11-27
[$TBD][1023817] High CVE-2019-13732: Use after free in WebAudio. Reported by Sergei Glazunov of Google Project Zero on 2019-11-12
[$TBD][1025466] High CVE-2019-13734: Out of bounds write in SQLite. Reported by "Team 0x34567a61" @Xbalien29 @leonwxqian on 2019-11-16
[$TBD][1025468] High CVE-2019-13735: Out of bounds write in V8. Reported by Gengming Liu and Zhen Feng from Tencent Keen Lab on 2019-11-16
[$TBD][1028863] High CVE-2019-13764: Type Confusion in V8. Reported by Wen Xu of SSLab, Georgia Tech on 2019-11-26
[$7500][1020899] Medium CVE-2019-13736: Integer overflow in PDFium. Reported by Anonymous on 2019-11-03
[$5000][1013882] Medium CVE-2019-13737: Insufficient policy enforcement in autocomplete. Reported by Mark Amery on 2019-10-12
[$5000][1017441] Medium CVE-2019-13738: Insufficient policy enforcement in navigation. Reported by Johnathan Norman and Daniel Clark of Microsoft Edge Team on 2019-10-23
[$3000][824715] Medium CVE-2019-13739: Incorrect security UI in Omnibox. Reported by xisigr of Tencent's Xuanwu Lab on 2018-03-22
[$2000][1005596] Medium CVE-2019-13740: Incorrect security UI in sharing. Reported by Khalil Zhani on 2019-09-19
[$2000][1011950] Medium CVE-2019-13741: Insufficient validation of untrusted input in Blink. Reported by Michał Bentkowski of Securitum on 2019-10-07
[$2000][1017564] Medium CVE-2019-13742: Incorrect security UI in Omnibox. Reported by Khalil Zhani on 2019-10-24
[$1000][754304] Medium CVE-2019-13743: Incorrect security UI in external protocol handling. Reported by Zhiyang Zeng of Tencent security platform department on 2017-08-10
[$1000][853670] Medium CVE-2019-13744: Insufficient policy enforcement in cookies. Reported by Prakash (@1lastBr3ath) on 2018-06-18
[$500][990867] Medium CVE-2019-13745: Insufficient policy enforcement in audio. Reported by Luan Herrera (@lbherrera_) on 2019-08-05
[$500][999932] Medium CVE-2019-13746: Insufficient policy enforcement in Omnibox. Reported by David Erceg on 2019-09-02
[$500][1018528] Medium CVE-2019-13747: Uninitialized Use in rendering. Reported by Ivan Popelyshev and André Bonatti on 2019-10-26
[$N/A][993706] Medium CVE-2019-13748: Insufficient policy enforcement in developer tools. Reported by David Erceg on 2019-08-14
[$N/A][1010765] Medium CVE-2019-13749: Incorrect security UI in Omnibox. Reported by Khalil Zhani on 2019-10-03
[$TBD][1025464] Medium CVE-2019-13750: Insufficient data validation in SQLite. Reported by "Team 0x34567a61" @Xbalien29 @leonwxqian on 2019-11-16
[$TBD][1025465] Medium CVE-2019-13751: Uninitialized Use in SQLite. Reported by "Team 0x34567a61" @Xbalien29 @leonwxqian on 2019-11-16
[$TBD][1025470] Medium CVE-2019-13752: Out of bounds read in SQLite. Reported by Wenxiang Qian of Tencent Blade Team on 2019-11-16
[$TBD][1025471] Medium CVE-2019-13753: Out of bounds read in SQLite. Reported by Wenxiang Qian of Tencent Blade Team on 2019-11-16
[$500][442579] Low CVE-2019-13754: Insufficient policy enforcement in extensions. Reported by Cody Crews on 2014-12-16
[$500][696208] Low CVE-2019-13755: Insufficient policy enforcement in extensions. Reported by Masato Kinugawa on 2017-02-25
[$500][708595] Low CVE-2019-13756: Incorrect security UI in printing. Reported by Khalil Zhani on 2017-04-05
[$500][884693] Low CVE-2019-13757: Incorrect security UI in Omnibox. Reported by Khalil Zhani on 2018-09-17
[$500][979441] Low CVE-2019-13758: Insufficient policy enforcement in navigation. Reported by Khalil Zhani on 2019-06-28
[$N/A][901789] Low CVE-2019-13759: Incorrect security UI in interstitials. Reported by Wenxu Wu (@ma7h1as) of Tencent Security Xuanwu Lab on 2018-11-05
[$N/A][1002687] Low CVE-2019-13761: Incorrect security UI in Omnibox. Reported by Khalil Zhani on 2019-09-10
[$N/A][1004212] Low CVE-2019-13762: Insufficient policy enforcement in downloads. Reported by csanuragjain (@csanuragjain) on 2019-09-16
[$TBD][1011600] Low CVE-2019-13763: Insufficient policy enforcement in payments. Reported by weiwangpp93 on 2019-10-05

We would also like to thank all security researchers that worked with us during the development cycle to prevent security bugs from ever reaching the stable channel.

As usual, our ongoing internal security work was responsible for a wide range of fixes:
  • [1032080] Various fixes from internal audits, fuzzing and other initiatives

Many of our security bugs are detected using AddressSanitizer, MemorySanitizer, UndefinedBehaviorSanitizer, Control Flow Integrity, libFuzzer, or AFL.


Interested in switching release channels?  Find out how here. If you find a new issue, please let us know by filing a bug. The community help forum is also a great place to reach out for help or learn about common issues.




Krishna Govind
Google Chrome


Lessons Learned from Developing ML for Healthcare

Posted by Yun Liu, Research Scientist and Po-Hsuan Cameron Chen, Research Engineer, Google Health

Machine learning (ML) methods are not new in medicine -- traditional techniques, such as decision trees and logistic regression, were commonly used to derive established clinical decision rules (for example, the TIMI Risk Score for estimating patient risk after a coronary event). In recent years, however, there has been a tremendous surge in leveraging ML for a variety of medical applications, such as predicting adverse events from complex medical records, and improving the accuracy of genomic sequencing. In addition to detecting known diseases, ML models can tease out previously unknown signals, such as cardiovascular risk factors and refractive error from retinal fundus photographs.

Beyond developing these models, it’s important to understand how they can be incorporated into medical workflows. Previous research indicates that doctors assisted by ML models can be more accurate than either doctors or models alone in grading diabetic eye disease and diagnosing metastatic breast cancer. Similarly, doctors are able to leverage ML-based tools in an interactive fashion to search for similar medical images, providing further evidence that doctors can work effectively with ML-based assistive tools.

In an effort to improve guidance for research at the intersection of ML and healthcare, we have written a pair of articles, published in Nature Materials and the Journal of the American Medical Association (JAMA). The first is for ML practitioners to better understand how to develop ML solutions for healthcare, and the other is for doctors who desire a better understanding of whether ML could help improve their clinical work.

How to Develop Machine Learning Models for Healthcare
In “How to develop machine learning models for healthcare” (pdf), published in Nature Materials, we discuss the importance of ensuring that the needs specific to the healthcare environment inform the development of ML models for that setting. This should be done throughout the process of developing technologies for healthcare applications, from problem selection, data collection and ML model development to validation and assessment, deployment and monitoring.

The first consideration is how to identify a healthcare problem for which there is both an urgent clinical need and for which predictions based on ML models will provide actionable insight. For example, ML for detecting diabetic eye disease can help alleviate the screening workload in parts of the world where diabetes is prevalent and the number of medical specialists is insufficient. Once the problem has been identified, one must be careful with data curation to ensure that the ground truth labels, or “reference standard”, applied to the data are reliable and accurate. This can be accomplished by validating labels via comparison to expert interpretation of the same data, such as retinal fundus photographs, or through an orthogonal procedure, such as a biopsy to confirm radiologic findings. This is particularly important since a high-quality reference standard is essential both for training useful models and for accurately measuring model performance. Therefore, it is critical that ML practitioners work closely with clinical experts to ensure the rigor of the reference standard used for training and evaluation.

Validation of model performance is also substantially different in healthcare, because the problem of distributional shift can be pronounced. In contrast to typical ML studies where a single random test split is common, the medical field values validation using multiple independent evaluation datasets, each with different patient populations that may exhibit differences in demographics or disease subtypes. Because the specifics depend on the problem, ML practitioners should work closely with clinical experts to design the study, with particular care in ensuring that the model validation and performance metrics are appropriate for the clinical setting.

Integration of the resulting assistive tools also requires thoughtful design to ensure seamless workflow integration, with consideration for measurement of the impact of these tools on diagnostic accuracy and workflow efficiency. Importantly, there is substantial value in prospective study of these tools in real patient care to better understand their real-world impact.

Finally, even after validation and workflow integration, the journey towards deployment is just beginning: regulatory approval and continued monitoring for unexpected error modes or adverse events in real use remains ahead.
Two examples of the translational process of developing, validating, and implementing ML models for healthcare based on our work in detecting diabetic eye disease and metastatic breast cancer.
Empowering Doctors to Better Understand Machine Learning for Healthcare
In “Users’ Guide to the Medical Literature: How to Read Articles that use Machine Learning,” published in JAMA, we summarize key ML concepts to help doctors evaluate ML studies for suitability of inclusion in their workflow. The goal of this article is to demystify ML, to assist doctors who need to use ML systems to understand their basic functionality, when to trust them, and their potential limitations.

The central questions doctors ask when evaluating any study, whether ML or not, remain: Was the reference standard reliable? Was the evaluation unbiased, such as assessing for both false positives and false negatives, and performing a fair comparison with clinicians? Does the evaluation apply to the patient population that I see? How does the ML model help me in taking care of my patients?

In addition to these questions, ML models should also be scrutinized to determine whether the hyperparameters used in their development were tuned on a dataset independent of that used for final model evaluation. This is particularly important, since inappropriate tuning can lead to substantial overestimation of performance, e.g., a sufficiently sophisticated model can be trained to completely memorize the training dataset and generalize poorly to new data. Ensuring that tuning was done appropriately requires being mindful of ambiguities in dataset naming, and in particular, using the terminology with which the audience is most familiar:
The intersection of two fields: ML and healthcare creates ambiguity in the term “validation dataset”. An ML validation set is typically used to refer to the dataset used for hyperparameter tuning, whereas a “clinical” validation set is typically used for final evaluation. To reduce confusion, we have opted to refer to the (ML) validation set as the “tuning” set.
Future outlook
It is an exciting time to work on AI for healthcare. The “bench-to-bedside” path is a long one that requires researchers and experts from multiple disciplines to work together in this translational process. We hope that these two articles will promote mutual understanding of what is important for ML practitioners developing models for healthcare and what is emphasized by doctors evaluating these models, thus driving further collaborations between the fields and towards eventual positive impact on patient care.

Acknowledgements
Key contributors to these projects include Yun Liu, Po-Hsuan Cameron Chen, Jonathan Krause, and Lily Peng. The authors would like to acknowledge Greg Corrado and Avinash Varadarajan for their advice, and the Google Health team for their support.

Source: Google AI Blog


Year in Search: Here’s what Aussies searched for in 2019

From bushfires to rugby and plant-based recipes – our Searches showed our greatest curiosities, quirks and cravings in 2019.



As the year comes to a close, it’s time to look back at the moments that had Aussies searching, wondering – and baking scones. It's been a year of battling bushfires, following elections and learning how to #kicklikeagirl. We welcomed a Royal Baby, bid farewell to some of the greats, and looked for answers during crises. Whether we were building decks, looking to vote or solving rubix cubes, we turned to Search when we wanted to learn, understand or needed a helping hand.

Here's a summary of five themes that defined Search in Australia in 2019:
Fires, floods and disasters

2019 was the year we navigated natural disasters, both on our shores and abroad. “Fires near me” was the highest Search query in Australia for the year – and our top trending News Searches included “NSW fires,” “Cyclone Oma,” and “Townsville flooding.” Beyond our borders, we showed our concern for the Amazon and asked why it was burning. We also tried to make sense of atrocities and attacks, searching for Christchurch and Sri Lanka – and why Hong Kong is protesting.
All about sport

Whether we were watching live, following our teams or looking up scores, our passion for sport came through in Search. We followed the Rugby, Cricket and Basketball World Cups – and tuned into the Ashes.

Female sports stars also shone bright in Search this year. Ash Barty topped the list of top trending Aussies overall, Tayla Harris was the top football player – while queries for “Women’s World Cup” were among the top trending Searches for sporting events in Australia.

Crafty, creative and curious

Aussies were full of questions and projects in 2019, both big and small. We embarked on DIY craft, beauty and home renovation projects – asking how to make beeswax wraps, face masks, and decorations. We certainly didn’t shy away from large-scale projects however, as the top trending DIY Search was “how to build a deck.”

Our questions ranged from practical, political, pop-cultural, scientific and religious. As Aussies hit the polling booths, we looked for help on how to vote. We also wondered how to pronounce “psalm,” what a VSCO girl and astrophysicist is – along with the timeless question, “How to win the Powerball.” Madonna was also a subject of our Search attention as we wondered why she wore an eye patch at Eurovision, which led the list of top trending “Why is…?” queries.
Colourful cravings

From scones to risotto, apple pie to fried rice, Aussies had a palette for all kinds of sweet, savoury and multicultural dishes this year. While we looked up hearty classics like osso buco and beef stroganoff, Searches for “plant based recipes” took the cake in 2019, leading the list of top trending recipes. And as the ketogenic craze continues, we also asked what the keto diet is.
New beginnings and goodbyes

Aussies showed their support for the Royal baby this year, as the world welcomed little Archie in May. We also bid farewell to politicians, fashion icons, sports stars and actors. We mourned the loss of local legend Bob Hawke – as well as global powerhouses, Karl Lagerfeld and Doris Day.

To dive into the data, check out Australia's full trending* lists for 2019:

Searches (Trending)

  1. Fires near me
  2. Rugby World Cup
  3. Cricket World Cup
  4. Election results Australia 2019
  5. Cameron Boyce
  6. Thanos
  7. Avengers Endgame
  8. Danny Frawley
  9. The fall of the Berlin wall
  10. Christchurch shooting

News (Trending)

  1. Election results Australia 2019
  2. Christchurch shooting
  3. NSW fires
  4. Cyclone Oma
  5. Sri Lanka
  6. NSW election
  7. Brexit
  8. Townsville flooding
  9. Royal baby
  10. Amazon fire

Sporting events (Trending)

  1. Rugby World Cup
  2. Cricket World Cup
  3. FIBA World Cup
  4. Cricket Australia
  5. Pakistan vs Australia
  6. Ashes score
  7. Copa America
  8. Ashes 4th test
  9. NRL grand final
  10. Women's World Cup

Global figures (Trending)

  1. James Charles
  2. Billie Eilish
  3. Jordyn Woods
  4. Greta Thunberg
  5. Keanu Reeves
  6. Julian Assange
  7. Lady Gaga
  8. Tsitsipas
  9. Bradley Cooper
  10. Prince Andrew

Australian public figures (Trending)

  1. Ash Barty
  2. Fraser Anning
  3. Israel Folau
  4. Cody Simpson
  5. Scott Morrison
  6. Bill Shorten
  7. Jack Vidgen
  8. George Pell
  9. Tayla Harris
  10. Chris Lilley

Loss (Trending)

  1. Cameron Boyce
  2. Danny Frawley
  3. Luke Perry
  4. Bob Hawke
  5. Annalise Braakensiek
  6. Karl Lagerfeld
  7. Keith Flint
  8. Doris Day
  9. Mike Willesee
  10. Ben Unwin

Recipes (Trending)

  1. Plant based recipes
  2. Scones recipes
  3. Beef stroganoff recipes
  4. MKR recipes 2019
  5. Apple pie recipes
  6. Frittata recipes
  7. Fried rice recipes
  8. Risotto recipes
  9. Hello Fresh recipes
  10. Osso buco recipes

How to…? (Trending)

  1. How to vote
  2. How to watch Game of Thrones
  3. How to vote labor 2019
  4. How to vote liberal
  5. How to pronounce psalm
  6. How to vote in NSW State Election
  7. How to vote early
  8. How to win Powerball
  9. How to opt out of my health
  10. How to solve a rubix cube

Why is..? (Trending)

  1. Why is Madonna wearing an eye patch at Eurovision
  2. Why is the Amazon burning
  3. Why is Australia Day on the 26th
  4. Why is the Press Club closing
  5. Why is Instagram not working
  6. Why is ASAP rocky in jail
  7. Why is Hong Kong protesting
  8. Why is Halloween celebrated
  9. Why is ANZAC day commemorated on 25 April
  10. Why is Turkey invading Syria

What is…? (Trending)

  1. What is area 51
  2. What is momo
  3. What is Brexit
  4. What is a VSCO girl
  5. What is a boomer
  6. What is keto diet
  7. What is an astrophysicist
  8. What is negative gearing
  9. What is Diwali
  10. What is vaping

Top related searches for “DIY” in 2019 in Australia (Most Searched)

  1. How to build a deck
  2. DIY face mask
  3. How to lay pavers
  4. DIY halloween costumes
  5. Beeswax wraps DIY
  6. DIY Christmas decorations
  7. DIY balloon garland
  8. DIY garage
  9. DIY lip scrub
  10. DIY dog wash

Posted by Camilla Ibrahim, Communications & Public Affairs Manager, Google Australia

* Trending Searches: What was hot in 2019? The "trending" queries are the searches that had the highest spike in traffic over a sustained period in 2019 as compared to 2018.

* Most Searched: What topped Google’s charts? The "most searched" queries are the most popular terms for 2019—ranked in order by volume of searches.

Launching a new Publisher Center

Today we are announcing the launch of Publisher Center to help publishers more easily manage how their content appears across Google products. Publisher Center merges two existing tools, Google News Producer and Google News Publisher Center, improving their user experience and functionality.
Publisher Center’s new features include a simpler way to manage your publication’s identity, like updating light and dark theme logos. It also provides an easier way for those that own multiple publications to organize and switch between them, particularly with improved permission settings that make it easier to collaborate with colleagues. Additionally, publishers can now point to the URLs for their website’s sections instead of RSS to configure sections in Google News. Content for News will now come directly from the web, just as it does for Search.
Publisher Center launches today in the existing four languages of the previous tools (English, Spanish, French, and German) and will expand to more languages soon. Learn more here.
Posted by Eric Silva, Product Manager

Year in Search 2019: Here’s what Kiwis searched for this year

As Kiwis we grinned, grew and grieved through Search this year, so as 2020 draws near, it’s time to reflect on the moments, people and existential questions that piqued our interest in 2019.

Sports tournaments, games and stars took the lead in trending searches across the country this year. We mused over musicians, paid close attention to fires, both here and abroad, and looked to Search to solve everyday problems like how to write a cover letter. Kiwis continued the obsession with keto, however ‘plant based diets’ surged to the top of our diet queries.

Here’s a summary of some themes that dominated our searches this year:

A Helpful Search
Answering the questions you either can’t or don’t want to ask someone else, Search provided helpful information to Kiwis on everything from cover letters to quinoa. Many of us wondered what is Easter, and Ramadan, and what is area 51?
Helping Kiwis with recipes, we cooked spag bol and guac with gusto, not to mention pikelets and pumpkin soup. (Doesn’t that sound like the worst dinner party ever?)

A Country of Two Halves
As we saw in 2018, Kiwi’s sports obsession continued throughout the year. Games of rugby, cricket matches, tennis tournaments and boxing champions all captured our hearts and minds. Six of the top 10 Kiwis searched are athletes, and Spark Sport and ‘Where to watch the Rugby World Cup’ helped the sports supremacy continue.

Millennial Moments
Generational divide came into stark contrast for us this year, as we collectively wondered, ‘what is a boomer?’ Millennials prevailed with Kiwi’s curiosity of local heroes Kane Williamson (let’s not talk about the cricket), Israel Adesanya and Lorde. Alongside this however, up and coming Gen Z’s, such as Billie Eilish, broke into the zeitgeist.

Local Events
Some moments found New Zealand on the world stage this year, and our Search habits show local interest in these. The Christchurch shooting and the Grace Millane case also drew international interest. We kept up with ongoing disputes at Ihumatao and a measles outbreak, and wondered what our minimum wage was.

A Changing World
News events of fires, terror attacks and cyclones all show the state of the world we’re trying to understand. Our searches show we came together for greater understanding, as we also queried how to wear a hijab and what is a mosque. Meanwhile breakout activist Greta Thunberg captured our attention, as did a newfound discovery of the ‘plant based diet’.

Check out the top trending* lists for New Zealand in 2019:

Overall

  1. Rugby World Cup
  2. New Zealand vs England
  3. Cricket World Cup
  4. Christchurch Shooting
  5. Stuff News NZ
  6. Spark Sport
  7. Australia vs Pakistan
  8. Australia Open 2019
  9. Disney Plus
  10. Grace Millane


Kiwis

  1. Israel Adesanya
  2. William Waiirua
  3. Ryan Fox
  4. Lorde
  5. Kane Williamson
  6. Anna Wilcox
  7. Sarah Dowie
  8. Robert Whittaker
  9. Sonny Bill Williams
  10. Scott McLaughlin


Loss

  1. Grace Millane
  2. Cameron Boyce
  3. Pua Magasiva
  4. Manning Smith
  5. Luke Perry
  6. Jeffrey Epstein
  7. Karl Lagerfeld
  8. Keith Flint
  9. Etika
  10. Jessi Combs


Global Figures

  1. James Charles
  2. Billie Eilish
  3. Prince Andrew
  4. Nipsey Hussle
  5. Greta Thunberg
  6. Jordyn Woods
  7. Israel Folau
  8. R Kelly
  9. Pewdiepie
  10. Marie Kondo


News Events

  1. Christchurch Shooting
  2. Nelson Fire
  3. The fall of the Berlin wall
  4. London Bridge attack
  5. Measles
  6. Auckland Fire
  7. Notre Dame
  8. Sri Lanka
  9. Ihumatao
  10. Cyclone Oma


What Is…

  1. What is area 51?
  2. What is black Friday all about?
  3. What is a mosque?
  4. What is lupus?
  5. What is a boomer?
  6. What is momo?
  7. What is minimum wage in NZ?
  8. What is easter?
  9. What is the time in India?
  10. What is ramadan?


How to…

  1. How to watch the Rugby World Cup
  2. How to watch Game of Thrones NZ
  3. How to wear a hijab
  4. How to solve a rubix cube
  5. How to write a cover letter
  6. How to cook quinoa
  7. How to get rid of fleas
  8. How to make french toast
  9. How to take a screenshot
  10. How to make cheese sauce


Recipes

  1. Spaghetti bolognese recipe
  2. Guacamole recipe
  3. Hot cross buns recipe
  4. Apple crumble recipe
  5. Pikelet recipe
  6. Playdough recipe
  7. Pancake recipe easy
  8. Fried rice recipe
  9. Lasagne recipe
  10. Pumpkin soup recipe


Diet Trends

  1. Plant based diet
  2. Fodmap diet
  3. Keto diet plan
  4. Diet doctor
  5. Mediterranean diet
  6. What is keto diet
  7. Dubrow diet
  8. Low carb diet
  9. Vegan diet
  10. Atkins diet


* Trending Searches: What was hot in 2019? The "trending" queries are the searches that had the highest spike in traffic over a sustained period in 2019 as compared to 2018.


Post content Posted by Carrie Jones, Communications Manager, Google New Zealand

Better password protections in Chrome – How it works

Posted by Patrick Nepper, Kiran C. Nair, Vasilii Sukhanov and Varun Khaneja, Chrome Team

Today, we announced better password protections in Chrome, gradually rolling out with release M79. Here are the details of how they work.


Warnings about compromised passwords
Google first introduced password breach warnings as a Password Checkup extension early this year. It compares passwords and usernames against over 4 billion credentials that Google knows to have been compromised. You can read more about it here. In October, Google built the Password Checkup feature into the Google Account, making it available from passwords.google.com.

Chrome’s integration is a natural next step to ensure we protect even more users as they browse the web. Here is how it works:
  • Whenever Google discovers a username and password exposed by another company’s data breach, we store a hashed and encrypted copy of the data on our servers with a secret key known only to Google.
  • When you sign in to a website, Chrome will send a hashed copy of your username and password to Google encrypted with a secret key only known to Chrome. No one, including Google, is able to derive your username or password from this encrypted copy.
  • In order to determine if your username and password appears in any breach, we use a technique called private set intersection with blinding that involves multiple layers of encryption. This allows us to compare your encrypted username and password with all of the encrypted breached usernames and passwords, without revealing your username and password, or revealing any information about any other users’ usernames and passwords. In order to make this computation more efficient, Chrome sends a 3-byte SHA256 hash prefix of your username to reduce the scale of the data joined from 4 billion records down to 250 records, while still ensuring your username remains anonymous.
  • Only you discover if your username and password have been compromised. If they have been compromised, Chrome will tell you, and we strongly encourage you to change your password.

You can control this feature in the “Sync and Google Services” section of Chrome Settings. Enterprise admins can control this feature using the Password​Leak​Detection​Enabled policy setting.


Real-time phishing protection: Checking with Safe Browsing’s blocklist in real time.
Chrome’s new real-time phishing protection is also expanding existing technology — in this case it’s Google’s well-established Safe Browsing.

Every day, Safe Browsing discovers thousands of new unsafe sites and adds them to the blocklists shared with the web industry. Chrome checks the URL of each site you visit or file you download against this local list, which is updated approximately every 30 minutes. If you navigate to a URL that appears on the list, Chrome checks a partial URL fingerprint (the first 32 bits of a SHA-256 hash of the URL) with Google for verification that the URL is indeed dangerous. Google cannot determine the actual URL from this information.

However, we’re noticing that some phishing sites slip through our 30-minute refresh window, either by switching domains very quickly or by hiding from Google's crawlers.

That’s where real-time phishing protections come in. These new protections can inspect the URLs of pages visited with Safe Browsing’s servers in real time. When you visit a website, Chrome checks it against a list stored on your computer of thousands of popular websites that are known to be safe. If the website is not on the safe-list, Chrome checks the URL with Google (after dropping any username or password embedded in the URL) to find out if you're visiting a dangerous site. Our analysis has shown that this results in a 30% increase in protections by warning users on malicious sites that are brand new.

We will be initially rolling out this feature for people who have already opted-in to “Make searches and browsing better” setting in Chrome. Enterprises administrators can manage this setting via the Url​Keyed​Anonymized​Data​Collection​Enabled policy settings.


Expanding predictive phishing protection
Your password is the key to your online identity and data. If this key falls into the hands of attackers, they can easily impersonate you and get access to your data. We launched predictive phishing protections to warn users who are syncing history in Chrome when they enter their Google Account password into suspected phishing sites that try to steal their credentials.

With this latest release, we’re expanding this protection to everyone signed in to Chrome, even if you have not enabled Sync. In addition, this feature will now work for all the passwords you have stored in Chrome’s password manager.

If you type one of your protected passwords (this could be a password you stored in Chrome’s password manager, or the Google Account password you used to sign in to Chrome) into an unusual site, Chrome classifies this as a potentially dangerous event.

In such a scenario, Chrome checks the site against a list on your computer of thousands of popular websites that are known to be safe. If the website is not on the safe-list, Chrome checks the URL with Google (after dropping any username or password embedded in the URL). If this check determines that the site is indeed suspicious or malicious, Chrome will immediately show you a warning and encourage you to change your compromised password. If it was your Google Account password that was phished, Chrome also offers to notify Google so we can add additional protections to ensure your account isn't compromised.

By watching for password reuse, Chrome can give heightened security in critical moments while minimizing the data it shares with Google. We think predictive phishing protection will protect hundreds of millions more people.

New Google Assistant features to customize your alarm clock

The Lenovo Smart Clock with the Google Assistant makes for a good gift this holiday season. Its compact size fits neatly on your nightstand, offers alarm suggestions based on your calendar, and gently wakes you up with a sunrise alarm that mimics the rising sun.

And with the latest software update rolling out globally today, your device can be customized to your environment. “Impromptu” is a new alarm option that gives you a ringtone that fits your situation, based on things like the time of day or weather. It’s powered by machine learning technology from our Magenta open source project. For example, if your alarm goes off early in the morning and the weather is less than 50 degrees, you may hear this ringtone.

Thanks to your continued feedback, you’ll also notice other improvements on your Smart Clock. We’ve fine-tuned the automatic brightness, added a music recommendation card to help you get to your playlists faster, and provided more granular control over how long your alarm lasts (now up to 60 minutes). 

And just in time for the holidays, you can find the Lenovo Smart Clock at a great deal from popular retailers like Best Buy, The Home Depot,Sam’s Club and Lenovo.com. 

Better password protections in Chrome

Many of us have encountered malware, heard of data breaches, or even been a victim of phishing, where a site tries to scam you into entering your passwords and other sensitive information. With all this considered, data security has become a top concern for many people worldwide. Chrome has safety protections built in, and now we're expanding those protections further. 

Chrome warns when your password has been stolen

When you type your credentials into a website, Chrome will now warn you if your username and password have been compromised in a data breach on some site or app. It will suggest that you change them everywhere they were used.

Keyword Blog - breach detection.png

If your credentials were compromised, we recommend to change them immediately.

Google first introduced this technology early this year as the Password Checkup extension. In October it became a part of the Password Checkup in your Google Account, where you can conduct a scan of your saved passwords anytime. And now it has evolved to offer warnings as you browse the web in Chrome. 

You can control it in Chrome Settings under Sync and Google Services. For now, we’re gradually rolling this out for everyone signed in to Chrome as a part of our Safe Browsing protections.

Phishing protection in real time

Google’s Safe Browsing maintains an ever-growing list of unsafe sites on the web and shares this information with webmasters, or other browsers, to make the web more secure. The list refreshes every 30 minutes, protecting 4 billion devices every day against all kinds of security threats, including phishing.

Graph.png

Safe Browsing list has been capturing an increasing number of phishing sites.

However, some phishing sites slip through that 30-minute window, either by quickly switching domains or by hiding from our crawlers. Chrome now offers real-time phishing protections on desktop, which warn you when visiting malicious sites in 30 percent more cases. Initially we will roll out this protection to everyone with the “Make searches and browsing better” setting enabled in Chrome. 

Expanding predictive phishing protections

If you're signed in to Chrome and have Sync enabled, predictive phishing protection warns you if you enter your Google Account password into a site that we suspect of phishing. This protection has been in place since 2017, and today we’re expanding the feature further.

Now we'll be protecting your Google Account password when you sign in to Chrome, even if Sync is not enabled. In addition, this feature will now work for all the passwords you store in Chrome’s password manager. Hundreds of millions more users will now benefit from the new warnings.

Keyword Blog - phishing.png

Chrome will show this warning when a user enters their Google Account password into a phishing page.

Sharing your device? Now it’s easier to tell whose Chrome profile you’re using 

We realize that many people share their computers or use multiple profiles. To make sure you always know which profile you’re currently using—for example, when creating and saving passwords with Chrome’s password manager—we’ve improved the way your profile is featured.

On desktop, you’ll see a new visual representation of the profile you’re currently using, so you can be sure you are saving your passwords to the right profile. This is a visual update and won’t change your current Sync settings. We’ve also updated the look of the profile menu itself: it now allows for easier switching and clearly shows if you are signed in to Chrome or not.

A3.gif

The new sign-in indicator.

From Munich with love

Many of these technologies were developed at the Google Safety Engineering Center (GSEC), a hub of privacy and security product experts and engineers based in Munich, which opened last May. GSEC is home to the engineering teams who build many of the safety features into the Chrome browser. We’ll continue to invest in our teams worldwide to deliver the safest personal browser experience to everyone, and we look forward to bringing more new features to strengthen the privacy and security of Chrome in 2020. 

All these features will be rolled out gradually over the next few weeks. Interested in how they work? You can learn more on Google Security blog.


Android 10 on Android TV

Posted by Paul Lammertsma, Developer Advocate

Technology has changed the way media and entertainment is accessed and consumed in the home. While the living room experience is evolving with the addition of smart devices, TVs still remain the largest and most frequently used screen for watching content.

When Android TV was first introduced in 2014, we set out to bring the best of Android into the connected home on the TV. We worked closely with the developer community to grow our content and app ecosystem and bring users the content they want. Since then, we’ve seen tremendous momentum with OEM and operator partners as well as consumer adoption worldwide.

Today, we are bringing Android API level 29 with the recent performance and security updates made with Android 10 to Android TV. We’re excited to provide faster updates through Project Treble and more secure storage with encrypted user data. TLS 1.3 by default also brings better performance benefits and is up to date with the TLS standard. In addition, Android 10 includes hardening for several security-critical areas of the platform.

ADT-3

To make sure developers have the ability to build and test Android TV app implementations on Android 10 prior to rollout, we’re introducing a new, developer-focused streaming media device called ADT-3.

With a quad-core A53, 2GB of DDR3 memory and 4Kp60 HDR HDMI 2.1 output, we’ve designed this pre-certified TV dongle with updates and security patches to help developers design for the next generation of Android TV devices. By providing a way to test on physical and up to date hardware, developers can better validate their Android TV app’s compatibility.

Android TV box and remote

ADT-3 will be made available to developers in the coming months for purchase online through an OEM partner.