Monthly Archives: August 2021

View embedded Office files in your documents

What’s changing 

You can now view embedded Microsoft Office files in your documents when you’re working with Office files in Google Docs, Sheets, and Slides.. This feature allows you to:  
  • View the files in preview mode, Copy an embedded file directly to Drive, or download it. 
  • View your embedded files from Office in your documents 

View your embedded files from Office in your documents
 View your embedded files from Office in your documents
Who’s impacted 
Admins and end users 

Why it matters 
We’ve heard your feedback that it’s important to be able to access embedded files within your Microsoft Office files. This feature enables you to access embedded Office files within your existing Office files from Docs, Sheets, and Slides for a seamless work experience. 

Getting started 

Rollout pace 

Availability 
Available to all Google Workspace customers, as well as G Suite Basic and Business customers. Also available to users with personal Google Accounts 

Resources

Chrome for Android Update

Hi, everyone! We've just released Chrome 93 (93.0.4577.62) for Android: it'll become available on Google Play over the next few days.

This release includes stability and performance improvements. You can see a full list of the changes in the Git log. If you find a new issue, please let us know by filing a bug.

Ben Mason
Google Chrome

Chrome for iOS Update

Hi, everyone! We've just released Chrome 93 (93.0.4577.39) for iOS: it'll become available on App Store in the next few hours.

This release includes stability and performance improvements. You can see a full list of the changes in the Git log. If you find a new issue, please let us know by filing a bug.

Harry Souders

Google Chrome

TAG Bulletin: Q3 2021

This bulletin includes coordinated influence operation campaigns terminated on our platforms in Q3 2021. It was last updated on August 31, 2021.


July 

  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Ukraine. This campaign uploaded content in Ukrainian and Russian that was supportive of Russia’s government and critical of the Ukrainian military. We received leads from FireEye that supported us in this investigation.
  • We blocked 10 domains from eligibility to appear on Google News surfaces and Discover as part of our investigation into coordinated influence operations linked to Russia. This campaign uploaded content in Russian that was critical of Ukraine’s government and supportive of Russia.
  • We terminated 2 YouTube channels as part of our investigation into coordinated influence operations linked to Iraq. This campaign uploaded content in Arabic that was supportive of Iran-backed militias and critical of the U.S. and its allies. Our findings are similar to findings reported by Facebook.
  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Jordan. This campaign uploaded content in Arabic that was supportive of the Jordanian government and critical of its opposition. Our findings are similar to findings reported by Facebook.
  • We terminated 15 YouTube channels as part of our investigation into coordinated influence operations linked to Algeria. This campaign uploaded content in Arabic that was supportive of the Algerian government and its military. Our findings are similar to findings reported by Facebook. We received leads from Graphika that supported us in this investigation.
  • We terminated 6 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was critical of certain local politicians in Campeche, Mexico. Our findings are similar to findings reported by Facebook.
  • We terminated 4 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was supportive of a member of the National Action Party). Our findings are similar to findings reported by Facebook.
  • We terminated 16 YouTube channels and 1 ads account as part of our investigation into coordinated influence operations linked to Sudan. This campaign uploaded content in Arabic that was supportive of the Muslim Brotherhood and critical of the current Sudanese government. Our findings are similar to findings reported by Facebook.
  • We terminated 850 YouTube channels as part of our ongoing investigation into coordinated influence operations linked to China. These channels mostly uploaded spammy content in Chinese about music, entertainment, and lifestyle. A very small subset uploaded content in Chinese and English about China’s COVID-19 vaccine efforts and social issues in the U.S. These findings are consistent with our previous reports.

TAG Bulletin: Q3 2021

This bulletin includes coordinated influence operation campaigns terminated on our platforms in Q3 2021. It was last updated on August 31, 2021.


July 

  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Ukraine. This campaign uploaded content in Ukrainian and Russian that was supportive of Russia’s government and critical of the Ukrainian military. We received leads from FireEye that supported us in this investigation.
  • We blocked 10 domains from eligibility to appear on Google News surfaces and Discover as part of our investigation into coordinated influence operations linked to Russia. This campaign uploaded content in Russian that was critical of Ukraine’s government and supportive of Russia.
  • We terminated 2 YouTube channels as part of our investigation into coordinated influence operations linked to Iraq. This campaign uploaded content in Arabic that was supportive of Iran-backed militias and critical of the U.S. and its allies. Our findings are similar to findings reported by Facebook.
  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Jordan. This campaign uploaded content in Arabic that was supportive of the Jordanian government and critical of its opposition. Our findings are similar to findings reported by Facebook.
  • We terminated 15 YouTube channels as part of our investigation into coordinated influence operations linked to Algeria. This campaign uploaded content in Arabic that was supportive of the Algerian government and its military. Our findings are similar to findings reported by Facebook. We received leads from Graphika that supported us in this investigation.
  • We terminated 6 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was critical of certain local politicians in Campeche, Mexico. Our findings are similar to findings reported by Facebook.
  • We terminated 4 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was supportive of a member of the National Action Party). Our findings are similar to findings reported by Facebook.
  • We terminated 16 YouTube channels and 1 ads account as part of our investigation into coordinated influence operations linked to Sudan. This campaign uploaded content in Arabic that was supportive of the Muslim Brotherhood and critical of the current Sudanese government. Our findings are similar to findings reported by Facebook.
  • We terminated 850 YouTube channels as part of our ongoing investigation into coordinated influence operations linked to China. These channels mostly uploaded spammy content in Chinese about music, entertainment, and lifestyle. A very small subset uploaded content in Chinese and English about China’s COVID-19 vaccine efforts and social issues in the U.S. These findings are consistent with our previous reports.

TAG Bulletin: Q3 2021

This bulletin includes coordinated influence operation campaigns terminated on our platforms in Q3 2021. It was last updated on August 31, 2021.


July 

  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Ukraine. This campaign uploaded content in Ukrainian and Russian that was supportive of Russia’s government and critical of the Ukrainian military. We received leads from FireEye that supported us in this investigation.
  • We blocked 10 domains from eligibility to appear on Google News surfaces and Discover as part of our investigation into coordinated influence operations linked to Russia. This campaign uploaded content in Russian that was critical of Ukraine’s government and supportive of Russia.
  • We terminated 2 YouTube channels as part of our investigation into coordinated influence operations linked to Iraq. This campaign uploaded content in Arabic that was supportive of Iran-backed militias and critical of the U.S. and its allies. Our findings are similar to findings reported by Facebook.
  • We terminated 7 YouTube channels as part of our investigation into coordinated influence operations linked to Jordan. This campaign uploaded content in Arabic that was supportive of the Jordanian government and critical of its opposition. Our findings are similar to findings reported by Facebook.
  • We terminated 15 YouTube channels as part of our investigation into coordinated influence operations linked to Algeria. This campaign uploaded content in Arabic that was supportive of the Algerian government and its military. Our findings are similar to findings reported by Facebook. We received leads from Graphika that supported us in this investigation.
  • We terminated 6 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was critical of certain local politicians in Campeche, Mexico. Our findings are similar to findings reported by Facebook.
  • We terminated 4 YouTube channels as part of our investigation into coordinated influence operations linked to Mexico. This campaign uploaded content in Spanish that was supportive of a member of the National Action Party). Our findings are similar to findings reported by Facebook.
  • We terminated 16 YouTube channels and 1 ads account as part of our investigation into coordinated influence operations linked to Sudan. This campaign uploaded content in Arabic that was supportive of the Muslim Brotherhood and critical of the current Sudanese government. Our findings are similar to findings reported by Facebook.
  • We terminated 850 YouTube channels as part of our ongoing investigation into coordinated influence operations linked to China. These channels mostly uploaded spammy content in Chinese about music, entertainment, and lifestyle. A very small subset uploaded content in Chinese and English about China’s COVID-19 vaccine efforts and social issues in the U.S. These findings are consistent with our previous reports.

Stable Channel Update for Desktop

The Chrome team is delighted to announce the promotion of Chrome 93 to the stable channel for Windows, Mac and Linux. This will roll out over the coming days/weeks.


Chrome 93.0.4577.63 contains a number of fixes and improvements -- a list of changes is available in the log. Watch out for upcoming Chrome and Chromium blog posts about new features and big efforts delivered in 93.

Security Fixes and Rewards

Note: Access to bug details and links may be kept restricted until a majority of users are updated with a fix. We will also retain restrictions if the bug exists in a third party library that other projects similarly depend on, but haven’t yet fixed.


This update includes 27 security fixes. Below, we highlight fixes that were contributed by external researchers. Please see the Chrome Security Page for more information.


[$20000][1233975] High CVE-2021-30606: Use after free in Blink. Reported by Nan Wang (@eternalsakura13) and koocola (@alo_cook) of 360 Alpha Lab on 2021-07-28

[$10000][1235949] High CVE-2021-30607: Use after free in Permissions. Reported by Weipeng Jiang (@Krace) from Codesafe Team of Legendsec at Qi'anxin Group on 2021-08-03

[$7500][1219870] High CVE-2021-30608: Use after free in Web Share. Reported by Huyna at Viettel Cyber Security on 2021-06-15

[$5000][1239595] High CVE-2021-30609: Use after free in Sign-In. Reported by raven (@raid_akame)  on 2021-08-13

[$N/A][1200440] High CVE-2021-30610: Use after free in Extensions API. Reported by Igor Bukanov from Vivaldi on 2021-04-19

[$20000][1233942] Medium CVE-2021-30611: Use after free in WebRTC. Reported by Nan Wang (@eternalsakura13) and koocola (@alo_cook) of 360 Alpha Lab on 2021-07-28

[$20000][1234284] Medium CVE-2021-30612: Use after free in WebRTC. Reported by Nan Wang (@eternalsakura13) and koocola (@alo_cook) of 360 Alpha Lab on 2021-07-29

[$15000][1209622] Medium CVE-2021-30613: Use after free in Base internals. Reported by Yangkang (@dnpushme) of 360 ATA on 2021-05-16

[$10000][1207315] Medium CVE-2021-30614: Heap buffer overflow in TabStrip. Reported by Huinian Yang (@vmth6) of Amber Security Lab, OPPO Mobile Telecommunications Corp. Ltd.  on 2021-05-10

[$5000][1208614] Medium CVE-2021-30615: Cross-origin data leak in Navigation. Reported by NDevTK on 2021-05-12

[$5000][1231432] Medium CVE-2021-30616: Use after free in Media. Reported by Anonymous on 2021-07-21

[$3000][1226909] Medium CVE-2021-30617: Policy bypass in Blink. Reported by NDevTK on 2021-07-07

[$3000][1232279] Medium CVE-2021-30618: Inappropriate implementation in DevTools. Reported by @DanAmodio and @mattaustin from Contrast Security on 2021-07-23

[$3000][1235222] Medium CVE-2021-30619: UI Spoofing in Autofill. Reported by Alesandro Ortiz on 2021-08-02

[$NA][1063518] Medium CVE-2021-30620: Insufficient policy enforcement in Blink. Reported by Jun Kokatsu, Microsoft Browser Vulnerability Research on 2020-03-20

[$NA][1204722] Medium CVE-2021-30621: UI Spoofing in Autofill. Reported by Abdulrahman Alqabandi, Microsoft Browser Vulnerability Research on 2021-04-30

[$NA][1224419] Medium CVE-2021-30622: Use after free in WebApp Installs. Reported by Jun Kokatsu, Microsoft Browser Vulnerability Research on 2021-06-28

[$10000][1223667] Low CVE-2021-30623: Use after free in Bookmarks. Reported by Leecraso and Guang Gong of 360 Alpha Lab on 2021-06-25

[$TBD][1230513] Low CVE-2021-30624: Use after free in Autofill. Reported by Wei Yuan of MoyunSec VLab on 2021-07-19


We would also like to thank all security researchers that worked with us during the development cycle to prevent security bugs from ever reaching the stable channel.

As usual, our ongoing internal security work was responsible for a wide range of fixes:

  • [1245324] Various fixes from internal audits, fuzzing and other initiatives


Many of our security bugs are detected using AddressSanitizer, MemorySanitizer, UndefinedBehaviorSanitizer, Control Flow Integrity, libFuzzer, or AFL.


Interested in switching release channels?  Find out how here. If you find a new issue, please let us know by filing a bug. The community help forum is also a great place to reach out for help or learn about common issues.



Prudhvikumar Bommana
Google Chrome

Introducing Omnimattes: A New Approach to Matte Generation using Layered Neural Rendering

Image and video editing operations often rely on accurate mattes — images that define a separation between foreground and background. While recent computer vision techniques can produce high-quality mattes for natural images and videos, allowing real-world applications such as generating synthetic depth-of-field, editing and synthesising images, or removing backgrounds from images, one fundamental piece is missing: the various scene effects that the subject may generate, like shadows, reflections, or smoke, are typically overlooked.

In “Omnimatte: Associating Objects and Their Effects in Video”, presented at CVPR 2021, we describe a new approach to matte generation that leverages layered neural rendering to separate a video into layers called omnimattes that include not only the subjects but also all of the effects related to them in the scene. Whereas a typical state-of-the-art segmentation model extracts masks for the subjects in a scene, for example, a person and a dog, the method proposed here can isolate and extract additional details associated with the subjects, such as shadows cast on the ground.

A state-of-the-art segmentation network (e.g., MaskRCNN) takes an input video (left) and produces plausible masks for people and animals (middle), but misses their associated effects. Our method produces mattes that include not only the subjects, but their shadows as well (right; individual channels for person and dog visualized as blue and green).

Also unlike segmentation masks, omnimattes can capture partially-transparent, soft effects such as reflections, splashes, or tire smoke. Like conventional mattes, omnimattes are RGBA images that can be manipulated using widely-available image or video editing tools, and can be used wherever conventional mattes are used, for example, to insert text into a video underneath a smoke trail.

Layered Decomposition of Video
To generate omnimattes, we split the input video into a set of layers: one for each moving subject, and one additional layer for stationary background objects. In the example below, there is one layer for the person, one for the dog, and one for the background. When merged together using conventional alpha blending, these layers reproduce the input video.

Besides reproducing the video, the decomposition must capture the correct effects in each layer. For example, if the person’s shadow appears in the dog’s layer, the merged layers would still reproduce the input video, but inserting an additional element between the person and dog would produce an obvious error. The challenge is to find a decomposition where each subject’s layer captures only that subject’s effects, producing a true omnimatte.

Our solution is to apply our previously developed layered neural rendering approach to train a convolutional neural network (CNN) to map the subject’s segmentation mask and a background noise image into an omnimatte. Due to their structure, CNNs are naturally inclined to learn correlations between image effects, and the stronger the correlation between the effects, the easier for the CNN to learn. In the above video, for example, the spatial relationships between the person and their shadow, and the dog and its shadow, remain similar as they walk from right to left. The relationships change more (hence, the correlations are weaker) between the person and the dog’s shadow, or the dog and the person’s shadow. The CNN learns the stronger correlations first, leading to the correct decomposition.

The omnimatte system is shown in detail below. In a preprocess, the user chooses the subjects and specifies a layer for each. A segmentation mask for each subject is extracted using an off-the-shelf segmentation network, such as MaskRCNN, and camera transformations relative to the background are found using standard camera stabilization tools. A random noise image is defined in the background reference frame and sampled using the camera transformations to produce per-frame noise images. The noise images provide image features that are random but consistently track the background over time, providing a natural input for the CNN to learn to reconstruct the background colors.

The rendering CNN takes as input the segmentation mask and the per-frame noise images and produces the RGB color images and alpha maps, which capture the transparency of each layer. These outputs are merged using conventional alpha-blending to produce the output frame. The CNN is trained from scratch to reconstruct the input frames by finding and associating the effects not captured in a mask (e.g., shadows, reflections or smoke) with the given foreground layer, and to ensure the subject’s alpha roughly includes the segmentation mask. To make sure the foreground layers only capture the foreground elements and none of the stationary background, a sparsity loss is also applied on the foreground alpha.

A new rendering network is trained for each video. Because the network is only required to reconstruct the single input video, it is able to capture fine structures and fast motion in addition to separating the effects of each subject, as seen below. In the walking example, the omnimatte includes the shadow cast on the slats of the park bench. In the tennis example, the thin shadow and even the tennis ball are captured. In the soccer example, the shadow of the player and the ball are decomposed into their proper layers (with a slight error when the player’s foot is occluded by the ball).

This basic model already works well, but one can improve the results by augmenting the input of the CNN with additional buffers such as optical flow or texture coordinates.

Applications
Once the omnimattes are generated, how can they be used? As shown above, we can remove objects, simply by removing their layer from the composition. We can also duplicate objects, by repeating their layer in the composition. In the example below, the video has been “unwrapped” into a panorama, and the horse duplicated several times to produce a stroboscopic photograph effect. Note that the shadow that the horse casts on the ground and onto the obstacle is correctly captured.

A more subtle, but powerful application is to retime the subjects. Manipulation of time is widely used in film, but usually requires separate shots for each subject and a controlled filming environment. A decomposition into omnimattes makes retiming effects possible for everyday videos using only post-processing, simply by independently changing the playback rate of each layer. Since the omnimattes are standard RGBA images, this retiming edit can be done using conventional video editing software.

The video below is decomposed into three layers, one for each child. The children’s initial, unsynchronized jumps are aligned by simply adjusting the playback rate of their layers, producing realistic retiming for the splashes and reflections in the water.

In the original video (left), each child jumps at a different time. After editing (right), everyone jumps together.

It’s important to consider that any novel technique for manipulating images should be developed and applied responsibly, as it could be misused to produce fake or misleading information. Our technique was developed in accordance with our AI Principles and only allows rearrangement of content already present in the video, but even simple rearrangement can significantly alter the effect of a video, as shown in these examples. Researchers should be aware of these risks.

Future Work
There are a number of exciting directions to improve the quality of the omnimattes. On a practical level, this system currently only supports backgrounds that can be modeled as panoramas, where the position of the camera is fixed. When the camera position moves, the panorama model cannot accurately capture the entire background, and some background elements may clutter the foreground layers (sometimes visible in the above figures). Handling fully general camera motion, such as walking through a room or down a street, would require a 3D background model. Reconstruction of 3D scenes in the presence of moving objects and effects is still a difficult research challenge, but one that has seen promising recent progress.

On a theoretical level, the ability of CNNs to learn correlations is powerful, but still somewhat mysterious, and does not always lead to the expected layer decomposition. While our system allows for manual editing when the automatic result is imperfect, a better solution would be to fully understand the capabilities and limitations of CNNs to learn image correlations. Such an understanding could lead to improved denoising, inpainting, and many other video editing applications besides layer decomposition.

Acknowledgements
Erika Lu, from the University of Oxford, developed the omnimatte system during two internships at Google, in collaboration with Google researchers Forrester Cole, Tali Dekel, Michael Rubinstein, William T. Freeman and David Salesin, and University of Oxford researchers Weidi Xie and Andrew Zisserman.

Thank you to the friends and families of the authors who agreed to appear in the example videos. The “horse jump low”, “lucia”, and “tennis” videos are from the DAVIS 2016 dataset. The soccer video is used by permission from Online Soccer Skills. The car drift video was licensed from Shutterstock.

Source: Google AI Blog


A new place for Black women in tech to tell their stories

During the summer of 2020, people all over the world demanded an end to police brutality against Black people and for action to be taken in the way Black people are seen and treated. This was accompanied by an awakening in the tech industry as well: A recognition that the tech community should play a major role in addressing racial bias and equity.  

This is part of why Google’s Women Techmakers decided to launch our Black Women In Tech storytelling campaign. 

Black women are underrepresented in the tech industry, and their contributions are not widely acknowledged and celebrated. The Black Women In Tech campaign will highlight the stories, experiences, and expertise of Black women in the American tech industry through things like community stories shared by Black women within and outside of Google.

We built this campaign by teaming up with Black illustrator Rachelle Baker, and Black stock photography company TONL. We also wanted to start a conversation about what “being in tech” means – and debunk the narrative that it only means you’re a developer if you live in Silicon Valley. On the Black Women In Tech website, you’ll find stories about program manager Yolanda Washington, a Bronx native, and Women Techmaker Madona Wambua based in Alabama. 

We also wanted to make sure that Black women interested in learning new skills could find the resources they needed. So we added a comprehensive list of the training resources Google provides for developers and founders.

Ultimately, we hope the campaign is an inspiration to the next generation of Black women considering a career in tech, and that Black women in the industry see themselves in these stories.

We know what equity should look like, but it takes the effort of everyone every day and at every step to achieve sustainable equity in the workplace.

To read the stories, discover the resources, and keep up with the campaign, visit the Black Women In Tech website.

A new place for Black women in tech to tell their stories

During the summer of 2020, people all over the world demanded an end to police brutality against Black people and for action to be taken in the way Black people are seen and treated. This was accompanied by an awakening in the tech industry as well: A recognition that the tech community should play a major role in addressing racial bias and equity.  

This is part of why Google’s Women Techmakers decided to launch our Black Women In Tech storytelling campaign. 

Black women are underrepresented in the tech industry, and their contributions are not widely acknowledged and celebrated. The Black Women In Tech campaign will highlight the stories, experiences, and expertise of Black women in the American tech industry through things like community stories shared by Black women within and outside of Google.

We built this campaign by teaming up with Black illustrator Rachelle Baker, and Black stock photography company TONL. We also wanted to start a conversation about what “being in tech” means – and debunk the narrative that it only means you’re a developer if you live in Silicon Valley. On the Black Women In Tech website, you’ll find stories about program manager Yolanda Washington, a Bronx native, and Women Techmaker Madona Wambua based in Alabama. 

We also wanted to make sure that Black women interested in learning new skills could find the resources they needed. So we added a comprehensive list of the training resources Google provides for developers and founders.

Ultimately, we hope the campaign is an inspiration to the next generation of Black women considering a career in tech, and that Black women in the industry see themselves in these stories.

We know what equity should look like, but it takes the effort of everyone every day and at every step to achieve sustainable equity in the workplace.

To read the stories, discover the resources, and keep up with the campaign, visit the Black Women In Tech website.