Tag Archives: Google AR and VR

Shopping for a beauty product? Try it on with Google.

I’ve been doing most of my shopping online this year and it seems I’m not alone. With the holidays right around the corner, more than 73 percent of U.S. shoppers are planning to buy online. There are plenty of perks with online shopping, from the convenience of doing it from your couch to the multitude of options right at your fingertips.

At the same time, many still crave that in-store experience, whether it’s seeing a product up close, trying it on or getting advice from in-store experts. So we’re bringing some of the benefits of in-store shopping to your phone, with new features launching this week in the U.S.

Visualize makeup looks and try on products 

If you’re shopping for beauty products, it’s difficult to make a decision on such personal items without trying them on. Say you want to know how sheer a lip gloss is, how much pigment is in an eyeshadow or what a product will look like on your skin tone, it can be difficult to understand these things without seeing the products up close. 

To make it easier for you to answer these questions from home, we’re working with data partners ModiFace and Perfect Corp to help you better visualize thousands of lipstick and eyeshadow shades from your favorite brands like L’Oreal, MAC Cosmetics, Black Opal and Charlotte Tilbury. Now, when you search for a lipstick or eyeshadow product, like L’Oreal’s Infallible Paints Metallic Eyeshadow, you can see what it looks like on a range of skin tones and compare shades and textures to help you find the right products.

How you can try on lipstick and eyeshadow

To help you find the perfect match, you can now also virtually try makeup products right from the Google app. Find the perfect nude lip by searching for MAC Powder Kiss Lipstick and quickly trying each shade in the collection. If you’re looking for a velvety matte lipstick for your holiday video calls with family and friends, search for NARS Powermatte Lip Pigment and find the color that matches your style.

Woman trying on lipstick

Get recs from experts and enthusiasts

Sometimes it’s helpful to get recommendations and see how products work for other people. Now as you browse Google Shopping on your phone or scroll through the Discover feed in the Google app, we'll show recommendations from beauty, apparel and home and garden enthusiasts and experts about their favorite products. If a product catches your eye, you can hear directly from experts on why they love the product and how they use it. 

Hear from professional makeup artist Jonet on what products can help you achieve that dewy fresh look or learn how Homesick Candles make the perfect holiday present.  Once you’ve found a product you love, you’ll be able to easily shop these recommendations. This new feature comes from work done by the Shoploop team, which was formerly part of Area 120, Google's in-house incubator.

We hope these new shopping tools will make it easier for you to find products you’ll love while shopping from home.

“The Mandalorian” in AR? This is the way.

In a galaxy far, far away, the Mandalorian and the Child continue their journey, facing enemies and rallying allies in the tumultuous era after the collapse of the Galactic Empire. But you don’t need a tracking fob to explore the world of the hit STAR WARS streaming series. Google and Lucasfilm have teamed up to bring iconic moments from the first season of “The Mandalorian” to life with “The Mandalorian” AR Experience (available on the Play Store for 5G Google Pixels and other select 5G Android phones) as fans follow the show’s second season. (Check your phone to see if it meets app requirements.)

Animated GIF showing a person's hand holding a Pixel phone while using the Mandalorian AR app.

From dinosaurs to astronauts, Google has been bringing objects and creatures to life with augmented reality. Now, people using compatible Android 5G devices can interact with heroes from the Mandalorian in their own space.

“The Mandalorian” AR Experience puts you in the shoes of a bounty hunter following the trail of Mando himself, Din Djarin and the Child. Explore the world of “The Mandalorian,” interact with characters in augmented reality and capture your very own scenes to share with friends.

To create this original experience, Google, Disney and Lucasfilm worked together to imagine a next-generation augmented reality app optimized for 5G devices. Our teams collaborated to build hyper-detailed models and life-like animations—all while packing scenes with fun surprises.

UsingARCore,Google’s developer platform for building augmented reality experiences, we created scenes that interact with your environment and respond to your surroundings. You can discover and unlock even more effects based on your actions. And thanks to the new ARCore Depth API, we also enabled occlusion, allowing 3D scenes to blend more naturally with our world.

Animated GIF showing the character the Mandalorian in AR standing in someone's kitchen on the screen of a Pixel phone.

New content will keep rolling out in the app each week onMando Mondays, so stay tuned—and Pixel owners should keep an eye out for additional exclusive content outside of the app as well.

Lucasfilm, the Lucasfilm logo, STAR WARS and related properties are trademarks and/or copyrights, in the United States and other countries, of Lucasfilm Ltd. and/or its affiliates. © & ™ 2020 Lucasfilm Ltd. All rights reserved.

Source: Android


Visual ways to search and understand our world

Whether you’re a student learning about photosynthesis or a parent researching the best cars for your growing family, people turn to Google with all sorts of curiosities. And we can help you understand in different ways—through text, your voice or even your phone’s camera. Today, as part of the SearchOn event, we’re announcing new ways you can use Google Lens and augmented reality (AR) while learning and shopping.

Visual tools to help you learn 

For many families, adjusting to remote learning hasn’t been easy, but tools like Google Lens can help lighten the load. With Lens, you can search what you see using your camera. Lens can now recognize 15 billion things—up from 1 billion just two years ago—to help you identify plants, animals, landmarks and more. If you’re learning a new language, Lens can also translate more than 100 languages, such as Spanish and Arabic, and you can tap to hear words and sentences pronounced out loud. 


If you’re a parent, your kids may ask you questions about things you never thought you’d need to remember, like quadratic equations. From the search bar in the Google app on Android and iOS, you can use Lens to get help on a homework problem. With step-by-step guides and videos, you can learn and understand the foundational concepts to solve math, chemistry, biology and physics problems.

Lens Homework

Sometimes, seeing is understanding. For instance, visualizing the inner workings of a plant cell or the elements in the periodic table in 3D is more helpful than reading about them in a textbook. AR brings hands-on learning home, letting you explore concepts up close in your space. Here’s how Melissa Brophy-Plasencio, an educator from Texas, is incorporating AR into her lesson plans.

Melissa Brophy-Plasencio, an educator from Texas, shares how she's using AR into her science lessons.

Shop what you see with Google Lens 

Another area where the camera can be helpful is shopping—especially when what you’re looking for is hard to describe in words. With Lens, you can already search for a product by taking a photo or screenshot. Now, we’re making it even easier to discover new products as you browse online on your phone. When you tap and hold an image on the Google app or Chrome on Android, Lens will find the exact or similar items, and suggest ways to style it. This feature is coming soon to the Google app on iOS.

Lens Shopping

Lens uses Style Engine technology which combines the world’s largest database of products with millions of style images. Then, it pattern matches to understand concepts like “ruffle sleeves” or “vintage denim” and how they pair with different apparel. 

Bring the showroom to you with AR

When you can’t go into stores to check out a product up close, AR can bring the showroom to you. If you’re in the market for a new car, for example, you’ll soon be able to search for it on Google and see an AR model right in front of you. You can easily check out what the car looks like in different colors, zoom in to see intricate details like buttons on the dashboard, view it against beautiful backdrops and even see it in your driveway. We’re experimenting with this feature in the U.S. and working with top auto brands, such as Volvo and Porsche, to bring these experiences to you soon.

ColorSwaps_Volvo_560_sq.gif

AR experience of the 2020 Volvo XC40 Recharge

Everyone’s journey to understand is different. Whether you snap a photo with Lens or immerse yourself in AR, we hope you find what you’re looking for...

Ladybug.gif

...and even have some fun along the way.

Source: Search


Play with Art Filter to learn about culture

What if you could learn about the story of the Girl with a Pearl Earring while wearing her earrings yourself? Or how about putting on a Japanese helmet to take you back to the time of Samurai traditions? Starting today, you can use “Art Filter” in the Google Arts & Culture app to become an artwork or try on iconic artifacts otherwise safely stored in museums. 


Thanks to our partners who make their amazing collections available online, we were able to create five educational and fun 3D-modelled augmented reality filters based on iconic paintings, objects and accessories from all over the world. Snap a video or image of yourself to become Van Gogh or Frida Kahlo’s self-portraits, or the famous Girl with a Pearl Earring. You can also step deep into history with a traditional Samurai helmet or a remarkable Ancient Egyptian necklace.

In this novel experience, each filter has been crafted carefully so that you can explore the artifacts in high-quality detail from every angle. Before you try on the filter, you’ll also see more about its history and context. 


Our team was guided by Google’s AI Principles to determine how we built this feature for everyone, with social benefit, security, privacy and scientific rigor top of mind. Art Filter applies Machine Learning based image processing because making this kind of AR feature possible needs a sophisticated anchoring of the virtual content to the real world. Through this approach, the artifacts are able to position themselves organically and smoothly on your head or react to your facial expression. Art Filter—similar to the popular Art Selfie feature—runs completely on your device; your videos and photos are not stored unless you choose to save them or share them with friends.  


To get started, open the free Google Arts & Culture app for Android or iOS and tap the rainbow camera icon at the bottom of the homepage. We are looking forward to seeing what creations you make with Art Filter and what you will learn about these artifacts. Share your favorite photos and videos using the hashtag #ArtFilter.

Travel digitally with Google on World Tourism Day

September 27 is World Tourism Day - a time to celebrate tourism's ability to promote meaningful exchanges between people around the world, have fun, recall how travel helps us all recharge – and make a real difference by supporting livelihoods and protecting our heritage. 

This year may have changed our ability to travel across the globe, but our desire to experience new cultures, see far-off places or discover hidden gems in our own backyard has not diminished. 

Today, Google Arts & Culture has brought together a new collection to help anyone choose their perfect virtual travel with thousands of museums and cultural destinations to explore. And with the help of our partner CyArk, we've launched on Google Search 37 cultural heritage sites from across the world in Augmented Reality (AR). Hop from your couch and search on your mobile phone to bring the Moai statues of Ahu Ature Huki, Rapa Nui (Easter Island), the Brandenburg Gate in Germany, or the Maya pyramid of Chichén Itzá, Mexico right into your living room.

  • ThomasJeffersonMemorial.png

    Thomas Jefferson Memorial and El Castillo, Chichen Itza projected in AR

  • BrandenburgGate.png

    Brandenburg Gate, Germany

  • GatewayOfIndia.png

    Gateway of India, Mumbai

You can read more about what it takes for CyArk to capture just one site in "Documenting the Thomas Jefferson Memorial" and discover how this work helps global conservation efforts communicate the impacts of climate change to iconic places like Rapa Nui.


Continue your journey on Google Arts & Culture

There are plenty more sites to visit virtually: let Google Arts & Culture be your guide to discover some of the world’s most amazing destinations, from the Wonders of Mexico, the USA, France and many more to some amazing city breaks, action-packed adventures and paradise escapes.


Let your favorite creator take you on a tour

Finally, travel like a local, and explore Andalucia with YouTube creator Kikillo, join a virtual walk around Milan with Instagram creator Federica di Nardo, or listen to the sounds of Florence with The Whispering Traveller.


All this, and more than 10,000 destinations and 2,000 collections are ready to be explored on Google Arts & Culture at g.co/culturaltravel. And if Augmented Reality really has you hooked, make sure to check out a few other cool things including Dinosaurs, the Skeletal System and Apollo 11 by looking them up in Google Search.

Travel back in time with AR dinosaurs in Search

Dinosaurs may have ruled the Earth millions of years ago, but with augmented reality (AR), you can turn your home into “Jurassic World.” We're partnering with Universal Brand Development, Amblin Entertainment and Ludia to bring 10 dinosaurs from the franchise film, “Jurassic World”, to Google Search. Watch the massive T. Rex stomp in your living room or gaze up at a majestic Brachiosaurus as it towers above a neighborhood tree. 

Search for a dinosaur on Google using a mobile device and tap “View in 3D” to rotate or zoom in and see it up close. You can then bring the dinosaur into your space with AR and adjust its size to understand how big it is in relation to the things around you. On Android devices, turn up your volume to hear the thudding footsteps and roars of each dinosaur. 

“Jurassic World” dinosaurs that are viewable in AR include: Tyrannosaurus Rex, Velociraptor, Triceratops, Spinosaurus, Stegosaurus, Brachiosaurus, Ankylosaurus, Dilophosaurus, Pteranodon, and Parasaurolophus.

  • Pteranodon.gif


  • TRex.gif


  • Brachiosauras.gif


A behind-the-scenes look at how
“Jurassic World” AR dinosaurs are made 

Using technology from Ludia’s “Jurassic World Alive” game, these AR dinosaurs are some of the most realistic models out there. Check out this video to see how an AR Brachiosaurus is made, including 3D modeling, texturing and animation.

“To create the 3D dinosaurs, our concept artists first did preliminary research to discover information about each creature,” says Camilo Sanin, Ludia’s Lead on Character Creations. “Not only did we draw research from various forms of literature, our artists also worked with paleontologists and the ‘Jurassic World’ team to make the assets as accurate and realistic as possible. Even the smallest of details, such as irregularities of skin color and patterns, are important.” 

Unlike some of Google’s AR animals, like a dog or tiger, dinosaurs pose a new technical challenge: their massive size. The new auto-scale feature on Android can now automatically calculate the distance between your phone and a surface in your space and resize the dinosaur so it fits on your phone screen. If you tap “View actual size,” AR tracking technology automatically repositions the dinosaur in your space to make room for it.

view actual size.gif

New auto-scale feature on Android

How to access and share

On Android, search for “dinosaur” or one of the 10 dinosaurs on the Google app or any Android browser and tap “View in 3D.” You can see 3D content on devices running with Android 7 and above and you can see AR content on ARCore-enabled devices. Easily explore all dinosaurs using the carousel format. 

On iOS, search for “dinosaur” or one of the 10 dinosaurs on the Google app or on Google.com with Chrome or Safari. 3D and AR content is available on devices running iOS 11 and above.

You can also create AR videos—or recreate your favorite scenes from the “Jurassic World” movies—with the recording option. Don’t forget to tag your photos and videos on social with #Google3D and #JurassicWorld. Safe traveling (back in time)!

100 Years of Bauhaus on Google Arts & Culture

Even if you’ve never heard of the Bauhaus movement, you’ve probably seen its influence all around you. From traffic signs to office furniture, the legendary design school changed the way our world looks and functions.  

One hundred years after the movement began in Germany, we’re still surrounded by Bauhaus ideas about art, technology and craftsmanship, which are reflected in Google Arts & Culture's newest collection—"Bauhaus Everywhere". The collection came together in partnership with the Bauhaus Dessau Foundation in Germany—as part of our multi-year digitization collaboration—and six other partners including the IIT Institute of Design or the Guggenheim Museum.

Bauhaus Everywhere

Bauhaus design aimed to improve people's lives through functional design. Well-known members of the school, such as its founder Walter Gropius, the controversial Hannes Meyer or Gunta Stölzl, as one of many female designers and artists, have a lasting influence on architecture, furniture design and even typefaces. 

This project digitizes over 10,000 objects, offers virtual tours of iconic buildings and exhibits over 400 artworks captured with our Art Camera. The result is over 45 online exhibitions curated by our seven partners featuring icons like the world known tubular steel armchair or imagery of “Africa's Finest Campus” and the (perhaps unexpectedly) best selling bauhaus design, wallpaper. 

There are also unique insights into the everyday student life of Bauhaus including the renowned Bauhaus parties and the forward thinking empowerment of women. And, because the school’s design principles spread far beyond Germany and Europe, we’ve created a Google Earth Voyager Tour to show how people as far away as Japan, India or Brazil were inspired by Bauhaus. 

  • Young people posing at the beach

    The Bauhaus Study Guide: Learn what it was like to study at this very unusual school. © Stiftung Bauhaus Dessau

  • 3D model of a round house .gif

    Explore three never-built Bauhaus buildings in Augmented Reality. Here theRundhaus by Carl Flieger.

  • Colourful expressionist painting

    Zoom into a high resolution capture of Wassily Kandinsky's masterwork “Yellow. Red. Blue” and learn about his condition that made him hear colors. © Stiftung Bauhaus Dessau


  • Image of a big concrete high-rise building

    Learn why the first international exhibition on Bauhaus took place in India and how the Bauhaus ideas influenced the look of Indian cities. © Johannes Schwartz, Het Nieuwe Instituut

  • A dining room with colorful furniture

    Follow the tracks of Bauhaus internationally and see what the Dutch De Stijl movement and Bauhaus had in common. © Johannes Schwartz


New shapes, materials and approaches to construction made Bauhaus proposals stand out. Its architectural designs  were especially known for their avantgarde approach. But many of these bold building plans stayed just that, and were never actually constructed. In collaboration with experts from the Bauhaus Dessau Foundation, the collection contains buildings that had only ever existed on paper and in the minds of their creators. 

Together we assembled archival sketches, scribbles and vague descriptions to create augmented reality models of three visionary structures. In the Google Arts & Culture app anyone can now explore “Round House” by Carl Fieger, “BAMBOS” by Marcel Breuer and “Court House” by Eduard Ludwig from inside and outside. 

László Moholy-Nagy, a teacher at the Bauhaus, put it this way: "Design is not a profession, design is an attitude." We hope you’ll see that the Bauhaus attitude is not just everywhere but, through this exhibit, also for everyone. 


A VR series about women with the “courage to question”

Many of us who fight for women’s rights have the privilege of knowing mighty women and girls. They are the courageous ones--those who insist on and fight for a future where women and girls are free from violence and can live out their full potential. Most of these women’s rights defenders are not widely known, yet work tirelessly at grave risk to themselves and their families. But they are dogged. They are fearless. They are unbowed. They are leaders. And on this International Women's Day, we celebrate them, and the impact that women have had all around the world.

Along with UN Women, and the civil society organizations Vital Voices and Global Fund for Women, we identified four women’s rights defenders who are building movements against mass incarceration, human trafficking, child marriage and sexual violence. Their stories are the stories of our virtual reality series, “Courage to Question.”

Courage to Question: Series Trailer

Now pardoned and free after 21 years of confinement, Alice Johnson discusses her wrenching as a mother behind bars, the impact of mass incarceration on women, and why she fights for the women she left behind.


Lydia Cacho shares her experience as a Mexican journalist, author, and human rights activist who, despite receiving multiple threats to her safety and life over the years, fights tirelessly to tell the stories of women and girls who have been trafficked. (link to video)


Asha Kowtal—General Secretary of the Dalit Women’s Rights movement in India—walks us through a day in the life of Dalit women. Formerly known as “untouchables,” they’re members of the lowest caste in India and are fighting back against systems of oppression. (link to video)


Chief Theresa Kachindamoto of the Dedza District in the central region of Malawi shares the story of her upbringing, the practice of child marriage in her country, and her fight to eradicate it. (link to video)


Captured by an all-female crew in VR180, these films make you feel like you’re actually there with these brave women, who are remaking a world that allows women and girls to be free and equal. The films are best viewed in a VR headset like Cardboard, but you can also view them on YouTube with your phone or desktop.


As a human rights lawyer, I know that so often human rights organizations receive the scraps of new technology. “Courage to Question” gives these four amazing women a platform to tell their stories and advance their human rights work.


We’ll be premiering these videos at the United Nations today for the opening ceremony of International Women’s Day. This kicks off an ongoing discussion around digital rights as women’s rights, and gender inclusion and equity in the context of tech. We’ll also be sharing that Google has signed the Women’s Empowerment Principles, which have been developed by the United Nations Global Compact and UN Women to help organizations advance and empower women in the workplace and beyond. These principles build on our ongoing commitment to a diverse and inclusive workplace for all, as well as support for education and economic opportunity for women globally.


In honor of all that has been fought for and hard-won—and all that is still left to do—we give thanks to these four women, and the many others, on whose shoulders we stand, and whose work we are grateful for.

Seeing art in a new way: VR tools let characters jump right in

We all know you’re not supposed to touch the pieces of art in a museum, but what if you could jump inside them? In YouTube creator SoKrispy’s latest VR video, Do Not Touch, the characters do just that: they literally dive into the artwork and become part of the scene.   

Do Not Touch GIF

A scene From SoKrispyMedia’s “Do Not Touch” VR Video

Visual effects this advanced usually require physical equipment like green screens that can be costly and difficult to set up. SoKrispy, however, was able to achieve many of the effects in post-production with Google’s Jump VR Video suite, along with Adobe Creative Suite, Unreal Engine, and Autodesk 3DS Max.

To make it look like the actors were actually inside the pictures, SoKrispyMedia used Jump’s "high-quality stitching" option, a feature that lets creators achieve a green screen effect without needing a physical screen. They also utilized this feature in one of their earlier works, “Video Game Vehicle,” after discovering in post-production that the physical green screen they had used while filming wasn’t big enough. High-quality stitching let them “fix it in post” with a few clicks, rather than spending days reshooting with a wraparound green screen.

DepthGreenscreen

Jump’s high-quality depth maps allow creators to get a green screen effect without a physical green screen

So how does this feature work? In addition to producing higher quality stitches, it lets creators generate razor-sharp depth maps that estimate the distance (or depth) of every pixel in the scene. This enables the use of editing software like Adobe Premiere or After Effects to extract elements or even composite a new scene in the background, all without a physical screen or meticulous manual rotoscoping. And in the latest release of After Effects, our partners at Adobe have made it even easier to leverage Jump's depth maps in post-production.

SoKrispy ZDepth in Adobe GIF.gif

Generating depth maps in Jump Manager gives you more flexibility in post-production

In addition to using depth maps, SoKrispyMedia used Jump’s high-quality stitches to realistically light computer generated (CG) objects with Image Based Lighting. Instead of having to capture separate light probes, creators can get the same effect with Jump’s high bit-depth 360° stitches. This saves valuable time on set and gives creators the ability to seamlessly integrate CG into live action footage with accurate lighting and reflections.

Car_IBL_Image.png

Side by side still of CG object’s from Video Game Vehicle with and without image based lighting

SoKrispyMedia also utilized a third, software-based post-production tool to enhance their Jump footage. Using a technique detailed on the Google AI blog called “style transfer," they applied artificial intelligence to their footage to transform the look of each character into the style of the painting they’ve jumped into.  

Style Transfer.gif

Another scene From “Do Not Touch”

Taken together, these software-based features—high-quality stitching, Image Based Lighting and style transfer—provide creators with new ways to share their vision with the world. As SoKrispyMedia’s Director and VFX supervisor Sam Wickert explains, “The most important aspect of these projects is to make VR content that is really worth watching in a headset, and these tools let us do that.” For audiences, this means we can look forward to traveling to all sorts of new destinations in the virtual world … from a video game environment to a museum where you actually jump into the paintings to wherever creators take us next.

VR Labs Open Doors of Opportunity for STEM Students

For students pursuing STEM degrees like biology, hands-on time in a lab can be as essential as time spent in the lecture hall or library. In fact, for many science-based degrees, it’s required. But getting access to a lab isn’t always easy. Many students don't live close enough to a lab facility or a university that offers their degree of choice. Others find it hard to get enough lab time because student demand is too high or their school can't afford to provide unlimited access.

Through its ability to take people anywhere, virtual reality can be a powerful resource for students who otherwise would not have access to the lab time they need to complete their degrees.  We partnered with science education company Labster to create more than 30 virtual labs on the Daydream platform, where students can do their lab work in VR without having to walk, drive, or fly to a campus.  These VR labs can be particularly useful to students and faculty at the rapidly-growing number of schools that offer online science degrees. 

Earlier this month, students in Arizona State University’s online B.S. in Biological Sciences program began working in these virtual labs for full course credit. Soon students at the University of Texas at San Antonio, McMaster University, and other institutions across North America and Europe will be able to do their lab work in VR as well.  

Through its ability to take people anywhere, virtual reality can be a powerful resource for students who otherwise would not have access to the lab time they need to complete their degrees.
COC_Holotable2.gif

UsingDaydream View or the Lenovo Mirage Solo with Daydream, students can do things that previously necessitated a physical presence in a lab, like examining organisms under a microscope and sequencing DNA. They can also do things that wouldn’t be possible in the physical world, like seeing and manipulating DNA at the molecular level and visiting Astakos IV, a newly discovered exoplanet being explored as a potential habitat for human beings.

Because there’s no time limit, students can review theories, concepts, and techniques as many times as they want. In addition, students receive personalized feedback in the app to help them understand which concepts they need to review, and which techniques need more practice.

COC_Fume_Hood.gif

We’re hoping to make the virtual lab experience available to more students worldwide, including undergraduates, graduate students, and even high schoolers.

If you’d like to bring the virtual lab experience to your school, you can learn more at labster.com/vr.