Tag Archives: Google AR and VR

Helping NASA and JPL bring the surface of Mars to your browser

On August 6, 2012, the Curiosity rover landed on Mars. Ever since, it’s been searching for evidence that Mars has ever been suitable for life. It’s also been photographing the Martian terrain in great detail. Scientists at NASA’s Jet Propulsion Lab use these photos to create a 3D model of Mars. It’s a one-of-a-kind scientific tool for planning future missions.

Today, we’re putting that same 3D model into an immersive experience for everyone to explore. We call it Access Mars, and it lets you see what the scientists see. Get a real look at Curiosity’s landing site and other mission sites like Pahrump Hills and Murray Buttes. Plus, JPL will continuously update the data so you can see where Curiosity has just been in the past few days or weeks. All along the way, JPL scientist Katie Stack Morgan will be your guide, explaining key points about the rover, the mission, and some of the early findings.

The experience is built using WebVR, a technology that lets you see virtual reality right in your browser, without installing any apps. You can try it on a virtual reality headset, phone, or laptop.

Check it out at g.co/accessmars.

And if you’re an educator, we’ve updated our Mars tour in Google Expeditions with highlights from this experience. To try it with your class or in self-guided mode, download the Expeditions app from Google Play or the App Store.

Source: The Official Google Blog

Look up to your role models in VR with “The Female Planet”

I grew up surrounded by strong female role models. My mom worked on Wall Street in the 1960s as an IBM Systems Engineer, one of the only women in her field. From day one, her stories about preparation, mental toughness, and teamwork helped me determine aspirations for my own career. When I was a bit older I had a summer internship and shadowed a female broadcast journalist at the top of her career, sparking my interest in media. Now in my role at Google, I want to use VR technology to make it easier for young women to shadow and learn from global female leaders.

We just released “The Female Planet,” a new VR video series from Google and Surround Vision, that captures the daily lives of amazing female role models around the world. “The Female Planet” delves into the personal and professional experiences of five extraordinary women with careers spanning technology, science, sports, and the arts. Follow in their footsteps as they take you through their day-to-day routines. You’ll see inside their workplaces, hear firsthand why they chose the paths they’re on now, and get to know what pushed them to keep moving forward.

Each episode features a different female leader, such as Tiera Fletcher, a Boeing aerospace engineer designing a rocket for the NASA mission to Mars, or Inna Braverman, an entrepreneur turning wave power into green energy. In the first episode, Golden Globe winner Gina Rodriguez of TV’s “Jane the Virgin” talks about how her father’s boxing lessons and her mom’s strength gave her the courage to only accept roles that empower women. She also takes you on the set of Season 4 of “Jane the Virgin” as she explains how Jane is helping to bust stereotypes.

To make you feel as if you’re really walking alongside Gina, Tiera, and Inna, the series was shot in 3D 360 with the new YI HALO camera and Google’s Jump technology. The YI HALO features 17 cameras in a 360 array and is paired with Jump to stitch together the images, resulting in high resolution 3D video where near things look near, and far things look far, so you’re truly immersed in the stories.

Tune in on YouTube to find your role model via virtual reality. Use Daydream View or Cardboard to check out the first episode on Google’s YouTube channel, with more episodes coming soon. And stay tuned for more Daydream and Jump productions later this fall.

Source: The Official Google Blog

Bringing Google Earth to Expeditions with Seurat

Editor’s Note: In this technical post we explain how Seurat, a developer tool from Google, is able to bring high-end graphics to mobile VR systems.

If students could see almost any place in the world with the high quality graphics of Google Earth VR, it would create new opportunities for exploration and learning. For this to work, it would need to be based on a system that’s accessible and easy to use for schools. Using a new tool called Seurat, we were able to do just that. We recently launched support for Google Earth scenes in Expeditions, which lets millions of students around the world experience some of the most beautiful places on the planet.

Mobile VR is easy to use, but it has less available computing power than PC-based systems. To make Google Earth VR hit the required framerates on mobile VR, we need to use techniques like occlusion culling. Google Earth has one of the largest 3D datasets available today, and we’re constantly updating it with new scans and applying reconstruction algorithms in our data centers. However, the pace of updates makes it challenging to apply recent research advances.

Fortunately, Seurat can help deal with scenes that are heavily occluded. We announced this tool at Google IO, and it’s a solution for high quality graphics on mobile VR:

Seurat is a scene simplification technology that optimizes the geometry and textures in your scene from a defined viewing volume (cube to the left above). It takes layered depth images (RGBA + Depth) as input and generates a textured mesh, targeting a configurable number of triangles and texture size. Having control of triangle count is beneficial for the Google Earth data, where the quality of the vertex data varies between locations. Hitting consistent framerates is important for VR, and the more reliable the scene geometry rendering time is, the easier it gets to organize per-frame budget for other elements, such as menus and overlays. Below is an example of generating Seurat input from Google Earth data.

As seen below, the mesh looks good inside the viewing volume. As soon as we move outside, we can quickly see the effects of the Seurat occlusion optimization. In this example, we used millions of triangles and 1000+ MB of texture data to generate the Seurat input. The output is a 20k triangles mesh (50:1 compression) with 5 MB texture data (200:1 compression).

In current writing, mobile VR only supports 3DoF (rotation only) tracking. 360 degree images, both mono and stereo, have been utilized in this environment with great success.

Seurat is a powerful tool for bringing immersive experiences with high fidelity graphics to mobile VR. Hear what our friends at ILM had to say about it, and if you want to check out these Expeditions with Earth VR imagery for yourself, download the Google Expeditions app on iOS or Android and enjoy them in your Cardboard or Daydream View. Seurat is still in development, but stay tuned for future updates!

Source: The Official Google Blog

The best hardware, software and AI—together

Today, we introduced our second generation family of consumer hardware products, all made by Google: new Pixel phones, Google Home Mini and Max, an all new Pixelbook, Google Clips hands-free camera, Google Pixel Buds, and an updated Daydream View headset. We see tremendous potential for devices to be helpful, make your life easier, and even get better over time when they’re created at the intersection of hardware, software and advanced artificial intelligence (AI).

Why Google?

These days many devices—especially smartphones—look and act the same. That means in order to create a meaningful experience for users, we need a different approach. A year ago, Sundar outlined his vision of how AI would change how people would use computers. And in fact, AI is already transforming what Google’s products can do in the real world. For example, swipe typing has been around for a while, but AI lets people use Gboard to swipe-type in two languages at once. Google Maps uses AI to figure out what the parking is like at your destination and suggest alternative spots before you’ve even put your foot on the gas. But, for this wave of computing to reach new breakthroughs, we have to build software and hardware that can bring more of the potential of AI into reality—which is what we’ve set out to do with this year’s new family of products.

Hardware, built from the inside out

We’ve designed and built our latest hardware products around a few core tenets. First and foremost, we want them to be radically helpful. They’re fast, they’re there when you need them, and they’re simple to use. Second, everything is designed for you, so that the technology doesn’t get in they way and instead blends into your lifestyle. Lastly, by creating hardware with AI at the core, our products can improve over time. They’re constantly getting better and faster through automatic software updates. And they’re designed to learn from you, so you’ll notice features—like the Google Assistant—get smarter and more assistive the more you interact with them.

You’ll see this reflected in our 2017 lineup of new Made by Google products:

The Pixel 2 has the best camera of any smartphone, again, along with a gorgeous display and augmented reality capabilities. Pixel owners get unlimited storage for their photos and videos, and an exclusive preview of Google Lens, which uses AI to give you helpful information about the things around you.
Google Home Mini brings the Assistant to more places throughout your home, with a beautiful design that fits anywhere. And Max is our biggest and best-sounding Google Home device, powered by the Assistant. And with AI-based Smart Sound, Max has the ability to adapt your audio experience to you—your environment, context, and preferences.
With Pixelbook, we’ve reimagined the laptop as a high-performance Chromebook, with a versatile form factor that works the way you do. It’s the first laptop with the Assistant built in, and the Pixelbook Pen makes the whole experience even smarter.
Our new Pixel Buds combine Google smarts and the best digital sound. You’ll get elegant touch controls that put the Assistant just a tap away, and they’ll even help you communicate in a different language.
The updated Daydream View is the best mobile virtual reality (VR) headset on the market, and the simplest, most comfortable VR experience.
Google Clips is a totally new way to capture genuine, spontaneous moments—all powered by machine learning and AI. This tiny camera seamlessly sends clips to your phone, and even edits and curates them for you.

Assistant, everywhere

Across all these devices, you can interact with the Google Assistant any way you want—talk to it with your Google Home or your Pixel Buds, squeeze your Pixel 2, or use your Pixelbook’s Assistant key or circle things on your screen with the Pixelbook Pen. Wherever you are, and on any device with the Assistant, you can connect to the information you need and get help with the tasks to get you through your day. No other assistive technology comes close, and it continues to get better every day.

Google’s hardware business is just getting started, and we’re committed to building and investing for the long run. We couldn’t be more excited to introduce you to our second-generation family of products that truly brings together the best of Google software, thoughtfully designed hardware with cutting-edge AI. We hope you enjoy using them as much as we do.

Source: Google Chrome

The best hardware, software and AI—together

Why Google?

Hardware, built from the inside out

You’ll see this reflected in our 2017 lineup of new Made by Google products:

The Pixel 2 has the best camera of any smartphone, again, along with a gorgeous display and augmented reality capabilities. Pixel owners get unlimited storage for their photos and videos, and an exclusive preview of Google Lens, which uses AI to give you helpful information about the things around you.
Google Home Mini brings the Assistant to more places throughout your home, with a beautiful design that fits anywhere. And Max is our biggest and best-sounding Google Home device, powered by the Assistant. And with AI-based Smart Sound, Max has the ability to adapt your audio experience to you—your environment, context, and preferences.
With Pixelbook, we’ve reimagined the laptop as a high-performance Chromebook, with a versatile form factor that works the way you do. It’s the first laptop with the Assistant built in, and the Pixelbook Pen makes the whole experience even smarter.
Our new Pixel Buds combine Google smarts and the best digital sound. You’ll get elegant touch controls that put the Assistant just a tap away, and they’ll even help you communicate in a different language.
The updated Daydream View is the best mobile virtual reality (VR) headset on the market, and the simplest, most comfortable VR experience.
Google Clips is a totally new way to capture genuine, spontaneous moments—all powered by machine learning and AI. This tiny camera seamlessly sends clips to your phone, and even edits and curates them for you.

Assistant, everywhere

Source: The Official Google Blog

The best hardware, software and AI—together

Why Google?

Hardware, built from the inside out

You’ll see this reflected in our 2017 lineup of new Made by Google products:

The Pixel 2 has the best camera of any smartphone, again, along with a gorgeous display and augmented reality capabilities. Pixel owners get unlimited storage for their photos and videos, and an exclusive preview of Google Lens, which uses AI to give you helpful information about the things around you.
Google Home Mini brings the Assistant to more places throughout your home, with a beautiful design that fits anywhere. And Max is our biggest and best-sounding Google Home device, powered by the Assistant. And with AI-based Smart Sound, Max has the ability to adapt your audio experience to you—your environment, context, and preferences.
With Pixelbook, we’ve reimagined the laptop as a high-performance Chromebook, with a versatile form factor that works the way you do. It’s the first laptop with the Assistant built in, and the Pixelbook Pen makes the whole experience even smarter.
Our new Pixel Buds combine Google smarts and the best digital sound. You’ll get elegant touch controls that put the Assistant just a tap away, and they’ll even help you communicate in a different language.
The updated Daydream View is the best mobile virtual reality (VR) headset on the market, and the simplest, most comfortable VR experience.
Google Clips is a totally new way to capture genuine, spontaneous moments—all powered by machine learning and AI. This tiny camera seamlessly sends clips to your phone, and even edits and curates them for you.

Assistant, everywhere

Source: The Official Google Blog

The best hardware, software and AI—together

Why Google?

Hardware, built from the inside out

You’ll see this reflected in our 2017 lineup of new Made by Google products:

The Pixel 2 has the best camera of any smartphone, again, along with a gorgeous display and augmented reality capabilities. Pixel owners get unlimited storage for their photos and videos, and an exclusive preview of Google Lens, which uses AI to give you helpful information about the things around you.
Google Home Mini brings the Assistant to more places throughout your home, with a beautiful design that fits anywhere. And Max is our biggest and best-sounding Google Home device, powered by the Assistant. And with AI-based Smart Sound, Max has the ability to adapt your audio experience to you—your environment, context, and preferences.
With Pixelbook, we’ve reimagined the laptop as a high-performance Chromebook, with a versatile form factor that works the way you do. It’s the first laptop with the Assistant built in, and the Pixelbook Pen makes the whole experience even smarter.
Our new Pixel Buds combine Google smarts and the best digital sound. You’ll get elegant touch controls that put the Assistant just a tap away, and they’ll even help you communicate in a different language.
The updated Daydream View is the best mobile virtual reality (VR) headset on the market, and the simplest, most comfortable VR experience.
Google Clips is a totally new way to capture genuine, spontaneous moments—all powered by machine learning and AI. This tiny camera seamlessly sends clips to your phone, and even edits and curates them for you.

Assistant, everywhere

Source: Google Chrome

Pixel 2 and Daydream View: new experiences in AR and VR

Virtual and augmented reality have the potential to make computing more immersive, intuitive and powerful. With augmented reality (AR), you can interact with digital objects that are integrated seamlessly in the real world around you. With virtual reality (VR), you can experience what it’s like to go anywhere. We just announced the new Pixel 2, which gives you access to amazing smartphone AR and high-quality mobile VR in the same device.

AR: Have fun and get things done

Aside from taking great photos and videos, the Pixel 2 camera is factory calibrated and optimized for AR. It enables robust tracking, even in low-light conditions, and 60 frames per second rendering of AR objects. That means you’ll be able to have really engaging AR experiences.

To start, we’re putting AR objects and characters at your fingertips in a new experience called AR Stickers. With it, you’ll be able to express yourself with playful emojis and set the scene for your very own stories. If you’re powering through an afternoon slump, you can take a picture with a tired cup of coffee from the Foodmoji pack. Or, if you’re feeling festive, shoot a happy birthday video for your friend with lots of AR balloons. These stickers give you new ways to be creative and capture how you’re feeling, so you can share with the people in your life. It works right in the camera, and it’ll be exclusive to the Pixel.

We’ll release more new AR Sticker packs in the future, so keep your eyes peeled around the changing seasons, holidays and big pop culture moments. You’ll be able to interact with your favorite characters from “Stranger Things,” and be part of the scene as Eleven faces off against the Demogorgon. Team up with R2-D2 to save the day with the "Star Wars: The Last Jedi" pack. Hang out with your favorite NBA Stars and the cast of "Saturday Night Live."

We’re previewing apps and experiences built by our partners with ARCore, the AR software development kit (SDK) for Android. With Houzz, you’ll be able to bring the showroom into your living room and see how different furniture, accent pieces and more fit into your space. Experience the real-time action of a League of Legends match through an AR map, built by Grab Games. And with LEGO, have fun building virtual models in AR with characters that come to life, without fear of stepping on that last 2x4 brick.

AR Stickers and lots of other AR experiences will be available in the coming months.

VR: More immersive with an all-new Daydream View

We’re also announcing a new Google Daydream View headset, which you can pair with Pixel 2 or another Daydream-ready phone for great VR.

With this new headset, we kept the best parts of the original and made them even better. The new Daydream View has high-performance lenses, which result in better image clarity and a wider field of view. It sports a premium two-tone fabric that makes the headset soft and light, and it comes in three new colors: Fog, Charcoal and Coral.

There’s so much to experience on Daydream. Last year, we launched with 25 apps and games. Daydream now has more than 250 titles, so you’ll never run out of things to do. Over the last year, we’ve also seen how much people love watching immersive VR video on apps like YouTube VR. And no wonder: great VR video puts you in the center of the action. It’s a totally different experience from watching on a flat screen.

We’re bringing you more of these amazing, only-in-VR moments with a slate of new and exclusive premium content for Daydream. Get closer than front row seats with intimate performances from your favorite stars like Ed Sheeran in “Austin City Limits Backstage.” Hear personal confessions from huge comics like Trevor Noah in “The Confessional,” a YouTube VR original from Felix & Paul. With “The Female Planet,” follow in the footsteps of inspiring women like Gina Rodriguez and Inna Braverman. Venture to all seven continents with the new series “Discovery TRVLR,” and blast off into space with IMAX 3D in Google Play Movies.

Learn more and buy

With Google Pixel 2, you’ll have easy access to both AR and VR. We can’t wait for you to get your hands on it and start exploring. Learn more and pre-order one at the Google Store.

The new Google Daydream View will be available later this year:

United States: Google Store, Verizon, Best Buy, AT&T, Amazon
Canada: Google Store, Best Buy, Rogers, Bell, Telus, Freedom, Videotron, Glentel
United Kingdom: Google Store, EE, Carphone Warehouse
Germany: Google Store, Deutsche Telekom, Vodafone, MediaMarkt, Saturn
Australia: Google Store, Telstra, JB Hi-Fi
India: Flipkart
Italy: Google Store
France: Google Store
Spain: Google Store
Japan: Google Store
Korea: Google Store

From now until the end of the year, your purchase of Daydream View will come with a bundle of top games to help get you started. (U.S., U.K. and Korea only).

Source: The Official Google Blog

Pixel 2 and Daydream View: new experiences in AR and VR

AR: Have fun and get things done

AR Stickers and lots of other AR experiences will be available in the coming months.

VR: More immersive with an all-new Daydream View

We’re also announcing a new Google Daydream View headset, which you can pair with Pixel 2 or another Daydream-ready phone for great VR.

Learn more and buy

With Google Pixel 2, you’ll have easy access to both AR and VR. We can’t wait for you to get your hands on it and start exploring. Learn more and pre-order one at the Google Store.

The new Google Daydream View will be available later this year:

United States: Google Store, Verizon, Best Buy, AT&T, Amazon
Canada: Google Store, Best Buy, Rogers, Bell, Telus, Freedom, Videotron, Glentel
United Kingdom: Google Store, EE, Carphone Warehouse
Germany: Google Store, Deutsche Telekom, Vodafone, MediaMarkt, Saturn
Australia: Google Store, Telstra, JB Hi-Fi
India: Flipkart
Italy: Google Store
France: Google Store
Spain: Google Store
Japan: Google Store
Korea: Google Store

From now until the end of the year, your purchase of Daydream View will come with a bundle of top games to help get you started. (U.S., U.K. and Korea only).

Source: The Official Google Blog

Daydream Labs: Interactive scenes with Blocks objects

Since the launch of Blocks, people have been enthusiastically creating and sharing their amazing models with the community. So we asked ourselves: what would it be like to use Blocks objects to create an entire interactive scene?

Turns out it’s possible. In an experiment our team built recently, we created a system that lets people make their own "Escape the Room" experience in VR. Every object in the game is made from Blocks objects, including typical stuff like a flashlight, desk, bookcase, and the obligatory keypad, but also even the room itself.

Throw in some lighting, and the result is a scene with exactly the cartoonishly spooky vibe we were going for. Not a room you'd want to be trapped in for too long!

To get everything to work, we had to define how objects interact. We could’ve just written that directly in our code, but our goal was to allow anybody to create these experiences—no programming knowledge required. So we created a simple system of triggers and actions that allows the creator to indicate what happens next in response to certain events.

The system can express concepts such as "when the battery object collides with the flashlight object, activate the light object." The light happens to be a spotlight located at the tip of the flashlight object, so when the player places the battery in the right place, a cone of light will shine forward and move with the flashlight.

Using this simple trigger/action system, we built a number of other puzzles in the room, like opening a locked chest with a key, placing a book in a sliding bookcase and figuring out the combination to enter on a keypad.

Combining Blocks objects to create interactive scenes was a lot of fun. Because Blocks has a consistent low-poly visual style, the result of our efforts was an engaging environment where everything fit well together, even though objects were made by many different people on our team.

We learned a few other things along the way. First, the ability to add interactivity to a scene is super important, and a wide range of interactive scenes can be built from the simple primitives we had set up with our trigger and action system. Most of the interactions could be expressed as collisions (key and lock, battery and flashlight, book and bookcase) and simple actions like showing/hiding or animating particular objects.

Next, setting up the rendering was almost no work at all, because Blocks objects are low-poly and work well with simple materials. We just used the standard diffuse shaders for the opaque surfaces and a simple translucent one for the glass surfaces. Combining that with an ambient light and a spotlight achieved the rendering effect that we wanted.

Last, we set up a simple animation system where we pre-recorded the motions of certain objects and expressed them as a sequence of transformations (position, rotation, scale). This rudimentary animation system worked well when moving solid objects like a bookcase or the lid of a chest, but we’d need something more elaborate if we were to do character animation, perhaps using what we learned from our experiments on animating Blocks models. What’s more, adjusting the colliders for the objects to ensure they interacted correctly required some manual tweaks. In order to scale this, it might be worth looking into automatically generating simple colliders for objects.

Scene building and interactivity with Blocks objects are exciting areas for experimentation, and we're looking forward to seeing what other applications developers will come up with in this space.