Tag Archives: Google AR and VR

Go behind the scenes with Austin City Limits: Backstage

Austin City Limits” needs little introduction. It’s the longest-running television music program in history, it’s helped launch the careers of iconic musicians like Willie Nelson (featured in the very first episode back in 1974), and it’s even enshrined in the Rock & Roll Hall of Fame. But, for all its history, the closest you can get is either in the crowd, or in front of your TV screen. We wanted to go further, and pay tribute to this legendary show’s 43rd season and its impact on pop culture. So we’re releasing a new virtual reality video series called “Austin City Limits: Backstage” in partnership with SubVRsive Media.

ACLZBB

“ACL Backstage” lets you explore the untold stories of the crew, the city, the fans and, of course, the musicians who make Austin City Limits possible—all in virtual reality. Venture backstage at Austin’s legendary Moody Theater to hear stories from some of your favorite artists. Then, watch and listen up close as they take the stage and play their hits under the bright lights. After that, you can take a whirlwind tour through the city’s thriving local music scene, where you’ll hear up-and-coming stars who might make it big one day.

ACLHero

ACL Backstage will have 10 episodes, each featuring a different artist or group.The first three are available now, with more coming soon:

  • Ed​ ​Sheeran” This is Ed Sheeran’s second ACL Live performance, and since he last took the stage in 2014, his career has skyrocketed. Now, with multiple Grammy wins and three platinum records under his belt, he reflects on his rise to the top of the charts. His passion for the music and his fans shine through in this episode.

  • Zac​ ​Brown​ ​Band​” Three-time Grammy Award-winning multi-platinum artists Zac Brown Band make a stop on their 2017 Welcome Home Tour to grace the ACL stage for the very first time. Sit backstage with the band as they chat about ACL’s rich history, and join them onstage for their lively show.

  • Unsung Heroes” Hear ACL stories directly from crew members, many of whom have been working the show for decades. They explain the ethos of Austin City Limits and why it remains so popular among musicians.

Use your Cardboard or Google Daydream View to check out all the videos on the ACL YouTube Channel. Kick back, hang with your favorite artists, and rock out.

Maximizing art assets for VR

In line with previous tips we shared about creating art assets for VR, we’ve put together a few specific techniques for VR artists to get the most out of polygons and UV space.


Tiling textures: Instead of making one giant texture that covers lots of space, create a smaller tileable texture that can be used in more spaces. The example below uses UV space with a tileable texture repeated eight times to create high resolution on this ladder.  Through mirroring and flipping, the user will likely not notice the reused UV spaces, but the overall look is higher-res.
vrart2ladder

Use more polygons: While keeping the poly count low is important, texture and pixel resolution is more valuable. Plan ahead when designing an asset to account for the fact that you may need to add more polygons or model an asset in a less polygonally-optimized way, because doing so will increase resolution and result in a more detailed and overall optimized asset. On the left is a 13k poly model which was intended to be 6” tall in VR.  When the user approached the model, it became pixelated and didn’t look good. Instead of using large sheets of UV space, the model on the right was built with more careful polygon placement. It’s a 9k poly with a higher resolution of textures that holds up to close inspection in VR.

vrships

Maximize that UV space: Strive for higher resolution by making larger UV spaces to achieve a higher resolution per asset.  Because almost every asset has a unique UV space, the image on the left uses the large green leaf area for the entire balloon, which looks pixelated at close distances in VR. By making larger and higher resolution UV spaces and duplicating the geometry instead of relying on unique UV sections, the final asset on the right looks sharper when viewed up close.

shiptexture

Use geometry to make something unique, not textures: One drawback with reusing UV space is that things can look repetitive. To make areas of your asset look more unique, use geometry to add a new color splash or add a few edge loops to bend or scale a duplicated asset into a different silhouette.

geocompare

Plan ahead when creating assets, so they can be reused and combined with other assets: A good workflow in the tight constraints of VR is to build the individual parts of a final asset and then assemble them. Always be considering whether or not that next unique part is needed at the cost of an overall smaller UV layout, thus less lower resolution per asset.  The entire final asset on the left was built with the parts on the right.

shipparts

Hopefully, these tips will be useful as you create and optimize your assets for VR. What are some of your favorite tips and tricks? Let’s get a conversation going; use #VRArtTips to share.

Resonance Audio: Multi-platform spatial audio at scale

As humans, we rely on sound to guide us through our environment, help us communicate with others and connect us with what’s happening around us. Whether walking along a busy city street or attending a packed music concert, we’re able to hear hundreds of sounds coming from different directions. So when it comes to AR, VR, games and 360 video, you need rich sound to create an engaging immersive experience that makes you feel like you’re really there. Today, we’re releasing a new spatial audio software development kit (SDK) called Resonance Audio. It’s based on technology from Google’s VR Audio SDK, and it works at scale across mobile and desktop platforms.

Resonance Audio

Experience spatial audio in our Audio Factory VR app for Daydream and SteamVR

Performance that scales on mobile and desktop

Bringing rich, dynamic audio environments into your VR, AR, gaming, or video experiences without affecting performance can be challenging. There are often few CPU resources allocated for audio, especially on mobile, which can limit the number of simultaneous high-fidelity 3D sound sources for complex environments. The SDK uses highly optimized digital signal processing algorithms based on higher order Ambisonics to spatialize hundreds of simultaneous 3D sound sources, without compromising audio quality, even on mobile. We’re also introducing a new feature in Unity for precomputing highly realistic reverb effects that accurately match the acoustic properties of the environment, reducing CPU usage significantly during playback.

Geometric Reverb
Using geometry-based reverb by assigning acoustic materials to a cathedral in Unity

Multi-platform support for developers and sound designers

We know how important it is that audio solutions integrate seamlessly with your preferred audio middleware and sound design tools. With Resonance Audio, we’ve released cross-platform SDKs for the most popular game engines, audio engines, and digital audio workstations (DAW) to streamline workflows, so you can focus on creating more immersive audio. The SDKs run on Android, iOS, Windows, MacOS and Linux platforms and provide integrations for Unity, Unreal Engine, FMOD, Wwise and DAWs. We also provide native APIs for C/C++, Java, Objective-C and the web. This multi-platform support enables developers to implement sound designs once, and easily deploy their project with consistent sounding results across the top mobile and desktop platforms. Sound designers can save time by using our new DAW plugin for accurately monitoring spatial audio that’s destined for YouTube videos or apps developed with Resonance Audio SDKs. Web developers get the open source Resonance Audio Web SDK that works in the top web browsers by using the Web Audio API.
DAW Plugin
DAW plugin for sound designers to monitor audio destined for YouTube 360 videos or apps developed with the SDK

Model complex sound environments

By providing powerful tools for accurately modeling complex sound environments, Resonance Audio goes beyond basic 3D spatialization. The SDK enables developers to control the direction acoustic waves propagate from sound sources. For example, when standing behind a guitar player, it can sound quieter than when standing in front. And when facing the direction of the guitar, it can sound louder than when your back is turned.

AcousticGif
Controlling sound wave directivity for an acoustic guitar using the SDK

Another SDK feature is automatically rendering near-field effects when sound sources get close to a listener’s head, providing an accurate perception of distance, even when sources are close to the ear. The SDK also enables sound source spread, by specifying the width of the source, allowing sound to be simulated from a tiny point in space up to a wall of sound. We’ve also released an Ambisonic recording tool to spatially capture your sound design directly within Unity, save it to a file, and use it anywhere Ambisionic soundfield playback is supported, from game engines to YouTube videos.

If you’re interested in creating rich, immersive soundscapes using cutting-edge spatial audio technology, check out the Resonance Audio documentation on our developer site. You can also experience spatial audio in our Audio Factory VR app for Daydream and SteamVR. Let us know what you think through GitHub, and show us what you build with #ResonanceAudio on social media; we’ll be resharing our favorites. 

Get a new perspective on the world with Discovery TRVLR

One of the richest ways to explore and learn about a place is to get to know it from someone who calls that place home. What if you could experience the subtle mysticism of standing in a pagoda in Vietnam watching warrior monks train, or hear the echoes of ancient Aztec gods in the rhythmic stomping of feet during a traditional Mexican cleansing ritual?

Virtual reality makes this possible, so you can go on adventures that transcend the mundane—all without booking a single airplane ticket. That’s the idea behind a new series we created in partnership with Discovery, called Discovery TRVLR.

Discovery TRVLR

Discovery has been showing you new parts of the world for over 35 years, and with Discovery TRVLR, you’ll get to explore it in vivid VR video filmed on Jump cameras. Discovery TRVLR takes you on voyages to all seven continents, and by focusing on locals, you’ll uncover the hidden stories of these places and learn things you’d never see in a travel guidebook.

Available today, the first chapter takes you to Auckland, New Zealand—hold on tight as you descend into the depths of a seemingly bottomless cave. New chapters will follow every week. In Hanoi, keep your eyes peeled for slumbering demons as you explore a ghostly forest. In Mexico City, venture south of the capital to the haunting Isla de las Muñecas (Island of the Dolls). In Yerevan, walk a tightrope with the last member of a generation of Armenian acrobats. In Cape Town, explore the underside of the city with a charming, though not strictly licensed, cab operator. And in La Paz, meet female wrestlers giving hope to domestic violence victims. Then cap it all off riding alongside a polar explorer through ice caves and the frozen tundra of Antarctica. 

Find all episodes of Discovery TRVLR on YouTube or the Discovery VR app on Daydream, and watch them with your Cardboard or Daydream View. Here’s hoping you catch your own glimpse of something truly special.

The She Word: Tory Voight’s climb through her career

Editor’s note: Last week we hosted a Women who VRock panel at the Google Pop Up space in Los Angeles, bringing together women across the VR industry. Tory Voight, an engineering program manager on the Google AR/VR team (and oil painter and rock climber on the side), moderated the discussion. Today shares her own perspective for the She Word, our Keyword series focused on amazing women at Google.

Panel2.png
Women who VRock

How do you explain your job at a dinner party?

Blog_HeadShot_ToryVoight.jpg

I work closely with engineers and creators to dream up new ideas and ways to use VR. I spend most of my time building the Artist in Residency (AiR) program, which engages creators to use our products in new ways and provide valuable product feedback to push VR forward.  

What advice would you give to women starting out in their careers?

If you’re interested in something, don’t hesitate to reach out. Many great relationships, lessons, and even job positions have resulted from doing just that. Back in 2015, I contacted the VR team and offered to lend 20 percent of my time to help out with Cardboard. I did that for a few months, and eventually got a full time job on the team. If I had never sent that first email, I wouldn’t have ended up where I am now. Offering to help with projects helps you understand the field you want to be in—and what to expect.

Who has been a strong female influence in your life?

From an early age, it was my mother, a single parent of five children who worked two or even three jobs to support our family. I learned the value of perseverance through the adversity we faced together, and through all the wonderful women I met in our various jobs—from cleaning houses, to working in a truck stop Wendy’s. They inspire me to give back in my career as a purpose-driven individual, and to work for a company like Google that builds tools to democratize experiences and opportunities, regardless of one’s background. That’s why I’m so excited about VR products we’re building—they give people access to experiences.

You've mentioned that giving back is important. How do you give back to the community?

IMG_2567.JPG
Tory and Sookie at the top. Red Rock Canyon National Conservation Area, NV.

As a rock climber, I love volunteering for the Yosemite Climbers Association’s “Facelift” program. I help pick up trash around Yosemite Valley (volunteers have collected more than 10 tons of trash over the past 14 years!), and I feel like this small contribution makes the park more enjoyable for everyone.


I also spend time mentoring young women in high school and college. When I was their age, I had no idea what I wanted to be when I grew up—higher education and a career seemed like an unobtainable, abstract idea for a kid from my background. Now I want to take my experience and help them understand what to expect and encourage them, despite any perceived odds or barriers. I give back in honor of all the individuals who pushed me think I could do anything, even if it seemed insurmountable at the time.

What’s an experience you’ve had in VR that really had an impact on you?

This is Bears Ears National Park” opened my eyes to how VR can be used to inform and build empathy over an issue. The park is stunningly beautiful, but continues to face a lot of political and environmental threats. And because I love being outdoors, this content had a particularly strong impact on me. I’m proud that we’re lending a hand to Bears Ears through our Jump Start program, which pairs filmmakers with the resources to create their own stories in VR.

Related Article

The She Word: Tea Uglow, a "pebble in the landslide"

We’re talking with dynamic female Googlers about who they are, what they do and why they inspire us.

Read Article

What do you find most challenging about working in VR? 

The exciting and the challenging thing about VR is that it’s still in its technical infancy. We're in a new field where there aren’t necessarily answers. We have to find them and validate them, and we're learning all the time. That’s why programs like AiR, and taking user feedback to heart while we play with concepts, are important.

Why is it important to have a wide variety of people and artists explore VR as a medium?

For the past year, I’ve worked closely with artists from different disciplines and mediums—graffiti artists, painters, illustrators, graphic designers, and cartoonists—in the AiR program. When building products, a diverse set of voices is essential to ensuring that those products are delightful and useful for everyone—a successful product simply can’t be achieved from a homogeneous atmosphere.

Poly: Browse, discover and download 3D objects and scenes

If you’re building for virtual and augmented reality, you need 3D objects for your apps. With Tilt Brush and Blocks, we’ve made 3D creation easier. Now, we want to enable creators and developers to build on everyone’s work. That’s why we created Poly: one place to browse, discover and download 3D objects. 

Poly lets you quickly find 3D objects and scenes for use in your apps, and it was built from the ground up with AR and VR development in mind. It’s fully integrated with Tilt Brush and Blocks, and it also allows direct OBJ file upload, so there’s lots to discover and use. Whether you’re creating an intense space walk in VR or a serene garden of AR flowers, you’ll find the ingredients you need in Poly.

Introducing Poly

Search thousands of free models for use in your AR or VR apps, including everything from a rocket ship to a synthesizer to an ice cream cone. Found a robot you like, but need it with four arms instead of two? You can remix many of the models you find. Click “like” to import a remixable object into Tilt Brush or Blocks and make changes; Poly will automatically credit and link to the original creation when you publish your remix.

PolyPlaces

In addition to being a great place for developers to find assets, anyone can use Poly to view 3D objects in a mobile or desktop browser. You can search for specific things, like a fox or a pizza. Once you discover something you like, you can create a shareable GIF or view it in VR using Cardboard or Daydream View.

So if you’re a developer building for VR, ARCore or ARKit, or you just want to explore cool 3D objects and scenes, Poly will help you quickly and easily find what you’re looking for. Get started today at poly.google.com, or sign up to preview our API when it’s available.

Swing into the World Series with Google Expeditions

Hey batter, batter, batter! It’s late October, and the World Series is shaping up for an amazing finish. Step up to the plate with five new Google Expeditions that we created in partnership with MLB.com.

These include virtual tours of Citi Field in New York and Oriole Park at Camden Yards in Baltimore, both of which are narrated by MLB Network’s Heidi Watney. You can also get behind-the-scenes access with career tours that showcase the lives of a baseball beat reporter and television broadcasters. We’re also bringing you a Statcast tour, so you can geek out Moneyball-style with the math and physics behind the game.

MLBEx

If you’re an educator, we’ve got all the bases covered. We worked with teachers to provide accompanying free lessons plans to go along with these tours. “MLB Statcast Exploration” and “Journalism, The Appeals, and MLB” can both be found at TES.com.

Check out these MLB tours and our catalogue of over 700 unique experiences on your own or in the classroom by downloading Expeditions on your iOS or Android devices.

Daydream Labs: Accessibility in VR

Virtual reality offers the ability to explore new worlds and have adventures without leaving home. We love the sense of freedom that VR offers, but it’s a technology that still relies mainly on visual cues—which makes it inaccessible to people with visual impairments. To bring these incredible experiences to visually impaired people, the technology needs to offer new tools. So we’ve been exploring how spatial audio cues can be used for navigating and interacting with virtual environments.

Compared to VR, other technologies offer more tools and help to visually-impaired people. One example is the alternative text found on website images, and another is Google TalkBack, which adds spoken, audible and haptic feedback to help visually-impaired people interact with their devices. With these technologies as inspiration, we created an audio tool aimed at making VR more accessible.

Accessibility in VR

Using an HTC Vive, we built a prototype of a 1:1 scale virtual room, recorded the name of every object in the room, and linked these audio labels to the individual objects—including the floor, walls and other features. Then, we made the user’s field of vision entirely black to simulate complete blindness. To enable navigation in the pitch-black room, we created a 3D audio laser system that includes a laser pointer extending from the Vive controller to select and play the audio labels, and an audio location control (touchpad click) to provide distance and direction to the last object aimed at by the laser pointer.

roomexvr

When a person aims the laser pointer at a virtual object and selects the audio location control, the VR system plays a short impulse response tone at location of the controller. Then the sound is played a few more times as it quickly progresses to the location of the virtual object. Because all audio is processed using the Google VR Spatial Audio plugin, each tone provides enough information to understand distance and relative location of the object in the virtual space.

To test our prototype, we challenged participants to find and pick up a toy laser gun within the virtual room, navigate to the window, and finally shoot at a duck moving outside the window. We ran six non-visually-impaired people through the prototype, and all of them were able to complete the challenge successfully. After the task was completed, four of them went through the experience again, this time able to see the room without vision impairment. Because they had navigated the room by sound, we found that they were already familiar with their surroundings.

DisPerVR

It’s a small step, but this experiment demonstrated that it’s possible to navigate and interact with a room in VR using only auditory cues. We hope others will also continue to explore ways to make VR accessible for everyone. There’s much more to do in this area!

You can find more details in our published technical disclosure.

Helping NASA and JPL bring the surface of Mars to your browser

On August 6, 2012, the Curiosity rover landed on Mars. Ever since, it’s been searching for evidence that Mars has ever been suitable for life. It’s also been photographing the Martian terrain in great detail. Scientists at NASA’s Jet Propulsion Lab use these photos to create a 3D model of Mars. It’s a one-of-a-kind scientific tool for planning future missions.


Today, we’re putting that same 3D model into an immersive experience for everyone to explore. We call it Access Mars, and it lets you see what the scientists see. Get a real look at Curiosity’s landing site and other mission sites like Pahrump Hills and Murray Buttes. Plus, JPL will continuously update the data so you can see where Curiosity has just been in the past few days or weeks. All along the way, JPL scientist Katie Stack Morgan will be your guide, explaining key points about the rover, the mission, and some of the early findings.


The experience is built using WebVR, a technology that lets you see virtual reality right in your browser, without installing any apps. You can try it on a virtual reality headset, phone, or laptop.


Check it out at g.co/accessmars.


And if you’re an educator, we’ve updated our Mars tour in Google Expeditions with highlights from this experience. To try it with your class or in self-guided mode, download the Expeditions app from Google Play or the App Store.

Source: Google Chrome


Helping NASA and JPL bring the surface of Mars to your browser

On August 6, 2012, the Curiosity rover landed on Mars. Ever since, it’s been searching for evidence that Mars has ever been suitable for life. It’s also been photographing the Martian terrain in great detail. Scientists at NASA’s Jet Propulsion Lab use these photos to create a 3D model of Mars. It’s a one-of-a-kind scientific tool for planning future missions.


Today, we’re putting that same 3D model into an immersive experience for everyone to explore. We call it Access Mars, and it lets you see what the scientists see. Get a real look at Curiosity’s landing site and other mission sites like Pahrump Hills and Murray Buttes. Plus, JPL will continuously update the data so you can see where Curiosity has just been in the past few days or weeks. All along the way, JPL scientist Katie Stack Morgan will be your guide, explaining key points about the rover, the mission, and some of the early findings.


The experience is built using WebVR, a technology that lets you see virtual reality right in your browser, without installing any apps. You can try it on a virtual reality headset, phone, or laptop.


Check it out at g.co/accessmars.


And if you’re an educator, we’ve updated our Mars tour in Google Expeditions with highlights from this experience. To try it with your class or in self-guided mode, download the Expeditions app from Google Play or the App Store.

Source: Education