Developing 3D apps is complicated—whether you’re using a native graphics API or enlisting the help of your favorite game engine, there are thousands of graphics commands that have to come together perfectly to produce beautiful 3D visuals on your phone, desktop or VR headsets.

To help developers diagnose rendering and performance issues with their Android and desktop applications, we’re releasing a new tool called GAPID (Graphics API Debugger). With GAPID, you can capture a trace of your application and step through each graphics command one-by-one. This lets you visualize how your final image is built and isolate calls with issues, so you spend less time debugging through trial and error until you find the source of the problem.

The goal of GAPID is to help you save time and get the most out of your GPU. To get started with GAPID, download it, take your favorite application, and capture a trace!

Over the past few years, many people have experienced virtual reality with headsets like Cardboard, Daydream View, and higher-end PC units like Oculus Rift and HTC Vive. Now, augmented reality has the potential to reach people right on their mobile devices. AR can bring information to you, and that digital information can enhance the experience you have with their physical space. However, AR is new, so creators need to think carefully when it comes to designing intuitive user interactions.

From our own explorations, we’ve learned a few things about design patterns that may be useful for creators as they consider mobile AR platforms. For this post, we revisited our learnings from designing for head-mounted displays, mobile virtual reality experiences, and depth-sensing augmented reality applications. First-party apps such as Google Earth VR and Tilt Brush allow users to explore and create with two positionally-tracked controllers. Daydream helped us understand the opportunities and constraints for designing immersive experiences for mobile. Mobile AR introduces a new set of interaction challenges. Our explorations show how we’ve attempted to adapt emerging patterns to address different physical environments and the need to hold the phone throughout an entire application session.

Key design considerations

Mobile constraints. Achieving immersive interactions is possible through a combination of the device's camera, real-world coordinates for digital objects, and input methods of screen-touch and proximity. Since mobile AR experiences typically require at least one hand to hold the phone at all times, it's important for interactions to be discoverable, intuitive, and easy to achieve with one or no hands. The mobile device is the user’s window into the augmented world, so creators must also consider ways to make their mobile AR experiences enjoyable and usable for varying screen sizes and orientations.

Mobile mental models and dimension-shifts. Content creators should keep in mind existing mental models of mobile AR users. 2D UI patterns, when locked to the user’s mobile screen, tend to lead to a more sedentary application experience; however, developers and designers can get creative with world-locked UI or other interaction patterns that encourage movement throughout the physical space in order to guide users toward a deeper and richer experience. The latter approach tends to be a more natural way to get users to learn and adapt to the 3D nature of their application session and more quickly begin to appreciate the value a mobile AR experience has to offer — such as observing augmented objects from many different angles.

Environmental considerations. Each application has a dedicated "experience space," which is a combination of the physical space and range of motion the experience requires. Combined with ARCore's ability to detect varying plane sizes or overlapping planes at different elevations, this opens the door to unique volumetric responsive design opportunities that allow creators to determine how digital objects should react or scale to the constraints of the user's mobile play space. Visual cues like instructional text or character animations can direct users to move around their physical spaces in order to reinforce the context switch to AR and encourage proper environment scanning.

Visual affordances. Advanced screen display and lighting technology makes it possible for digitally rendered objects to appear naturally in the user’s environment. Volumetric UI patterns  can complement a 3D mobile AR experience, but it’s still important that they stand out as interactive components so users get a sense of selection state and functionality. In addition to helping users interact with virtual objects in their environment, it’s important to communicate the planes that the mobile device detects in order to manage the users’ expectations for where digital items can be placed.

Mobile AR 2D interactions. With mobile AR, we’ve seen applications of a 2D screen-locked UI which gives users a “magic-hand” pattern to engage with the virtual world via touch inputs. The ability to interact with objects from a distance can be very empowering for users. However, because of 2D UI patterns' previous association with movement-agnostic experiences, users are less likely to move around. If physical movement is a desired form of interaction, mobile AR creators can consider ways to more immediately use plane detection, digital object depth, and phone-position to motivate exploration of a volumetric space. But be wary of too much 2D UI, as it can break immersion and disconnect the user from the AR experience.

Mobile AR immersive interactions. To achieve immersion, we focused on core mobile AR interaction mechanics ranging from object interaction, browsing, information display, and visual guidance. It's possible to optimize for readability, usability, and scale by considering ways to use a fixed position or dynamic scaling for digital objects. Using a reticle or raycast from the device is one way to understand intent and focus, and designers and developers may find it appropriate to have digital elements scale or react based on where the camera is pointing. Having characters react with an awareness to how close the user is, or revealing more information about an object as a user approaches, are a couple great examples of how creators can use proximity cues to reward exploration and encourage interaction via movement.

What’s next?

These are some early considerations for designers. Our team will be publishing guidelines for mobile AR design soon. There are so many unique problems that mobile AR can solve and so many delightful experiences it can unlock. We’re looking forward to seeing what users find compelling and sharing what we learn along the way, too. In the meantime, continue making and breaking things!

Images in this post by Chris Chamberlain

With cold weather outside and the holidays fast approaching, now’s the perfect time to take a moment and kick back with Daydream. We’re launching an update to make it easier to discover great content both in and out of VR, plus loads of new VR experiences to keep you entertained.

Find great VR content easily

With the latest Daydream app, you can now see what’s new and trending on the Google Play store in VR. And when you’re on the go without your headset, you can still search for apps and games, or explore content categories that refresh weekly right in the app on your phone.

Watch immersive VR videos

There's more and more VR content available all the time. From music and comedy to travel, check out some of our recent favorites.

Go behind the scenes with your favorite musicians

• Why Don't We: Step inside the house of Why Don't We in VR180. Take a behind-the-scenes tour with band members as they walk you through their house and talk about the milestones and memories made in their house, including songs recorded, deals signed and videos shot.

• Austin City Limits: Backstage - Episodes 1 to 3: Venture backstage at Austin’s legendary Moody Theater to hang with your favorite artists like Ed Sheeran, Zac Brown Band, and Unsung Heroes.
  

• The Confessional - Episode 4: Hear from Judd Apatow in the latest episode of “The Confessional,” a series where today’s biggest comedy stars share intimate (and funny) stories from their lives in a virtual confessional booth.

• Discovery TRVLR - Continents 1 to 4: Travel around the world—starting with Auckland, Hanoi, Mexico City and Yerevan—and connect with personalities and adventurers across the globe.
  

• Female Planet - Episode 2: Discover the world of Inna Braverman, co-founder of Eco Wave Power, a company that converts wave power into usable energy.

• Jump Start: See what happens when filmmakers get Jump cameras to pursue their creative projects. So far, these filmmakers have launched VR content covering a wide range of topics—everything from exotic animals in LA and archeological site visits to dance moves.
  

• YouTube Creator VR Lab: Watch the newest creations from up and coming VR filmmakers: take a lap around the track with Toyota factory driver Ken Gushi, imagine what Los Angeles will look like in the future, or follow the life of a college student who’s also an undocumented immigrant.
  
  

Explore new apps—and get favorites on sale

Developers have been busy building some awesome new experiences for Daydream. Here are some of our favorites:

A timeless classic now in VR

• Rez Infinite: Blast through waves of enemies and giant transforming bosses, with colors and sounds that sync and blend to the beat of the app’s techno soundtrack.
  

Coming soon to Daydream

• ASTEROIDS!: Following the initial teaser, the highly-anticipated full-length film will be launching next week. From the director of Madagascar comes the follow-up episode to the Emmy award-winning film INVASION!. The film follows the journey of quirky aliens Mac and Cheez (voiced by Elizabeth Banks), and their sidekick Peas as they traverse the challenges of space.
  
• Flutter VR: Explore the beauty of the Amazonian rainforest as you discover real butterfly species and see the environment change from day to night and rain to sunshine as you meet and discover new creatures. 
  

Some original VR favorites are now 50% off for a limited time, too (discounts valid 12/7/17 to 12/14/17). Explore imaginary worlds with Beartopia and So Let Us Melt, embark on new adventures with Hunters Gate and Spark of Light, and hone your reflexes with Ultrawings, Drift, The Arcslinger, and Fishing Star VR.

We have lots more new features and exciting experiences in the works for 2018. If you’re new to Daydream, or you haven’t tried it in a while, now’s a great time to jump in. And remember, you can always bring your friends and family along on your VR adventures by casting what you see to a TV.

Today we're making it even easier for developers to find and use 3D objects and scenes for their VR and AR apps with the Poly API.

Poly lets creators and developers browse, find, and download 3D objects and scenes for use in their apps. It’s fully integrated with Blocks and Tilt Brush, and even allows you to upload your own models, so there are plenty of options to choose from.

We want to make the process of finding the right 3D assets for your projects faster and more flexible. With the new Poly API, you can access our growing collection of Creative Commons 3D assets and interact directly with Poly to search, download, and import objects dynamically across desktop, mobile, virtual reality, and augmented reality.

If you’re using Unity or Unreal Engine to develop your apps, we also created the Poly Toolkit, an evolution of Tilt Brush Toolkit. With it, you can import 3D objects and scenes from Poly directly into a project, thanks to the API.

And with samples for both ARCore and ARKit, our developer site provides you with everything you need to use Poly assets in your AR experiences.

To put the Poly API and Toolkit to the test, we partnered with a few talented developers to show just how compelling their apps can become with a Poly API integration. Check out how Mindshow, TheWaveVR, Unity EditorXR, and many others have already integrated with the API:

Starting today, you can find all types of assets for your applications, and easily search for remixable, free assets licensed under a Creative Commons license by keyword, category, format, popularity or date uploaded. You can even filter by model complexity, or give people a personalized experience by letting them sign into your app with their Google account to access any assets they’ve uploaded or liked on Poly.

Ready to get started? Visit our developer page to see instructions on how to use the API and download our sample apps and toolkits.

Grab a Daydream headset and get ready to jump into the cockpit of one of the world’s fastest motor sports in virtual reality: The Red Bull Air Race World Championship, a time trial using planes to race a low-level slalom track marked by 25-meter-high, air-filled pylons. It’s fast, exciting, and requires nerves of steel to win. The lightweight race planes hit speeds up to 370 kph while enduring forces of up to 10G. This combination of high speed, low altitude and extreme maneuverability makes the sport accessible only to the world's most exceptional pilots. But with Daydream, Google’s platform for high-quality, mobile VR, you can now get right into the air-racing action—both live and on demand.

The Red Bull Air Race LIVE VR app uses real-time flight data directly from the pilot’s cockpit and re-creates the races in a 360° virtual reality experience. You can check out races that have already taken place, or join the action live during the upcoming Red Bull Air Race World Championship starting with Abu Dhabi on February 2, 2018.

The VR experience is powered by telemetry data like latitude, longitude, roll, pitch and airspeed to generate breathtaking perspectives and a live visualization of the pilots’ view onto an environment guided by Google Maps footage. You can also track the speed of the pilot and how much G Force they’re pulling.

To develop the experience, we worked with Red Bull Air Race to capture data from more than 40 flights at the 2017 World Championship season finale at the legendary Indianapolis racetrack in the US. At a dedicated Red Bull Air Race LIVE VR booth, more than 1,000 fans were among the first to virtually hit the skies. One of the highlights was the outstanding flight of Yoshihide Muroya from Japan, who set the track record in his final run on his way to being crowned World Champion 2017.

If you want to join Yoshihide and the other pilots in the cockpit, just download the “Red Bull Air Race LIVE VR App” in the Google Play Store, drop your Pixel or any other Daydream-ready phone into the headset and get started. See you at the 2018 World Championship.

It’s one thing to read a news report about polar ice caps melting, but it’s another to hear the wind whipping against a towering glacier as you watch huge chunks of ice break off and tumble into the water below. With VR video, it’s possible to experience things that are rare or places that are far away or even impossible to get to. And short of actually being there, there’s no better way to understand the great challenges and opportunities of our world. Virtual reality can help people learn more and feel a part of important but distant problems.

Awareness is the first step toward driving social change, but organizations and changemakers often lack the resources or knowledge to use VR to shine a light on the causes they care about. That’s why we’re launching Daydream Impact to help organizations, creators, and changemakers make their programs even more powerful using virtual reality.

Daydream Impact focuses on three common bottlenecks we have identified: a lack of training on how to create VR video, difficulties accessing camera equipment and tools to showcase their content, and little exposure to how VR has been used creatively to tackle big challenges.

To help provide changemakers with training, we’ve created a VR filmmaking course on Coursera, which anyone can take. The course begins by outlining basic hardware requirements and pre-production checklists, and it shares tips for getting the best VR footage including best practices from other creators. The training also covers all the post-production work required to create the video and concludes with guidance on how to publish and promote the video.

We've been working with several organizations to help shape and refine this program, and they've already created VR content and programs that’s helping them advocate for their causes:

• Harmony Labs partnered with media makers, researchers, and other experts to create three VR anti-bullying pieces and pilot these experiences in schools, ultimately creating a toolkit to highlight learnings about using VR for social impact.

• Springbok Cares worked with Daydream Impact to study how VR can reduce anxiety for cancer patients during treatment, and how to integrate large-scale VR programs into hospital environments. At the same time, they also launched a program to provide virtual reality entertainment to hospital patients and staff.

• Eastern Congo Initiative partnered with Daydream Impact to create a VR film and Expeditions that explore the struggles in the Congo and the resilience of its people. Through these platforms, ECI offers policymakers, donors and students an immersive experience and an emotional connection to the cause. By doing so, ECI believes that these partnerships, education efforts and advocacy will inform and inspire change for the Congolese people.

• The Rising Seas project and oceanographer Juliette Finzi Hart are leveraging 360 videos, depth maps and VR simulations to let people experience their future coastlines now, believing that allowing people to see the future they want to be will inspire them to take action—today.

We’ll share more on upcoming projects and case studies in 2018 with World Wildlife Fund & Condition One, UNAIDS, the International Committee of the Red Cross, Starlight Children’s Foundation, Protect our Winters, and Novo Media.

It’s our hope that Daydream Impact will help organizations tell their stories more effectively, or even change the way they operate, using VR. In a world full of information, virtual reality can help advocates inspire, connect, and bring change.

Earlier this year, 3D modeler Jarlan Perez joined the Blocks team for a two-week sprint. The goal of his time with the team was to create a fully immersive virtual reality game in just two weeks using Blocks and Unreal Engine, two tools that have significantly influenced his process as a modeler and game enthusiast.

The result was “Blocks Isle,” the first level of a game that takes you on a journey to find your long lost friend in a sci-fi land of wonder. To win, you must solve a puzzle using hidden clues and interactions throughout the experience.

You start out on a strange desert island. After uncovering some clues and pulling a handy lever, a rocky pathway opens for exploration. Up ahead, hidden radios and books reveal clues to solve the puzzle.

We caught up with Jarlan to hear more about his process and advice for other developers building immersive experiences using Blocks and Unreal Engine 4.

Brittany: Tell us about using Blocks and Unreal to develop a game in such a short amount of time.

Jarlan: Tag teaming both pieces of software worked very well! Blocks allowed me to visualize and be in the space during the modeling and conceptual phase. Unreal is like giving an artist magical powers: I’m able to fully build a proof of concept and implement functionality without having to be a professional programmer.

I found myself spending part of the day in Blocks experimenting with concepts and the rest in Unreal creating basic functionality for those ideas. This method allowed for rapid prototyping and was later beneficial when populating the space with art assets.

What tips and tricks did you uncover that made it easy to build your game?

Another general tip, the snap trigger is your friend! I’ve used it for most of my modeling in Blocks to snap and place assets.

How did you experiment with different ideas and concepts?

Without Blocks, how might this process have been different?

After all is said and done, I still had to take the geometry from Blocks and bring it into a 3D program for unwrapping and lightmap baking.

That said, even though I am proficient in traditional 3D modeling, I think the project would have taken longer to put together without Blocks. Blocks helped me take out some steps in the process. Traditionally I’d model out the scene and export pieces as I went, bringing them into the engine, placing them, and moving around to get a sense of how the space feels. All that got combined inside Blocks. Oh, and not to mention color exploration. If I wanted to try out colors I’d also have to create materials and place them on each asset during the in-engine test which takes more time. I can easily preview all of that in Blocks.

What advice would you give to other game developers about using these tools?

Keep exploring and always stay hungry. Be on the lookout for new tools that can improve your process and don’t be afraid of trying something new. If it doesn’t work out, it’s ok. We learn so much more from the challenges we take on than from the ones we don’t face by walking the easy path.

There are some amazing low poly games and artists out there. I think many artists would benefit from making models in VR using Blocks. If I was able to finish this project in two weeks, I can only imagine what a small team could do. Give it a try, and post your creations or questions using #MadeWithBlocks.

If you’d like to experience Blocks Isle on the HTC Vive, you can download the game.