Tag Archives: arcore

How We Made the CES 2024 AR Experience: Android Virtual Guide, powered by Geospatial Creator

Posted by Kira Rich – Senior Product Marketing Manager, AR and Bradford Lee – Product Marketing Manager, AR

Navigating a large-scale convention like CES can be overwhelming. To enhance the attendee experience, we've created a 360° event-scale augmented reality (AR) experience in our Google booth. Our friendly Android Bot served as a digital guide, providing:

  • Seamless wayfinding within our booth, letting you know about the must try demos
  • Delightful content, only possible with AR, like replacing the Las Vegas Convention Center facade with our Generative AI Wallpapers or designing an interactive version of Android on Sphere for those who missed it in real life
  • Helpful navigation tips and quick directions to transportation hubs (Monorail, shuttle buses)

In partnership with Left Field Labs and Adobe, we used Google’s latest AR technologies to inspire developers, creators, and brands on how to elevate the conference experience for attendees. Here’s a behind the scenes look at how we used Geospatial Creator, powered by ARCore and Photorealistic 3D Tiles from Google Maps Platform, to promote the power and usefulness of Google on Android.

Moving image showing end-to-end experience
Using Google’s Geospatial Creator, we helped attendees navigate CES with the Android Bot as a virtual guide, providing helpful and delightful immersive tips on what to experience in our Google Booth.


Tools We Used


Geospatial Creator in Adobe Aero Pre-Release

Geospatial Creator in Adobe Aero enables creators and developers to easily visualize where in the real-world they want to place their digital content, similar to how Google Earth visualizes the world. With Geospatial Creator, we were able to bring up Las Vegas Convention Center in Photorealistic 3D Tiles from Google Maps Platform and understand the surroundings of where the Google Booth would be placed. In this case, the booth did not exist in the Photorealistic 3D Tiles because it was a temporary build for the conference. However, by utilizing the 3D model of the booth and the coordinates of where it would be built, we were able to easily estimate and visualize the booth inside of Adobe Aero and build the experience around it seamlessly, including anchoring points for the digital content and the best attendee viewing points for the experience.


"At CES 2024, the Android AR experience, created in partnership with the Google AR, Android, and Adobe teams, brought smiles and excitement to attendees - ultimately that's what it's all about. The experience not only showcased the amazing potential of app-less AR with Geospatial Creator, but also demonstrated its practical applications in enhancing event navigation and engagement, all accessible with a simple QR scan." 
– Yann Caloghiris, Executive Creative Director at Left Field Labs
Moving image of developer timelapse
Adobe Aero provided us with an easy way to visualize and anchor the AR experience around the 3D model of the Google Booth at the Las Vegas Convention Center.

With Geospatial Creator, we had multiple advantages for designing the experience:

  • Rapid iteration with live previews of 3D assets and high fidelity visualization of the location with Photorealistic 3D Tiles from Google Maps Platform were crucial for building a location-based, AR experience without having to be there physically. 
  • Easy selection of the Las Vegas Convention Center and robust previews of the environment, as you would navigate in Google Earth, helped us visualize and develop the AR experience with precision and alignment to the real world location.

In addition, Google Street View imagery generated a panoramic skybox, which helped visualize the sight lines in Cinema 4D for storyboards. We also imported this and Photorealistic 3D Tiles from Google Maps Platform into Unreal Engine to visualize occlusion models at real world scale.

In Adobe Aero, we did the final assembly of all 3D assets and created all interactive behaviors in the experience. We also used it for animating simpler navigational elements, like the info panel assets in the booth.

AR development was primarily done with Geospatial Creator in Adobe Aero. Supplementary tools, including Unreal Engine and Autodesk Maya, were used to bring the experience to life.

Adobe Aero also supports Google Play Instant apps and App Clips1, which means attendees did not have to download an app to access the experience. They simply scanned a QR code at the booth and launched directly into the experience, which proved to be ideal for onboarding users and reducing friction especially at a busy event like CES.

Unreal Engine was used to bring in the Photorealistic 3D Tiles, allowing them to build the 3D animated Android Bot that really interacted closely with the surrounding environment. This approach was crucial for previews of the experience, allowing us to understand sight lines and where to best locate content for optimal viewing from the Google booth.

Autodesk Maya was used to create the Android Bot character, environmental masks, and additional 3D props for the different scenes in the experience. It was also used for authoring the final materials.

Babylon exporter was used for exporting from Autodesk Maya to glTF format for importing into Adobe Aero.

Figma was used for designing flat user interface elements that could be easily imported into Adobe Aero.

Cinema 4D was used for additional visualization and promotional shots, which helped with stakeholder alignment during the development of the experience.



Designing the experience

During the design phase, we envisioned the AR experience to have multiple interactions, so attendees could experience the delight of seeing precise and robust AR elements blended into the real world around them. In addition, they could experience the helpfulness of contextual information embedded into the real objects around them, providing the right information at the right time.

Image of Creative storyboard
To make the AR experience more engaging for attendees, we created several possibilities for people to interact with their environment (click to enlarge).

Creative storyboarding

Creating an effective storyboard for a Geospatial AR experience using Adobe Aero begins with a clear vision of how the digital overlays interact with the real-world locations.

Left Field Labs started by mapping out key geographical points at the Las Vegas Convention Center location where the Google booth was going to stand, integrating physical and digital elements along the way. Each scene sketched in the storyboard illustrated how virtual objects and real-world environments would interplay, ensuring that user interactions and movements felt natural and intuitive.


“Being able to pin content to a location that’s mapped by Google and use Photorealistic 3D Tiles in Google’s Geospatial Creator provided incredible freedom when choosing how the experience would move around the environment. It gave us the flexibility to create the best flow possible.” 
– Chris Wnuk, Technical Director at Left Field Labs

Early on in the storyboarding process, we decided that the virtual 3D Android Bot would act as the guide. Users could follow the Bot around the venue by turning around in 360°, but staying at the same vantage point. This allowed us to design the interactive experience and each element in it for the right perspective from where the user would be standing, and give them a full look around the Google Booth and surrounding Google experiences, like the Monorail or Sphere.

The storyboard not only depicted the AR elements but also considered user pathways, sightlines, and environmental factors like time of day, occlusion, and overall layout of the AR content around the Booth and surrounding environment.

We aimed to connect the attendees with engaging, helpful, and delightful content, helping them visually navigate Google Booth at CES.

User experience and interactivity

When designing for AR, we have learned that user interactivity and ensuring that the experience has both helpful and delightful elements are key. Across the experience, we added multiple interactions that allowed users to explore different demo stations in the Booth, get navigation via Google Maps for the Monorail and shuttles, and interact with the Android Bot directly.

The Android brand team and Left Field Labs created the Android character to be both simple and expressive, showing playfulness and contextual understanding of the environment to delight users while managing the strain on users’ devices. Taking an agile approach, the team iterated on a wide range of both Android and iOS mobile devices to ensure smooth performance across different smartphones, form factors such as foldables, as well as operating system versions, making the AR experience accessible and enjoyable to the widest audience.

testing content in Adobe Aero
With Geospatial Creator in Adobe Aero, we were able to ensure that 3D content would be accurate to specific locations throughout the development process.

Testing the experience

We consistently iterated on the interactive elements based on location testing. We performed two location tests: First, in the middle of the design phase, which helped us validate the performance of the Visual Positioning Service (VPS) at the Las Vegas Convention Center. Second, at the end of the design phase and a few days before CES, which further validated the placement of the 3D content and enabled us to refine any final adjustments once the Google booth structure was built on site.


“It was really nice to never worry about deploying. The tracking on physical objects and quickness of localization was some of the best I’ve seen!” 
– Devin Thompson, Associate Technical Director at Left Field Labs

Attendee Experience

When attendees came to the Google Booth, they saw a sign with the QR code to enter the AR experience. We positioned the sign at the best vantage point at the booth, ensuring that people had enough space around them to scan with their device and engage in the AR experience.

Sign with QR code to scan for entry at the Google Booth, Las Vegas Convention Center
By scanning a QR code, attendees entered directly into the experience and saw the virtual Android Bot pop up behind the Las Vegas Convention Center, guiding them through the full AR experience.

Attendees enjoyed seeing the Android Bot take over the Las Vegas Convention Center. Upon initializing the AR experience, the Bot revealed a Generative AI wallpaper scene right inside of a 3D view of the building, all while performing skateboarding tricks at the edge of the building’s facade.

Moving image of GenAI Wallpaper scene
With Geospatial Creator, it was possible for us to “replace” the facade of the Las Vegas Convention Center, revealing a playful scene where the Android Bot highlighted the depth and occlusion capabilities of the technology while showcasing a Generative AI Wallpaper demo.

Many people also called out the usefulness of seeing location-based, AR content with contextual information, like navigation through Google Maps, embedded into interesting locations around the Booth. Interactive panels overlaid around the Booth then introduced key physical demos located at each station around the Booth. Attendees could quickly scan the different themes and features demoed, orient themselves around the Booth, and decide which area they wanted to visit first.


“I loved the experience! Maps and AR make so much sense together. I found it super helpful seeing what demos are in each booth, right on top of the booth, as well as the links to navigation. I could see using this beyond CES as well!” 
– CES Attendee
Moving image Booth navigation
The Android Bot helped attendees visually understand the different areas and demos at the Google Booth, helping them decide what they wanted to go see first.

From the attendees we spoke to, over half of them engaged with the full experience. They were able to skip parts of the experience that felt less relevant to them and focus only on the interactions that added value. Overall, we’ve learned that most people liked seeing a mix of delightful and helpful content and they felt excited to explore the Booth further with other demos.

Moving image of people navigating augmented reality at CES
Many attendees engaged with the full AR experience to learn more about the Google Booth at CES.

Photo of Shahram Izadi watching a demonstration of the full Geospatial AR experience at CES.
Shahram Izadi, Google’s VP and GM, AR/XR, watching a demonstration of the full Geospatial AR experience at CES.

Location-based, AR experiences can transform event experiences for attendees who desire more ways to discover and engage with exhibitors at events. This trend underscores a broader shift in consumer expectations for a more immersive and interactive world around them and the blurring lines between online and offline experiences. At events like CES, AR content can offer a more immersive and personalized experience that not only entertains but also educates and connects attendees in meaningful ways.

To hear the latest updates about Google AR, Geospatial Creator, and more follow us on LinkedIn (@GoogleARVR) and X (@GoogleARVR). Plus, visit our ARCore and Geospatial Creator websites to learn how to get started building with Google’s AR technology.


1Available on select devices and may depend on regional availability and user settings.

Congratulations to the winners of Google’s Immersive Geospatial Challenge

Posted by Bradford Lee – Product Marketing Manager, Augmented Reality, and Ahsan Ashraf – Product Marketing Manager, Google Maps Platform

In September, we launched Google's Immersive Geospatial Challenge on Devpost where we invited developers and creators from all over the world to create an AR experience with Geospatial Creator or a virtual 3D immersive experience with Photorealistic 3D Tiles.

"We were impressed by the innovation and creativity of the projects submitted. Over 2,700 participants across 100+ countries joined to build something they were truly passionate about and to push the boundaries of what is possible. Congratulations to all the winners!" 

– Shahram Izadi, VP of AR at Google

We judged all submissions on five key criteria:

  • Functionality - How are the APIs used in the application?
  • Purpose - What problem is the application solving?
  • Content - How creative is the application?
  • User Experience - How easy is the application to use?
  • Technical Execution - How well are you showcasing Geospatial Creator and/or Photorealistic 3D Tiles?

Many of the entries are working prototypes, with which our judges thoroughly enjoyed experiencing and interacting. Thank you to everyone who participated in this hackathon.



From our outstanding list of submissions, here are the winners of Google’s Immersive Geospatial Challenge:


Category: Best of Entertainment and Events

Winner, AR Experience: World Ensemble

Description: World Ensemble is an audio-visual app that positions sound objects in 3D, creating an immersive audio-visual experience.


Winner, Virtual 3D Experience: Realistic Event Showcaser

Description: Realistic Event Showcaser is a fully configurable and immersive platform to customize your event experience and showcase its unique location stories and charm.


Winner, Virtual 3D Experience: navigAtoR

Description: navigAtoR is an augmented reality app that is changing the way you navigate through cities by providing a 3 dimensional map of your surroundings.



Category: Best of Commerce

Winner, AR Experience: love ya

Description: love ya showcases three user scenarios for a special time of year that connect local businesses with users.



Category: Best of Travel and Local Discovery

Winner, AR Experience: Sutro Baths AR Tour

Description: This guided tour through the Sutro Baths historical landmark using an illuminated walking path, information panels with text and images, and a 3D rendering of how the Sutro Baths swimming pool complex would appear to those attending.


Winner, Virtual 3D Experience: Hyper Immersive Panorama

Description: Hyper Immersive Panorama uses real time facial detection to allow the user to look left, right, up or down, in the virtual 3D environment.


Winner, Virtual 3D Experience: The World is Flooding!

Description: The World is Flooding! allows you to visualize a 3D, realistic flooding view of your neighborhood.


Category: Best of Productivity and Business

Winner, AR Experience: GeoViz

Description: GeoViz revolutionizes architectural design, allowing users to create, modify, and visualize architectural designs in their intended context. The platform facilitates real-time collaboration, letting multiple users contribute to designs and view them in AR on location.



Category: Best of Sustainability

Winner, AR Experience: Geospatial Solar

Description: Geospatial Solar combines the Google Geospatial API with the Google Solar API for instant analysis of a building's solar potential by simply tapping it.


Winner, Virtual 3D Experience: EarthLink - Geospatial Social Media

Description: EarthLink is the first geospatial social media platform that uses 3D photorealistic tiles to enable users to create and share immersive experiences with their friends.


Honorable Mentions

In addition, we have five projects that earned honorable mentions:

  1. Simmy
  2. FrameView
  3. City Hopper
  4. GEOMAZE - The Urban Quest
  5. Geospatial Route Check

Congratulations to the winners and thank you to all the participants! Check out all the amazing projects submitted. We can't wait to see you at the next hackathon.

How We Made SPACE INVADERS: World Defense, an AR game powered by ARCore

Posted by Dereck Bridie, Developer Relations Engineer, ARCore and Bradford Lee, Product Marketing Manager, Augmented Reality

To celebrate the 45th anniversary of “SPACE INVADERS,” we collaborated with TAITO, the Japanese developer of the original arcade game, and UNIT9 to launch “SPACE INVADERS: World Defense,” an immersive game that takes advantage of the most advanced location-based AR technology. Players around the world can go outside to explore their local neighborhoods, defend the Earth from virtual Space Invaders that spawn from nearby structures, and score points by taking them down – all with augmented reality.

The game is powered by our latest ARCore technology - Geospatial API, Streetscape Geometry API, and Geospatial Creator. We’re excited to show you behind the scenes of how the game was developed and how we used our newest features and tools to design the first of its kind procedural, global AR gameplay.

Geospatial API: Turn the world into a playground

Geospatial API enables you to attach content remotely to any area mapped by Google Street View and create richer and more robust immersive experiences linked to real-world locations on a global scale. SPACE INVADERS: World Defense is available in over 100 countries in areas with high Visual Positioning Service (VPS) coverage in Street View, adapting the gameplay to busy urban environments as well as smaller towns and villages.

For players who live in areas without VPS coverage, we have recently updated the game to include our new mode called Indoor Mode, which allows you to defend the Earth from Space Invaders in any setting or location - indoors or outdoors.

Indoor Mode
The new Indoor Mode in Space Invaders brings the immersive gameplay to any indoor building setting

Creating the initial user flow

ARCore Geospatial API uses camera images from the user’s device to scan for feature points and compares those to images from Google Street View in order to precisely position the device in real-world space.

Geospatial API
Geospatial API is based on VPS with tens of billions of images in Street View to enable developers to build world-anchored experiences remotely in over 100 countries

This requires the user to hold up their phone and pan around the area such that enough data is collected to accurately position the user. To do this, we employed a clever technique to get users to scan the area, by requiring them to track the spaceship in the camera’s field of view.

Start of Game spaceship
To get started, follow the spaceship to scan your local surroundings

Using this user flow, we continually check whether the Geospatial API has gathered enough data for a high quality experience:

if (earthManager.EarthTrackingState == TrackingState.Tracking) {         var yawAcc = earthManager.CameraGeospatialPose.OrientationYawAccuracy;         var horiAcc = earthManager.CameraGeospatialPose.HorizontalAccuracy;         bool yawIsAccurate = yawAcc <= 5;         bool horizontalIsAccurate = horiAcc <= 10; return yawIsAccurate && horizontalIsAccurate; }

Transforming the environment into the playground

After scanning the nearby area, the game uses mesh data from the Streetscape Geometry API to algorithmically make playing the game in different locations a unique experience. Every real-world location has its own topography and city layout, affecting the gameplay in its own unique way.

Space Invaders played in diferent locations
Gameplay is varied depending on your location - from towns in Czech Republic (left) to cities in New York (right)

In the game, SPACE INVADERS can spawn from buildings, so we constructed test cases using building geometry obtained from different parts of the world. This ensures that the game would perform optimally in diverse environments from local villages to bustling cities.

Portal Placement
A visualization of how the algorithm would place portals in the real-world

Entering the Invader’s dimension

From our research studies, we learned that it can be tiring for users to keep holding their hands up for a prolonged period of time for an augmented reality experience. This knowledge influenced our gameplay development - we created the Invader’s dimension to give players a chance to relax their phone arm and improve user comfort.

Our favorite ‘wow’ moment that really shows you the power of the Geospatial API is the transition between real-world AR and virtually generated, 3D dimensions.

Transition AR to 3D
Gameplay transition from real-world AR to 3D dimension

This effect is achieved by blending the camera feed with the virtual environment shader that renders the buildings and terrain in the distinct wireframe style.

Portal Transition Editor
The Invader’s dimension appears around the player in the Unity Editor, seamlessly transitioning between the two modes

After the player enters the Invader’s dimension, the player’s spaceship flies through an algorithmically generated path through their local neighborhood. This is done by creating a depth image of the user’s environment from an overhead camera. In this image, the red channel represents buildings and the blue channel represents space that could potentially be used for the flight path. This image is then used to generate a grid with points that the path should follow, and an A* search algorithm is used to solve for a path that follows all the points.

Finally, the generated A-Star path is post-processed to smooth out any potential jittering, sharp turns and collisions.

To smooth out the spaceship’s pathway, the jitter is removed by sampling the path over a set interval of nodes. Then, we determine if there are any sharp turns on a path by analyzing the angles along the path. If a sharp turn is present, we introduce two additional points to round it out. Lastly we see if this smoothed path would collide with any obstacles, and adjust it to fly over them if detected.

Depth Composite on the left and 3D Path on the right
A visualization of the depth map and a generated sample path in the Invader’s dimension

Creating a global gaming experience

A key takeaway from building the game was that the complexity of the contextual generation required worldwide testing. With Unity, we brought multiple environments into test cases, which allowed us to rapidly iterate and validate changes to these algorithms. This gave us confidence to deploy the game globally.

Visualizing SPACE INVADERS using Geospatial Creator

We used Geospatial Creator, powered by ARCore and Photorealistic 3D Tiles from Google Maps Platform, to validate how virtual content, such as Space Invaders, would appear next to specific landmarks within Tokyo in Unity.

Japan 3D Tiles
With Photorealistic 3D Tiles, we were able to visualize Invaders in specific locations, including the Tokyo Tower in Japan

Future updates and releases

Since the game’s launch, we have heard our players’ feedback and have been actively updating and improving the gameplay experience.

  • We have added a new gameplay mode, Indoor Mode, which allows all players without VPS coverage or players who do not want to use AR mode to experience the game.
  • To encourage users to play the game in AR, scores have been rebalanced to reward players who play outside more than players who play indoors.

Download the game on Android or iOS today and join the ranks of an elite Earth defender force to compete in your neighborhood for the highest score. To hear the latest game updates, follow us on Twitter (@GoogleARVR) to hear how we are improving the game. Plus, visit our ARCore and Geospatial Creator websites to learn how to get started building with Google’s AR technology.

Build transformative augmented reality experiences with new ARCore and geospatial features

  Posted by Eric Lai, Group Product Manager

With ARCore, Google’s platform for building augmented reality experiences, we continue to enhance the ways we interact with information and experience the people and things around us. ARCore is now available on 1.4 billion Android devices and select features are also available on compatible iOS devices, making it the largest cross-device augmented reality platform.

Last year, we launched the ARCore Geospatial API, which leverages our understanding of the world through Google Maps and helps developers build AR experiences that are more immersive, richer, and more useful. We further engaged with all of you through global hackathons, such as the ARCore Geospatial API Challenge, where we saw a number of high quality submissions across a number of use cases, including gaming, local discovery, and navigation.

Today, we are introducing new ARCore Geospatial capabilities, including Streetscape Geometry API, Geospatial Depth API, and Scene Semantics API to help you build transformative, world-scale immersive experiences.


Introducing Streetscape Geometry API

With the new Streetscape Geometry API, you can interact, visualize, and transform building geometry around the user. The Streetscape Geometry API makes it easy for developers to build experiences that interact with real world geometry, like reskinning buildings, power more accurate occlusion, or just placing a virtual asset on a building, by providing a 3D mesh within a 100m radius of the user’s mobile device location.

moving image showing streetscape geometry
Streetscape Geometry API provides a 3D mesh of nearby buildings and terrain geometry

You can use this API to build immersive experiences like transforming building geometry into live plants growing on top of them or using the building geometry as a feature in your game by having virtual balls bounce off and interact with them.

Streetscape Geometry API is available on Android and iOS.


Introducing Rooftop Anchors and Geospatial Depth

Previously, we launched Geospatial anchors which allow developers to place stable geometry at exact locations using latitude, longitude, and altitude. Over the past year, we added Terrain anchors which are placed on Earth's terrain, using only longitude and latitude coordinates, with the altitude being calculated automatically.

Today we are introducing a new type of anchor: Rooftop anchors. Rooftop anchors let you anchor digital content securely to building rooftops, respecting the building geometry and the height of buildings.

moving image showing rooftop anchors
Rooftop anchors make it easier to
anchor digital content to building rooftops
moving image showing geospatial depth
Geospatial depth combines
real time depth measurement from
users' device with Streetscape Geometry data
to generate a depth map of up to 65 meters

In addition to new anchoring features, we are also leveraging the Streetscape Geometry API to improve one of the most important capabilities in AR: Depth. Depth is critical to enable more realistic occlusion or collision of virtual objects in the real world.

Today, we are launching Geospatial Depth. It combines the mobile device real time depth measurement with Streetscape Geometry data to improve depth measurements using building and terrain data providing depth for up to 65m. With Geospatial Depth you can build increasingly realistic geospatial experiences in the real world.

Rooftop Anchors are available on Android and iOS. Geospatial Depth is available on Android.


Introducing Scene Semantics API

The Scene Semantics API uses AI to provide a class label to every pixel in an outdoor scene, so you can create custom AR experiences based on the features in an area around your user. At launch, twelve class labels are available, including sky, building, tree, road, sidewalk, vehicle, person, water and more.

moving image showing streetscape geometry
Scene Semantics API uses AI to provide accurate labels for different features that are present in a scene outdoors

You can use the Scene Semantics API to enable different experiences in your app. For example, you can identify specific scene components, such as roads and sidewalks to help guide a user through the city, people and vehicles to render realistic occlusions, the sky to create a sunset at any time of the day, and buildings to modify their appearance and anchor virtual objects.

The Scene Semantics API is available on Android.


Mega Golf: The game that brings augmented mini-golf to your neighborhood

To help you get started, we’re also releasing Mega Golf, an open source demo that helps you experience the new APIs in action. In Mega Golf you will use buildings in your city to bounce off and propel a golf ball towards a hole while avoiding 3D virtual obstacles. This open source demo is available on GitHub. We're excited to see what you can do with this project.

moving image showing streetscape geometry
Mega Golf uses Streetscape Geometry API to transform neighborhoods into a playable mini golf course where players use nearby buildings to bounce and propel a golf ball towards a hole

With these new ARCore features improvements and the new Geospatial Creator in Adobe Aero and Unity, we’ll make it easier than ever for developers and creators to build realistic augmented reality experiences that delight and provide utility for users. Get started today at g.co/ARCore. We’re excited to see what you create when the world is your canvas, playground, gallery, or more!

Create world-scale augmented reality experiences in minutes with Google’s Geospatial Creator

Posted by Stevan Silva, Senior Product Manager

ARCore, our augmented reality developer platform, provides developers and creators alike with simple yet powerful tools to build world-scale and room-scale immersive experiences on 1.4 billion Android devices.

Since last year, we have extended coverage of the ARCore Geospatial API from 87 countries to over 100 countries provided by Google’s Visual Positioning System and the expansion of Street View coverage, helping developers build and publish more transformative and robust location-based, immersive experiences. We continue to push the boundaries of introducing helpful applications and delightful new world-scale use cases, whether it's the innovative hackathon submissions from the ARCore Geospatial API Challenge or our partnership with Gorillaz, where we transformed Times Square and Piccadilly Circus into a music stage to witness Gorillaz play in a larger-than-life immersive performance.

One thing we’ve consistently heard from you over the past year is to broaden access to these powerful resources and ensure anyone can create, visualize, and deploy augmented reality experiences around the world.

Introducing Geospatial Creator


Today, we are launching Geospatial Creator, a tool that helps anyone easily visualize, design, and publish world-anchored immersive content in minutes straight from platforms you already know and love — Unity or Adobe Aero.

Easily visualize, create, and publish augmented reality experiences with Geospatial Creator in Unity (left) and Adobe Aero (right)

Geospatial Creator, powered by ARCore and Photorealistic 3D Tiles from Google Maps Platform, enables developers and creators to easily visualize where in the real-world they want to place their digital content, similar to how Google Earth or Google Street View visualize the world. Geospatial Creator also includes new capabilities, such as Rooftop anchors, to make it even easier to anchor virtual content with the 3D Tiles, saving developers and creators time and effort in the creation process.

These tools help you build world-anchored, cross-platform experiences on supported devices on both Android and iOS. Immersive experiences built in Adobe Aero can be shared via a simple QR code scan or link with no full app download required. Everything you create in Geospatial Creator can be experienced in the physical world through real time localization and real world augmentation.


With Geospatial Creator, developers and creators can now build on top of Photorealistic 3D Tiles from Google Maps Platform (left) which provide real time localization and real time augmentation (right)

When the physical world is augmented with digital content, it redefines the way people play, shop, learn, create, shop and get information. To give you an idea of what you can achieve with these tools, we’ve been working with partners in gaming, retail, and local discovery including Gap, Mattel, Global Street Art, Singapore Tourism Board, Gensler, TAITO, and more to build real world use cases.

SPACE INVADERS: World Defense immersive game turns the world into a playground

Later this summer you’ll be able to play one of the most acclaimed arcade games in real life, in the real world. To celebrate the 45 year anniversary of the original release, TAITO will launch SPACE INVADERS: World Defense. The game, powered by ARCore and Geospatial Creator, is inspired by the original gameplay where players will have to defend the Earth from SPACE INVADERS in their neighborhood. It will combine AR and 3D gameplay to deliver a fully contextual and highly engaging immersive experience that connects multi-generations of players.



Gap and Mattel transform a storefront into an interactive immersive experience

Gap and Mattel will transform the iconic Times Square Gap Store into an interactive Gap x Barbie experience powered by Geospatial Creator in Adobe Aero. Starting May 23, customers will see the store come to life with colors and shapes and be able to interact with Barbie and her friends modeling the new limited edition Gap x Barbie collection of clothing.

moving image of Gap by Mattel

Global Street Art brings street art to a new dimension with AR murals

Google Arts & Culture partnered with Global Street Art and three world-renowned artists to augment physical murals in London (Camille Walala), Mexico City (Edgar Saner), and Los Angeles (Tristan Eaton). The artists used Geospatial Creator in Adobe Aero to create the virtual experience, augmenting physical murals digitally in AR and bringing to life a deeper and richer story about the art pieces.



Singapore Tourism Board creates an immersive guided tour to explore Singapore

Google Partner Innovation team partnered with Singapore Tourism Board to launch a preview of an immersive Singapore guided tour in their VisitSingapore app. Merli, Singapore's tourism mascot, leads visitors on an interactive augmented tour of the city’s iconic landmarks and hidden gems, beginning with the iconic Merlion Park and engaging visitors with an AR symphony performance at Victoria Theatre and Concert Hall. The full guided tour is launching later this summer, and will help visitors discover the best local hawker food, uncover the city's history through scenes from the past, and more.


Gensler helps communities visualize new urban projects

Gensler used Geospatial Creator in Adobe Aero to help communities easily envision what new city projects might look like for the unhoused. The immersive designs of housing projects allows everyone to better visualize the proposed urban changes and their social impact—ultimately bringing suitable shelter to those who need it.

moving image of city projects from Gensler

Geospatial Creator gives anyone the superpower of creating world scale AR experience remotely. Both developers and creators can build and publish immersive experiences in minutes in countries where Photorealistic 3D Tiles are available. In just a few clicks, you can create applications that help communities, delight your users, and provide solutions to businesses. Get started today at goo.gle/geospatialcreator. We’re excited to see what you create when the world is your canvas, playground, gallery, or more!

Unlock new use cases and increase developer velocity with the latest ARCore updates

Posted by Ian Zhang, Product Manager, AR & Zeina Oweis, Product Manager, AR

Two phones showing animated screens

ARCore was created to provide developers with simple yet powerful tools to seamlessly blend the digital and physical worlds. Over the last few years, we’ve seen developers create apps that entertain, engage, and help people in different ways–from letting fans interact with their favorite characters, to placing virtual electronics and furniture for the perfect home setup and beyond.

At I/O this year, we continue on the mission of improving and building AR developer tools. With the launch of ARCore 1.24, we’re introducing the Raw Depth API and the Recording and Playback API. These new APIs will enable developers to create new types of AR experiences and speed up their development cycles.

Increase AR realism and precision with depth

When we launched the Depth API last year, hundreds of millions of Android devices gained the ability to generate depth maps in real time without needing specialized depth sensors. Data in these depth maps was smoothed, filling in any gaps that would otherwise occur due to missing visual information, making it easy for developers to create depth effects like occlusion.

The new ARCore Raw Depth API provides more detailed representations of the geometry of objects in the scene by generating “raw” depth maps with corresponding confidence images. These raw depth maps include unsmoothed data points, and the confidence images provide the confidence of the depth estimate for each pixel in the raw depth map.

4 examples of ARCore Raw Depth API

Improved geometry from the Raw Depth API enables more accurate depth measurements and spatial awareness. In the ARConnect app, these more accurate measurements give users a deeper understanding of their physical surroundings. The AR Doodads app utilizes raw depth’s spatial awareness to allow users to build realistic virtual Rube Goldberg machines.

ARConnect by PHORIA (left) and AR Doodads by Jam3 (right) use the improved geometry from the Raw Depth AP

ARConnect by PHORIA (left) and AR Doodads by Jam3 (right) use the improved geometry from the Raw Depth API

The confidence image in the Raw Depth API allows developers to filter depth data in real time. For example, TikTok’s newest effect enables users to upload an image and wrap it onto real world objects. The image conforms to surfaces where there is high confidence in the underlying depth estimate. The ability for developers to filter for high confidence depth data is also essential for 3D object and scene reconstruction. This can be seen in the 3D Live Scanner app, which enables users to scan their space and create, edit, and share 3D models.

TikTok by TikTok Pte. Ltd. (left) and 3D Live Scanner by Lubos Vonasek Programmierung (right) use confidence images from the ARCore Raw Depth API

TikTok by TikTok Pte. Ltd. (left) and 3D Live Scanner by Lubos Vonasek Programmierung (right) use confidence images from the ARCore Raw Depth API

We’re also introducing a new type of hit-test that uses the geometry from the depth map to provide more hit-test results, even in low-texture and non-planar areas. Previously, hit-test worked best on surfaces with lots of visual features.

Hit Results with Planes (left)

Works best on horizontal, planar surfaces with 

good texture

Hit Results with Depth (right)

Gives more results, even on non-planar or
low-texture areas

The lifeAR app uses this improved hit-test to bring AR to video calls. Users see accurate virtual annotations on the real-world objects as they tap into the expertise of their social circle for instant help to tackle everyday problems.

lifeAR by TeamViewer uses the improved depth hit-test

As with the previous Depth API, these updates leverage depth from motion, making them available on hundreds of millions of Android devices without relying on specialized sensors. Although depth sensors such as time-of-flight (ToF) sensors are not required, having them will further improve the quality of your experiences.

In addition to these apps, the ARCore Depth Lab has been updated with examples of both the Raw Depth API and the depth hit-test. You can find those and more on the Depth API documentation page and start building with Android and Unity today.

Increase developer velocity and post-capture AR

A recurring pain point for AR developers is the need to continually test in specific places and scenarios. Developers may not always have access to the location, lighting will change, and sensors won’t catch the exact same information during every live camera session.

The new ARCore Recording and Playback API addresses this by enabling developers to record not just video footage, but also IMU and depth sensor data. On playback, this same data can be accessed, enabling developers to duplicate the exact same scenario and test the experience from the comfort of their workspace.

DiDi used the Recording and Playback API to build and test AR directions in their DiDi-Rider app. They were able to save 25% on R&D and testing costs, 60% on travel costs, and accelerated their development cycle by 6 months.

DiDi-Rider by Didi Chuxing saves on development resources with the Recording and Playback API

DiDi-Rider by Didi Chuxing saves on development resources with the Recording and Playback API

In addition to increasing developer velocity, recording and playback unlocks opportunities for new AR experiences, such as post-capture AR. Using videos enables asynchronous AR experiences that remove time and place constraints. For instance, when visualizing AR furniture, users no longer have to be in their home. They can instead pull up a video of their home and accurately place AR assets, enabling them to “take AR anywhere”.

Jump AR by SK Telecom uses the Recording and Playback API to transport scenes from South Korea right into users’ homes to augment with culturally relevant volumetric and 3D AR content.

JumpAR by SKT uses Recording and Playback to bring SouthKorea to your home

JumpAR by SKT uses Recording and Playback to bring SouthKorea to your home

VoxPlop! by Nexus Studios is experimenting with the notion of Spatial Video co-creation, where users can reach in and interact with a recorded space rather than simply placing content on top of a video. The Recording and Playback API enables users to record videos, drop in 3D characters and messages, and share them with family and friends.

VoxPlop! by Nexus Studios uses the Recording and Playback API to experiment with Spatial Video co-creation

VoxPlop! by Nexus Studios uses the Recording and Playback API to experiment with Spatial Video co-creation

Learn more and get started with the Recording and Playback API docs.

Get started with ARCore today

These latest ARCore updates round out a robust set of powerful developer tools for creating engaging and realistic AR experiences. With over a billion lifetime installs and 850 million compatible devices, ARCore makes augmented reality accessible to nearly everyone with a smartphone. We're looking forward to seeing how you innovate and reach more users with ARCore. To learn more and get started with the new APIs, visit the ARCore developer website.

Improving shared AR experiences with Cloud Anchors in ARCore 1.20

Posted by Eric Lai, Product Manager, Augmented Reality

Augmented reality (AR) can help you explore the world around you in new, seemingly magical ways. Whether you want to venture through the Earth’s unique habitats, explore historic cultures or even just find the shortest path to your destination, there’s no shortage of ways that AR can help you interact with the world.

That’s why we’re constantly improving ARCore — so developers can build amazing AR experiences that help us reimagine what’s possible.

In 2018, we introduced the Cloud Anchors API in ARCore, which lets people across devices view and share the same AR content in real-world spaces. Since then, we’ve been working on new ways for developers to use Cloud Anchors to make AR content persist and more easily discoverable.

Create long-lasting AR experiences

Last year, we previewed persistent Cloud Anchors, which lets people return to shared AR experiences again and again. With ARCore 1.20, this feature is now widely available to Android, iOS, and Unity mobile developers.

Developers all over the world are already using this technology to help people learn, share and engage with the world around them in new ways.

MARK, which we highlighted last year, is a social platform that lets people leave AR messages in real-world locations for friends, family and their community to discover. MARK is now available globally and will be launching the MARK Hope Campaign in the US to help people raise funds for their favorite charities and have their donations matched for a limited time.

AR photo

MARK by People Sharing Streetart Together Limited

REWILD Our Planet is an AR nature series produced by Melbourne based studio PHORIA. The experience is based on the Netflix original documentary series Our Planet. REWILD uses Ultra High Definition Video alongside AR content to let you venture into earth’s unique habitats and interact with endangered wildlife. It originally launched in museums, but can now be enjoyed on your smartphone in your living room. As episodes of the show are released, persistent Cloud Anchors allow you to return to the same spot in your own home to see how nature is changing.

AR image

REWILD Our Planet by PHORIA

Changdeok ARirang is an AR tour guide app that combines the power of SK Telecom’s 5G with persistent Cloud Anchors. Visitors at Changdeokgung Palace in South Korea are guided by the legendary Haechi to relevant locations where they can experience historical and cultural high fidelity AR content. Changdeok ARirang at Home was also launched so that this same experience can be accessed from the comfort of your couch.

AR image

Changdeok ARirang by SK Telecom

In Sweden, SJ Labs, the innovation arm of Swedish Railways, together with Bontouch, their tech innovation partner, uses persistent Cloud Anchors to help passengers find their way at Central Station in Stockholm, making it easier and faster for them to make their train departures.

AR image

SJ Labs by SJ – Swedish Railways

Coming soon, Lowe’s Persistent View will let you design your home in AR with the help of an expert. You’ll be able to add furniture and appliances to different areas of your home to see how they’d look, and return to the experience as many times as needed before making a purchase.

AR example

Lowe’s Persistent View powered by Streem

If you’re interested in building AR experiences that last over time, you can learn more about persistent Cloud Anchors in our docs.

Call for collaborators: test a new way to find AR content

As developers use Cloud Anchors to attach more AR experiences to the world, we also want to make it easier for people to discover them. That’s why we’re working on earth Cloud Anchors, a new feature that uses AR and global localization—the underlying technology that powers Live View features on Google Maps—to easily guide users to AR content. If you’re interested in early access to test this feature, you can apply here.

Some earth Cloud Anchors concepts

Invite ancient creatures to your living room with AR

What does it feel like to stare into some of the oldest eyes on earth? With augmented reality (AR) and Google Arts & Culture, now you can find out: Meet Cambropachycope, an ancient crustacean with a distinctive pointy head covered in tiny eyes. In collaboration with institutions such as Moscow’s State Darwin Museum and London’s Natural History Museum, we’ve brought a menagerie of prehistoric animals back to digital life. Thanks to AR, you can see them up close through your phone. 


In addition to Cambropachycope, you can also meet the oldest large filter feeder, the fish that swims poorly, or the largest animal ever to live on Earth. Make sure to snap a picture or a video so you can show how these creatures compare in size to the Felis catus or Canis familiaris that roams your living room.
  • aegirocassis.gif

    Aegirocassis - a 480m-year-old marine animal

  • AR-GAC-cambroachycope.gif

    Cambropachycope - owner of the oldest preserved eye

  • AR-GAC-Shark (1).gif

    Silvertip shark - a ferocious apex predator

  • 2 - Whale.png

    Blue whale - the largest animal to ever live on Earth

  • AR-GAC-Apollo.gif

    Apollo 11 Module -  living quarters during most of the first manned lunar landing mission

If unusual critters aren’t your thing, we’ve also recreated a collection of unusual cultural artifacts for you to experience in AR. Meet the pre-Inca “smiling god” Lanzón from circa 500 BCE, or see how the Apollo 11 Command Module looks in your backyard—along with a spacesuit, of course. Or, choose from among thousands of paintings to decorate your space, from Frida Kahlo’s self portraits to The Kiss.


To start learning about culture, history and nature in new dimensions, explore our collection of objects in AR and download the Google Arts & Culture app, available for free on Android and iOS

A new wave of AR Realism with the ARCore Depth API

Posted by Rajat Paharia, Product Lead, AR Platform

Since the launch of ARCore, our developer platform for building augmented reality (AR) experiences, we've been focused on providing APIs that help developers seamlessly blend the digital and physical worlds.

At the end of last year, we announced a preview of the ARCore Depth API, which uses our depth-from-motion algorithms to generate a depth map with a single RGB camera. Since then, we’ve been working with select collaborators to explore how depth can be used across a range of use cases to enhance AR realism.

Today, we're taking a major step forward and announcing the Depth API is available in ARCore 1.18 for Android and Unity, including AR Foundation, across hundreds of millions of compatible Android devices.

Generate a depth map without specialized hardware to unlock capabilities like occlusion

As we highlighted last year, a key capability of the Depth API is occlusion: the ability for digital objects to accurately appear behind real world objects. This makes objects feel as if they’re actually in your space, creating a more realistic AR experience.

Illumix, the game studio behind Five Nights at Freddy’s AR: Special Delivery, uses occlusion to deepen the realism of the experience by allowing certain characters to hide behind objects for more startling jump scares.

Play Five Nights at Freddy’s AR: Special Delivery

While occlusion is an important capability, the ARCore Depth API unlocks more ways to increase realism and enables new interaction types. The ARCore Depth Lab spurred more ideas on how depth can be used, including realistics physics, surface interactions, environmental traversal, and more. Developers can now build on these ideas through the open sourced GitHub project.

Experiment with ARCore Depth Lab on the Google Play Store

The designers and engineers at Snap Inc. integrated several of these ideas into a set of Snapchat Lenses including the Dancing Hotdog and a new Android exclusive Undersea World Lens.

See how depth can add a layer of realism to your Snapchat experience

Snapchat Lens Creators can now download an ARCore Depth API template to create depth-based experiences for compatible Android devices. Sam Hare, Research Engineering Manager at Snap Inc, expressed his excitement, “We’re beginning to understand what kinds of depth capabilities are exciting for developers to build with. This single integration point streamlines and simplifies the development process and enables Lens Studio developers to easily take advantage of advanced depth capabilities.”

Another app that combines occlusion with other depth capabilities is Lines of Play, an Android experiment from the Google Creative Lab. Lines of Play lets users create domino art in AR, and uses depth information to showcase both occlusion and collisions. Design elaborate domino creations, topple them over and watch them collide with the furniture and walls in your room.

Watch as domino pieces topple into each other and onto your walls with Lines of Play

In addition to gaming and self-expression, depth can also be used to unlock new utility use cases. For example, the TeamViewer Pilot app, a remote assistance solution that enables AR annotations on video calls, uses depth to better understand the environment so experts around the world can more precisely apply real time 3D AR annotations for remote support and maintenance.

3D annotations help experts accurately highlight details in the TeamViewer Pilot app

Later this year, you will be able to try more depth-enabled AR experiences such as SKATRIX by Reality Crisis and SPLASHAAR by ForwARdgames, that use surface interactions and environmental traversal as they make rich use of the environment around you.

Check out surface interactions and environmental traversal in SKATRIX, and SPLASHAAR

While depth sensors, such as time-of-flight (ToF) sensors, are not required for the Depth API to work, having them will further improve the quality of experiences. Dr. Soo Wan Kim, Camera Technical Product Manager at Samsung commented on the future that the Depth API and ToF unlocks saying, “Depth will enrich user's AR experience in many perspectives. It will reduce scanning time, and can detect planes fast, even low textured planes. These will bring seamless experiences to users who will be able to use AR apps more easily and frequently.” In the coming months, Samsung will update their Quick Measure app to use the ARCore Depth API on the Galaxy Note10+ and Galaxy S20 Ultra.

Accurately measure with Quick Measure

To learn more and get started with the ARCore Depth API, get the SDK and visit the ARCore developer website.

Meet humanity’s first artists through virtual reality

Editor’s Note: France’s Chauvet Cave contains some of the world’s oldest prehistoric drawings. It’s so delicate that it’s closed to the public, but thanks to our partner, the Syndicat Mixte de la Grotte Chauvet, you can now step into the world of our ancient ancestors through Google Search’s augmented reality feature as well as virtual reality. One of these ancient ancestors, who has asked to remain anonymous, has time-traveled 36,000 years to share what the cave was like back then. 

We began our journey to the big cave days ago. Today we arrive and settle near the stone arch that spans the river. We light a fire, signalling to our people up near the caves that we’re here. We’ve brought small stone tools with us to sew the arrowheads we use for hunting. Perhaps we’ll be able to trade them.

There’s plenty of moonlight, so once we’ve made camp I venture out, hiking up to the cave’s entrance to greet the others. The children are still awake, playing with their toys but also listening intently to the lions roaring in the distance. There used to be bears living here too, but they’re long gone.

The closer I get to its entrance, the more the dark cave seems to draw me in, so I light a torch and step inside. After a short walk, the fire illuminates where we—and those before us—have left our marks. Here, someone scraped the clay, exposed the limestone and painted their world, long before we arrived. My favorites are the horses—I think one is afraid, another is playing, and a third one, the curious one, has pricked up its ears inquiringly.

Near the familiar mammoth, a new image catches my eye—perhaps some of our young hunters have depicted this lion to celebrate their success.

The fresco is so enormous, it’s impossible to take it all in. I step back to try and comprehend its meaning. There are cave lions, reindeer and stags, all seeming to move in the play of light and shadow. Just a few lines, drawn by practiced hands, and somehow we have a masterpiece.

Then there are the handprints left by those who came before us. I stand on my toes and stretch to match my own hand to the imprints on the cold rock, and suddenly I feel compelled to leave my mark too. I’ve never been chosen as a painter, but I’m alone and feeling daring, so I dip my hand into the red paint that’s been left out, rise back to my toes, and add my handprint to the others on the wall. 

As it dries, I draw back and watch as the animals and the handprints fade into the darkness. Who knows how long they’ve all been here, and how long they’ll remain?

Another note from the editor: if you enjoyed hearing from our anonymous cave ancestor, check out the following images of the cave she described, or find out more in Google Arts & Culture’s latest exhibit “Chauvet: Meet our Ancestors.”


  • Panoramic view of the a river and mountains

    Panoramic view of the Pont d’Arc, a stone arch over the river with the Chauvet Cave mountains in the background. © David Huguet

  • Phone screen displaying a AR model

    When you search for the Chauvet Cave on Google Search, the AR in Search features  let you view and interact with a 3D model of a fresco in the cave. Use your smartphone to place it directly into your own space, to get a sense of scale and detail.

  • A torch lights up a cave painting_3.jpg

    “Chauvet - The Dawn of Art” is a virtual reality experience available on HTC vive that guides visitors through the Chauvet Cave. 

    Narrated by Daisy Ridley (English version) or Cécile de France (French version), it shows you 12 stations within the cave. No worries if you don’t have a VR headset - this Youtube 360 video version is equally immersive.


  • Image of a cave painting with felines

    The “alcove of the felines” is the central part of the Horses Fresco, one of the masterpieces of Chauvet Cave. 

    © David Huguet


  • 3D model of a Bear Skull.png

    This is the 3D model of a bear skull that was placed on a  block of stone in the cave. It was staged deliberately, meant to be seen this way, and has not moved for 36,000 years.