Tag Archives: arcore

Six ways to take Playground home for the holidays with Pixel

In October, we launched Playground on the Pixel 3 and Pixel 3 XL, giving you the power to create and play with the world around you through your camera. Playground helps you bring more of your imagination to your photos and videos with interactive Playmoji—characters that react to each other and to you—and tell a richer story by adding animated stickers and creative captions. Starting today, Playground is available on Pixel, Pixel XL, Pixel 2 and Pixel 2 XL, so now all of #teampixel can join in on the fun.

Just in time for the holidays, we’re also introducing festive new Playmoji and stickers soon that can help bring your photos and videos to life. Whether you’re celebrating at home or hitting the road, here are six ways you can take Playground home for the holidays with the Pixel camera.

1. Share your journey.Whether your holiday travels take you away on a plane, train or automobile, getting there is all the fun with the speedy new Travel Playmoji pack. Document your adventures from the window seat, or spice up snaps from your road trip.

SunnySanta _ Device Frame.gif

2. Send a virtual postcard. Wish loved ones a happy holiday from wherever you are with Playmoji, stickers and captions. Or say it all in a selfie by posing with characters who react to you. Once you’ve lined up the perfect shot, sharing with Playground is easy—just a few taps straight from the camera.

winter

3. Let your creativity snow. Brrr! Chillier temperatures mean a chance to use snowy Weather Playmoji and the Winter Playmoji pack during your favorite seasonal activities like ice skating and hockey.

Oy to the World

4. Put your spin on a scene. Oh, dreidel, dreidel, dreidel! We made you out of…pixels. Hanukkah may be over, but you can use these themed stickers year-round, complete with dancing menorah candles, spinning dreidels and latkes galore.

ar ornaments

5. Deck the halls. Make an old tradition new again by challenging your family to a tree decorating contest with Playground. One tree, endless possibilities!

christmascheer

6. Tell an imaginative story. Add some flair to holiday scenes with the jolly new “Christmas Cheer” stickers. Throw a Santa hat on your friend, stick mistletoe where it belongs or place a one-of-a-kind “gift” under the tree.

holiday hello

You can spread the holiday cheer with Playground this season by sharing your creations with #teampixel. We can’t wait to see what you create!

Creating More Realistic AR experiences with updates to ARCore & Sceneform

Posted by Ashish Shah, Product Manager, Google AR & VR

The magic of augmented reality is in the way it blends the digital and the physical worlds. For AR experiences to feel truly immersive, digital objects need to look realistic -- as if they were actually there with you, in your space. This is something we continue to prioritize as we update ARCore and Sceneform, our 3D rendering library for Java developers.

Today, with the release of ARCore 1.6, we're bringing further improvements to help you build more realistic and compelling experiences, including better plane boundary tracking and several lighting improvements in Sceneform.

With 250M devices now supporting ARCore, developers can bring these experiences to an even larger and growing user base.

More Realistic Lighting in Sceneform

Previous versions of Sceneform defaulted to optimizing ambient light as yellow. Version 1.6 defaults to neutral and white. This aligns more closely to the way light appears in the real world, making digital objects look more natural. You can see the differences below.

Left side image: Sceneform 1.5Right side image: Sceneform 1.6

This change will also make objects rendered with Sceneform look as if they're affected more naturally by color and lighting in the surrounding environment. For example, if you're viewing an AR object at sunset, it would appear to be illuminated by the red and orange hues, just like real objects in the scene.

In addition, we've updated Sceneform's built-in environmental image to provide a more neutral scene for your app. This will be most noticeable when viewing reflections in smooth metallic surfaces.

Adding screen capture and recording to the mix

To help you further improve quality and engagement in your AR apps, we're adding screen capture and recording to Sceneform. This is something a number of developers have requested to help with demo recording and prototyping. It can also be used as an external facing feature, allowing your users to share screenshots and videos on social media more easily, which can help get the word out about your app.

You can access this functionality through the surface mirroring API for the SceneView class. The API allows you to display the Sceneform view on a device's screen at the same time it's being rendered to another surface (such as the input surface for the Android MediaRecorder).

Learn more and get started

The new updates to Sceneform and ARCore are available today. With these new versions also comes support for new devices, such as the Samsung Galaxy A3 and the Huawei P20 Lite, that will join the list of ARCore-enabled devices. More information is available on the ARCore developer website.

An art gallery in your pocket: See Vermeer’s paintings in augmented reality

Over 28 years ago, two art thieves dressed as police officers made their way into the Isabella Stewart Gardner Museum in Boston and stole multiple artworks, including a Vermeer painting that was one of only 36 attributed to the artist. With an estimated value of over $200 million, “The Concert” remains one of the most expensive missing items on the FBI’s list of stolen art. With the rest of Vermeer’s masterpieces scattered across 17 collections in seven countries, people have never had the opportunity to see all of Vermeer’s works in one place. And since some of his works are now too fragile to travel, they’ll have to remain where they are indefinitely.

But now, you can experience all of Vermeer's known artworks in one place for the first time. Thanks to the Mauritshuis museum in the Netherlands and other cultural institutions guarding Vermeer’s legacy, they’re available in Pocket Gallery, a brand new feature on the Google Arts & Culture app. Pocket Gallery uses augmented reality, so you can pull out your phone and step into a virtual exhibition space to see all of his works, curated by experts from the Mauritshuis. All 36 of his paintings—including the missing masterpiece and the famous “Girl with a Pearl Earring”—hang lifesize and perfectly lit. As you step closer, you’ll see each painting in stunning detail and can learn more about each piece.   

The Art Camera—our ultra-high resolution robotic camera made specifically for artworks—was deployed to several galleries around the world, creating the highest-ever resolution image of eight of Vermeer’s masterpieces for your zooming pleasure. You can also dive into “in painting tours” of each Vermeer’s 36 works and enjoy the guided insights into artworks like Girl with a Flute. In addition to Vermeer’s paintings, you’ll be able to explore several expert stories that shed light on Vermeer's art, legacy or mysterious life—for instance, you can hear from Tracy Chevalier, author of the bestseller “Girl with a Pearl Earring.”

Today, Vermeer resonates in pop culture references around the world. Justin Richburg—who recently created the character designs for Childish Gambino’s music video “Feels like Summer”—conceived an original piece of art that bridges time and cultures: “Icons” reimagines Vermeer in the 21st century, and shows how the subjects of his paintings have become icons themselves.

justinrichburg

Icons by Justin Richburg.

You can experience Vermeer’s work in a variety of formats—whether it’s an interactive coloring book on Instagram or an original series with YouTube Creators. To see Vermeer’s paintings hanging where they currently are, you can also check out Street View photography in galleries worldwide to navigate the halls of the Frick Collection (New York) and Rijksmuseum (Amsterdam). Visit g.co/meetvermeer, join the conversation with #MeetVermeer or download the app on either iOSor Android to try out Pocket Gallery.

Bust ghosts in the newest game built with Maps—Ghostbusters World

Earlier this year, we introduced a new way for game developers to create real world games using information about the world from Google Maps. It enables game studios to easily reimagine our world as whatever they can dream up and helps them find the best places in the world for players to fuel up or start a mission.


With Ghostbusters World™, the newest game built with Google Maps, you can grab your virtual proton pack and bust ghosts—all as you explore a game world built on the Google Maps you know and love. Brought to you by Sony Pictures Consumer Products, Ghost Corps, publisher FourThirtyThree Inc. (4:33), and developer Next Age, Ghostbusters World is available for free on Google Play and the App Store now.

                                                   

As a Ghostbuster, your mission is to “bust” ghosts to keep the world safe and ghost-free (just in time for Halloween, in case you’re superstitious). Lurking among 3D buildings, landmarks and parks, you’ll find hundreds of ghosts from all dimensions of the Ghostbusters franchise like Wes Pinker, Splat and Achira—in addition to fan favorites like Stay Puft and Slimer. Catch them in your proton beam to drain their energy and then capture them in your containment unit. As you advance in the game, you’ll gain access to the latest in spectral neutralization and trapping technology.

                                                     

Because some ghosts are just too strong to take down on your own (would you want to face Stay Puft solo?), you can team up with nearby Ghostbusters in multiplayer boss raids. Not a team player? No problem. If competition is what you’re after, just build up your ghost team (the ghosts you capture and store in your bank) and enter battles against other Ghostbusters around the world to gain valuable resources needed to make your ghosts stronger.

                                                      

For those Ghostbusters who delight in the story—not just the action—there’s an all original story mode featuring your favorite classic characters. And if you’re feeling festive (or daring) this Halloween, there’s an AR Mode (built with ARCore on Android) that lets you blur the lines between ghostly fantasy and reality.

                                                        

If you want to do your part to make sure your local streets are ghost-free this Halloween, try Ghostbusters World. Download it now from Google Play and the App Store.


Source: Google LatLong


Introducing new APIs to improve augmented reality development with ARCore

Posted by Clayton Wilkinson, Developer Platforms Engineer

Today, we're releasing updates to ARCore, Google's platform for building augmented reality experiences, and to Sceneform, the 3D rendering library for building AR applications on Android. These updates include algorithm improvements that will let your apps consume less memory and CPU usage during longer sessions. They also include new functionality that give you more flexibility over content management.

Here's what we added:

Supporting runtime glTF loading in Sceneform

Sceneform will now include an API to enable apps to load gITF models at runtime. You'll no longer need to convert the gITF files to SFB format before rendering. This will be particularly useful for apps that have a large number of gITF models (like shopping experiences).

To take advantage of this new function -- and load models from the cloud or local storage at runtime -- use RenderableSource as the source when building a ModelRenderable.

 private static final String GLTF_ASSET = "https://github.com/KhronosGroup/glTF-Sample-Models/raw/master/2.0/Duck/glTF/Duck.gltf";

// When you build a Renderable, Sceneform loads its resources in the background while returning
// a CompletableFuture. Call thenAccept(), handle(), or check isDone() before calling get().
ModelRenderable.builder()
.setSource(this, RenderableSource.builder().setSource(
this,
Uri.parse(GLTF_ASSET),
RenderableSource.SourceType.GLTF2).build())
.setRegistryId(GLTF_ASSET)
.build()
.thenAccept(renderable -> duckRenderable = renderable)
.exceptionally(
throwable -> {
Toast toast =
Toast.makeText(this, "Unable to load renderable", Toast.LENGTH_LONG);
toast.setGravity(Gravity.CENTER, 0, 0);
toast.show();
return null;
});

Publishing the Sceneform UX Library's source code

Sceneform has a UX library of common elements like plane detection and object transformation. Instead of recreating these elements from scratch every time you build an app, you can save precious development time by taking them from the library. But what if you need to tailor these elements to your specific app needs? Today we're publishing the source code of the UX library so you can customize whichever elements you need.

An example of interactive object transformation, powered by an element in the Sceneform UX Library.

Adding point cloud IDs to ARCore

Several developers have told us that when it comes to point clouds, they'd like to be able to associate points between frames. Why? Because when a point is present in multiple frames, it is more likely to be part of a solid, stable structure rather than an object in motion.

To make this possible, we're adding an API to ARCore that will assign IDs to each individual dot in a point cloud.

These new point IDs have the following elements:

  • Each ID is unique. Therefore, when the same value shows up in more than one frame, you know that it's associated with the same point.
  • Points that go out of view are lost forever. Even if that physical region comes back into view, a point will be assigned a new ID.

New devices

Last but not least, we continue to add ARCore support to more devices so your AR experiences can reach more users across more surfaces. These include smartphones as well as -- for the first time -- a Chrome OS device, the Acer Chromebook Tab 10.

Where to find us

You can get the latest information about ARCore and Sceneform on https://developers.google.com/ar/develop

Ready to try out the samples or have issues, then visit our projects hosted on GitHub:

Save dinosaurs from extinction in a game world built with Google Maps

JurassicWorldAlive1

Back in March we announced a product to help developers build games using the information Google Maps knows about the world around you. It enables game studios to easily reimagine our world as a medieval fantasy, a bubble gum candy land, or a zombie-infested post-apocalyptic city. It even helps them find the best places for gameplay––whether it’s a landmark where a player refuels or a park where they must go to complete a mission––no matter where in the world players are.


Just in time for summer and the release of Jurassic World: Fallen Kingdom™, you can explore a virtual world built with Google Maps while saving dinosaurs from extinction. Ludia and Universal’s Jurassic World™ Alive is available for free on Google Play and the App Store now.


Ludia used Google Maps’ rich and accurate data to create a game world where dinosaurs have escaped Jurassic World on Isla Nublar to roam freely in cities and neighborhoods around the world. As a member of the Dinosaur Protection Group, your mission is to save dinosaurs from another extinction––and you do that by exploring the world around you. Nestled among 3D buildings, roads, landmarks, and parks, you’ll find dinosaurs, track them with drones, collect DNA samples to level up, and create hybrid dinosaurs in your lab. Once you’ve assembled your own roster of prehistoric animals, you can battle other players to defend against threats to your mission.

JurassicWorldAlive2

To perform your duties as part of the Dinosaur Protection Group, you’ll need to earn rewards like in-game currency and battery life for your drone by finding supply drops––all strategically placed in fun (and appropriate) places using Google Maps’ in-depth understanding of real world places.

JurassicWorldAlive3

The game uses ARCore, Google's platform for building augmented reality experiences, to power an exciting (and terrifying!) AR mode that lets you get up close and personal with dinosaurs in your collection. To get to the AR mode, tap the collection icon at the bottom of your screen, select a dinosaur, and then tap the round AR icon on the right hand side.

So if you’re still bummed that dinosaurs went extinct millions of years ago, try Jurassic World Alive and spend some quality time with prehistoric beasts in your own natural habitat.

Creating AR Experiences for I/O: Our Process

Posted by Karin Levi, Product Marketing, ARCore

A few weeks ago at Google I/O we released a major update to ARCore, Google's AR development platform. We added new APIs like Cloud Anchors, that enable multi-user, collaborative AR experiences and Augmented Images that enable activation of 2D images into 3D objects. All of these updates are going to change the way we use AR today and enable developers to create richer, more immersive AR apps.

With these new capabilities, we decided to put our platform to the test. So we built real experiences to showcase how these all come to life. All demos were presented at the I/O AR & VR sandbox area. We open sourced them to make sure you can see how simple it is to build these experiences. We're pretty happy with how they turned out and would love to share with you some learning and insights from behind the scenes.

Light Board - Multiplayer game

Light Board is an AR multiplayer tabletop game where two players on floating game boards launch colored projectiles at each other.

While building Light Board it was important for us to keep in mind who the end users are. We wanted it to be a simple/fun game for developers to try out while visiting the I/O sandbox. The developers would only have a couple minutes to play while passing through, so it needed to allow players (even non-gamers) to pick it up and play with very little setup.

The artwork for Light Board was a major focus. Our mission for the look of the game was to align with the design and decor of I/O 2018. This way, our app would feel like an extension of everything the attendees saw around them. As a result, our design philosophy had 3 goals; bright accent colors, simple graphic shapes and natural physical materials.

Left: Design for AR/VR Sandbox at I/O 2018. Right: Key art for Light Board game boards

The artwork was created in Maya and Cinema 4D. We created physically based materials for our models using Substance Painter. Just as continuous iteration is crucial for engineering, it is also important when creating art assets. With that in mind, we kept careful track of our content pipeline, even for this relatively simple project. This allowed us to quickly try out different looks and board styles before settling on our final design.

On the engineering front we selected the Unity game engine as our dev environment. Unity gives us a couple of important advantages. First, it is easy to get great looking 3D graphics up and running right away. Second, the engine component is already complete, so we could immediately start iterating on gameplay code. As with the artwork, this allowed us to test gameplay options before we made a final decision. Additionally, Unity gave us support for both Android and iOS with only a little extra work.

To handle the multiplayer aspect we used Firebase Realtime Database. We were concerned with network performance at the event, and felt that the persistent nature of a database would make it more tolerant of poor networks. As it turned out, it worked very well and we got the ability to quit and rejoin games for free!

We had a lot of fun building Light Board and we hope people can use it as an example of how easy it can be to not only build AR apps, but to use really cool features like Cloud Anchors. Please check out our open source repo and give Light Board a try!

Just a line - Draw with your friends

In March, we released Just a Line, an Android app that lets you draw in the air with your phone. It's a simple experiment meant to showcase the power of ARCore. At Google I/O, we added Cloud Anchors to the app so that two people can draw at once in the same space, even if one of them is using Android and the other iOS.

Both apps were built natively: The Android version was written in Android Studio, and the iOS version was built in xCode. ARCore's Cloud Anchors enable Just a Line to pair two phones, allowing users to draw simultaneously in a shared space. Pairing works across Android and iOS devices, and drawings are synchronized live through a Firebase Realtime Database. You can find the open-source code for iOS here and for Android here.

Illusive Images - Art exhibition comes to life

"Illusive Images" demo is an augmented gallery consisting of 3 artworks, each exploring a different augmented image use case and user experience. As one walks from side to side, around the object, or gazes in a specific direction, 2D artworks are married with 3D, inviting the viewer to enter into the space of the artwork spanning well beyond the physical frame.

Due to the visual design nature of our augmented images, we experimented a lot with creating databases with varying degrees of features. In order to get the best results, we iterated quickly by resizing the canvas for the artwork. We also moved and stretched the brightness and contrast levels. These variations helped to achieve the most optimal image without compromising design intent.

The app was built in Unity with ARCore, with the majority of assets created in Cinema 4D. Mograph animations were imported into Unity as an fbx, and driven entirely by the position of the user in relation to the artwork. An example project can be found here.

To make your development experience easier, we open sourced all the demos our team built. We hope you find this useful! You can also visit our website to learn more and start building AR experiences today.

ARCore: Augmented reality at Android scale

Posted by Dave Burke, VP, Android Engineering

With more than two billion active devices, Android is the largest mobile platform in the world. And for the past nine years, we've worked to create a rich set of tools, frameworks and APIs that deliver developers' creations to people everywhere. Today, we're releasing a preview of a new software development kit (SDK) called ARCore. It brings augmented reality capabilities to existing and future Android phones. Developers can start experimenting with it right now.

We've been developing the fundamental technologies that power mobile AR over the last three years with Tango, and ARCore is built on that work. But, it works without any additional hardware, which means it can scale across the Android ecosystem. ARCore will run on millions of devices, starting today with the Pixel and Samsung's S8, running 7.0 Nougat and above. We're targeting 100 million devices at the end of the preview. We're working with manufacturers like Samsung, Huawei, LG, ASUS and others to make this possible with a consistent bar for quality and high performance.

ARCore works with Java/OpenGL, Unity and Unreal and focuses on three things:

  • Motion tracking: Using the phone's camera to observe feature points in the room and IMU sensor data, ARCore determines both the position and orientation (pose) of the phone as it moves. Virtual objects remain accurately placed.
  • Environmental understanding: It's common for AR objects to be placed on a floor or a table. ARCore can detect horizontal surfaces using the same feature points it uses for motion tracking.
  • Light estimation: ARCore observes the ambient light in the environment and makes it possible for developers to light virtual objects in ways that match their surroundings, making their appearance even more realistic.

Alongside ARCore, we've been investing in apps and services which will further support developers in creating great AR experiences. We built Blocks and Tilt Brush to make it easy for anyone to quickly create great 3D content for use in AR apps. As we mentioned at I/O, we're also working on Visual Positioning Service (VPS), a service which will enable world scale AR experiences well beyond a tabletop. And we think the Web will be a critical component of the future of AR, so we're also releasing prototype browsers for web developers so they can start experimenting with AR, too. These custom browsers allow developers to create AR-enhanced websites and run them on both Android/ARCore and iOS/ARKit.

ARCore is our next step in bringing AR to everyone, and we'll have more to share later this year. Let us know what you think through GitHub, and check out our new AR Experiments showcase where you can find some fun examples of what's possible. Show us what you build on social media with #ARCore; we'll be resharing some of our favorites.