Tag Archives: Pixel

See your world differently with Playground and Google Lens on Pixel 3

Our phone’s camera has become an important part of our lives. We use it to capture moments that matter to us, like memories with our families, and the small things we don’t want to forget, like handwritten notes or parking tickets. And now, thanks to advancements in computer vision, our camera can understand our world and help in new ways.

Today we introduced our most helpful camera yet on Pixel 3 and Pixel 3 XL, so you can turn your world into a playground and do more with what you see.

Introducing Playground, a new way to create and play

From reenacting scenes in “Star Wars: The Last Jedi” to hanging out with Eleven from “Stranger Things,” you’re already having fun with AR Stickers. Today, we’re taking a big step forward by introducing Playground, a new mode in the Pixel camera to create and play with the world around you. It helps you bring more of your imagination to a scene with cameos from your favorite superheroes, stickers that animate around you and fun captions that put words where the action is.

Playground brings you more powerful AR experiences and uses AI to recommend content for expressing yourself in the moment. You can make your photos and videos come to life with Playmoji—characters that react to each other and to you—and tell a richer story by adding fun captions or animated stickers you’ll recognize from Gboard.

Comp-1.gif

You can snap photos on the back camera (left) or take selfies with Playmoji that respond to your facial expressions (right).

Playground also works in selfie mode, so now you can up your selfie game by standing next to characters you love, like Iron Man from the Marvel Cinematic Universe. New packs for Weather, Pets, Sports and Signs let you have everyday fun, and coming later this year, you'll be able to sharpen your dance skills with moves from Childish Gambino.

Iron Man _ Blog%2FSocial (1).gif

Marvel Studios’ Avengers

And for those moments when you aren’t quite sure where to start, Playground makes real-time suggestions to recommend content based on the scene you’re in. Are you walking your dog? Cooking in the kitchen? Gardening in the backyard? Playground uses advanced computer vision and machine learning to recommend relevant Playmoji, stickers and captions to populate the scene.

AR_Playground_Garden.gif

Scene suggestions

Once you’ve got your perfect shot, sharing with Playground is easy—with just a few taps straight from the camera.

Do more with what you see with Google Lens

Last year with Pixel 2, we introduced Google Lens to help you do more with what you see, whether its finding an outfit you like, copying and searching text, or identifying that cute dog in the park. This year, we’ve integrated Lens even more into Pixel 3 for a faster, more helpful experience.

LensKeynote_Menu_1.gif

Lens Suggestions

Lens is already a great way to quickly call a number, visit a URL, scan a QR code, and add an email address to your contacts. On Pixel 3, we’re introducing Lens Suggestions to bring these common actions right into the main Pixel camera. Simply point your camera, and with a single tap, call or save a phone number on a takeout menu, send an email right from a flyer, or open an address in Google Maps. This is all done on-device and takes advantage of Pixel Visual Core, so it doesn’t require an internet connection.

We’ve also made it even easier to get to the full Lens experience on Pixel 3—just long press in the camera. It’s a fast and convenient new way to access Lens.

Lens in Overview.gif

Lens from recent apps screen on Pixel 3

Finally, we’re making Lens accessible from the recent apps screen on Pixel 3, so regardless of the app you’re in, Lens can help. Just drag up from the home button and long press on an image. It’s great for when you see an outfit you like while browsing the web or scrolling through an app.

With Google Lens and Playground, Pixel 3’s camera lets you do more with what you see and have fun with the world around you. Google Lens is available on all Pixel devices, with Playground on Pixel 3 first and coming to Pixel 1 and 2 soon. Preorder Pixel 3 or learn more on the Google Store.

Accessories made for you


There's a lot of new hardware from Google coming your way, and so are new accessories to go along with them. We're launching a variety of beautiful and useful accessories make charging your phone, listening to your favorite song, and looking stylish with your device better than ever.

From our signature fabric cases, to smart bulbs connected to your Google Assistant, we’ve created lots of compatible accessories designed to pair perfectly with your Google devices and help you put your personal stamp on your devices.

Wireless charging, redefined: meet Pixel Stand

Ever tried to use your phone while it’s plugged into a wall charging? Us too. It can be a mess. With Pixel Stand, we’re solving that problem by tilting the angle of the screen to give you the info you need while your Pixel 3 or Pixel 3XL phone is charging.

Pixel Stand & Pixel 3 Alarm.jpg

Pixel Stand makes it easier to receive notifications, listen to music and access the web with help from the Google Assistant while your Pixel 3 and Pixel 3 XL are charging. Features include:

  • An enhanced music experience on your Pixel 3. See your favorite artists' album covers on display while listening to their music. 
  • While docked, turn your phone into a digital photo frame with Google Photos and relive your favorite moments with family and friends.
  • Quick access to your Google Assistantso you can get a rundown of your day, the traffic for your commute and listen to your favorite podcast just by saying “Hey Google, good morning.”
  • Smart home integrations to help you manage your home. If you have a Nest Hello video doorbell, you can get a live feed on your Pixel screen when docked.
  • A gentle wake-up with Pixel Stand’s Sunrise Alarm, which mimics a sunrise on your screen over the course of 15 minutes, brightening your room so you can get on with your day.
04_Dreamliner_ZeroState_swipeup.gif

Rich digital audio, and the Google Assistant in your ear

Last year, we introduced Pixel Buds—wireless headphones that give you instant access to the best features available from the Google Assistant. This year, we’ve included a pair of wired Pixel USB-C Earbuds in the box with Pixel 3 and Pixel 3 XL—the first wired USB-C headphones to include access to the Google Assistant. These not only provide rich audio quality, but also allow the Google Assistant read notifications, give in-ear directions, and provide real-time translation through Google Translate, optimized for Pixel users. You’ll also get a USB-C to 3.5mm adapter, so you can use any other 3.5mm compatible headphones with your Pixel 3.

Protect your Pixel, just in case

We’ve updated our line of fabric cases with stylish new colors, or you can customize your own My Case with a personal photo, curated design, or stylized map. Perfectly made for your phone, our cases work with Qi wireless charging and Active Edge™.

Something for everyone

Our Made for Google line of accessories includes more than 400 compatible add-ons for your Pixel, Pixelbook, Pixel Slate and Google Home Mini, from more than 40 popular brands, including:

  • Cases by brands like Bellroy, Incipio, Speck, Nomad, Otterbox and Sonix so your Pixel and Pixel Slate are covered for any occasion.
  • A wide variety of audio products, from Libratone's USB-C earbuds to Skullcandy’s on-ear cans. All of these sound great and look even better.
To see more of our Made by Google and Made for Google accessories, head over to Google Store.

Smart Compose comes to Pixel 3 and four new languages

In May, we introduced Smart Compose in Gmail on the web, which uses machine learning to help you draft emails faster. It has already saved people from typing over 1 billion characters each week and now it can help you write emails even faster while you’re on the go. We’re bringing Smart Compose to the Gmail mobile app, starting with Pixel 3. Smart Compose will offer writing suggestions as you type and if you like what you see, just swipe right to use it.

Demo 1 animation.gif

Smart Compose has already rolled out globally in English. In the coming months, we’ll bring the experience to four new languages: Spanish, French, Italian and Portuguese. We’ll also bring it to more mobile devices beginning in 2019.

We’re excited to bring Smart Compose to more places and more languages to help you do more with Gmail—faster. Want to make it even better? Keep the feedback coming using the “Help & feedback” link on the web or mobile app.

Smart Compose comes to Pixel 3 and four new languages

In May, we introduced Smart Compose in Gmail on the web, which uses machine learning to help you draft emails faster. It has already saved people from typing over 1 billion characters each week and now it can help you write emails even faster while you’re on the go. We’re bringing Smart Compose to the Gmail mobile app, starting with Pixel 3. Smart Compose will offer writing suggestions as you type and if you like what you see, just swipe right to use it.

Demo 1 animation.gif

Smart Compose has already rolled out globally in English. In the coming months, we’ll bring the experience to four new languages: Spanish, French, Italian and Portuguese. We’ll also bring it to more mobile devices beginning in 2019.

We’re excited to bring Smart Compose to more places and more languages to help you do more with Gmail—faster. Want to make it even better? Keep the feedback coming using the “Help & feedback” link on the web or mobile app.

Source: Gmail Blog


Smart Compose comes to Pixel 3 and four new languages

In May, we introduced Smart Compose in Gmail on the web, which uses machine learning to help you draft emails faster. It has already saved people from typing over 1 billion characters each week and now it can help you write emails even faster while you’re on the go. We’re bringing Smart Compose to the Gmail mobile app, starting with Pixel 3. Smart Compose will offer writing suggestions as you type and if you like what you see, just swipe right to use it.

Demo 1 animation.gif

Smart Compose has already rolled out globally in English. In the coming months, we’ll bring the experience to four new languages: Spanish, French, Italian and Portuguese. We’ll also bring it to more mobile devices beginning in 2019.

We’re excited to bring Smart Compose to more places and more languages to help you do more with Gmail—faster. Want to make it even better? Keep the feedback coming using the “Help & feedback” link on the web or mobile app.

Source: Gmail Blog


New ways to experience Made by Google products

Today, we told you about what’s coming in our latest family of #MadebyGoogle products. But what's a line-up of shiny new products without a plethora of ways for the world to experience, try and buy them? As we continue to build products for everyone, we’re exploring helpful new ways to get our products to everyone.

The Google Hardware Store pop-ups

Starting on October 18, New Yorkers and Chicagoans can try out and buy our new products at a pop-up shop in each city—the only place you can shop Google products in a fully Google-made experiential space. Our pop ups will be open October 18 through December 31, so if you’re in Chicago (Bucktown at 1704 N. Damen) or NYC (SoHo at 131 Green Street), come visit us.

The Google Store and Enjoy

You can now pre-order and shop all of our products via the online Google Store, including the Pixel 3 / 3XL (that works with all major carriers). And as of October 18, folks in the Bay Area can buy the new Pixel 3 / 3XL and get it delivered as soon as three hours and expertly set up via the Enjoy service. You can also get the Pixel 2XL, Pixelbook and Google Home Max via Enjoy delivery now. We’re bringing the Google Store to you!

Google Store Retail 2018.jpg

Google Store + Enjoy bring the expertise of the Google Store to you.

b8ta

Made by Google products are part of an interactive shopping experience in five b8ta stores across the country, including Austin, Corte Madera, Houston, San Francisco, Tysons Corner, and will be available in two new b8ta stores in Short Hills, NJ and Scottsdale, AZ opening later this year. As a part of the unique in-store experience, customers can test out and shop Google’s Home products in interactive home-like vignettes. Visit a store and demo products with one of b8ta’s experts.
b8ta

Made by Google products are now part of a new interactive shopping experience in b8ta.

goop

goop is joining forces with Made by Google products to offer the Google Home smart speaker family across the U.S. in permanent goop Lab stores and goop GIFT pop-ups this holiday season. Abroad, customers can shop at the goop London pop-up which opened this past September. Keep an eye out for more information from goop + Made by Google later this month.

And as for the future...

Google Home Mini and Wing

It’s a bird, it’s a plane. It’s Google Home Mini being delivered by drone! You read that right—along with Wing (an Alphabet company), we’re pushing the boundaries of conventional delivery. As a part of a small, localized test, Google Home Minis were recently dropped off at customers’ homes only 10 minutes after ordering. Although not a reality today, imagine the possibilities in years to come… 

Google Pixel 3: Make every day more extraordinary

Today we’re introducing Pixel 3 and Pixel 3XL, the new smartphones from Google. Pixel brings you the best of Google in a phone, powered by AI to deliver more helpful, thoughtful and enjoyable experiences. That means a phone that answers for you when a telemarketer calls, a camera that uses AI to make sure you never miss the shot, and a more helpful visual and audio experience while charging, powered by the Google Assistant.

Brilliant photos every time and super-charged selfies

We’re taking more photos on our phones than ever before, but we still often miss the perfect moment. Pixel 3 helps you get that perfect shot on the first try.

Here’s how the best camera gets even better with Pixel 3:

  • Capture smiles, not blinks:A feature we call Top Shot uses AI to help you capture the perfect photo every time. When you take a motion photo, it captures alternate shots in HDR+, then recommends the best one—even if it’s not exactly when you hit the shutter—looking for those where everyone is smiling, with eyes open, and facing the camera.

  • Get better zoom:When you zoom in on a phone camera, the image looks grainy. Super Res Zoom is a computational photography technique, traditionally used for astronomy and scientific imaging, that produces sharp details when you zoom.

  • No light; no problem:Pixel 3 lets you take natural-looking photos in dark surroundings, all without a flash. With Night Sight, coming soon to Pixel 3, you can take bright, detailed, colorful shots around the campfire, in a moonlit forest, or after you close out the bar.

  • No selfie stick required:Get everyone in the picture with Group Selfie, which gives you 184 percent more room in your photo for friends and scenery.

  • Look … no hands! Photobooth mode uses AI to recognize that when you’re smiling or making a funny expression, you’re ready for a selfie. It snaps the photo on its own so you don’t need to reach for the shutter button—a good option for candids.  

  • Even more stunning portraits, front and back:When you take photos in Portrait Mode, you can change the blurriness of the background, or change the part of the picture in focus, after the fact. Google Photos can also make the subject of your photo pop by leaving them in color, while changing the background to black and white.

  • Create and play:In Playground, you can make photos, selfies and videos come to life by adding your favorite superheroes, animated stickers and fun captions. In celebration of Marvel Studios’ 10 Year Anniversary, you’ll enjoy seeing the characters from the Marvel Cinematic Universe (exclusively on Pixel) react to each other and to you. New packs for Weather, Pets, Sports and Signs let you have everyday fun, and coming later this year, you'll be able to sharpen your dance skills with moves from Childish Gambino.

  • Super smooth video:When you want to capture something that won’t stop moving—think an adorable toddler or your new puppy—Motion Auto Focus will make sure your Pixel 3 camera stays in sharp focus automatically, as you record. And if you happen to be taking a selfie video while walking or moving around, Pixel 3 brings you front-facing video stabilization.  

Pixel_recommended.gif

Unlimited storage for all of your photos and videos

With Pixel 3, you can save all your favorite moments with free, unlimited photo and video storage in original resolution*. It’s hassle-free, you don’t have to think about back-ups. Come back to Google Photos later and search for the beach photos you took on your Pixel 3, and they’ll pop right up.

Your AI-powered sidekick

AI in Pixel 3 enables new features that make your day-to-day actions simpler and easier.

If you want to know more about something you’re looking at, use Google Lens, built right into the Pixel 3 camera. To scan and translate text, find similar styles of clothing, or identify popular plants and animals, you can now long press in the Pixel 3 camera to easily open Lens. When you point your camera at information you want to remember or don’t feel like typing in—like a URL or QR code on a flyer or an email address on a business card—Google Lens suggests what to do next, like creating a new contact.

You can count on even more help across other apps too, including Gmail’s Smart Compose, now available for mobile on Pixel 3. Smart Compose suggests phrases in your emails so that you can draft them faster, on the go. Gboard, the keyboard built into your Pixel 3, will recommend GIFs, stickers and more, to make your conversations fun and engaging. Both are available first in English. 

The Google Assistant is also baked into Pixel 3 to help you find answers and control your phone and compatible smart home devices—all with a simple squeeze or by using your voice. This year we have two new Assistant features coming to Pixel.

First, starting out in English in the U.S., Pixel 3’s on-device AI helps you screen phone calls and avoid spam calls. Imagine you’re at dinner with family or in a meeting at work and a call from an unknown caller comes in. Just tap on “Screen call” to find out who's calling and why, as well as other information (as prompted by you). You'll immediately see a transcript of the caller's responses so that you can then decide whether to pick up, respond by tapping a quick reply (e.g., “I’ll call you back later”), or mark the call as spam and dismiss. Processing the call details on-device means these experiences are fast, private to you, and use up less battery.

Pixel3_ScreeningCall.gif

Second, Pixel users in the U.S. will be the first to get access to an experimental new Google Assistant feature, powered by Duplex technology, which helps you complete real-world tasks over the phone, like calling a restaurant to book a table. This feature will initially be available later this year in New York, Atlanta, Phoenix and the San Francisco Bay Area to help people book restaurant reservations and will roll out to other U.S. cities in the future.

As we develop new calling technologies, we believe it’s critical that we help people understand the context of the conversation. We’ll disclose to businesses receiving the call that they're speaking to an automated system, and we have developed controls to protect against spam and abuse, as well as the ability for a business to opt out of receiving calls. For Call Screen, we will also let the caller know that a screening service is being used.

Digital Wellbeing

Our phones, while probably the most important tech in our lives, shouldn’t control our lives. So Digital Wellbeing, a suite of tools to help you find your own balance with technology, is built into Pixel 3. It includes a dashboard to help you understand how you spend time on your phone, the ability to set time limits on specific apps, and a new Wind Down mode to help you get to sleep at night by gently transitioning your display to a grayscale screen. When you don’t want to be bothered by rings or notifications, just flip to Shhh—an easy gesture that turns on Do Not Disturb and minimizes distractions.

Fast and wireless charging

Pixel 3 comes with an 18 Watt fast charger in the box, which can give you seven hours of use in 15 minutes of charging. With our AI-powered Adaptive Battery technique, Pixel 3 prioritizes battery power for your most important apps to make your phone last all day.

Alongside Pixel 3, we’re also introducing Pixel Stand, our new, Qi compliant wireless charger (sold separately). While charging in the Pixel Stand, your phone turns into a smart visual and audio experience powered by the Google Assistant, similar to Google Home Hub. It answers your questions, plays music, helps you control smart home devices, transitions into a photo frame when idle, and much more. If you set an alarm, your screen will gently brighten over 15 minutes before your alarm goes off, mimicking the sunrise and helping you wake up naturally. Pixel 3 also comes with dual front-firing speakers tuned by a GRAMMY®-winning music producer to turn your phone into a powerful speaker. Customers who activate a Pixel 3 or Pixel 3 XL by December 31, 2018 can get six months of free YouTube Music Premium.

Pixel 3 is IP68 water- and dust-resistant and has a security chip custom-designed by Google called Titan M, making it the most secure phone we've built yet. Titan M enhances mobile security by protecting your unlock credentials, disk encryption, app data, and the integrity of the operating system code itself. Powered by Android 9 Pie, Pixel 3 comes with the latest Android operating system.

Pixel3 Specs.jpg

Pixel 3 is available for pre-order now from Verizon, Project Fi, and Google Store unlocked.

*Free, unlimited online original-quality storage for photos/videos uploaded from Pixel 3 to Google Photos through 1/31/2022, and those photos/videos will remain free at original quality. g.co/help/photostorage

Google hardware. Designed to work better together.

This year marks Google’s 20th anniversary—for two decades we’ve been working toward our mission to organize the world’s information and make it universally accessible and useful for everybody. Delivering information has always been in our DNA. It’s why we exist. From searching the world, to translating it, to getting a great photo of it, when we see an opportunity to help people, we’ll go the extra mile. We love working on really hard problems that make life easier for people, in big and small ways.

There’s a clear line from the technology we were working on 20 years ago to the technology we’re developing today—and the big breakthroughs come at the intersection of AI, software and hardware, working together. This approach is what makes the Google hardware experience so unique, and it unlocks all kinds of helpful benefits. When we think about artificial intelligence in the context of consumer hardware, it isn’t artificial at all—it’s helping you get real things done, every day. A shorter route to work. A gorgeous vacation photo. A faster email response. 

So today, we’re introducing our third-generation family of consumer hardware products, all made by Google:

  • For life on the go, we’re introducing the Pixel 3 and Pixel 3 XL—designed from the inside out to be the smartest, most helpful device in your life. It’s a phone that can answer itself, a camera that won’t miss a shot, and a helpful Assistant even while it’s charging.

  • For life at work and at play, we’re bringing the power and productivity of a desktop to a gorgeous tablet called Pixel Slate. This Chrome OS device is both a powerful workstation at the office, and a home theater you can hold in your hands.

  • And for life at home we designed Google Home Hub, which lets you hear and see the info you need, and manage your connected home from a single screen. With its radically helpful smart display, Google Home Hub lays the foundation for a truly thoughtful home.

Please visit our updated online store to see the full details, pricing and availability

The new Google devices fit perfectly with the rest of our family of products, including Nest, which joined the Google hardware family at the beginning of this year. Together with Nest, we’re pursuing our shared vision of a thoughtful home that isn’t just smart, it’s also helpful and simple enough for everyone to set up and use. It's technology designed for the way you live.

Ivy Ross + Hardware Design

Our goal with these new products, as always, is to create something that serves a purpose in people’s lives—products that are so useful they make people wonder how they ever lived without them. The simple yet beautiful design of these new devices continues to bring the smarts of the technology to the forefront, while providing people with a bold piece of hardware.

Our guiding principle

Google's guiding principle is the same as it’s been for 20 years—to respect our users and put them first. We feel a deep responsibility to provide you with a helpful, personal Google experience, and that guides the work we do in three very specific ways:

  • First, we want to provide you with an experience that is unique to you. Just like Google is organizing the world’s information, the combination of AI, software and hardware can organize your information—and help out with the things you want to get done. The Google Assistant is the best expression of this, and it’s always available when, where, and however you need it.

  • Second, we’re committed to the security of our users. We need to offer simple, powerful ways to safeguard your devices. We’ve integrated Titan™ Security, the system we built for Google, into our new mobile devices. Titan™ Security protects your most sensitive on-device data by securing your lock screen and strengthening disk encryption.

  • Third, we want to make sure you’re in control of your digital wellbeing. From our research, 72 percent of our users are concerned about the amount of time people spend using tech. We take this very seriously and have developed new tools that make people’s lives easier and cut back on distractions.

A few new things made by Google

With these Made by Google devices, our goal is to provide radically helpful solutions. While it’s early in the journey, we’re taking an end-to-end approach to consumer technology that merges our most innovative AI with intuitive software and powerful hardware. Ultimately, we want to help you do more with your days while doing less with your tech—so you can focus on what matters most.

Google’s Next Generation Music Recognition



In 2017 we launched Now Playing on the Pixel 2, using deep neural networks to bring low-power, always-on music recognition to mobile devices. In developing Now Playing, our goal was to create a small, efficient music recognizer which requires a very small fingerprint for each track in the database, allowing music recognition to be run entirely on-device without an internet connection. As it turns out, Now Playing was not only useful for an on-device music recognizer, but also greatly exceeded the accuracy and efficiency of our then-current server-side system, Sound Search, which was built before the widespread use of deep neural networks. Naturally, we wondered if we could bring the same technology that powers Now Playing to the server-side Sound Search, with the goal of making Google’s music recognition capabilities the best in the world.

Recently, we introduced a new version of Sound Search that is powered by some of the same technology used by Now Playing. You can use it through the Google Search app or the Google Assistant on any Android phone. Just start a voice query, and if there’s music playing near you, a “What’s this song?” suggestion will pop up for you to press. Otherwise, you can just ask, “Hey Google, what’s this song?” With this latest version of Sound Search, you’ll get faster, more accurate results than ever before!
Now Playing versus Sound Search
Now Playing miniaturized music recognition technology such that it was small and efficient enough to be run continuously on a mobile device without noticeable battery impact. To do this we developed an entirely new system using convolutional neural networks to turn a few seconds of audio into a unique “fingerprint.” This fingerprint is then compared against an on-device database holding tens of thousands of songs, which is regularly updated to add newly released tracks and remove those that are no longer popular. In contrast, the server-side Sound Search system is very different, having to match against ~1000x as many songs as Now Playing. Making Sound Search both faster and more accurate with a substantially larger musical library presented several unique challenges. But before we go into that, a few details on how Now Playing works.

The Core Matching Process of Now Playing
Now Playing generates the musical “fingerprint” by projecting the musical features of an eight-second portion of audio into a sequence of low-dimensional embedding spaces consisting of seven two-second clips at 1 second intervals, giving a segmentation like this:
Now Playing then searches the on-device song database, which was generated by processing popular music with the same neural network, for similar embedding sequences. The database search uses a two phase algorithm to identify matching songs, where the first phase uses a fast but inaccurate algorithm which searches the whole song database to find a few likely candidates, and the second phase does a detailed analysis of each candidate to work out which song, if any, is the right one.
  • Matching, phase 1: Finding good candidates: For every embedding, Now Playing performs a nearest neighbor search on the on-device database of songs for similar embeddings. The database uses a hybrid of spatial partitioning and vector quantization to efficiently search through millions of embedding vectors. Because the audio buffer is noisy, this search is approximate, and not every embedding will find a nearby match in the database for the correct song. However, over the whole clip, the chances of finding several nearby embeddings for the correct song are very high, so the search is narrowed to a small set of songs which got multiple hits.
  • Matching, phase 2: Final matching: Because the database search used above is approximate, Now Playing may not find song embeddings which are nearby to some embeddings in our query. Therefore, in order to calculate an accurate similarity score, Now Playing retrieves all embeddings for each song in the database which might be relevant to fill in the “gaps”. Then, given the sequence of embeddings from the audio buffer and another sequence of embeddings from a song in the on-device database, Now Playing estimates their similarity pairwise and adds up the estimates to get the final matching score.
It’s critical to the accuracy of Now Playing to use a sequence of embeddings rather than a single embedding. The fingerprinting neural network is not accurate enough to allow identification of a song from a single embedding alone — each embedding will generate a lot of false positive results. However, combining the results from multiple embeddings allows the false positives to be easily removed, as the correct song will be a match to every embedding, while false positive matches will only be close to one or two embeddings from the input audio.

Scaling up Now Playing for the Sound Search server
So far, we’ve gone into some detail of how Now Playing matches songs to an on-device database. The biggest challenge in going from Now Playing, with tens of thousands of songs, to Sound Search, with tens of millions, is that there are a thousand times as many songs which could give a false positive result. To compensate for this without any other changes, we would have to increase the recognition threshold, which would mean needing more audio to get a confirmed match. However, the goal of the new Sound Search server was to be able to match faster, not slower, than Now Playing, so we didn’t want people to wait 10+ seconds for a result.

As Sound Search is a server-side system, it isn’t limited by processing and storage constraints in the same way Now Playing is. Therefore, we made two major changes to how we do fingerprinting, both of which increased accuracy at the expense of server resources:
  • We quadrupled the size of the neural network used, and increased each embedding from 96 to 128 dimensions, which reduces the amount of work the neural network has to do to pack the high-dimensional input audio into a low-dimensional embedding. This is critical in improving the quality of phase two, which is very dependent on the accuracy of the raw neural network output.
  • We doubled the density of our embeddings — it turns out that fingerprinting audio every 0.5s instead of every 1s doesn’t reduce the quality of the individual embeddings very much, and gives us a huge boost by doubling the number of embeddings we can use for the match.
We also decided to weight our index based on song popularity - in effect, for popular songs, we lower the matching threshold, and we raise it for obscure songs. Overall, this means that we can keep adding more (obscure) songs almost indefinitely to our database without slowing our recognition speed too much.

Conclusion
With Now Playing, we originally set out to use machine learning to create a robust audio fingerprint compact enough to run entirely on a phone. It turned out that we had, in fact, created a very good all-round audio fingerprinting system, and the ideas developed there carried over very well to the server-side Sound Search system, even though the challenges of Sound Search are quite different.

We still think there’s room for improvement though — we don’t always match when music is very quiet or in very noisy environments, and we believe we can make the system even faster. We are continuing to work on these challenges with the goal of providing the next generation in music recognition. We hope you’ll try it the next time you want to find out what song is playing! You can put a shortcut on your home screen like this:
Acknowledgements
We would like to thank Micha Riser, Mihajlo Velimirovic, Marvin Ritter, Ruiqi Guo, Sanjiv Kumar, Stephen Wu, Diego Melendo Casado‎, Katia Naliuka, Jason Sanders, Beat Gfeller, Christian Frank, Dominik Roblek, Matt Sharifi and Blaise Aguera y Arcas‎.

Source: Google AI Blog


Fill in the blanks with #teampixel

Sometimes the best photos are the ones that don’t tell the whole story, daring you to fill in the blanks. And when it comes to simple, yet thought-provoking imagery, there’s no one better than #teampixel. 

Consider this week’s mix of minimalist photos—a white fluff dreaming cat dreams, a floating jellyfish heading deeper down under, and a peep of curiosity before descending the stairs. What’s going on in those photos? And what about behind the camera? You decide:

The next time you capture something with your Pixel, tag it with #teampixel and you could see your work featured on @google, @madebygoogle and The Keyword.