Tag Archives: Ask a Techspert

Ask a Techspert: How does motion sensing work?

Editor’s Note: Do you ever feel like a fish out of water? Try being a tech novice and talking to an engineer at a place like Google. Ask a Techspert is a series on the Keyword asking Googler experts to explain complicated technology for the rest of us. This isn’t meant to be comprehensive, but just enough to make you sound smart at a dinner party. 

Thanks to my allergies, I’ve never had a cat. They’re cute and cuddly for about five minutes—until the sneezing and itching set in. Still, I’m familiar enough with cats (and cat GIFs) to know that they always have a paw in the air, whether it’s batting at a toy or trying to get your attention. Whatever it is they’re trying to do, it often looks like they’re waving at us. So imagine my concern when I found out that you can now change songs, snooze alarms or silence your phone ringing on your Pixel 4 with the simple act of waving. What if precocious cats everywhere started unintentionally making us sleep late by waving their paws?

Fortunately, that’s not a problem. Google’s motion sensing radar technology—a feature called Motion Sense in the Pixel 4—is designed so that only human hands, as opposed to cat paws, can change the tracks on your favorite playlist. So how does this motion sensing actually work, and how did Google engineers design it to identify specific motions? 

To answer my questions, I found our resident expert on motion sensors, Brandon Barbello. Brandon is a product manager on our hardware team and he helped me unlock the mystery behind the motion sensors on your phone, and how they only work for humans. 

When I’m waving my hand in front of my screen, how can my phone sense something is there? 

Brandon tells me that your Pixel phone has a chip at the top with a series of antennas, some of which emit a radio signal and others that receive “bounce backs” of the same signal the other antenna emitted. “Those radio signals go out into the world, and then they hit things and bounce back. The receiver antennas read the signals as they bounce back and that’s how they’re able to sense something has happened. Your Pixel actually has four antennas: One that sends out signals, and three that receive.”

What happens after the antenna picks up the motion? 

According to Brandon, when the radio waves bounce back, the computer in your phone begins to process the information. “Essentially, the sensor picks up that you’re around, and that triggers your phone to keep an eye out for the relevant gestures,” he says.

How does the Pixel detect that a motion is a swipe and not something else? 

With the motion sensing functions on the Pixel, Brandon and his team use machine learning to determine what happened. “Those radio waves get analyzed and reduced into a series of numbers that can be fed into the machine learning models that detect if a reach or a swipe has just happened,” Brandon says. “We collected millions of motion samples to pre-train each phone to recognize intentional swipes. Specifically, we’ve trained the models to detect motions that look like they come from a human hand, and not, for instance, a coffee mug passing over the phone as you put it down on the table.”

What will motion sensors be capable of in the future? 

Brandon told me that he and his team plan to add more gestures to recognize beyond swiping, and that specific movements could be connected to more apps. “In the future, we want to create devices that can understand your body language, so they’re more intuitive to use and more helpful,” he tells me. 

At the moment, motion-sensing technology is focused on the practical, and there’s still improvements to be made and new ground to cover, but he says this technology can also be delightful and fun—like on the Pixel’s gesture-controlled Pokémon Live Wallpaper. Overall, motion sensing technology helps you use your devices in a whole new way, and that will keep changing as the tech advances. "We're just beginning to see the potential of motion sensing," Brandon says.

Ask a Techspert: What is machine learning?

Editor’s Note: Do you ever feel like a fish out of water? Try being a tech novice and talking to an engineer at a place like Google. Ask a Techspert is a series on the Keyword asking Googler experts to explain complicated technology for the rest of us. This isn’t meant to be comprehensive, but just enough to make you sound smart at a dinner party. 

Imagine you’re going to the grocery store to buy ice cream. If you’re an ice cream lover like me, this probably happens regularly. Normally, I go to the store closest to my home, but every so often I opt to go to a different one, in search of my ice-cream white whale: raspberry chocolate chip. 

When you’re in a new store searching for your favorite-but-hard-to-find flavor of ice cream, you might not know exactly where it is, but you’ll probably know that you should head toward the refrigerators, it’s in the aisle labeled frozen foods and that it’s probably not in the same section as the frozen pizza.

My ability to find ice cream in a new store is not instinctive, even though it feels like it. It is the result of years of memories navigating the many sections and aisles of different grocery stores, using visual cues like refrigerators or aisle signs to figure out if I am on the right track. 

Today, when we hear about “machine learning,” we’re actually talking about how Google teaches computers to use existing information to answer questions like: Where is the ice cream? Or, can you tell me if my package has arrived on my doorstep? For this edition of Ask a Techspert, I spoke with Rosie Buchanan, who is a senior software engineer working on Machine Perception within Google Nest. 

She not only helped explain how machine learning works, she also told me that starting today, Nest Aware subscribers can receive a notification when their Nest Hello, using machine learning, detects that a package has been delivered. 

What is machine learning? 

I’ll admit: Rosie came up with the food metaphor. She told me that when you’re looking for something to eat, you have a model in your head. “You learn what to eat by seeing, smelling, touching and by using your prior experience with similar things,” she says. “With machine learning, we’re teaching the computer how to do something, often with better accuracy than a person, based on past understanding.” 

How do you get a machine to learn? 

Rosie and her team teach machines through supervised learning. To help Nest cameras identify packages, they use data that they know contains the “right answers,” which in this case are photos of packages. They then input these data sets to the computer so that it can create an algorithmic model based on the images they provided. This is called a training job, and it requires hundreds of thousands of images. “Over time, the computer is able to independently identify a delivered package without assistance,” Rosie says. 

How do you figure out what to make a machine learn? 

Rosie told me that package detection was one of the most requested features from Nest Hello users. “In particular, we’re trying to solve problems based on what users want,” she says. “Home safety and security is a huge area for our users.” By bringing package delivery notifications to Nest Aware, Rosie and her team have found a use for machine learning that eliminates the tedious task of waiting around for your delivery. 

Do you need a massive supercomputer to do machine learning? 

That depends on whether you’re creating a machine learning model or using it. If you’re a developer like Rosie, you’ll need some powerful computers. But if you want to see whether there’s a package on your doorstep, you don’t need more than a video doorbell. "When engineers develop a machine learning model, it can take a ton of computing power to teach it what it needs to know,” Rosie says. “But once it's trained, a machine learning model doesn't necessarily take up a lot of space, so it can run basically anywhere, like in your smart doorbell."

Can machines understand some things that we humans can’t? 

According to Rosie, yes. “We can often describe the things we’re learning,” she says, “but there are things we can’t describe, and machines are good at understanding these observations.” It’s called black box learning: We can tell the model is learning something but we can’t quite tell what it is. 

A great example of this is when a package arrives at your doorstep. Rosie’s team shows the network lots of pictures of packages, and lots of pictures of other things (trees, dogs, bananas, you name it). They tell the network which images are packages and which ones are not. The network is made up of different nodes, each trying to learn how to identify a package on its own. One node might learn that many packages are brown, and another might notice that many are rectangular. 

“These nodes work together to start putting together a concept of what a package is, eventually coming up with a concept of ‘packageness’ that we as humans might not even understand,” Rosie says. “At the end, we don't actually know exactly what the network learned as its definition of ‘packageness,’ whether it's looking for a brown box, a white bag or something else.” With machine learning, teams can show a network a new picture and it may tell us there’s a package in it, but we can’t fully know exactly how it made that decision. 

What’s the best part about working on machine learning? 

Rosie, who’s been at Google for over five years, says it’s all about working on the unknown. “We get to work on problems that we don’t know are actually solvable,” she says. “It’s exciting to get started on something while knowing that it might not be feasible.” 

So will machine learning be able to identify that raspberry chocolate chip is the best flavor of ice cream ever created? Probably not. We’ll still need human knowledge to confirm that. But machine learning will help us in other ways, like waiting around for a package to be delivered so you can take that precious time to peruse the frozen foods section. 

Ask a Techspert: How does Wi-Fi actually work?

Editor’s Note: Do you ever feel like a fish out of water? Try being a tech novice and talking to an engineer at a place like Google. Ask a Techspert is a series on the Keyword asking Googler experts to explain complicated technology for the rest of us. This isn’t meant to be comprehensive, but just enough to make you sound smart at a dinner party. 

How do you define a best friend?  Is it that someone who understands your needs? Or maybe it’s the person who is there through your ups and downs. Or, perhaps, does it require a special ability to allow your electronic devices to connect to the web without cords? 

While there aren’t many people who immediately consider wireless routers their bestie, according to a recent study commissioned by Google and conducted by Kelton Research, 57 percent of respondents say their Wi-Fi is like their best friend. In fact, 25 percent compared Wi-Fi to their significant other, and 68 percent said they’d be lonelier without Wi-Fi. And respondents said they’d rather suffer annoying situations like long lines at the DMV than deal with spotty Wi-Fi connections. 

Certainly, Wi-Fi is part of our daily lives, but how does it actually work? For this edition of “Ask A Techspert,” I spoke with Sanjay Noronha, a product manager at Google Nest and our resident expert on Wi-Fi and routers, to learn more about how the technology behind Wi-Fi works and about the future of home networks.

  • Wi-Fi Survey 1.jpg
  • Wi-Fi Survey 2.jpg
  • Wi-Fi Survey 3.jpg
  • Wi-Fi Survey 4.jpg

How does Wi-Fi even work? 

“It’s like listening to the radio, but two-way. Instead of just receiving sound like we do with AM or FM, Wi-Fi also lets you send data, like an email or a post to social media,” Sanjay told me. “Wi-Fi sends the data over radio waves quickly and reliably so that the thing you’re trying to do, or video you’re trying to stream, or game you’re trying to play, happens in a seamless way so you’re not stuck to your wall with an ethernet cable.” 

Wi-Fi operates on 2.4GHz and 5GHz radio frequencies. Think of those numbers like tuning your car to 97.9 FM to hear your favorite station. Except you don’t actually need to set anything yourself. Your Wi-Fi router decides which radio station to put your devices on so you can watch YouTube videos on your smartphone or take a video call while moving around your house. Multiple Wi-Fi networks can exist on the same frequencies, which is why you might see your neighbors’ networks when you try to connect on your device. (And respondents to our survey know this well: 13 percent said they have tried to connect to another network in their area, and five percent have asked their neighbors if they could tap into their Wi-Fi.)

Why does my Wi-Fi slow down at certain times? 

The overwhelming majority (81 percent) of router users in our survey have experienced issues with their home Wi-Fi. Among people who experience issues, half reported dealing with a slow connection, and 43 percent report slower speeds during certain times of day. 

I live in New York City and sometimes, particularly at night, my Wi-Fi gets particularly slow. And that’s because other New Yorkers are trying to stream their favorite TV shows, too. “That’s Wi-Fi congestion,” Sanjay told me. “If you have multiple Wi-Fi networks operating at once in the same area, they’re all using the same frequency ranges.” 

But if you use Google Wifi, there’s a way to avoid that problem. Wi-Fi was originally built for only 2.4 GHz, then newer Wi-Fi technology also added 5 GHz channels.  (If you see a wireless network with the number 5 at the end, that’s what that means.) That means you sometimes may have to pick which one to connect to when you’re online. But with Google Wifi, the experience is simplified. Users just connect to one network and are automatically moved between channels with a technology called “band steering.” Google Wifi also seamlessly selects the Wi-Fi frequencies it uses,  depending on the congestion, so you can binge-watch without interruption. 

How come some parts of my home get better Wi-Fi? 

According to Sanjay, that depends on your router. “A single router is like a lightbulb,” he says, noting a lightbulb has a limited range of light, and a router has a limited range of signal. “Just like you have multiple lightbulbs throughout your house, we want to make it easy for you to put in multiple routers.” 

Google Wifi is “mesh technology,” and it enables you to get better Wi-Fi by putting additional Wi-Fi routers throughout your home. So it’s like having multiple lightbulbs in your house, instead of expecting one lightbulb by your front door to illuminate your attic. Having a mesh system helps spread Wi-Fi signals throughout your home, wherever you’re using Wi-Fi. 

“Even though Wi-Fi has been around for many years, many people still experience Wi-Fi that cuts out,” Sanjay says. “We’re applying our years of experience to make Wi-Fi even more accessible everywhere in your home, not just in the room with the router.” 

Wi-Fi survey

Even though Wi-Fi might be like your best friend, some people have an odd way of showing it. According to our study, router users go to great lengths to hide their routers. Over two in five router users confess they’ve attempted to hide their networking device because of its appearance. So, we designed Google Wifi to look different from a traditional router. Instead of clunky cords and external antennas, Google Wifi is sleek and compact, so you may not mind having it hang out on you counter or shelf for the best connection possible. That way  you can hang out with your best friend, anywhere in the house, without worrying about making the place look neat.