Tag Archives: Pixel

Take Your Best Selfie Automatically, with Photobooth on Pixel 3



Taking a good group selfie can be tricky—you need to hover your finger above the shutter, keep everyone’s faces in the frame, look at the camera, make good expressions, try not to shake the camera and hope no one blinks when you finally press the shutter! After building the technology behind automatic photography with Google Clips, we asked ourselves: can we bring some of the magic of this automatic picture experience to the Pixel phone?

With Photobooth, a new shutter-free mode in the Pixel 3 Camera app, it’s now easier to shoot selfies—solo, couples, or even groups—that capture you at your best. Once you enter Photobooth mode and click the shutter button, it will automatically take a photo when the camera is steady and it sees that the subjects have good expressions with their eyes open. And in the newest release of Pixel Camera, we’ve added kiss detection to Photobooth! Kiss a loved one, and the camera will automatically capture it.

Photobooth automatically captures group shots, when everyone in the photo looks their best.
Photobooth joins Top Shot and Portrait mode in a suite of exciting Pixel camera features that enable you to take the best pictures possible. However, unlike Portrait mode, which takes advantage of specialized hardware in the back-facing camera to provide its most accurate results, Photobooth is optimized for the front-facing camera. To build Photobooth, we had to solve for three challenges: how to identify good content for a wide range of user groups; how to time the shutter to capture the best moment; and how to animate a visual element that helps users understand what Photobooth sees and captures.

Models for Understanding Good Content
In developing Photobooth, a main challenge was to determine when there was good content in either a typical selfie, in which the subjects are all looking at the camera, or in a shot that includes people kissing and not necessarily facing the camera. To accomplish this, Photobooth relies on two distinct models to capture good selfies—a model for facial expressions and a model to detect when people kiss.

We worked with photographers to identify five key expressions that should trigger capture: smiles, tongue-out, kissy/duck face, puffy-cheeks, and surprise. We then trained a neural network to classify these expressions. The kiss detection model used by Photobooth is a variation of the Image Content Model (ICM) trained for Google Clips, fine tuned specifically to focus on kissing. Both of these models use MobileNets in order to run efficiently on-device while continuously processing the images at high frame rate. The outputs of the models are used to evaluate the quality of each frame for the shutter control algorithm.

Shutter Control
Once you click the shutter button in Photobooth mode, a basic quality assessment based on the content score from the models above is performed. This first stage is used as a filter that avoids moments that either contain closed eyes, talking, or motion blur, or fail to detect the facial expressions or kissing actions learned by the models. Photobooth temporally analyzes the expression confidence values to detect their presence in the photo, making it robust to variations in the output of machine learning (ML) models. Once the first stage is successfully passed, each frame is subjected to a more fine-grained analysis, which outputs an overall frame score.

The frame score considers both facial expression quality and the kiss score. As the kiss detection model operates on the entire frame, its output can be used directly as a full-frame score value for kissing. The face expressions model outputs a score for each identified expression. Since a variable number of faces may be present in each frame, Photobooth applies an attention model using the detected expressions to iteratively compute an expression quality representation and weight for each face. The weighting is important, for example, to emphasize the expressions in the foreground, rather than the background. The model then calculates a single, global score for the quality of expressions in the frame.

The final image quality score used for triggering the shutter is computed by a weighted combination of the attention based facial expression score and the kiss score. In order to detect the peak quality, the shutter control algorithm maintains a short buffer of observed frames and only saves a shot if its frame score is higher than the frames that come after it in the buffer. The length of the buffer is short enough to give users a sense of real time feedback.

Intelligence Indicator
Since Photobooth uses the front-facing camera, the user can see and interact with the display while taking a photo. Photobooth mode includes a visual indicator, a bar at the top of the screen that grows in size when photo quality scores increase, to help users understand what the ML algorithms see and capture. The length of the bar is divided into four distinct ranges: (1) no faces clearly seen, (2) faces seen but not paying attention to the camera, (3) faces paying attention but not making key expressions, and (4) faces paying attention with key expressions.

In order to make this indicator more interpretable, we forced the bar into these ranges, which prevented the bar scaling from being too rapid. This resulted in smooth variability of the bar length as the quality score changes and improved the utility. When the indicator bar reaches a length representative of a high quality score, the screen flashes to signify that a photo was captured and saved.
Using ML outputs directly as intelligence feedback results in rapid variation (left), whereas specifying explicit ranges creates a smooth signal (right).
Conclusion
We’re excited by the possibilities of automatic photography on camera phones. As computer vision continues to improve, in the future we may generally trust smart cameras to select a great moment to capture. Photobooth is an example of how we can carve out a useful corner of this space—selfies and group selfies of smiles, funny faces, and kisses—and deliver a fun and useful experience.

Acknowledgments
Photobooth was a collaboration of several teams at Google. Key contributors to the project include: Kojo Acquah, Chris Breithaupt, Chun-Te Chu, Geoff Clark, Laura Culp, Aaron Donsbach, Relja Ivanovic, Pooja Jhunjhunwala, Xuhui Jia, Ting Liu, Arjun Narayanan, Eric Penner, Arushan Raj, Divya Tyam, Raviteja Vemulapalli, Julian Walker, Jun Xie, Li Zhang, Andrey Zhmoginov, Yukun Zhu.

Source: Google AI Blog


Suit up with Marvel Studios’ Avengers: Endgame and Pixel

Want to defeat a villain like Thanos and save the world?


Now #teampixel can, with a little help from Marvel Studios and Playground, a creative mode in the Pixel camera that gives you the power to create and play with the world around you using augmented reality. Just in time for the upcoming release of Marvel Studios’ “Avengers: Endgame,” in theaters April 26, today we’re adding to our collection of Playmoji from the Marvel Cinematic Universe with five new characters: War Machine, Thor, Black Widow, Rocket and Captain Marvel.


The heroes join Iron Man, Captain America, Hulk, Nebula and Okoye in Playground, so now you can make even more epic scenes come to life by adding the interactive characters to your photos and videos. Thanks to ARCore’s motion tracking, light estimation and ability to understand the real world, the Playmoji look and feel lifelike, and react to your facial expressions in real time.

You don’t need superhero strength or a suit of armor to unleash the power of Playground—you just need a Pixel and the newest Playmoji joining the Marvel Studios’ Avengers: Endgame pack.


For some added fun, we reimagined the Marvel Cinematic Universe by exploring what would happen if Pixel 3 was launched into a world in need of a little assistance, alongside the Avengers.

Pixel 3 + Marvel Studios‘ Avengers: Endgame

So whether you’re suiting up to defeat Thanos or getting ready to supercharge your selfie, start saving the world alongside your favorite Playmoji using Playground today. Show us how you’re assembling to defeat Thanos on social with #pixelendgame.

See Brighter in the Dark with Pixel at Coachella

This weekend, Coachella festival-goers will wander the desert, heading to and from performances by some of their favorite artists. If you're one of them, head over to Brighter in the Dark—a tech and music installation we created with Childish Gambino.


The installation—a multisensory sight and sound experience that uses a mixture of light and music to give attendees a sneak peek into Childish Gambino’s creative world—uses the power of Pixel’s camera features to provide an interactive and Instagrammable moment for festival-goers.

Fans can also swing by Brighter in the Dark to get a sneak peek of Childish Gambino’s  augmented reality (AR) app that we’re previewing onsite at Coachella on Pixel 3. Not heading to Indio? Stay tuned for more info, as the AR app is coming to more devices soon!

Whether you’re bringing your Pixel with you to weekend one or weekend two, here are a few ways your camera can help you capture the beauty of the desert and the sights of Coachella.

Night Sight: Use Pixel’s low-light mode, Night Sight, to capture the best evening photos at Coachella, and experience the cool visuals at Brighter in the Dark.

Motion Auto Focus: Don’t let your dancing prevent you from taking amazing  photos! Your Pixel can be your mosh-pit companion—with Motion Auto Focus, videos and photos that you take of your favorite headliners will stay in focus, even while the performers are moving around the stage.

Group Selfie: Traveling in a pack to Coachella? Get your whole crew in the picture with Pixel 3’s wide angle selfie cam in Group Selfie mode.

Google Lens: Inspired by a fellow fan’s festival fashion? Use Lens with a long-press on the Camera’s viewfinder to search for similar items.

Don’t forget to share your Brighter in the Dark experience with #teampixel!

Rove around “Mars on Earth” in Street View

Devon Island, a desolate land mass in Canada’s Arctic with a polar climate and treacherous terrain, is the largest uninhabited island on Earth. Yet the factors that make the island unlivable also make it indispensable to the scientists and researchers who work there—its climate and landscape are the closest thing to Mars that can be found on Earth.  

Mars on Earth: A Visit to Devon Island

Now anyone can visit "Mars on Earth" in Street View. Last year, I received a special invite from Dr. Pascal Lee, chairman of the Mars Institute and director of the Haughton-Mars Project, to visit Devon Island and learn about the research done there. We spent three months preparing for the expedition, and after 72 hours on seven flights, found ourselves at basecamp surrounded by an untouched landscape.

Devon Island, much like a future base on Mars, lacks the infrastructure we take for granted. All the supplies needed for camp—food, gasoline, tools and personal supplies—must be brought along on each excursion, and all the waste packed up and brought back to the mainland. At the research base, everyone has their job. Even Dr. Lee’s dog KingKong has a responsibility—he’s there to serve as an advance warning in case a polar bear wanders into camp.


Every morning, before heading out to collect Street View on ATVs, we would brief as a group to make sure everybody knew the plan that day: who was leading, who would ride rear, and who was staying at camp to cook and handle maintenance. This provided a real insight into how humans who will go to Mars will explore the new planet: detailed planning and preparation is key.

Visit Devon Island in Google Earth

Visit Devon Island on Google Earth

Throughout the week, we rode to some of the places of most interest to NASA’s research and exploration: Haughton Crater, an impact crater 20-kilometers in diameter; Astronaut Canyon, similar to many of the V-shaped, winding valleys on Mars; and the ancient lake beds of Breccia Hills. What strikes you most about Devon Island is how vast and desolate everything is. Yet every rock, hill and canyon tells a story. Breccia Hills, for example, is filled with shatter cones, rocks created by meteor impact millions of years ago.

We were also able to capture our experience on a Pixel 3, shooting the first-ever documentary filmed on Pixel to showcase just how majestic, and sometimes trying, training for a Mars Mission on Devon Island can be.


Explore “Mars on Earth” and learn about the work being done there in a new Google Earth guided tour.

Ask a Techspert: Why am I getting so many spam calls?

Editor’s Note: Do you ever feel like a fish out of water? Try being a tech novice and talking to an engineer at a place like Google. Ask a Techspert is a new series on the Keyword asking Googler experts to explain complicated technology for the rest of us. This isn’t meant to be comprehensive, but just enough to make you sound smart at a dinner party.

Growing up, I was taught to say “Schottenfels residence” when answering the phone. It was the polite way of doing things. When the phone rang, it was usually family, friends and, yes, the occasional telemarketer on the other side of the line. Then things changed. Personal calls moved to mobile phones, and the landline became the domain of robocalls. My cell was a sanctuary, free of the pesky automated dialers that plague the landlines of yore. Until recently.

Today, it feels like the only phone calls I get are spam calls. And I know I’m not alone. According to a recent Google survey, half of respondents received at least one spam call per day, and one third received two or more per day.

And people are answering those calls. More than one third of respondents worry that a call from an unknown number is a call about a loved one, and another third think it could be a call from a potential loved one, so they pick up. And almost everyone agrees: Spam calls are the worst. In fact, 75 percent of those surveyed think spam calls are more annoying than spam texts or emails.

So what’s the deal with spam calls? And how can we stop them from happening? For the latest edition of Ask a Techspert, I spoke to Paul Dunlop, the product manager for the Google Phone App, to better understand why, all of the sudden, spam calls are happening so frequently, and what tools, like Pixel’s Call Screen feature, you can use  to avoid the headache.

Why spam calls are more common lately

According to Paul, voice-over IP (VoIP) is the culprit. These are phone calls made using the web instead of a traditional telephone line, and today they're cheaper and easier than ever to use. “Using VoIP technology, spammers place phone calls over the Internet and imitate a different phone number,” Paul says. “It used to be that they had a fixed number, and you could block that number. Now with VoIP, spammers have the ability to imitate any phone number.” Paul says this became possible when companies, which wanted to call customers from call centers, made it so one general 1-800 number for a business showed up on caller IDs. So what started as a common-sense solution ended up becoming an easy loophole for spammers.

This is called spoofing, and there’s nothing in phone systems—the infrastructure of telephones—that can prevent spam callers from imitating numbers. “You can actually be spammed by your own phone number,” Paul says. “But the most common is neighborhood spam, using your area code and the first three digits of your phone number, which increases the likelihood you’ll answer.”

How Pixel can help you avoid picking up spam calls

A video explaining the Call Screen feature on Pixel phones

Enter Call Screen, a feature on Pixel phones that helps protect you from spam calls by giving you more information before you decide to answer. Before you have to pick up, Call Screen asks the caller to say why they’re calling and, with the help of the Google Assistant, translates the message into text so you can decide whether or not to answer. All of this happens “on device,” meaning it protects your privacy while it makes sure you get the message as fast as possible.

“Call Screen gives you that bit of protection from those spam calls, and helps you make sure you don’t miss those really important calls,” Paul explains. “It’s only one piece of the puzzle though.”

The future of fighting spam calls

But what about the problem of spam calls at large? Paul and other industry techsperts look to technology called STIR/SHAKEN to address that spoof phone number technology, which the FTC is in the process of approving. And, yes, they are acronyms: STIR for “Secure Telephone Identity Revisited” and SHAKEN for “Signature-based Handling of Asserted information using toKENs.”

This new technology allows cell phone networks to authenticate calls by validating that the number associated with each phone call is legitimate. You can then know that the caller is a real person using a real phone number.

According to our survey respondents, spam calls are the worst type of call you can get. With new advances in technology, however, the number two most annoying group of callers—exes —might just take the top spot.

Roses are red, violets are blue: six Pixel camera tips, just for you

No matter what your plans are this Valentine’s Day, you’ll probably end up taking a few photos to celebrate or capture the moment—and that's where Pixel's camera comes in. Pixel 3's camera has tools that can help you capture and get creative with your V-Day photos. Here are six tips for our beloved #teampixel.

1. Virtual Valentines

Playground is a creative mode in the Pixel camera that helps you create and play with the world around you. You can send a virtual Valentine, or make your photos and videos stand out with the new Love Playmoji pack and two sticker packs. Capture and celebrate the love in the air today and year-round with interactive hearts, fancy champagne glasses, animated love notes or lovebirds.


2. A V-Day Vision

Your Valentine always stands out to you. So make them the center of focus with Portrait Mode, and watch as the background fades into a beautiful blur… just like the world does when you’re together.

3. Mood Lighting

Romantic dinner date? Use Night Sight to capture the mood when the lights are dim. Pixel’s camera can capture the highlights of your Valentine’s celebrations, even in low light.

Night Sight

4. Picture Perfect Palentines

If you’re celebrating with your Palentines, Group Selfie Cam on Pixel 3 gives everyone the love they deserve in your group selfie.

5. Search at First Sight

The technology that lets you search what you see is baked right into Pixel 3’s camera. See a shirt that would look great on your Galentine? Use Google Lens to find something similar online. Want to know what that flower is in your bouquet? Use Google Lens to identify it. Making a last-minute dinner reservation at that restaurant on the corner? Use Google Lens suggestions to dial the number on their sign with just a tap in the Pixel camera.

Lens

6. Sharing is Caring

With unlimited original quality photo and video storage using Google Photos on Pixel 3, you can snap as many shots as you want. From there, you can turn them into a movie or set up a live album, so you can relive (and share) your favorite Palentines’ moments year-round with your friends.

So whether you’re celebrating Valentine’s, Palentine’s or Galentine’s day, Pixel 3’s camera can help you capture your favorite moments with your favorite people.

From Doodling to Design: A Q&A with Nicola Formichetti

Nicola Formichetti is a designer, stylist and bonafide fashion icon. He’s created some of the most beautiful and awe-inspiring red carpet looks of the past decade—and now, he’s also the latest addition to #teampixel, thanks to his new collection of My Cases designed for Pixel 3 and Pixel 2 users.  


Nicola’s effortless and cool style pairs nicely with the sleek phone design, bringing a bright pop of color to Pixel devices. We caught up with Nicola to learn more about what inspired him to create his My Case collection and how technology shapes his current work.


What excited you about designing a My Case Artist collection?

I love that I was able to do my own design very quickly using Google tech—I used the Pixelbook and Pixelbook Pen to design this collection. I’m just one example of what fun you can have drawing with these new tools.


How does technology play a role in your creative process?

Technology is at the core of everything I do, from capturing my inspiration at the very beginning of the process to sharing the final product.


Tell us about what inspired your design of the phone cases?

When I was creating the concept for these cases with the Pixelbook Pen, I was reminded of when I was a kid and would doodle...everywhere! First on notebooks, then on walls, then eventually I began graffitiing. I’ve used pens on clothes before, but not on a phone case so it was an exciting process.  


What’s your favorite feature on Pixel 3?

The camera on the Pixel 3 is amazing and is able to capture photos and videos at an amazing quality. It’s perfect for taking a great selfie!


Welcome to #teampixel, Nicola!


Childish Gambino dances into Playground on Pixel

Playground gives you the power to create and play through your Pixel camera using augmented reality. You can bring your photos and videos to life by adding interactive characters called Playmoji to what’s around you, and now there’s a new Playmoji inspired by recording artist Childish Gambino. You can add him to your photos or videos by simply pointing your camera and dropping him into the scene.

childish gambino 2

Examples of Playmoji inspired by recording artist Childish Gambino. 

We worked closely with Childish Gambino and his music video choreographer, Sherrie Silver, to make sure the Playmoji’s dance moves rival those of Childish Gambino himself. By using ARCore’s motion tracking, light estimation, and ability to understand the real world, his Playmoji looks and feels lifelike, whether he’s in front of you or in a selfie next to you. He even reacts to your facial expressions in real time thanks to machine learning—try smiling or frowning in selfie mode and see how he responds.

gambino_better.gif

The Childish Gambino Playmoji pack features unique moves that map to three different songs: “Redbone,” “Summertime Magic,” and “This is America.” Pixel users can start playing with them today using the camera on their Pixel, Pixel XL, Pixel 2, Pixel 2 XL, Pixel 3 and Pixel 3 XL.

Google Pixel 3 | Playmoji Dance-Off

Think you have moves too? We want to bring you in on the fun so we're inviting #teampixel to a dance challenge. When you use the Childish Gambino Playmoji, bust out your best dance moves alongside him. The #pixeldanceoff is on—get moving!

Find more balance in your life this year, with help from Google

With our phones constantly buzzing and our inboxes filling up, it can sometimes feel like we’re always logged in. It’s easy to forget the importance of making deliberate choices about when we want to use our phones, and to know when we can take a much-needed break from screens.

Looking for more balance in your life this year? Here are some tools that will help you better understand how you’re currently using your phone, get more out your tech and carve out time to be a little more zen in 2019.

1. Take a look at your Digital Wellbeing dashboard.

Android Time Spent feature

The Digital Wellbeing dashboard on Android devices helps you understand how frequently you currently use different apps, how many notifications you receive and how often you unlock your phone. By looking at your usage over time, it’s easy to think about whether you’re getting value from the time spent on each activity and make changes.

2. Cut down on all that scrolling with app timers.

Android app timer

Once you’ve identified apps you’d like to use less often, you can set up app timers so your phone will nudge you when you’re close to your self-imposed limit. The app icon button will then gray out, with a notification to remind you of your goal, when you’ve exhausted the time limit you’ve set for yourself.

3. Use Flip to Shhh on Pixel 3.

Shhh mode on Pixel

For Pixel 3 users out there, if you turn your phone over on a table — like when you’re at dinner — your device automatically enters Do Not Disturb mode so you can focus on being present, not mindlessly checking sports scores or playing a game.

4. Create more family time with Family Link and the YouTube Kids app.

Digital Wellbeing for family and kids

If you have kids, Family Link and the YouTube Kids app allow you to set the digital ground rules for everyone in the household. With Family Link, you can view your children's activity, manage their apps, find apps recommended by teachers, set screen time limits and even lock their devices when it’s dinner or “go outside and play” time.

With the YouTube Kids app, you can decide whether or not your kids can use YouTube Kids search, keep tabs on the videos they’re watching and even block videos or channels you don’t want them to see—along with setting time limits for how long they can play with the app.

5. Get stuff done quickly and focus on what matters to you.

Great technology should improve your life, not distract from it, and a bunch of Google tools are here to help. The Google Assistant offers you downtime from screens by letting you to use your voice to send messages, control smart home devices and play music when you just want to chill. Google Photos automatically stylizes your photos for you, Android Auto minimizes distractions while you’re driving and Gmail’s Smart Compose already helps people save over a billion characters every week by suggesting words and phrases for you as you write.

6. Practice mindfulness and take a break.

Try searching for “mindfulness” in Google Play to download relaxing apps like Headspace, Calm, and many others to kickstart your wellbeing journey. You can also say to your Google Assistant, “I want to meditate” to get a bunch of app recommendations and healing sounds, and the recently updated Google Fit app now has guided breathing exercises for you, too.

7. Keep up with the #GetFitWithGoogle challenge.

With all this extra time, you might even have time to sneak in an extra run this week. We’re now three weeks into the #GetFitWithGoogle global challenge, with just one week to go as our influencer teams race to earn the most Heart Points during January with Google Fit.

Congrats to Colombia for holding onto the lead going into the final week!

Get Fit With Google leaderboard, week 3

Keep an eye on the #GetFitWithGoogle hashtag on Instagram and follow the teams below to follow their fitness journeys. Will Team Switzerland make a final dash for the line? Just one week to go before we announce the overall winners.

Don’t forget to share your own Heart Points progress using #GetFitWithGoogle to help others like you stay motivated.


Source: Android


Get more shut-eye in 2019 with help from Google

After a long day, it should be easy to hit the hay at night. But far too often, just as you’re about to drift off, you decide to check why your phone just buzzed...and you’re back to square one.

According to the National Sleep Foundation, most adults should aim to get between seven and nine hours of sleep every night. Many of us, including yours truly, usually don’t get there.

If you want a fresh start in the new year, here are some tech tips and healthy habits that will help you catch more zzzzs (and get better at counting sheep) in 2019.

1. Set up a bedtime schedule and stick to it with Wind Down.

Wind Down

If you find yourself endlessly scrolling through social feeds or trying to finish just one more level of your favorite game late at night, Wind Down on your Android phone can help you take back control. Wind Down automatically turns on Do Not Disturb and makes your phone less interesting by turning everything grayscale to help you get to sleep at the time you want.

2. Set up Quiet Hours on YouTube.

Quiet Hours on YouTube

Users on both iOS and Android devices can ask the YouTube app to silently send notifications to their phone during a specified time period each day. That means no more sounds or vibrations while you sleep. By default, all sounds and vibrations will be disabled between 10pm and 8am, but you can customize the start and end times to suit your schedule. And don’t worry, updates from your favorite creators will still be right there for you when you wake up.

3. Lull yourself to sleep with soothing sounds.

While complete silence is crucial for some people, others prefer consistent ambient noise to help them get to sleep. If you say “OK Google, help me relax”, the Google Assistant will randomly pick from more than 10 soothing sounds to quietly deliver you to the Land of Nod. You can also pick specific sounds by saying “play fireplace sounds” or “play white noise” once you work out which sounds work best for you.

4. Keep your phone out of reach, and out of mind.

Sunrise Alarm

5. Turn off your lights, and your Wi-Fi.

If you have trouble getting your kids to sleep, Google Wifi has family-friendly controls that allow you to schedule a regular Wi-Fi pause on your kids' devices. That way, your kids aren’t sneakily playing online games under the covers. (And neither are you.)

6. Keep up with the #GetFitWithGoogle challenge.

With all this extra sleep, you’ll have even more energy to spend on your other New Year resolutions like getting fit. For those following along at home from last week, here’s an update on the #GetFitWithGoogle global challenge — a four-week-long competition to see which country’s team of social influencers can earn the most Heart Points during January with Google Fit.

Congrats to Team Colombia for taking the lead on the global leaderboard after week 2!

Get Fit With Google leaderboard

Keep an eye on the #GetFitWithGoogle hashtag on Instagram and follow the teams below to follow their fitness journeys.

Don’t forget to share your own Heart Points progress using #GetFitWithGoogle to help others like you stay motivated.