Editor’s note: It’s hard to think of a more “analog” game than rock-paper-scissors. But this summer, one Googler decided to use TensorFlow, Google’s open source machine learning system, to build a machine that could play rock-paper-scissors. For more technical details and source code, see the original post on the Google Cloud Big Data and Machine Learning Blog.
This summer, my 12-year-old son and I were looking for a science project to do together. He’s interested in CS and has studied programming with Scratch, so we knew we wanted to do something involving coding. After exploring several ideas, we decided to build a rock-paper-scissors machine that detects a hand gesture, then selects the appropriate pose to respond: rock, paper, or scissors.
But our rock-paper-scissors machine had a secret ingredient: TensorFlow, which in this case runs a very simple ML algorithm that detects your hand posture through an Arduino micro controller connected to the glove.
To build the machine’s hardware, we used littleBits—kid-friendly kits that include a wide variety of components like LEDs, motors, switches, sensors, and controllers—to attach three bend sensors to a plastic glove. When you bend your fingers while wearing the glove, the sensors output an electric signal. To read the signals from the bend sensors and control the machine’s dial, we used an Arduino and Servo module.
After putting together the hardware component, we wrote the code to read data from the sensors. The Arduino module takes the input signal voltage it receives from the glove, then converts those signals to a series of three numbers.
The next step was to determine which combination of three numbers represents rock, paper or scissors. We wanted to do it in a way that could be flexible over time—for example if we wanted to capture more than three hand positions with many more sensors. So we created a linear model—a simple algebraic formula that many of you might have learned in high school or college—and used machine learning in TensorFlow to solve the formula based on the given sensor data and the expected results (rock, paper or scissors). What’s cool about this is that it’s like automated programming—we specify the input and output, and the computer generates the most important transformation in the middle.
Finally, we put it all together. Once we determine the hand’s posture, the Servo controls the machine hand to win the game. (Of course, if you were feeling competitive, you could always program the machine to let YOU win every time… but we would never do that. ?)
Rock-paper-scissors probably isn’t what comes to mind when you think about ML, but this project demonstrates how ML can be useful for all kinds of programmers, regardless of the task—reducing human coding work and speeding up calculations. In this case, it also provided some family fun!
