Today’s blog features Philip Haeusser, a PhD Research Intern at Google. Read on to learn about his projects, publishing at Google, coding, and his internship's impact. Enjoy!
So tell us about yourself and your PhD topic…
Hi! My name is Philip, and I’m a third-year PhD student in computer science at TU Munich, supervised by Daniel Cremers. I am working in the field of computer vision, the discipline where we teach computers to understand images and videos. To a computer, images and videos are nothing but a huge collection of meaningless numbers. If you represent them as colors, a human is immediately able to tell what’s in the picture.
In order to get a computer to achieve the same, I train neural networks — a family of models that can be interpreted as instances of a “mini visual cortex.” The goal is to map the many numbers that make up an image to something more meaningful, such as a class label like “cat.” Neural networks are amazing at this. I have worked on problems like optical flow (“what changes from one video frame to the next?”) or domain adaptation [“how can we use knowledge (labels) from one domain (e.g. handwritten images) on another domain (e.g. house numbers from Google street view)?”].
When I’m not doing research, I work on my YouTube channel “Phil’s Physics” where I present experiments and talk about science.
How did you get to work in this area?
In 2014, I was completing my Master’s in physics at the University of California in Santa Cruz. I was part of an interdisciplinary team working on retina implants for blind people. In one of our experiments, we had to deal with a lot of data that was very expensive to get — but we couldn’t use all of it because our data processing pipeline was not complex enough. So I started to read about machine learning and neural networks. I got immediately hooked and reached out to professors who were working in this area. It was a great honor to get invited to present my work to Daniel Cremers, who then offered me a PhD position at his chair.
Why did you apply for an internship at Google and how supportive was your PhD advisor?
The field of deep learning is moving very fast. Almost every week, a new paper on some new groundbreaking neural network or training trick appears. More often than not, the authors work at Google. That got me interested in the kind of work that Google is doing in this field. At a summer school, I met Olivier Bousquet, who gave an amazing talk about the Google Brain team. He told me about research internships at Google, and then I applied. My PhD advisor liked the idea, because it’s always good to get new perspectives, to connect with people and to engage in exchange, particularly in a new field like deep learning. Plus, Google has the resources to facilitate experiments that are computationally unfeasible at many universities.
What project was your internship focused on?
I had the honor to be working with Alexander Mordvintsev, one of the creators of DeepDream. The project was on a novel method of training neural networks with unlabeled data and semi-supervised learning.
We developed a new method that we called “Learning by Association.” It’s similar to the “association game,” where you’re told a word and you respond with the first thing that you associate with it. After a few “iterations,” you usually get very funny “association chains.”
We did something similar: We trained a neural network to produce representations (neural activation patterns) that allow for associations, too. Associations from labeled data to unlabeled data. Imagine an association chain from an example of the labeled batch to an example of the unlabeled batch. Then, you make a second association from unlabeled to labeled data. That would then be an “association cycle.” You can now compare the label of the example that you ended up at with the label of the example at the beginning of the cycle. The goal is to make consistent association cycles, meaning that the labels are the same. We formulated this as a cost function and showed that this technique works extremely well for training classification networks with less labeled data.
Did you publish at Google during your internship?
Yes, I wrote a paper and submitted it to CVPR, the biggest computer vision conference in the world. I even presented on this topic at the 2017 CVPR conference. And Google funded my conference trip, so this internship has not only brought me great industry experience, but also a publication that I am very proud of.
How closely connected was the work you did during your internship to your PhD topic?
My host and I were exploring many topics in the beginning and this particular one fits perfectly in my PhD. Besides, I think that even projects orthogonal to a PhD topic can help a lot, with regard to coding and project management skills.
Did you write your own code?
Tons! I am particularly grateful for the code reviews that helped me improve code quality and made me think more about style and scalability. I am also very happy that Google allowed me to open-source my code, as is common practice in the research world. This way, I can continue to work on the topic and share code with researchers all across the globe. I was able to write a follow-up paper on “Associative Domain Adaptation,” which just got accepted for ICCV.
This is your second internship at Google. What were the reasons to come back to Google Zurich?
There are so many interesting research projects at Google that only one internship is not enough :-). No, seriously, I was offered a really interesting project on the team of Sylvain Gelly. I had already met him and his colleagues last year. They are amazing people, and I couldn’t miss out on the chance to work with them.
What key skills have you gained from your time at Google?
I think my code quality improved a lot. At Google, you have the chance to learn from brilliant coders with a lot of experience. They write smart code, fast. And they help you do the same.
But I also connected with many people from different teams and backgrounds. I met amazing product managers who gave me insights into how they design successful products. I was even able to work with them on new YouTube features since I am also a YouTube creator. And last but not least, I worked on my metabolic endurance with the Tough Mudder team at Google’s internal fitness center.
What impact has this internship experience had on your PhD?
Looking at it in retrospect, my internship was an important milestone for my PhD. I didn’t apply with the expectation of it being a total game changer, but I think I returned back to university with a ton of new ideas and inspiration.
Looking back on your experiences now: Why should a PhD student apply for an internship at Google? Any advice to offer?
When you’re in the middle of your PhD, there will be this moment when you think you’re trapped and you need to see something new. Or when you realize that you would like to try an experiment that requires 1000 GPUs. Or when you think you know it all and you want to challenge yourself. Or when you think about the time after your PhD and you wonder what it would be like to write code at scale and make a huge impact. I think there are many reasons for an internship. Google is a fun place to try yourself out and maybe come home with a nice paper, new friends, new ideas, or even a job offer.