Tag Archives: Awards

The NeurIPS 2018 Test of Time Award: The Trade-Offs of Large Scale Learning

Progress in machine learning (ML) is happening so rapidly, that it can sometimes feel like any idea or algorithm more than 2 years old is already outdated or superseded by something better. However, old ideas sometimes remain relevant even when a large fraction of the scientific community has turned away from them. This is often a question of context: an idea which may seem to be a dead end in a particular context may become wildly successful in a different one. In the specific case of deep learning (DL), the growth of both the availability of data and computing power renewed interest in the area and significantly influenced research directions.

The NIPS 2008 paper “The Trade-Offs of Large Scale Learning” by Léon Bottou (then at NEC Labs, now at Facebook AI Research) and Olivier Bousquet (Google AI, Zürich) is a good example of this phenomenon. As the recent recipient of the NeurIPS 2018 Test of Time Award, this seminal work investigated the interplay between data and computation in ML, showing that if one is limited by computing power but can make use of a large dataset, it is more efficient to perform a small amount of computation on many individual training examples rather than to perform extensive computation on a subset of the data. This demonstrated the power of an old algorithm, stochastic gradient descent, which is nowadays used in pretty much all applications of DL.

Optimization and the Challenge of Scale
Many ML algorithms can be thought of as the combination of two main ingredients:
  • A model, which is a set of possible functions that will be used to fit the data.
  • An optimization algorithm which specifies how to find the best function in that set.
Back in the 90’s the datasets used in ML were much smaller than the ones in use today, and while artificial neural networks had already led to some successes, they were considered hard to train. In the early 2000’s, with the introduction of Kernel Machines (SVMs in particular), neural networks went out of fashion. Simultaneously, the attention shifted away from the optimization algorithms that had been used to train neural networks (stochastic gradient descent) to focus on those used for kernel machines (quadratic programming). One important difference being that in the former case, training examples are used one at a time to perform gradient steps (this is called “stochastic”), while in the latter case, all training examples are used at each iteration (this is called “batch”).

As the size of the training sets increased, the efficiency of optimization algorithms to handle large amounts of data became a bottleneck. For example, in the case of quadratic programming, running time scales at least quadratically in the number of examples. In other words, if you double your training set size, your training will take at least 4 times longer. Hence, lots of effort went into trying to make these algorithms scale to larger training sets (see for example Large Scale Kernel Machines).

People who had experience with training neural networks knew that stochastic gradient descent was comparably easier to scale to large datasets, but unfortunately its convergence is very slow (it takes lots of iterations to reach an accuracy comparable to that of a batch algorithm), so it wasn’t clear that this would be a solution to the scaling problem.

Stochastic Algorithms Scale Better
In the context of ML, the number of iterations needed to optimize the cost function is actually not the main concern: there is no point in perfectly tuning your model since you will essentially “overfit” to the training data. So why not reduce the computational effort that you put into tuning the model and instead spend the effort processing more data?

The work of Léon and Olivier provided a formal study of this phenomenon: by considering access to a large amount of data and assuming the limiting factor is computation, they showed that it is better to perform a minimal amount of computation on each individual training example (thus processing more of them) rather than performing extensive computation on a smaller amount of data.

In doing so, they also demonstrated that among various possible optimization algorithms, stochastic gradient descent is the best. This was confirmed by many experiments and led to a renewed interest in online optimization algorithms which are now in extensive use in ML.

Mysteries Remain
In the following years, many variants of stochastic gradient descent were developed both in the convex case and in the non-convex one (particularly relevant for DL). The most common variant now is the so-called “mini-batch” SGD where one considers a small number (~10-100) of training examples at each iteration, and performs several passes over the training set, with a couple of clever tricks to scale the gradient appropriately. Most ML libraries provide a default implementation of such an algorithm and it is arguably one of the pillars of DL.

While this analysis provided a solid foundation for understanding the properties of this algorithm, the amazing and sometimes surprising successes of DL continue to raise many more questions for the scientific community. In particular, the role of this algorithm in the generalization properties of deep networks has been repeatedly demonstrated but is still poorly understood. This means that a lot of fascinating questions are yet to be explored which could lead to a better understanding of the algorithms currently in use and the development of even more efficient algorithms in the future.

The perspective proposed by Léon and Olivier in their collaboration 10 years ago provided a significant boost to the development of the algorithm that is nowadays the workhorse of ML systems that benefit our lives daily, and we offer our sincere congratulations to both authors on this well-deserved award.

Source: Google AI Blog

Looking for Europe’s top entrepreneurs: The 2018 Digital Top 50 Awards

Posted by Torsten Schuppe, Vice President, Marketing

Tech entrepreneurs are changing the world through their own creativity and passion. To celebrate Europe's thriving developers and the entrepreneurial scene and honor the most promising tech companies, in 2016 we founded the Digital Top 50 Awards, in association with McKinsey and Rocket Internet.

The 2018 edition of the awards are now open for applications and companies with a digital product or service from the EU and from EFTA countries can apply on the Digital Top 50 website until April 1, 2018.

All top 50 companies will receive free tickets and showcase space at Tech Open Berlin on June 20-21 2018, where the final winners in each category will be announced. The winner in the Tech for Social Impact category will be granted a cash prize of 50,000 euros, and all five winners will be provided with support from the founding partners to scale their businesses further—through leading professional advice, structured consulting and coaching programs, as well as access to a huge network of relevant industry contacts.

Helping people embrace new digital opportunities is at the heart of our Grow with Google initiative in Europe. With the DT50 awards, we hope to recognize a new generation of startups and scale-ups, and help them grow further and realize their dreams.

Introducing the Mobile Excellence Award to celebrate great work on Mobile Web

Posted by Shane Cassells, mSite Product Lead, EMEA

We recently partnered with Awwwards, an awards platform for web development and web design, to launch a Mobile Excellence Badge on awwwards.comand a Mobile Excellence Award to recognize great mobile web experiences.

Starting this month, every agency and digital professional that submits their website to Awwwards can be eligible for a Mobile Excellence Badge, a guarantee of the performance of their mobile version. The mobile website's performance will be evaluated by a group of experts and measured against specific criteria based on Google's mobile principles on speed and usability. When a site achieves a minimum score, it will be recognized with the new Mobile Excellence Badge. All criteria are listed at the Mobile Guidelines.

The highest scoring sites with the Mobile Excellence Badge will be nominated for Mobile Site of the Week. One of them will then go on to win Mobile Site of the Month.

All Mobile Sites of the Month will be candidate for Mobile Site of the Year, with the winner receiving a physical award at the Awwwards Conference in Berlin, 8-9 February 2018.

In a time where mobile is playing a dominant role in how people access the web, it is necessary that web developers and web designers build websites that meet users' expectations. Today, 53% of mobile site visits are abandoned if pages take longer than 3 seconds to load1 and despite the explosion of mobile usage, performance and usability of existing mobile sites remain poor and are far from meeting those expectations. At the moment, the average page load time is 22s globally2, which represents a massive missed opportunity for many companies knowing the impact of speed on conversion and bounce rates3.

If you created a great mobile web experience and want it to receive a Mobile Excellence Badge and compete for the Mobile Excellence Award submit your request here.


  1. Google Data, Aggregated, anonymized Google Analytics data from a sample of mWeb sites opted into sharing benchmark data, n=3.7K, Global, March 2016 

  2. Google Research, Webpagetest.org, Global, sample of more than 900,000 mWeb sites across Fortune 1000 and Small Medium Businesses. Testing was performed using Chrome and emulating a Nexus 5 device on a globally representative 3G connection. 1.6Mbps download speed, 300ms Round-Trip Time (RTT). Tested on EC2 on m3.medium instances, similar in performance to high-end smartphones, Jan. 2017. 

  3. Akamai.com, Online Retail Experience Report 2017 

And the award goes to…

Today, Google's Andrei Broder, Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, and Andrew Tomkins, along with their coauthors, Farzin Maghoul, Raymie Stata, and Janet Wiener, have received the prestigious 2017 Seoul Test of Time Award for their classic paper “Graph Structure in the Web”. This award is given to the authors of a previous World Wide Web conference paper that has demonstrated significant scientific, technical, or social impact over the years. The first award, introduced in 2015, was given to Google founders Larry Page and Sergey Brin.

Originally presented in 2000 at the 9th WWW conference in Amsterdam, “Graph Structure in the Web” represents the seminal study of the structure of the World Wide Web. At the time of publication, it received the Best Paper Award from the WWW conference, and in the following 17 years proved to be highly influential, accumulating over 3,500 citations.

The paper made two major contributions to the study of the structure of the Internet. First, it reported the results of a very large scale experiment to confirm that the indegree of Web nodes is distributed according to a power law. To wit, the probability that a node of the Web graph has i incoming links is roughly proportional to 1/i2.1. Second, in contrast to previous research that assumed the Web to be almost fully connected, “Graph Structure in the Web” described a much more elaborate structure of the Web, which since then has been depicted with the iconic “bowtie” shape:
Original “bowtie” schematic from “Graph Structure in the Web”
The authors presented a refined model of the Web graph, and described several characteristic classes of Web pages:
  • the strongly connected core component, where each page is reachable from any other page,
  • the so-called IN and OUT clusters, which only have unidirectional paths to or from the core,
  • tendrils dangling from the two clusters, and tubes connecting the clusters while bypassing the core, and finally
  • disconnected components, which are isolated from the rest of the graph.
Whereas the core component is fully connected and each node can be reached from any other node, Broder et al. discovered that as a whole the Web is much more loosely connected than previously believed, while the probability that any two given pages can be reached from one another is just under 1/4.
Ravi Kumar, presenting the original paper in Amsterdam at WWW 2000
Curiously, the original study was done back in 1999 on two Altavista crawls having 200 million pages and 1.5 billion links. Today, Google indexes over 100 billion links merely within apps, and overall processes over 130 trillion web addresses in its web crawls.

Over the years, the power law was found to be characteristic of many other Web-related phenomena, including the structure of social networks and the distribution of search query frequencies. The description of the macroscopic structure of the Web graph proposed by Broder et al. provided a solid mathematical foundation for numerous subsequent studies on crawling and searching the Web, which profoundly influenced the architecture of modern search engines.

Hearty congratulations to all the authors on the well-deserved award!

And the winner of the $1 Million Little Box Challenge is…CE+T Power’s Red Electrical Devils

In July 2014, Google and the IEEE launched the $1 Million Little Box Challenge, an open competition to design and build a small kW-scale inverter with a power density greater than 50 Watts per cubic inch while meeting a number of other specifications related to efficiency, electrical noise and thermal performance. Over 2,000 teams from across the world registered for the competition and more than 80 proposals qualified for review by IEEE Power Electronics Society and Google. In October 2015, 18 finalists were selected to bring their inverters to the National Renewable Energy Laboratory (NREL) for testing.
Today, Google and the IEEE are proud to announce that the grand prize winner of the $1 Million Little Box Challenge is CE+T Power’s Red Electrical Devils. The Red Electrical Devils (named after Belgium’s national soccer team) were declared the winner by a consensus of judges from Google, IEEE Power Electronics Society and NREL. Honorable mentions go to teams from Schneider Electric and Virginia Tech’s Future Energy Electronics Center.
CE+T Power’s Red Electrical Devils receive $1 Million Little Box Challenge Prize
Schneider, Virginia Tech and The Red Electrical Devils all built 2kW inverters that passed 100 hours of testing at NREL, adhered to the technical specifications of the competition, and were recognized today in a ceremony at the ARPA-E Energy Innovation Summit in Washington, DC. Among the 3 finalists, the Red Electric Devils’ inverter had the highest power density and smallest volume.

Impressively, the winning team exceeded the power density goal for the competition by a factor of 3, which is 10 times more compact than commercially available inverters! When we initially brainstormed technical targets for the Little Box Challenge, some of us at Google didn’t think such audacious goals could be achieved. Three teams from around the world proved decisively that it could be done.

Our takeaway: Establish a worthy goal and smart people will exceed it!

Congratulations again to CE+T Power’s Red Electrical Devils, Schneider Electric and Virginia Tech’s Future Energy Electronics and sincere thanks to our collaborators at IEEE and NREL. The finalist’s technical approach documents will be posted on the Little Box Challenge website until December 31, 2017. We hope this helps advance the state of the art and innovation in kW-scale inverters.