Tag Archives: Google App Engine

How we brought the latest version of Python to App Engine and Cloud Functions

At Cloud Next 2018, we added Python 3.7 support to Cloud Functions and now we’ve announced Python 3.7 support for the App Engine standard environment. These new runtimes allow you to write Python functions and apps using the latest version of Python and the rich ecosystem of packages available on Python Packaging Index (PyPI).

This new runtime marks a significant update to App Engine and was enabled by new open source software that we recently released: gVisor and FTL.

Python, straight from the source

Running Python 3.7 on App Engine and Cloud Functions required us to fundamentally rethink our infrastructure. Traditionally, meeting Google Cloud’s security requirements meant that we had to run a modified version of the Python interpreter. However, using a modified interpreter constrained some language features and only allowed us to support a limited set of whitelisted Python libraries.

Thanks to gVisor, a container sandbox that provides improved security and process isolation, we can now run the unmodified Python 3.7.0 interpreter. We’ve done extensive testing to make sure Python 3.7 is compatible with gVisor. As part of our compatibility testing, we run Python’s full suite of language tests, and tests for Python packages that are popular on PyPI. We’re committed to ensuring that everything you’ve come to know and love about Python is supported on our platform.

Seamless deployments

Most importantly, this change in our infrastructure makes it easier to take advantage of Python’s vast ecosystem. As a developer, you just add project dependencies to a requirements.txt file and deploy.

During deployment, FTL, a tool for building containers, fetches dependencies listed in your requirements.txt file and installs them alongside your app or function. FTL also includes a short-lived dependency cache, which speeds up repeated deployments if no changes are detected in your requirements.txt file. This is particularly useful if you find just need to re-deploy because you found a typo.

Keeping up with the Pythonistas

In making these changes, we also decided to expand the list of system packages that are included with each runtime’s Ubuntu 18.04 distribution. We think that will make life just a little bit easier for developers working with the latest release of Python.

Looking forward, we’re excited about how these changes will allow us to keep up with the Python community’s progress as they release new versions and libraries. Please let us know what you think and if you run into any challenges.

You can learn more about how to get started with it on App Engine and Cloud Functions in our documentation. We can’t wait to see what you build with Python 3.7.

By Stewart Reichling, Product Manager

A sizzling open source release for the Australian Election site

Originally posted on the Geo Developers Blog

One of the best parts of my job at Google is 20 percent time. While I was hired to help developers use Google’s APIs, I value the time I'm afforded to be a student myself—to learn new technologies and solve real-world problems. A few weeks prior to the recent Australian election an opportunity presented itself. A small team in Sydney set their sights on helping the 15 million voters stay informed of how to participate, track real-time results, and (of course) find the closest election sausage sizzle!


Our team of designers, engineers and product managers didn't have an immediate sense of how to attack the problem. What we did have was the power of Google’s APIs, programming languages, and Cloud hosting with Firebase and Google Cloud Platform.



The result is a mish-mash of some technologies we'd been wanting to learn more about. We're open sourcing the ausvotes.withgoogle.com repository to give developers a sense of what happens when you get a handful of engineers in a room with a clear goal and a immovable deadline.

The Election AU 2016 repository uses:

  • Go from Google App Engine instances to serve the appropriate level of detail for users' viewport queries from memory at very low latency, and
  • Dart to render the live result maps on top of Google Maps JavaScript API using Firebase real time database updates.

A product is only as good as the attention and usage is receives. Our team was really happy with the results of our work:

  • 406,000 people used our maps, including 217,000 on election day.
  • We had 139 stories in the media.
  • Our map was also embedded in major news websites, such as Sky News.

Complete setup and installation instructions are available in the GitHub README.

By Brett Morgan, Developer Programs Engineer

Announcing Google Cloud Platform Education Grants for computer science

Posted by Bram Bout, Director, Google for Education

(Cross-posted on the Google for Education Blog.)

While university students are on their summer holidays, internships or jobs, their professors are already hard at work planning for fall courses. These course maps will be at the center of student learning, research and academic growth. Google was founded on the basis of the work that Larry and Sergey did as computer science students at Stanford, and we understand the critical role that teachers play in fostering and inspiring the innovation we see today and will see in the years to come. That’s why we’re excited to offer Google Cloud Platform Education Grants for computer science.

Starting today, university faculty in the United States who teach courses in computer science or related subjects can apply for free credits for their students to use across the full suite of Google Cloud Platform tools, like App Engine and the Cloud Machine Learning platform. These credits can be used any time during the 2016-17 academic year and give students access to the same tools and infrastructure used by Google engineers.
Students like Duke University undergrad Brittany Wenger are already taking advantage of cloud computing. After watching several women in her family suffer from breast cancer, Brittany used her knowledge of artificial intelligence to create Cloud4Cancer, an artificial neural network built on top of Google App Engine. By analyzing uploaded scans of benign and malignant breast cancer tumors, Cloud4Cancer has learned to distinguish between healthy and unhealthy tissue. It’s providing health care professionals with a powerful diagnostic tool in the fight against cancer.

Google Cloud Platform offers a range of tools and services that are unique among cloud providers. The tool that Brittany used — Google App Engine — lets you simply build and run an application without having to configure custom infrastructure. Our Machine Learning platform allows you to build models for any type of data, at any size, and TensorFlow provides access to an open-source public software library (tinker with that extensive data here). Students will also be able to get their hands on one of Cloud Platform’s most popular new innovations: the Cloud Vision API, which allows you to incorporate Google’s state-of-the-art image recognition capabilities into the most basic web or mobile app.

We look forward to seeing the creative ways that computer science students will use their Google Cloud Platform Education Grants, and will share stories along the way on this blog.

Computer science faculty in the United States can apply here for Education Grants. Students and others interested in Cloud Platform for Higher Education, should complete this form to register and stay up to date with the latest from Cloud Platform. For more information on Cloud Platform and its uses for higher education, visit our Google Cloud Platform for Higher Education site.

GRAS helps build sustainable and ethical supply chains using Google Maps APIs and Google Cloud Platform

Posted by Dr. Norbert Schmitz, Managing Director of Meo Carbon Solutions

Editor's note: This is the first in a series of “Mapping a better world” blog posts highlighting ways in which organizations are using location data to affect positive local and global change. Google Maps APIs continues to create opportunities and tools to support our community.

Today we hear from Dr. Norbert Schmitz, managing director of Meo Carbon Solutions. Read how Meo Carbon Solutions and Google for Work Premier Partner Wabion used Google Maps APIs and Google Cloud Platform to develop Global Risk Assessment Services (GRAS). The tool provides reliable information about the ecological and social risks of expanding agriculture into natural habitats.

In the European Union, companies that sell biofuels must get certifications to show that producing their fuel — often made from agricultural crops — does not cause deforestation, the loss of biodiversity or the loss of carbon stocks. We established GRAS to provide a single tool to gather and visualize this data required to support a credible certification.

Through GRAS, we’ve made this information accessible not only to governments and NGOs, but also to businesses, financial institutions and individuals. For example, a U.S. company buying soybean oil from Brazil can use GRAS to verify the ecological and social risk exposure of the mills and the farmers supplying the mills.

We decided the most effective way to present this complex set of information was to build a web app that would overlay data from multiple sources on top of a map. Using this tool, auditors can compare before-and-after maps of a certain area, and based on changes to the habitat, either grant or deny certifications.

To build GRAS, we partnered with IT consultant Wabion, a Google for Work Premier Partner. After testing several map solutions, we chose Google Maps because of its high performance, ability to easily integrate data from multiple sources, flexible APIs, solid support and large user community.


The GRAS website combines mapping information with data from dozens of government agencies, nongovernmental organizations (NGOs) and other global databases. It uses the Google Maps JavaScript API to display the base maps for the site. The API also visualizes layers and more than 100 types of data — including agriculture, deforestation and social welfare — on top of the maps.

Users can upload and visualize their own data sets. The Google Maps Geocoding API verifies map locations, and the Google Maps Places API autocomplete makes it easy for site users to choose locations that they want to learn about.
A GRAS visualization of biodiversity risk in Brazil



GRAS is powered by the Google Cloud Platform — specifically, Google App Engine and Google Compute Engine. Combining these services with the Google Maps APIs lets us handle geodata in a 10-terabyte database, which hosts more than 100 layers, many of them extremely large.

We’ve recently expanded GRAS beyond our original mission to provide insights for other industries by expanding the data layered on top of maps. We’ve included new information on biodiversity, land-use changes, and available carbon stock. We’ve also been able to incorporate numerous measurements of social health — ranging from the Global Slavery Index to the Global Hunger Index and the UNICEF index of access to drinking water and sanitation.

Through GRAS, we’ve made this information accessible not only to governments and NGOs, but also to businesses, financial institutions and individuals. For example, a U.S. company buying soybean oil from Brazil can use GRAS to verify the ecological and social risk exposure of the mills and the farmers supplying the mills.

Far exceeding our initial vision, the GRAS website combines transparency with the power of technology to help users in wide-ranging industries, —from food, to chemicals and energy — operate environmentally and socially sound supply chains.


Sports Authority handles 2,000 transactions per second with Google Cloud Platform

Posted by Jon Byrum, Product Marketing Manager, Google Cloud Platform

(Cross-posted on the Google Cloud Platform Blog.)

Athletic gear, much like all apparel categories, is quickly shifting to an online sales business. Sports Authority, seeing the benefits that cloud could offer around agility and speed, turned to Google Cloud Platform to help it respond to its customers faster.

In 2014, Sports Authority’s technical team was asked to build a solution that would expose all in-store product inventory to its ecommerce site, sportsauthority.com, allowing customers to see local store availability of products as they were shopping online. That’s nearly half a million products to choose from in over 460 stores across the U.S. and Puerto Rico.

This use case posed a major challenge for the company. Its in-store inventory data was “locked” deep inside a mainframe. Exposing millions of products to thousands of customers, 24 hours a day, seven days a week would not be possible using this system.

The requirements for a new solution included finding the customer’s location, searching the 90 million record inventory system and returning product availability in just the handful of stores nearest in location to that particular customer. On top of that, the API would need to serve at least 50 customers per second, while returning results in less than 200 milliseconds.

Choosing the right cloud provider

At the time this project began, Sports Authority had already been a Google Apps for Work (Gmail, Google Sites, Docs) customer since 2011. However, it had never built any custom applications on Google Cloud Platform.

After a period of due diligence checking out competing cloud provider options, Sports Authority decided that Google App Engine and Google Cloud Datastore had the right combination of attributes — elastic scaling, resiliency and simplicity of deployment — to support this new solution.

Through the combined efforts of a dedicated project team, business partners and three or four talented developers, it was able to build a comprehensive solution on Cloud Platform in about five months. It consisted of multiple modules: 1) batch processes, using Informatica to push millions of product changes from its IBM mainframe to Google Cloud Storage each night, 2) load processes — python code running on App Engine, which spawn task queue jobs to load Cloud Datastore, and 3) a series of SOAP and REST APIs to expose the search functionality to its ecommerce website.

Sports Authority used tools including SOAPUI and LOADUI to simulate thousands of virtual users to measure the scalability of SOAP and REST APIs. It found that as the number of transactions grew past 2,000 per second, App Engine and Cloud Datastore continued to scale seamlessly, easily meeting its target response times.

The company implemented the inventory locator solution just in time for the 2014 holiday season. It performed admirably during that peak selling period and continues to do so today.
This screenshot shows what customers see when they shop for products on the website — a list of local stores, showing the availability of any given product in each store



When a customer finds a product she's interested in buying, the website requests inventory availability from Sports Authority’s cloud API, which provides a list of stores and product availability to the customer, as exhibited in the running shoe example above.

In-store kiosk

As Sports Authority became comfortable building solutions on Cloud Platform, it opened its eyes to other possibilities for creating new solutions to better serve its customers.

For example, it recently developed an in-store kiosk, which allows customers to search for products that may not be available in that particular store. It also lets them enroll in the loyalty program and purchase gift cards. This kiosk is implemented on a Google Chromebox, connected to a web application running on App Engine.
This image shows the in-store kiosk that customers use to locate products available in other stores. 




Internal store portal

Additionally, it built a store portal and task management system, which facilitates communication between the corporate office and its stores. This helps the store team members plan and execute their work more efficiently, allowing them to serve customers better when needs arise. This solution utilizes App Engine, Cloud Datastore and Google Custom Search, and was built with the help of a local Google partner, Tempus Nova.
This screenshot shows the internal store portal that employees use to monitor daily tasks.




Learning how to build software in any new environment such as Cloud Platform takes time, dedication and a willingness to learn. Once up to speed, the productivity and power of Google Cloud Platform allowed the Sports Authority team to work like a software company and build quickly while wielding great power.