Tag Archives: Developer

How to use App Engine Blobstore (Module 15)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud

Introduction and background

In our ongoing Serverless Migration Station mini-series aimed at helping developers modernize their serverless applications, one of the key objectives for Google App Engine developers is to upgrade to the latest language runtimes, such as from Python 2 to 3 or Java 8 to 17. Another goal is to demonstrate how to move away from App Engine legacy APIs (now referred to as "bundled services") to Cloud standalone replacement services. Once this has been accomplished, apps are much more portable, making them flexible enough to:

Developers building web apps that provide for user uploads or serve large files like videos or audio clips can benefit from convenient "blob" storage backing such functionality, and App Engine's Blobstore serves this specific purpose. As mentioned above, moving away from proprietary App Engine services like Blobstore makes user apps more portable. The original underlying Blobstore infrastructure eventually merged with the Cloud Storage service anyway, so it's logical to move completely to Cloud Storage when convenient, and this content is inform on this process.

Showing App Engine users how to use its Blobstore service
In today's Module 15 video, we begin this journey by showing users how to add Blobstore usage to a sample app, setting us up for our next move to Cloud Storage in Module 16. Similar videos in this series adding use of an App Engine bundled service start with a Python 2 sample app that has already migrated web frameworks from webapp2 to Flask, but not this time.

Blobstore for Python 2 has a dependency on webapp, the original App Engine micro framework replaced by webapp2 when the Python 2.5 runtime was deprecated in favor of 2.7. Because the Blobstore handlers were left "stuck" in webapp, it's better to start with a more generic webapp2 app prior to a Flask migration. This isn't an issue because we modernize this app completely in Module 16 by:

  • Migrating from webapp2 (and webapp) to Flask
  • Migrating from App Engine NDB to Cloud NDB
  • Migrating from App Engine Blobstore to Cloud Storage
  • Migrating from Python 2 to Python (2 and) 3

We'll go into more detail in Module 16, but it suffices to say that once those migrations are complete, the resulting app becomes portable enough for all the possibilities mentioned at the top.

Adding use of Blobstore

The original sample app registers individual web page "visits," storing visitor information such as the IP address and user agent, then displaying the most recent visits to the end-user. In today's video, we add one additional feature: allowing visitors to optionally augment their visits with a file artifact, like an image. Instead of registering a visit immediately, the visitor is first prompted to provide the artifact, as illustrated below.

The updated sample app's new artifact prompt page

The end-user can choose to do so or click a "Skip" button to opt-out. Once this process is complete, the same most recent visits page is then rendered, with one difference: an additional link to view a visit artifact if one's available.

The sample app's updated most recent visits page

Below is pseudocode representing the core part of the app that was altered to add Blobstore usage, namely new upload and download handlers as well as the changes required of the main handler. Upon the initial GET request, the artifact form is presented. When the user submits an artifact or skips, the upload handler POSTs back to home ("/") via an HTTP 307 to preserve the verb, and then the most recent visits page is rendered as expected. There, if the end-user wishes to view a visit artifact, they can click a "view" link where the download handler which fetches and returns the corresponding artifact from the Blobstore service, otherwise an HTTP 404 if the artifact wasn't found. The bolded lines represent the new or altered code.

Adding App Engine Blobstore usage to sample app

Wrap-up

In this "migration," we added Blobstore usage to support visit artifacts to the Module 0 baseline sample app and arrived at the finish line with the Module 15 sample app. To get hands-on experience doing it yourself, do the codelab by hand and follow along with the video. Then you'll be ready to upgrade to Cloud Storage should you choose to do so. 

In Fall 2021, the App Engine team extended support of many of the bundled services to 2nd generation runtimes (that have a 1st generation runtime), meaning you are no longer required to migrate to Cloud Storage when porting your app to Python 3. You can continue using Blobstore in your Python 3 app so long as you retrofit the code to access bundled services from next-generation runtimes. 

If you do want to move to Cloud Storage, Module 16 is next. You can also try its codelab to get a head start. All Serverless Migration Station content (codelabs, videos, source code [when available]) can be accessed at its open source repo. While our content initially focuses on Python users, the Cloud team is working on covering other language runtimes, so stay tuned. For additional video content, check out our broader Serverless Expeditions series.

Use the Cloud Search Query API to set Suggest Filters to enhance Cloud Search results

What’s changing 

We’re introducing Suggest Filters for Cloud Search. Using the Cloud Search Query API, admins can specify a filter condition that will be pre-applied to keyword suggestions as user types a query. This will surface more relevant suggestions, helping reduce the time users spend searching. 



Who’s impacted 

Admins, developers and end users 



Why it’s important 

With suggestion filters, admins can configure suggestions based on the use cases for a given search application, reducing irrelevant suggestions. For example, admins can add a suggestion such as a country, which will surface suggestions based on documents that align the filters. 


Getting started 

  • Admins: See our developer documentation here and here for more information about creating a suggestion filter.
  • End users: There is no end user action required, you will automatically see relevant suggested filters as you type a query. 

Rollout pace 

  • This feature is available now for all users. 

Availability 

  • Available to Google Cloud Search Customers 

Resources 

Migrating from App Engine Memcache to Cloud Memorystore (Module 13)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud

Introduction and background

The previous Module 12 episode of the Serverless Migration Station video series demonstrated how to add App Engine Memcache usage to an existing app that has transitioned from the webapp2 framework to Flask. Today's Module 13 episode continues its modernization by demonstrating how to migrate that app from Memcache to Cloud Memorystore. Moving from legacy APIs to standalone Cloud services makes apps more portable and provides an easier transition from Python 2 to 3. It also makes it possible to shift to other Cloud compute platforms should that be desired or advantageous. Developers benefit from upgrading to modern language releases and gain added flexibility in application-hosting options.

While App Engine Memcache provides a basic, low-overhead, serverless caching service, Cloud Memorystore "takes it to the next level" as a standalone product. Rather than a proprietary caching engine, Cloud Memorystore gives users the option to select from a pair of open source engines, Memcached or Redis, each of which provides additional features unavailable from App Engine Memcache. Cloud Memorystore is typically more cost efficient at-scale, offers high availability, provides automatic backups, etc. On top of this, one Memorystore instance can be used across many applications as well as incorporates improvements to memory handling, configuration tuning, etc., gained from experience managing a huge fleet of Redis and Memcached instances.

While Memcached is more similar to Memcache in usage/features, Redis has a much richer set of data structures that enable powerful application functionality if utilized. Redis has also been recognized as the most loved database by developers in StackOverflow's annual developers survey, and it's a great skill to pick up. For these reasons, we chose Redis as the caching engine for our sample app. However, if your apps' usage of App Engine Memcache is deeper or more complex, a migration to Cloud Memorystore for Memcached may be a better option as a closer analog to Memcache.

Migrating to Cloud Memorystore for Redis featured video

Performing the migration

The sample application registers individual web page "visits," storing visitor information such as IP address and user agent. In the original app, the most recent visits are cached into Memcache for an hour and used for display if the same user continuously refreshes their browser during this period; caching is a one way to counter this abuse. New visitors or cache expiration results new visits as well as updating the cache with the most recent visits. Such functionality must be preserved when migrating to Cloud Memorystore for Redis.

Below is pseudocode representing the core part of the app that saves new visits and queries for the most recent visits. Before, you can see how the most recent visits are cached into Memcache. After completing the migration, the underlying caching infrastructure has been swapped out in favor of Memorystore (via language-specific Redis client libraries). In this migration, we chose Redis version 5.0, and we recommend the latest versions, 5.0 and 6.x at the time of this writing, as the newest releases feature additional performance benefits, fixes to improve availability, and so on. In the code snippets below, notice how the calls between both caching systems are nearly identical. The bolded lines represent the migration-affected code managing the cached data.

Switching from App Engine Memcache to Cloud Memorystore for Redis

Wrap-up

The migration covered begins with the Module 12 sample app ("START"). Migrating the caching system to Cloud Memorystore and other requisite updates results in the Module 13 sample app ("FINISH") along with an optional port to Python 3. To practice this migration on your own to help prepare for your own migrations, follow the codelab to do it by-hand while following along in the video.

While the code migration demonstrated seems straightforward, the most critical change is that Cloud Memorystore requires dedicated server instances. For this reason, a Serverless VPC connector is also needed to connect your App Engine app to those Memorystore instances, requiring more dedicated servers. Furthermore, neither Cloud Memorystore nor Cloud VPC are free services, and neither has an "Always free" tier quota. Before moving forward this migration, check the pricing documentation for Cloud Memorystore for Redis and Serverless VPC access to determine cost considerations before making a commitment.

One key development that may affect your decision: In Fall 2021, the App Engine team extended support of many of the legacy bundled services like Memcache to next-generation runtimes, meaning you are no longer required to migrate to Cloud Memorystore when porting your app to Python 3. You can continue using Memcache even when upgrading to 3.x so long as you retrofit your code to access bundled services from next-generation runtimes.

A move to Cloud Memorystore and today's migration techniques will be here if and when you decide this is the direction you want to take for your App Engine apps. All Serverless Migration Station content (codelabs, videos, source code [when available]) can be accessed at its open source repo. While our content initially focuses on Python users, we plan to cover other language runtimes, so stay tuned. For additional video content, check out our broader Serverless Expeditions series.

Google Workspace Updates Weekly Recap – May 20, 2022

New updates 


There are no new updates to share this week. Please see below for a recap of published announcements. 


Previous announcements


The announcements below were published on the Workspace Updates blog earlier this week. Please refer to the original blog posts for complete details.



Migrate your classic Google Sites before December 1, 2022 
We’re extending the previously announced timeline to give Google Workspace customers more time to migrate from classic Google Sites to new Google Sites: 
  • Starting December 1, 2022 (previously June 1, 2022), you will no longer be able to edit any remaining classic Sites in your domain. 
  • Starting January 1, 2023 (previously July 1, 2022), Classic Sites will no longer be viewable unless they are converted to new Google Sites. 
Learn more here and here. 


New and updated third-party DevOps integrations for Google Chat, including PagerDuty 
There are now a variety of additional DevOps integrations that allow you to act on common workflows directly in Google Chat. | Learn more. 


Export log data in near-real time to BigQuery
Exported log data streams are now in near-real time (under 10 minutes), improving upon the previous process which returned log data that could be up to three days old. | Learn more here and here. 


AppSheet Enterprise Standard and Enterprise Plus available as add-ons to Google Workspace editions
Google Workspace customers can now purchase AppSheet Enterprise Standard and Enterprise Plus as add-ons by contacting their Google Cloud sales representative or through the Google Workspace Partner network. | Learn more. 


Use Connected Sheets with VPC-SC protected data, improved Cloud Audit Logs for Connected Sheets events 
BigQuery datasets that are behind a perimeter created by VPC Service Controls can now be accessed using Connected Sheets. We’ve also made improvements to the Connected Sheets logging in the Cloud Audit Logs. | Learn more. 


New banners in Google Chat protect against malicious links 
In Google Chat, you can now see banners warning against potential phishing and malware messages coming from users with personal Google Accounts to help protect users against malicious actors, keeping data safe. | Learn more. 


For a recap of announcements in the past six months, check out What’s new in Google Workspace (recent releases).

How to use App Engine Memcache in Flask apps (Module 12)

Posted by Wesley Chun

Background

In our ongoing Serverless Migration Station series aimed at helping developers modernize their serverless applications, one of the key objectives for Google App Engine developers is to upgrade to the latest language runtimes, such as from Python 2 to 3 or Java 8 to 17. Another objective is to help developers learn how to move away from App Engine legacy APIs (now called "bundled services") to Cloud standalone equivalent services. Once this has been accomplished, apps are much more portable, making them flexible enough to:

In today's Module 12 video, we're going to start our journey by implementing App Engine's Memcache bundled service, setting us up for our next move to a more complete in-cloud caching service, Cloud Memorystore. Most apps typically rely on some database, and in many situations, they can benefit from a caching layer to reduce the number of queries and improve response latency. In the video, we add use of Memcache to a Python 2 app that has already migrated web frameworks from webapp2 to Flask, providing greater portability and execution options. More importantly, it paves the way for an eventual 3.x upgrade because the Python 3 App Engine runtime does not support webapp2. We'll cover both the 3.x and Cloud Memorystore ports next in Module 13.

Got an older app needing an update? We can help with that.

Adding use of Memcache

The sample application registers individual web page "visits," storing visitor information such as the IP address and user agent. In the original app, these values are stored immediately, and then the most recent visits are queried to display in the browser. If the same user continuously refreshes their browser, each refresh constitutes a new visit. To discourage this type of abuse, we cache the same user's visit for an hour, returning the same cached list of most recent visits unless a new visitor arrives or an hour has elapsed since their initial visit.

Below is pseudocode representing the core part of the app that saves new visits and queries for the most recent visits. Before, you can see how each visit is registered. After the update, the app attempts to fetch these visits from the cache. If cached results are available and "fresh" (within the hour), they're used immediately, but if cache is empty, or a new visitor arrives, the current visit is stored as before, and this latest collection of visits is cached for an hour. The bolded lines represent the new code that manages the cached data.

Adding App Engine Memcache usage to sample app

Wrap-up

Today's "migration" began with the Module 1 sample app. We added a Memcache-based caching layer and arrived at the finish line with the Module 12 sample app. To practice this on your own, follow the codelab doing it by-hand while following the video. The Module 12 app will then be ready to upgrade to Cloud Memorystore should you choose to do so.

In Fall 2021, the App Engine team extended support of many of the bundled services to next-generation runtimes, meaning you are no longer required to migrate to Cloud Memorystore when porting your app to Python 3. You can continue using Memcache in your Python 3 app so long as you retrofit the code to access bundled services from next-generation runtimes.

If you do want to move to Cloud Memorystore, stay tuned for the Module 13 video or try its codelab to get a sneak peek. All Serverless Migration Station content (codelabs, videos, source code [when available]) can be accessed at its open source repo. While our content initially focuses on Python users, we hope to one day cover other language runtimes, so stay tuned. For additional video content, check out our broader Serverless Expeditions series.

Create Spaces and Add Members with the Google Chat API, available in Developer Preview

What’s changing

Using the Google Chat API, you can now programmatically create new Spaces and add members to those Spaces. This functionality is available in preview – developers can apply for access through our Google Workspace Developer Preview Program.

Google Workspace administrators can also use the API controls in Admin Console if they would also like to restrict access to Google Chat data.


Who’s impacted

Admins and Developers

Why you’d use it 

While it’s easy to create new Spaces and add members directly in Google Chat, there are cases where Spaces can be filled with many topics and side conversations, making it difficult to keep track of important information. Using the new API functionality, you can set up new spaces that focus on a specific topic, team, or project. For example, an on-call app can automatically create a space when an outage has been detected.

Getting started


Availability

  • Available to all Google Workspace customers, as well as legacy G Suite Basic and Business customers


Using document popularity in Cloud Search rankings

Quick Summary 

We’re improving search rankings in Cloud Search by taking into account document popularity. This means documents that have been clicked on by a large number of users will be prioritized in the overall search rankings. By surfacing popular and useful documents that match a user’s query, this will help reduce the effort and time required to find relevant documents. 


Getting started 

  • Admins: The feature is available by default. Use this guide to learn more about the use of Document Popularity in Cloud Search ranking. Note: You need to use the default redirect URLs for search results when using Cloud Search Query API for this feature to work.
  • End Users: No effort needed.

Rollout pace 


Availability 

  • Available to all Google Cloud Search customers 
  • Available to Google Workspace Business Plus, Enterprise Standard, Enterprise Plus, and Education Plus customers 
  • Not available to Google Workspace Essentials, Business Starter, Business Standard, Enterprise Essentials, Education Fundamentals, Frontline, and Nonprofits, as well as G Suite Basic and Business customers 

Resources 

Develop Google Workspace add-ons to attach files to Calendar events from third-party services

Quick summary 

Google Workspace developers can now create Google Workspace add-ons that attach files to a Google Calendar event from any third-party service. This feature enables developers to create add-ons that support attachments from a wide range of sources beyond Google Drive, such as digital whiteboard, content creation, or file management tools. 


Users who have the relevant add-ons installed will be able to attach files from these sources to a Calendar event, and attendees can view the event with the attachment on the web or on mobile.



Attach files from a third-party service to a Calendar event


After attaching files in Calendar on the web, users can view the event with the attachment on the web or on mobile. 


Getting started 


Rollout pace 

  • This feature is available now for all developers and users. 

Availability 

  • Available to all Google Workspace customers, as well as legacy G Suite Basic and Business customers 

Resources 

Create surveys, quizzes, and more using the Google Forms API, now generally available

What’s changing

The Google Forms API provides programmatic access for managing Google Forms and acting on responses— empowering developers to build powerful integrations on top of Forms.

Who’s impacted 

Admins and developers

Why you’d use it 

The Google Forms API provides programmatic access to manage Forms and receive responses, supporting the development of a variety of powerful integrations. For example, you can use the API to develop real-time dashboards or data visualizations; trigger business workflows incorporating project management, CRM, or LMS tools; or auto-generate forms from question banks or other data sets. 

The API is useful for a variety of tasks such as:

  • Creating and modifying forms or quizzes
  • Retrieving form responses or quiz grades
  • Reading form content and metadata
  • Receiving push notifications for form or quiz responses or form structure updates 


Getting started




Rollout pace

Integration availability for end users
  • This feature is available now for all users.

Availability


  • Available to all Google Workspace customers, as well as legacy G Suite Basic and Business customers and users with personal Google Accounts

Resources


Google Forms API now available in open beta

What’s changing 

At Google Cloud Next 2021, we announced the Google Forms API Beta, which provides programmatic access for managing Google Forms and acting on responses— empowering developers to build powerful integrations on top of Forms. 

The Google Forms API is now rolling out as an Open Beta which means developers who are part of our Early Adopter Program can make their integrations available to the public. We’ll no longer require individual end-user accounts to be allowlisted. Developers should keep in mind, however, that their integrations are in Beta. 

Developers can apply to join our Early Adopter Program and begin developing using the Google Forms API by filling out this form. 

See below for more information. 

Who’s impacted 

Admins and developers 

Why you’d use it 

The Google Forms API provides programmatic access to manage Forms and receive responses, supporting the development of a variety of powerful integrations. For example, the API could be utilized to develop real-time dashboards or data visualizations, trigger business workflows incorporating project management, CRM, or LMS tools, or auto-generate forms from question banks or other data sets. 

The API is useful for a variety of tasks such as: 

  • Creating and modifying forms or quizzes 
  • Retrieving form responses or quiz grades 
  • Reading form content and metadata 
  • Receiving push notifications for form or quiz responses and updates 



Getting started 

Rollout pace 

Integration availability for end users 

Availability 

Resources