Tag Archives: firestore

Extending support for App Engine bundled services (Module 17)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud


App Engine initially launched in 2008, providing a suite of bundled services making it convenient for applications to access a database (Datastore), caching service (Memcache), independent task execution (Task Queue), Google Sign-In authentication (Users), or large "blob" storage (Blobstore), or other companion services. However, apps leveraging those services can only run their apps on App Engine.

To increase app portability and help Google move towards its goal of having the most open cloud on the market, App Engine launched its 2nd-generation service in 2018, initially removing those legacy services. The newer platform allows developers to upgrade apps to the latest language runtimes, such as moving from Python 2 to 3 or Java 8 to 11 (and today, Java 17). One of the major drawbacks to the 1st-generation runtimes is that they're customized, proprietary, and restrictive in what you can use or can't.

Instead, the 2nd-generation platform uses open source runtimes, meaning ability to follow standard development practices, use common/known idioms, and have fewer restrictions of 3rd-party libraries, and obviating the need to copy or "vendor" them with your code. Unfortunately, to use these newer runtimes, migrating away from App Engine services were required because while you could upgrade language releases, there was no access to bundled services, breaking apps or requiring complete rewrites, making it a showstopper for many users.

Due to their popularity and the desire to ease the upgrade process for customers, the App Engine team restored access to most (but not all) of those services in Fall 2021. Today's Serverless Migration Station video demonstrates how to continue usage of bundled services available to Python 3 developers.

Showing App Engine users how to use bundled services on Python 3

Performing the upgrade

Modernizing the typical Python 2 App Engine app looks something like this:
  1. Migrate from the webapp2 framework (not available in Python 3)
  2. Port from Python 2 to 3, preserve use of bundled services
  3. Optional migration to Cloud standalone or similar 3rd-party services

The first step is to move to a standard Python web framework like Flask, Django, Pyramid, etc. Below is some pseudocode from Migration Module 1 demonstrating how to migrate from webapp2 to Flask:

codeblocks for porting Python 2 sample app from webapp2 to Flask
Step 1: Port Python 2 sample app from webapp2 to Flask

The key changes are bolded in the above code snippets. Notice how the App Engine NDB code [the Visit class definition plus store_visit() and fetch_visits() functions] are unaffected by this web framework migration. The full webapp2 code sample can be found in the Module 0 repo folder while the completed migration to Flask sample is located in the Module 1 repo folder.

After your app has ported frameworks, you're free to upgrade to Python 3 while preserving access to the bundled services if your app uses any. Below is pseudocode demonstrating how to upgrade the same sample app to Python 3 as well as the code changes needed to continue to use App Engine NDB:

codeblocks for porting sample app to Python 3, preserving use of NDB bundled service
Step 2: Port sample app to Python 3, preserving use of NDB bundled service

The original app was designed to work under both Python 2 and 3 interpreters, so no language changes were required in this case. We added an import of the new App Engine SDK followed by the key update wrapping the WSGI object so the app can access the bundled services. As before, the key updates are bolded. Some updates to configuration are also required, and those are outlined in the documentation and the (Module 17) codelab.

The NDB code is also left untouched in this migration. Not all of the bundled services feature such a hands-free migration, and we hope to cover some of the more complex ones ahead in Module 22. Java, PHP, and Go users have it even better, requiring fewer or no code changes at all. The Python 2 Flask sample is located in the Module 1 repo folder, and the resulting Python 3 app can be found in the Module 1b repo folder.

The immediate benefit of step two is the ability to upgrade to a more current version of language runtime. This leaves the third step of migrating off the bundled services as optional, especially if you plan on staying on App Engine for the long-term.

Additional options

If you decide to migrate off the bundled services, you can do so on your own timeline. It should be a consideration should you ever want to move to modern serverless platforms such as Cloud Functions or Cloud Run, to lower-level platforms because you want more control, like GKE, our managed Kubernetes service, or Compute Engine VMs.

Step three is also where the rest of the Serverless Migration Station content may be useful:

*code samples and codelabs available; videos forthcoming

As far as moving to modern serverless platforms, if you want to break apart a large App Engine app into multiple microservices, consider Cloud Functions. If your organization has added containerization as part of your software development workflow, consider Cloud Run. It's suitable for apps if you're familiar with containers and Docker, but even if you or your team don't have that experience, Cloud Buildpacks can do the heavy lifting for you. Here are the relevant migration modules to explore:


    Early App Engine users appreciate the convenience of the platform's bundled services, and after listening to user feedback, adding them back to 2nd-generation runtimes is another way we can help developers modernize their apps. Whether upgrading to newer language runtimes to stay on App Engine and continue to use its bundled services, migrating to Cloud standalone products, or shifting to other serverless platforms, the Google Cloud team aims to provide the tools to help streamline your modernization efforts.

    All Serverless Migration Station content (codelabs, videos, source code [when available]) can be accessed at its open source repo. While our content initially focuses on Python users, the Cloud team is working on covering other language runtimes, so stay tuned. Today's video features a special guest to provide a teaser of what to expect for Java. For additional video content, check out the broader Serverless Expeditions series.

    Make Payments with Google Pay and Firebase

    Posted by Stephen McDonald, Developer Relations Engineer, Google Pay

    Connect Multiple Payment Gateways with Google Pay and Firebase

    We recently launched a series of open source samples demonstrating the server-side integration between Google Pay and a variety of Payment Service Providers (PSPs). These samples also show how to create a unified interface for integrating multiple PSPs, making integrations as easy as possible by reducing the time investment in integrating multiple APIs and client libraries.

    A recent study by 451 Research showed that for merchants with over 50% of sales occurring online, 69% of them used multiple PSPs. We first demonstrated with the aforementioned samples how you can implement a consistent interface to multiple PSPs, streamlining your codebase while also providing more flexibility for the future. We've now taken this one step further and brought this unified PSP interface to the Firebase platform, by way of a Firebase Extension for Google Pay, making it easier than ever to integrate Google Pay with one or more PSPs.

    Google Pay Firebase Extension

    Firebase Extensions are open source pre-packaged bundles of code that developers can easily pull into their apps, and are designed to increase productivity, and provide extended functionality to your apps without the need to research, write, or debug code on your own. Following this line, the Google Pay Firebase Extension brings the unified PSP interface to developers' Firebase apps.

    With the Google Pay Firebase Extension installed, you can pass a payment token from the Google Pay API to your Cloud Firestore database. The extension will listen for a request written to the path defined during installation, and then send the request to the PSP's API. It will then write the response back to the same Firestore node.

    Open Source

    Like all Firebase Extensions, the Google Pay Firebase Extension is entirely open source, so you can modify the code yourself to change the functionality as you see fit, or even contribute your changes back via pull requests - the sky's the limit.

    Furthermore, as the extension is backed by the aforementioned PSP samples project, the same set of PSPs are supported. Want to see your favorite PSP supported? Head on over to the PSP samples project which contains instructions for adding it.

    Summing it up

    Whether you're new to Google Pay or Firebase, or an existing user of either, the new Google Pay extension is designed to save you even more time and effort when integrating Google Pay and any number of Payment Service Providers with your application.

    Get started with the extension today in the Firebase console.

    What do you think? Follow us on Twitter for the latest updates @GooglePayDevs

    Do you have any questions? Let us know in the comments below or tweet using #AskGooglePayDevs.

    Introducing Cloud Firestore: Our New Document Database for Apps

    Originally posted by Alex Dufetel on the Firebase Blog

    Today we're excited to launch Cloud Firestore, a fully-managed NoSQL document database for mobile and web app development. It's designed to easily store and sync app data at global scale, and it's now available in beta.

    Key features of Cloud Firestore include:

    • Documents and collections with powerful querying
    • iOS, Android, and Web SDKs with offline data access
    • Real-time data synchronization
    • Automatic, multi-region data replication with strong consistency
    • Node, Python, Go, and Java server SDKs

    And of course, we've aimed for the simplicity and ease-of-use that is always top priority for Firebase, while still making sure that Cloud Firestore can scale to power even the largest apps.

    Optimized for app development

    Managing app data is still hard; you have to scale servers, handle intermittent connectivity, and deliver data with low latency.

    We've optimized Cloud Firestore for app development, so you can focus on delivering value to your users and shipping better apps, faster. Cloud Firestore:

    • Synchronizes data between devices in real-time. Our Android, iOS, and Javascript SDKs sync your app data almost instantly. This makes it incredibly easy to build reactive apps, automatically sync data across devices, and build powerful collaborative features -- and if you don't need real-time sync, one-time reads are a first-class feature.
    • Uses collections and documents to structure and query data. This data model is familiar and intuitive for many developers. It also allows for expressive queries. Queries scale with the size of your result set, not the size of your data set, so you'll get the same performance fetching 1 result from a set of 100, or 100,000,000.
    • Enables offline data access via a powerful, on-device database. This local database means your app will function smoothly, even when your users lose connectivity. This offline mode is available on Web, iOS and Android.
    • Enables serverless development. Cloud Firestore's client-side SDKs take care of the complex authentication and networking code you'd normally need to write yourself. Then, on the backend, we provide a powerful set of security rules so you can control access to your data. Security rules let you control which users can access which documents, and let you apply complex validation logic to your data as well. Combined, these features allow your mobile app to connect directly to your database.
    • Integrates with the rest of the Firebase platform. You can easily configure Cloud Functions to run custom code whenever data is written, and our SDKs automatically integrate with Firebase Authentication, to help you get started quickly.

    Putting the 'Cloud' in Cloud Firestore

    As you may have guessed from the name, Cloud Firestore was built in close collaboration with the Google Cloud Platform team.

    This means it's a fully managed product, built from the ground up to automatically scale. Cloud Firestore is a multi-region replicated database that ensures once data is committed, it's durable even in the face of unexpected disasters. Not only that, but despite being a distributed database, it's also strongly consistent, removing tricky edge cases to make building apps easier regardless of scale.

    It also means that delivering a great server-side experience for backend developers is a top priority. We're launching SDKs for Java, Go, Python, and Node.js today, with more languages coming in the future.

    Another database?

    Over the last 3 years Firebase has grown to become Google's app development platform; it now has 16 products to build and grow your app. If you've used Firebase before, you know we already offer a database, the Firebase Realtime Database, which helps with some of the challenges listed above.

    The Firebase Realtime Database, with its client SDKs and real-time capabilities, is all about making app development faster and easier. Since its launch, it has been adopted by hundred of thousands of developers, and as its adoption grew, so did usage patterns. Developers began using the Realtime Database for more complex data and to build bigger apps, pushing the limits of the JSON data model and the performance of the database at scale. Cloud Firestore is inspired by what developers love most about the Firebase Realtime Database while also addressing its key limitations like data structuring, querying, and scaling.

    So, if you're a Firebase Realtime Database user today, we think you'll love Cloud Firestore. However, this does not mean that Cloud Firestore is a drop-in replacement for the Firebase Realtime Database. For some use cases, it may make sense to use the Realtime Database to optimize for cost and latency, and it's also easy to use both databases together. You can read a more in-depth comparison between the two databases here.

    We're continuing development on both databases and they'll both be available in our console and documentation.

    Get started!

    Cloud Firestore enters public beta starting today. If you're comfortable using a beta product you should give it a spin on your next project! Here are some of the companies and startups who are already building with Cloud Firestore:

    Get started by visiting the database tab in your Firebase console. For more details, see the documentation, pricing, code samples, performance limitations during beta, and view our open source iOS and JavaScript SDKs on GitHub.

    We can't wait to see what you build and hear what you think of Cloud Firestore!