Posted by Jennifer Kohl, Global Program Manager, Google Developer Communities
The Google Developer Groups Spotlight series interviews inspiring leaders of community meetup groups around the world. Our goal is to learn more about what developers are working on, how they’ve grown their skills with the Google Developer Group community, and what tips they might have for us all.
We recently spoke with Ilias Papachristos, Google Developer Group Cloud Thessaloniki Lead in Greece. Check out our conversation with Ilias on Cloud architecture, reading official documentation, and suggested resources to help developers grow professionally.
Tell us a little about yourself?
I’m a family man, ex-army helicopter pilot, Kendo sensei, beta tester at Coursera, Lead of the Google Developer Group Cloud Thessaloniki community, Google Cloud Professional Architect, and a Cloud Board Moderator on the Google Developers Community Leads Platform (CLP).
I love outdoor activities, reading books, listening to music, and cooking for my family and friends!
Can you explain your work in Cloud technologies?
Over my career, I have used Compute Enginefor an e-shop, AutoML Tables for an HR company, and have architected the migration of a company in Mumbai. Now I’m consulting for a company on two of their projects: one that uses Cloud Run and another that uses Kubernetes.
Both of them have Cloud SQLand the Kubernetes project will use the AI Platform. We might even end up usingDataflowwith BigQuery for the streaming and Scheduler or Manager, but I’m still working out the details.
I love the chance to share knowledge with the developer community. Many days, I open my PC, read the official Google Cloud blog, and share interesting articles on the CLP Cloud Board and GDG Cloud Thessaloniki’s social media accounts. Then, I check Google Cloud’s Medium publication for extra articles. Read, comment, share, repeat!
How did the Google Developer Group community help your Cloud career?
My overall knowledge of Google Cloud has to do with my involvement with Google Developer Groups. It is not just one thing. It’s about everything! At the first European GDG Leads Summit, I met so many people who were sharing their knowledge and offering their help. For a newbie like me it was and still is something that I keep in my heart as a treasure
I’ve also received so many informative lessons on public speaking from Google Developer Group and Google Developer Student Club Leads. They always motivate me to continue talking about the things I love!
What has been the most inspiring part of being a part of your local Google Developer Group?
Collaboration with the rest of the DevFest Hellas Team! For this event, I was a part of a small group of 12 organizers, all of whom never had hosted a large meetup before. With the help of Google Developer Groups, we had so much fun while creating a successful DevFest learning program for 360 people.
What are some technical resources you have found the most helpful for your professional development?
Besides all of the amazing tricks and tips you can learn from the Google Cloud training team and courses on the official YouTube channel, I had the chance to hear a talk by Wietse Venema on Cloud Run. I also have learned so much about AI from Dale Markovitz’s videos on Applied AI. And of course, I can’t leave out Priyanka Vergadia’s posts, articles, and comic-videos!
Official documentation has also been a super important part of my career. Here are five links that I am using right now as an Architect:
I am a member of the Digital Analytics community in Thessaloniki, Greece. Their organizer asked me to write articles to start motivating young people. I translated one of the blogs into English and published it on Medium. The Lead of GDG Thessaloniki read them and asked me to become a facilitator for a Cloud Study Jams (CSJ) workshop. I accepted and then traveled to Athens to train three people so that they could also become CSJ facilitators. At the end of the CSJ, I was asked if I wanted to lead a Google Developer Group chapter. I agreed. Maria Encinar and Katharina Lindenthal interviewed me, and I got it!
What would be one piece of advice you have for someone looking to learn more about a specific technology?
Learning has to be an amusing and fun process. And that’s how it’s done with Google Developer Groups all over the world. Join mine, here. It’s the best one. (Wink, wink.)
Want to start growing your career and coding knowledge with developers like Ilias? Then join a Google Developer Group near you, here.
Posted by Wesley Chun, Developer Advocate, Google Cloud
Next generation service
Since its initial launch in 2008 as the first product from Google Cloud, Google App Engine, our fully-managed serverless app-hosting platform, has been used by many developers worldwide. Since then, the product team has continued to innovate on the platform: introducing new services, extending quotas, supporting new languages, and adding a Flexible environment to support more runtimes, including the ability to serve containerized applications.
With many original App Engine services maturing to become their own standalone Cloud products along with users' desire for a more open cloud, the next generation App Engine launched in 2018 without those bundled proprietary services, but coupled with desired language support such as Python 3 and PHP 7 as well as introducing Node.js 8. As a result, users have more options, and their apps are more portable.
With the sunset of Python 2, Java 8, PHP 5, and Go 1.11, by their respective communities, Google Cloud has assured users by expressing continued long-term support of these legacy runtimes, including maintaining the Python 2 runtime. So while there is no requirement for users to migrate, developers themselves are expressing interest in updating their applications to the latest language releases.
Each module represents a single modernization technique. Some are strongly recommended, others less so, and, at the other end of the spectrum, some are quite optional. We will guide you as far as which ones are more important. Similarly, there's no real order of modules to look at since it depends on which bundled services your apps use. Yes, some modules must be completed before others, but again, you'll be guided as far as "what's next."
More specifically, modules focus on the code changes that need to be implemented, not changes in new programming language releases as those are not within the domain of Google products. The purpose of these modules is to help reduce the friction developers may encounter when adapting their apps for the next-generation platform.
Central to the migration modules are the codelabs: free, online, self-paced, hands-on tutorials. The purpose of Google codelabs is to teach developers one new skill while giving them hands-on experience, and there are codelabs just for Google Cloud users. The migration codelabs are no exception, teaching developers one specific migration technique.
Developers following the tutorials will make the appropriate updates on a sample app, giving them the "muscle memory" needed to do the same (or similar) with their applications. Each codelab begins with an initial baseline app ("START"), leads users through the necessary steps, then concludes with an ending code repo ("FINISH") they can compare against their completed effort. Here are some of the initial modules being announced today:
Web framework migration from webapp2 to Flask
Updating from App Engine ndb to Google Cloud NDB client libraries for Datastore access
Upgrading from the Google Cloud NDB to Cloud Datastore client libraries
Moving from App Engine taskqueue to Google Cloud Tasks
Containerizing App Engine applications to execute on Cloud Run
Examples
What should you expect from the migration codelabs? Let's preview a pair, starting with the web framework: below is the main driver for a simple webapp2-based "guestbook" app registering website visits as Datastore entities:
A "visit" consists of a request's IP address and user agent. After visit registration, the app queries for the latest LIMIT visits to display to the end-user via the app's HTML template. The tutorial leads developers a migration to Flask, a web framework with broader support in the Python community. An Flask equivalent app will use decorated functions rather than webapp2's object model:
The framework codelab walks users through this and other required code changes in its sample app. Since Flask is more broadly used, this makes your apps more portable.
The second example pertains to Datastore access. Whether you're using App Engine's ndb or the Cloud NDB client libraries, the code to query the Datastore for the most recent limit visits may look like this:
def fetch_visits(limit): 'get most recent visits' query = Visit.query() visits = query.order(-Visit.timestamp).fetch(limit) return (v.to_dict() for v in visits)
If you decide to switch to the Cloud Datastore client library, that code would be converted to:
The query styles are similar but different. While the sample apps are just that, samples, giving you this kind of hands-on experience is useful when planning your own application upgrades. The goal of the migration modules is to help you separate moving to the next-generation service and making programming language updates so as to avoid doing both sets of changes simultaneously.
As mentioned above, some migrations are more optional than others. For example, moving away from the App Engine bundled ndb library to Cloud NDB is strongly recommended, but because Cloud NDB is available for both Python 2 and 3, it's not necessary for users to migrate further to Cloud Datastore nor Cloud Firestore unless they have specific reasons to do so. Moving to unbundled services is the primary step to giving users more flexibility, choices, and ultimately, makes their apps more portable.
Next steps
For those who are interested in modernizing their apps, a complete table describing each module and links to corresponding codelabs and expected START and FINISH code samples can be found in the migration module repository. We are also working on video content based on these migration modules as well as producing similar content for Java, so stay tuned.
In addition to the migration modules, our team has also setup a separate repo to support community-sourced migration samples. We hope you find all these resources helpful in your quest to modernize your App Engine apps!
Posted by Biswajeet Mallik, Program Manager, Google Developers India.
Image from Cloud Community Days India
Earlier this year, ten Google Developer Groups in India came together to host Google Cloud Community Days India, a two day event helping developers study for their upcoming Cloud Certification exams. To address the rising demand for professional certifications, the virtual event hosted over 63,000 developers, covered four main exam areas, and welcomed nine speakers. This was the second edition to the event series which started in 2019 in India.
By providing expert learning materials and mentorship, the event uniquely prepared developers for the Associate Cloud Engineer, Professional Data Engineer, Professional Cloud Machine Learning Engineer, and Professional Cloud Architect exams. Learn more below.
Acing the four key certifications
The Cloud Community Days event focused on helping developers study for four milestone certifications, tailored to engineers at four different stages of their career. The goal: help Google Developer Group members obtain the right credentials to improve their job prospects.
The event broke participants into breakout sessions based on which exam they were preparing to take. Since the certifications targeted professionals of all skill levels, study groups ranged from early career associates to late career executives. The learning groups were organized around the following certifications:
Associate Cloud Engineer:
This learning session was created to help early career developers complete the first stepping stone exam. In particular, learning materials and speakers were curated to guide participants who had no prior experience, or very little, working on the Google Cloud Platform.
Workshops were mainly dedicated to assisting programmers who were familiar with building different applications but wished to show employers that they could deploy them on Google Cloud Platform.
The next group brought together were data practitioners with special interests in data visualization and decision making. Workshops and learning activities helped these developers hone their large scale data and data driven decision making abilities.
Improving these skills are essential for passing the Professional Data Engineers certification and growing a programmer’s early career.
For these sessions, the Google Developer Group Cloud community paired experienced programmers with a significant interest in ML to form their study groups. The main driver in these learning activities was to help seasoned developers gain a deeper understanding of how to utilize Google Cloud ML services.
With significant emphasis being placed on machine learning in the ecosystem right now, Google Developer Group community leaders felt this certification could help developers make the leap into new leadership roles.
Lastly, this event paired experienced Cloud executives and professionals working in leading capacities for their organizations. For these sessions, speakers and activities had a specific scope: help high level professions be at the forefront of Google Cloud Platforms innovative capabilities.
Specifically, the Professional Cloud Architect Certification was created to help senior software engineers better design, scale and develop highly secure and robust applications.
Overall, the community put together these resources to help developers feel more confident in their abilities, obtain tangible credentials, and in turn increase access to better job opportunities. As two participants recalled the event,
“The session on Qwiklabs was so helpful, and taught me how to anticipate problems and then solve them. Cloud Community Days inspired me to take the next step with DevOps and Google Cloud.”
“This was the first time I attended the Google Developer Group event! It is an awesome package for learning in one place. All the fun activities were engaging and the panelist discussion was also very insightful. I feel proud to be a part of this grand GDG event.”
Start learning with Google Developer Groups
With Google Developer Groups, find a space to learn alongside a group of curious developers, all coming together to advance their careers from withinside a caring community of peers.
Want to know more about what Cloud Community days were like? Then watch their live recording below.
Ready to find a community event near you? Then get started at gdg.community.dev
Posted by Louis Wasserman, Software Engineer and James Ward, Developer Advocate
Kotlin is now the fourth "most loved" programming language with millions of developers using it for Android, server-side / cloud backends, and various other target runtimes. At Google, we've been building more of our apps and backends with Kotlin to take advantage of its expressiveness, safety, and excellent support for writing asynchronous code with coroutines.
Since everything in Google runs on top of gRPC, we needed an idiomatic way to do gRPC with Kotlin. Back in April 2020 we announced the open sourcing of gRPC Kotlin, something we'd originally built for ourselves. Since then we've seen over 30,000 downloads and usage in Android and Cloud. The community and our engineers have been working hard polishing docs, squashing bugs, and making improvements to the project; culminating in the shiny new 1.0 release! Dive right in with the gRPC Kotlin Quickstart!
For those new to gRPC & Kotlin let's do a quick runthrough of some of the awesomeness. gRPC builds on Protocol Buffers, aka "protos" (language agnostic & high performance data interchange) and adds the network protocol for efficiently communicating with protos. From a proto definition the servers, clients, and data transfer objects can all be generated. Here is a simple gRPC proto:
message HelloRequest { string name = 1; }
message HelloReply { string message = 1; }
service Greeter { rpc SayHello (HelloRequest) returns (HelloReply) {} }
In a Kotlin project you can then define the implementation of the Greeter's SayHello service with something like:
You'll notice that the function has `suspend` on it because it uses Kotlin's coroutines, a built-in way to handle async / reactive IO. Check out the server example project.
With gRPC the client "stubs" are generated making it easy to connect to gRPC services. For the protoc above, the client stub can be used in Kotlin with:
val stub = GreeterCoroutineStub(channel) val request = HelloRequest.newBuilder().setName("world").build() val response = stub.sayHello(request) println("Received: ${response.message}")
In this example the `sayHello` method is also a `suspend` function utilizing Kotlin coroutines to make the reactive IO easier. Check out the client example project.
Kotlin also has an API for doing reactive IO on streams (as opposed to requests), called Flow. gRPC Kotlin generates client and server stubs using the Flow API for stream inputs and outputs. The proto can define a service with unary streaming or bidirectional streaming, like:
This example just transforms each `HelloRequest` item on the flow to an item in the output / `HelloReply` Flow.
The bidirectional stream client is similar to the coroutine one but instead it passes a Flow to the `sayHello` stub method and then operates on the returned Flow:
val stub = GreeterCoroutineStub(channel) val helloFlow = flow { while(true) { delay(1000) emit(HelloRequest.newBuilder().setName("world").build()) } }
In this example the client sends a `HelloRequest` to the server via Flow, once per second. When the client gets items on the output Flow, it just prints them. Check out the bidi-streaming example project.
As you've seen, creating data transfer objects and services around them is made elegant and easy with gRPC Kotlin. But there are a few other exciting things we can do with this...
Android Clients
Protobuf compilers can have a "lite" mode which generates smaller, higher performance classes which are more suitable for Android. Since gRPC Kotlin uses gRPC Java it inherits the benefits of gRPC Java's lite mode. The generated code works great on Android and there is a `grpc-kotlin-stub-lite` artifact which depends on the associated `grpc-protobuf-lite`. Using the generated Kotlin stub client is just like on the JVM. Check out the stub-android example and android example.
GraalVM Native Image Clients
The gRPC lite mode is also a great fit for GraalVM Native Image which turns JVM-based applications into ahead-of-time compiled native images, i.e. they run without a JVM. These applications can be smaller, use less memory, and start much faster so they are a good fit for auto-scaling and Command Line Interface environments. Check out the native-client example project which produces a nice & small 14MB executable client app (no JVM needed) and starts, connects to the server, makes a request, handles the response, and exits in under 1/100th of a second using only 18MB of memory.
Google Cloud Ready
Backend services created with gRPC Kotlin can easily be packaged for deployment in Kubernetes, Cloud Run, or really anywhere you can run docker containers or JVM apps. Cloud Run is a cloud service that runs docker containers and scales automatically based on demand so you only pay when your service is handling requests. If you'd like to give a gRPC Kotlin service a try on Cloud Run:
We are super excited to have reached 1.0 for gRPC Kotlin and are incredibly grateful to everyone who filed bugs, sent pull requests, and gave the pre-releases a try! There is still more to do, so if you want to help or follow along, check out the project on GitHub.
Also huge shoutouts to Brent Shaffer, Patrice Chalin, David Winer, Ray Tsang, Tyson Henning, and Kevin Bierhoff for all their contributions to this release!
In this unprecedented time in history, we’ve seen millions of businesses across India adapt to the new normal of working remotely and learning from home. It is human nature to connect, and video conferencing plays a pivotal role here.
We saw more than 3 million new users connecting on Google Meet every day as of this month, spending over 3 billion minutes a day together. That’s a massive 30x jump from the numbers we saw earlier in January. And today, we’re making our premium video conferencing tool, Google Meet free for everyone, with availability rolling out over the coming weeks.
Stay home, stay safe, stay connected
Starting May, anyone with an email address can create a Google Account and use Meet to schedule, join or start secure video meetings with anyone — whether it’s a virtual dance class, a weekly book club, neighborhood meetings, or any other reason to connect with your community, friends, and family.
Until today, Google Meet was only available as part of G Suite, our collaboration and productivity solution for businesses, organizations and schools. Now, it is freely available on the web at meet.google.com and via mobile apps for iOSorAndroid. And if you use Google Calendar, you can easily start or join from there too.
We’re also rolling out new features including tiled layout for larger calls, the option to present a Chrome tab (instead of just presenting their window or entire screen), low-light mode and eventually noise cancellation.
Secure by design
Meet is designed, built and operated to be secure at scale — for everyone. We employ a vast array of safe-by-default measures to keep your meetings safe without doing a thing, everytime. We don't require or ask for any plugins to be installed, reducing the amount of software users and businesses need to patch with security updates on their machines.
We also ensure that only authorised users can use and access Meet services by using a 2-Step Verification option for account — making them secure and convenient. Google Meet users can enroll their accounts in our Advanced Protection Program (APP), which provides our strongest protections available against phishing and account hijacking, and is specifically designed for the highest-risk accounts.
Helping businesses collaborate with G Suite
We’re not just connecting over video. We’ve also seen huge spikes in the use of our entire G Suite offering as more people create, share, and connect together while working remotely. Earlier this year we marked another major milestone — surpassing six million paying businesses and organisations who use G Suite.
“Employees are able to access every business application via Google Cloud Platform and continue to communicate as usual not only between themselves but also with customers, vendors and other stakeholders with G Suite. The Google Cloud team is always accessible and supportive to help us ease things. The use of collaborative tools has facilitated important human contact and responsiveness in an unprecedented time of remote work,” said V M Samir, Group CIO, Rustomjee, a leading real estate company in Mumbai.
Mathan Babu Kasilingam, CISO of National Payments Corporation of India, an umbrella organisation for all retail payment systems in India says, “Google Meet has played a good role in helping our teams stay connected. It’s great to see that it is possible to work across various remote locations and manage to carry on business as usual through video conferencing.”
Here is what TR Chadha & Co, one of India’s prominent chartered accountancy firms, had to say. “G Suite has been a lifeline for the teams for the past month since we have transitioned to a work from home set-up due to the pandemic. With G Suite, our teams can securely log-in from any device to work wherever they are at any time,” said Gautam Kumar, IT Manager, TR Chadha.
Securely stay connected and productive not just today but also in the future
We’re carefully rolling out Meet incrementally over the coming weeks to ensure we can provide everyone with the reliability and security they expect from Google. This means you might not be able to create meetings right away, but you can sign up to be notified when it’s available.
Meetings are limited to 60 minutes for the free product, though we will not enforce this time limit until after September 30. Creating a trusted meeting space is important, so being mindful when sharing meeting links in public forums can help create a safe experience for all attendees. For more tips on how to use Meet securely and effectively, visit our Help Center.
Posted by Karan Bajwa, Managing Director, Google Cloud India
At Google Cloud, our mission is to accelerate every organisation’s ability to transform through data-powered innovation with leading infrastructure, platform, industry solutions and expertise designed to meet our customers where they are on their journey to the cloud.
Our cloud regions bring Google Cloud Platform (GCP) services to global organizations in industries like media and entertainment, retail and manufacturing, helping them drive growth, differentiation, and innovation for their business. As our customers in India grow and diversify, we continue to advance and invest in our cloud infrastructure to help regulated industries such as healthcare and financial services, as well as public sector organisations across India achieve their goals.
Introducing our next cloud region in India
Today, we are thrilled to share our plans to open a Delhi cloud region. This will be our second cloud region in India since we launched our Mumbai region in 2017, and will expand our existing network of eight regions in Asia Pacific and 22 regions globally today.
Projected to launch in 2021, our Delhi cloud region will have three zones to protect against service disruptions. The region will also include a portfolio of key GCP products, offer lower latency to nearby users, and, when combined with the existing region in Mumbai, enables geographically separate in-country disaster recovery for our customers’ mission critical applications.
For customers in Delhi and beyond—and businesses and organisations yet to try our services—the new region presents an opportunity to take advantage of our big data and infrastructure services onshore while staying compliant with India’s data laws and regulations.
Here’s what some of our customers have to say about the value of our expanding GCP infrastructure brings to their business:
“Cloud is the technology that enables us to achieve scale and reach. Today there are countless data points available about rural consumers which enable us to personalize our products to serve them better. With access to faster compute power, we can also on-board consumers more efficiently. Our rural businesses have clocked a disbursement CAGR of 60% over the past three years."
-- Sunil Prabhune, Chief Executive-Rural Finance, and Group Head-Digital, IT and Analytics, L&T Financial Services
“We knew we made the right choice when we selected Dialogflow Enterprise Edition as our strategic, long-term platform to run intelligent chatbots across our inhouse processes and functions. A Delhi cloud region shows we are working with a cloud provider that shares our commitment to high quality in-market and multinational services.” -- Mukesh Rathi, CIO & Chief Digital Officer, Dr. Reddy's Laboratories
“Buyers and suppliers can already access our marketplace much faster than previously with Google Cloud, and this has a positive impact on customer engagement, time spent and the entire user journey. We are extremely excited about the potential of a second GCP region in India to help us provide an even better experience to the businesses that use IndiaMART.” -- Amarinder S Dhaliwal, Chief Product Officer, IndiaMART
“Google Cloud is transforming the cloud market similar to what NoBroker is doing by providing C2C platform for real estate needs without any brokerages. A second Google Cloud region in India will give us access to secure infrastructure closure to customers as we recently launched NoBroker services in NCR region. It will also help us retain customer data within India - meeting our compliance requirements.” -- Akhil Gupta, Co Founder and Chief Technology Officer, NoBroker
2020 is already shaping up to be a great year for Google Cloud in India, and we look forward to helping businesses and governments solve their most complex challenges and create value for years to come.
Posted by Rick Harshman, Managing Director, Google Cloud Asia Pacific
Since the first days of Google Cloud Platform (GCP), Google has been at the forefront of making your applications more observable. Beyond Stackdriver, our most visible impact in this space is OpenTelemetry, which we initiated in 2017 (as OpenCensus) and has grown into a huge community that includes the majority of APM / monitoring vendors and cloud platforms.
While OpenTelemetry allows developers to easily capture distributed traces and metrics from their own services, there’s also a need to trace requests as they propagate through components that developers don’t directly control, like managed services, load balancers, network hardware, etc. To solve this we co-defined a prototype HTTP header that these components can rely on, gathered partners, and moved the work into the W3C.
This work is now complete, and the W3C Trace Context format is now an official standard. Once implemented in GCP, this will make our services even easier to manage, both with Stackdriver and other third party distributed tracing tools. We explain more in the official post on the W3C blog, which I’ve copied below:
The W3C Distributed Tracing working group has moved the Trace Context specification to the next maturity level. The specification is already being adopted and implemented by many platforms and SDKs. This article describes the Trace Context specification and how it improves troubleshooting and monitoring of modern distributed apps.
W3C Trace Context specification defines the format for propagating distributed tracing context between services. Distributed tracing makes it easy for developers to find the causes of issues in highly-distributed microservices applications by tracking how a single interaction was processed across multiple services. Each step of a trace is correlated through an ID that is passed between services, and W3C Trace Context now defines a standard for these context propagation headers.
Until now, different tracing systems have defined their own headers. Examples include Zipkin’s B3 format and X-Google-Cloud-Trace. Adopting a common context propagation format has been long desired by developers, APM vendors, and cloud platform hosts, as compatibility provides numerous benefits:
Web and RPC frameworks that use this standard to provide context propagation out of the box will also offer cross-service log correlation, even for developers who haven’t set up distributed tracing.
API producers can record the trace IDs of requests from API consumers and provide additional spans or metadata to their customers for a given traced request. Producers can also correlate customer trace IDs to internal traces when debugging technical issues raised by consumers.
Networking infrastructure (proxies, load balancers, routers, etc.) can both ensure that context propagation headers are not removed from requests passing through them, and can record spans or logs for a given trace, without having to support multiple vendor-specific formats. Potential examples of these include router appliances, cloud load balancers, and sidecar proxies like Envoy.
Instrumentation can be further decoupled from a developer’s choice of APM vendor. For example, using both OpenTelemetry and a given vendor’s agents, a developer can instrument different services in an application, and traces will flow through the system and be processed correctly by the vendor’s backend.
Web browsers and other clients can use these identifiers to correlate their telemetry with traces collected from backend services. This functionality is currently being defined.
To address this effort, a group of cloud providers, open source contributors, and APM vendors started defining a standard HTTP context propagation header that would replace their homegrown formats. This specification has been discussed and iterated on over the past two years, and the group working on it has grown significantly over that time. Sponsors include Google, Microsoft, Dynatrace, and New Relic (W3C members), and the group was officially moved into the W3C in 2018 for the work to proceed under the guidance of an official standards body and to spur even greater adoption.
TraceContext has since been adopted by OpenTelemetry (which enables it by default and also serves as the reference implementation), Azure services, Dynatrace, Elastic, Google Cloud Platform, Lightstep, and New Relic. We are tracking adoption in this list.
This first phase of work has focused on HTTP, as it is commonly used and has no built-in affordances for trace context propagation (gRPC and some newer RPC systems do). The same group of committee members are also working to define trace context propagation in other formats, starting with AMQP and MQTT for IoT; other upcoming topics include context propagation from clients and web browsers.
Posted by Paul Dille, Senior Software Developer, Carnegie Mellon University CREATE Lab, and Chris Herwig, Geo Data Engineer, Google Earth Outreach Six years ago, we first introduced Google Earth Timelapse, a global, zoomable time-lapse video that lets anyone explore our changing planet’s surface—from the global scale to the local scale. Earth Timelapse consists of 83 million multi-resolution overlapping video tiles, which are made interactively explorable through the open-source Time Machine client software developed at Carnegie Mellon University’s CREATE Lab. At its core, Google Earth Timelapse is an example of how organizing information can make it more accessible and useful, turning petabytes of satellite imagery into an interactive experience that shows the dynamic changes occurring across space and time. In April, we introduced several updates to Timelapse, including two additional years of imagery to the time-series visualization, which now spans from 1984 to 2018, with visual upgrades that make exploring more accessible and intuitive. We are especially excited that this update includes support for mobile and tablet devices, which are quickly overtaking desktop computers as the dominant source of app traffic.
Building the Global Visualization Making a planetary-sized time-lapse video required a significant amount of pixel crunching in Earth Engine, Google's cloud platform for petabyte-scale geospatial analysis. The new release followed a process similar to what we did in 2013, but at a significantly greater scale—turning 15 million satellite images acquired over the last three and a half decades from the USGS/NASA Landsat and European Sentinel programs into 35 cloud-free 4-terapixel images of the planet—one for each year from 1984 to 2018.
At its native resolution, the Timelapse visualization is a 4 terapixel video (that's four trillion pixels), which would take about 12 days to download on a 95 Mb/s internet connection. Most computers would have difficulty playing a video of this size, let alone with an interactive, zoomable interface. The problem is even more severe for a mobile device.
A solution was pioneered by Google Maps in 2004 with the map pyramiding technique. Before that time, navigating a map required the use of directional arrows to pan and zoom, with each step requiring the page to reload. The map pyramiding technique assembles the full map image displayed on-screen from tens of small 256x256 pixel non-overlapping image tiles in an array, with new tiles fetched as needed at an appropriate resolution as the user pans and zooms across the map.
A traditional Mercator map pyramid contains non-overlapping image tiles.
This works very well for maps made of static images, but less so for pyramids of video tiles, such as those used by Timelapse, since it requires a web browser to keep up to 16 videos in sync while interacting with the visualization. The solution is embodied in CREATE Lab’s open source Time Machine software: create much larger video tiles that can cover the entire screen and only show one whole-screen tile at a time. The tiles create a pyramid, where sibling tiles overlap with their neighbors to provide a seamless transition between tiles while panning and zooming. Though the overlapping tiles require the use of about 16x more videos, this pyramid structure enables the use of Timelapse on mobile devices by minimizing the amount of data required for visualization.
In our newest release, the global video pyramid consists of 83 million videos across 13 zoom levels, which required about 2 million CPU hours distributed across thousands of machines in Google Cloud to generate.
Earth Timelapse uses a pyramid of overlapping video tiles.
Time Travel, Wherever You Are Prior to April's update, ~30% of visitors to the Timelapse visualization were on mobile devices and didn't actually experience the visualization; instead they saw a YouTube playlist of locations in Timelapse. Until recently, the hardware and CPUs for phones and tablets could not decode videos fast enough without significant delays when someone attempted to zoom in or pan across a video, making mobile exploration unpleasant, if not impossible. In addition, in order for the visualization to be smooth as you pan and zoom, each video that is loaded must sync to the previously playing video and begin playing automatically. But, until only recently, mobile browser vendors had disabled video autoplay at the browser level for bandwidth reasons.
Now that mobile browser vendors have re-enabled video autoplay, we are able to take advantage of current mobile hardware and CPU capabilities, while leveraging the pyramid mapping technique’s efficient use of data, to enable Timelapse on mobile.
Redesigning Timelapse for Exploration Across Devices Timelapse is a tool for exploration, so we designed for immersiveness, devoting as much real estate as possible to the map. On the other hand, it's not just a map, but a map of videos. So we kept controls visible, like pausing and restarting the timeline or choosing highlights, by leveraging Material Design with simple, clean lines and clear focal areas.
Navigate with Google Maps using the new "Maps Mode" toggle.
To explore, you need to know where you are or where somewhere else is, so the new interface includes a new "Maps Mode" toggle that lets the user navigate with Google Maps. We also built in scalability to the timeline element of the UI, so that new features added in the future, such as lengthening the time-lapse or adding options for different time increments, won’t break the design. The timeline also allows the user to go backwards in time—an interesting way to compare the present with the past.
For desktop browsers supporting WebGL, we also added a new WebGL viewer to the open source project, which loads and synchronizes multiple videos to fill the screen at optimal resolution. The aesthetic improvement of this is nontrivial, with >4x better resolution.
What's next We're excited about the abundance of freely available, openly licensed satellite imagery and remote sensing data available, enabling new visualizations across time, space, and the visual and non-visual spectrum. We've found it's often the data combined with supplemental layers, such as the World Database on Protected Areas (WDPA) boundaries, that can spark new insights. For example, seeing the visual connection between declining home ownership and shifts in the city of Pittsburgh's racial makeup tells a story about inequality that numbers on a page simply cannot. Visual evidence can transcend language and cultural barriers and, we hope, generate productive conversations about our global challenges.
Acknowledgements Randy Sargent, Senior Systems Scientist, Carnegie Mellon University CREATE Lab and the Google Earth Engine team
Posted by Amit Ganesh, VP Engineering & Dan McGrath, Product Manager
As modern application development moves away from managing infrastructure and toward a serverless future, we're pleased to announce the general availability of Cloud Firestore, our serverless, NoSQL document database. We're also making it available in 10 new locations to complement the existing three, announcing a significant price reduction for regional instances, and enabling integration with Stackdriver for monitoring.
Cloud Firestore is a fully managed, cloud-native database that makes it simple to store, sync, and query data for web, mobile, and IoT applications. It focuses on providing a great developer experience and simplifying app development with live synchronization, offline support, and ACID transactions across hundreds of documents and collections. Cloud Firestore is integrated with both Google Cloud Platform (GCP) and Firebase, Google's mobile development platform. You can learn more about how Cloud Firestore works with Firebase here. With Cloud Firestore, you can build applications that move swiftly into production, thanks to flexible database security rules, real-time capabilities, and a completely hands-off auto-scaling infrastructure.
Cloud Firestore does more than just core database tasks. It's designed to be a complete data backend that handles security and authorization, infrastructure, edge data storage, and synchronization. Identity and Access Management (IAM) and Firebase Auth are built in to help make sure your application and its data remain secure. Tight integration with Cloud Functions, Cloud Storage, and Firebase's SDK accelerates and simplifies building end-to-end serverless applications. You can also easily export data into BigQuery for powerful analysis, post-processing of data, and machine learning.
Building with Cloud Firestore means your app can seamlessly transition from online to offline and back at the edge of connectivity. This helps lead to simpler code and fewer errors. You can serve rich user experiences and push data updates to more than a million concurrent clients, all without having to set up and maintain infrastructure. Cloud Firestore's strong consistency guarantee helps to minimize application code complexity and reduces bugs. A client-side application can even talk directly to the database, because enterprise-grade security is built right in. Unlike most other NoSQL databases, Cloud Firestore supports modifying up to 500 collections and documents in a single transaction while still automatically scaling to exactly match your workload.
What's new with Cloud Firestore
New regional instance pricing. This new pricing takes effect on March 3, 2019 for most regional instances, and is as low as 50% of multi-region instance prices.
Data in regional instances is replicated across multiple zones within a region. This is optimized for lower cost and lower write latency. We recommend multi-region instances when you want to maximize the availability and durability of your database.
SLA now available. You can now take advantage of Cloud Firestore's SLA: 99.999% availability for multi-region instances and 99.99% availability for regional instances.
New locations available. There are 10 new locations for Cloud Firestore:
Multi-region
Europe (eur3)
North America (Regional)
Los Angeles (us-west2)
Montréal (northamerica-northeast1)
Northern Virginia (us-east4)
South America (Regional)
São Paulo (southamerica-east1)
Europe (Regional)
London (europe-west2)
Asia (Regional)
Mumbai (asia-south1)
Hong Kong (asia-east2)
Tokyo (asia-northeast1)
Australia (Regional)
Sydney (australia-southeast1)
Cloud Firestore is now available in 13 regions.
Stackdriver integration (in beta). You can now monitor Cloud Firestore read, write and delete operations in near-real time with Stackdriver.
More features coming soon. We're working on adding some of the most requested features to Cloud Firestore from our developer community, such as querying for documents across collections and incrementing database values without needing a transaction.
As the next generation of Cloud Datastore, Cloud Firestore is compatible with all Cloud Datastore APIs and client libraries. Existing Cloud Datastore users will be live-upgraded to Cloud Firestore automatically later in 2019. You can learn more about this upgrade here.
Adding flexibility and scalability across industries
Cloud Firestore is already changing the way companies build apps in media, IoT, mobility, digital agencies, real estate, and many others. The unifying themes among these workloads include: the need for mobility even when connectivity lapses, scalability for many users, and the ability to move quickly from prototype to production. Here are a few of the stories we've heard from Cloud Firestore users.
When opportunity strikes...
In the highly competitive world of shared, on-demand personal mobility via cars, bikes, and scooters, the ability to deliver a differentiated user experience, iterate rapidly, and scale are critical. The prize is huge. Skip provides a scooter-sharing system where shipping fast can have a big impact. Mike Wadhera, CTO and Co-founder, says, "Cloud Firestore has enabled our engineering and product teams to ship at the clock-speed of a startup while leveraging Google-scale infrastructure. We're delighted to see continued investment in Firebase and the broader GCP platform."
Another Cloud Firestore user, digital consultancy The Nerdery, has to deliver high-quality results in a short period of time, often needing to integrate with existing third-party data sources. They can't build up and tear down complicated, expensive infrastructure for every client app they create. "Cloud Firestore was a great fit for the web and mobile applications we built because it required a solution to keep 40,000-plus users apprised of real-time data updates," says Jansen Price, Principal Software Architect. "The reliability and speed of Cloud Firestore coupled with its real-time capabilities allowed us to deliver a great product for the Google Cloud Next conferences."
Reliable information delivery
Incident response company Now IMS uses real-time data to keep citizens safe in crowded places, where cell service can get spotty when demand is high. "As an incident management company, real-time and offline capabilities are paramount to our customers," says John Rodkey, Co-founder. "Cloud Firestore, along with the Firebase Javascript SDK, provides us with these capabilities out of the box. This new 100% serverless architecture on Google Cloud enables us to focus on rapid application development to meet our customers' needs instead of worrying about infrastructure or server management like with our previous cloud."
Regardless of the app, users want the latest information right away, without having to click refresh. The QuintoAndar mobile application connects tenants and landlords in Brazil for easier apartment rentals. "Being able to deliver constantly changing information to our customers allows us to provide a truly engaging experience. Cloud Firestore enables us to do this without additional infrastructure and allows us to focus on the core challenges of our business," says Guilherme Salerno, Engineering Manager at QuintoAndar.
Real-time, responsive apps, happy users
Famed broadsheet and media company The Telegraph uses Cloud Firestore so registered users can easily discover and engage with relevant content. The Telegraph wanted to make the user experience better without having to become infrastructure experts in serving and managing data to millions of concurrent connections. "Cloud Firestore allowed us to build a real-time personalized news feed, keeping readers informed with synchronized content state across all of their devices," says Alex Mansfield-Scaddan, Solution Architect. "It allowed The Telegraph engineering teams to focus on improving engagement with our customers, rather than becoming real-time database and infrastructure experts."
On the other side of the Atlantic, The New York Times used Cloud Firestore to build a feature in The Times' mobile app to send push notifications updated in real time for the 2018 Winter Olympics. In previous approaches to this feature, scaling had been a challenge. The team needed to track each reader's history of interactions in order to provide tailored content for particular events or sports. Cloud Firestore allowed them to query data dynamically, then send the real-time updates to readers. The team was able to send more targeted content faster.
Delivering powerful edge storage for IoT devices
Athlete testing technology company Hawkin Dynamics was an early, pre-beta adopter of Cloud Firestore. Their pressure pads are used by many professional sports teams to measure and track athlete performance. In the fast-paced, high-stakes world of professional sports, athletes can't wait around for devices to connect or results to calculate. They demand instant answers even if the WiFi is temporarily down. Hawkin Dynamics uses Cloud Firestore to bring real-time data to athletes through their app dashboard, shown below.
"Our core mission at Hawkin Dynamics is to help coaches make informed decisions regarding their athletes through the use of actionable data. With real-time updates, our users can get the data they need to adjust an athlete's training on a moment-by-moment basis," says Chris Wales, CTO. "By utilizing the powerful querying ability of Cloud Firestore, we can provide them the insights they need to evaluate the overall efficacy of their programs. The close integrations with Cloud Functions and the other Firebase products have allowed us to constantly improve on our product and stay responsive to our customers' needs. In an industry that is rapidly changing, the flexibility afforded to us by Cloud Firestore in extending our applications has allowed us to stay ahead of the game."
Getting started with Cloud Firestore
We've heard from many of you that Cloud Firestore is helping solve some of your most timely development challenges by simplifying real-time data and data synchronization, eliminating server-side code, and providing flexible yet secure database authentication rules. This reflects the state of the cloud app market, where developers are exploring lots of options to help them build better and faster while also providing modern user experiences. This glance at Stack Overflow questions gives a good picture of some of these trends, where Cloud Firestore is a hot topic among cloud databases.
We've seen close to a million Cloud Firestore databases created since its beta launch. The platform is designed to serve databases ranging in size from kilobytes to multiple petabytes of data. Even a single application running on Cloud Firestore is delivering more than 1 million real-time updates per second to users. These apps are just the beginning. To learn more about serverless application development, take a look through the archive of the recent application development digital conference.
Posted by Jeremy Neuner, Head of Launchpad San Francisco
Google's Developer Relations team is opening a new innovation space at 543 Howard St. in San Francisco. By working with more than a million developers and startups we've found that something unique happens when we interact with our communities face-to-face. Talks, meetups, workshops, sprints, bootcamps, and social events not only provide opportunities for Googlers to authentically connect with users but also build trust and credibility as we form connections on a more personal level.
The space will be the US home of Launchpad, Google's startup acceleration engine. Founded in 2016 the Launchpad Accelerator has seen 13 cohorts graduate across 5 continents, reaching 241 startups. In 2019, the program will bring together top Google talent with startups from around the world who are working on AI-enabled solutions to problems in financial technology, healthcare, and social good.
In addition to its focus on startups, the Google innovation space will offer programming designed specifically for developers and designers throughout the year. For example, in tandem with the rapid growth of Google Cloud Platform, we will host hands-on sessions on Kubernetes, big data and AI architectures with Google engineers and industry experts.
Finally, we want the space to serve as a hub for industry-wide Developer Relations' diversity and inclusion efforts. And we will partner with groups such as Manos Accelerator and dev/Mission to bring the latest technologies to underserved groups.
We designed the space with a single credo in mind, "We must continually be jumping off cliffs and developing our wings on the way down." The flexible design of the space ensures our community has a place to learn, experiment, and grow.
For more information about our new innovation space, click here.
