Author Archives: Google Developers

How a student leader cultivates on-campus diversity and leadership in Australia

Posted by Matthew Ranocchiari, Developer Relations Community Manager, Australia & New Zealand

Banner image shows Milindi Kodikara smiling with text that reads How a student leader cultivates on-campus diversity and leadership in Australia

To get familiar with her local community, International student Milindi Kodikara, originally from Sri Lanka, joined almost every tech club at RMIT University in Melbourne, Australia. She immediately identified an opportunity to establish a more diverse and inclusive tech club to prepare students to work in the tech industry. Using both her instincts and passion for community building, she set out on a mission to re-establish a Google Developer Student Club (GDSC) at RMIT University.

“I applied to become the GDSC Lead at RMIT University to create a better, brighter, and stronger community where our students (regardless of age, gender, sexuality, or race) can thrive, learn new skills, and enjoy their time with new friends,” Kodikara says.

Preparing students for tech careers

After starting the club, Milindi sought the advice of Matthew Ranocchiari (Google Developer Relations Community Manager for the region) who then suggested Milindi form a team, set goals, and plan activities for the semester. Milindi assembled a team of seven students hailing from Malaysia, the Philippines, Indonesia, Bangladesh, and Sri Lanka. They brought a mix of technical and non-technical backgrounds. Each shared an eagerness to learn about technology, help other students build projects, and develop leadership skills.

“ I was looking for a team of passionate individuals to make a change at RMIT University.”

Screenshot from a Google Video Chat shows the GDSC core team at RMIT University. They are all smiling at the camera

Social media officer Chaamudi Kodikara, helped set up the club’s various channels, which currently engage 400+ students every day. Design experts Isaac (Yi Jie) Chuah, Jacqueline Ann Lim, and Andrea Gocheco; event managers Sheryl Mantik and Kowsar Rahman Sadit; and a tech expert Prottay Karim each round out the group.

Planning a “Study Jam-packed” semester

When asking students what they wanted from the new club, most students said they were seeking an opportunity to learn how to land a tech job through networking, get a sense of what tech jobs are like day-to-day, and explore various technologies. Most of all, they wanted to feel empowered. With these goals in mind, the team planned multiple events per month, including an inaugural virtual Games Night, a series of lightning talks (called Geeky Google Tech Talks), and Cloud Study Jams.

“As Cloud is already a field that is in high demand, we thought our community could benefit from learning this for their own career success,” explains Milindi.

Embracing school spirit and multiple learning opportunities

To ensure students would be prepared for workshops on Artificial Intelligence, Machine Learning, and robotics, the club held beginner-friendly programs that introduced students to the topics. They launched these in collaboration with RMIT's Society for Women in Information Technology, RMIT's Programming Club and additional partners and sponsors. The club’s HackVision hackathon, featured games, mentoring sessions, and workshops on topics such as public speaking, pitching ideas, ideation, UX/UI design, and software architecture.

With a wide variety of workshops that drew many attendees, the events and facilitators helped students with their technical queries throughout the semester. Some facilitators such as industry professional Thomas Frantz, Google Chrome Engineer, Jakub Młynarczyk, and Patrick Haralabidis of the Melbourne TensorFlow User Group stay connected with the students. Their connections and willingness to help solidifies the importance of mentorship. For example, one instructor led a machine learning workshop, while another one walked students through the complex algorithm-related questions asked during technical job interviews. Milindi herself led a live robotics demo where she programmed Nao Bots to dance to music and showed other students how to program the robots.

“These workshops helped them expand their tech skills to unexplored areas and help them feel confident at interviews. Our speakers showed students what is possible with these new technologies and machines. The time and effort spent on this event was truly rewarding seeing how much all the participants learned and experienced,” Milindi says.

Collage style image made up of images of GDSC RMIT members, memes, emojis, and graphics created by the group

Representation and professional development

Professional development remains important too, notes Milindi. “We here at GDSC RMIT have quite a diverse community, and we wanted to make sure our community was well prepared for the hardships of finding success.” She says, "as a female international student, I've had to apply to hundreds of companies when I was looking for internships, so I understand the struggle of applying for jobs, especially the frustration that comes with being rejected, in most cases, for reasons beyond your control, like visa issues.”

In this spirit, the club collaborated with RMIT's Society for Women in Information Technology to hold an event called "ROAR!” to empower female and underrepresented students. The event included an #IamRemarkable workshop with Googlers. Additionally, the club encouraged students to develop their resumes and LinkedIn profiles and offered a Personal Branding workshop, giving students multiple opportunities to showcase their talents and abilities.

Preparing a diverse student body for tech careers

Milindi expresses gratitude for having the opportunity to reinvigorate the community around RMIT University’s Google Developer Student Club. She relishes the ability to bring diverse students from many countries and gender identities together in the service of developing new skills, learning from each other, and ultimately landing jobs in the fast-growing tech industry.

"My amazing Core Team and I have given GDSC at RMIT University a new chapter and made a safe space for tech nerds of all ages, races, genders, and sexualities to feel welcome to be themselves and strive to be successful," Milindi says. "I am grateful for this fantastic opportunity to serve our community and help make the world a better place.”

For more information on this GDSC chapter visit the site here. If you’re a student and would like to join a Google Developer Student Club community, look for a chapter near you here, or visit the program page to learn more about starting one in your area.

Year in review: the Google Workspace Platform 2021

Posted by Charles Maxson, Developer Advocate

In 2021, we saw many changes and improvements to the Google Workspace Platform geared at helping developers build new solutions to keep up with the challenges of how we worked, like hybrid and fully remote office work. More than ever, we needed tools for virtual collaboration and digital processes to keep our work going. As paper processes in the office were less viable and we continued to go see digital transformations become necessary, many new custom solutions like desk reservation systems and automated test logging have evolved.

2021 was also a year for Platform milestones, Google Workspace grew to more than 3 billion users globally, we reached more than 5,300 public apps in the Google Workspace Marketplace, and we crossed over 4.8 billion apps installed (up from 1 billion in 2020)! We were also busy bringing Platform innovations and improving our developer experience to help building for Google Workspace easier and faster. Here’s a look at some of the key enhancements the Google Workspace Platform brought to the developer community.

Google Cloud Champion Innovators program

Community building is one of the most effective ways to support developers, which is why we created Google Cloud Innovators.This new community program was designed for developers and technical practitioners using Google Cloud and we welcome everyone.

And when we say everyone, it’s not just professional developers, data scientists, or student developers and hobbyists, we also mean non-technical end users. The growing Google community has something for everyone.

GWAO Alternate Runtimes goes GA

Google Workspace Add-ons are customized applications that tightly integrate with Google Workspace applications, and can be found in the Google Workspace Marketplace, or built specifically for your own domain. The development of these applications were limited to using Apps Script, our native scripting language for the Google Workspace Platform. With the launch of Alternate Runtimes you can now develop add-ons with your preferred hosting infrastructure, development tool chain, source control system, coding language, and code libraries; it was a highly requested update from the developer community, opening up the Platform to many new developer scenarios.

Card Builder UI Application

The GWAO Card Builder tool allows you to visually design the user interfaces for your Google Workspace Add-ons and Google Chat apps projects. It is a must-have for Google Workspace developers using either Apps Script or Alternate Runtimes, enabling you to prototype and design Card UIs super fast without hassle and errors of hand coding JSON or Apps Script on your own.

Card Builder tool for building Google Workspace Add-ons and Chat Apps

Recommended for Google Workspace

This program showcases a selection of market-leading applications built by software vendors across a wide range of categories, including project management, customer support, and finance in our Google Workspace Marketplace. These apps undergo rigorous usability and security testing to make sure they meet our requirements for high quality integrations. They must also have an exemplary track record of user satisfaction, reliability, and privacy.

Recommended for Google Workspace program showcases high quality applications

Chat Slash Commands and Dialogs

Slash commands simplify the way users interact with your Chat bot, offering them a visual leading way to discover and execute your bot’s primary features. As a developer, slash commands are straightforward to implement, and essential in offering a better bot experience. In addition to Slash Commands, Dialogs were a new capability introduced to the Chat App framework that allows developers to build user interfaces to capture inputs and parameters in a structured, reliable way. This was a tremendous step forward for bot usability because it simplified and streamlined the process of users interacting with bot commands. Now with dialogs, users can be led visually to supply inputs via prompts, versus having to rely on wrapping bot commands with natural language inputs.

Forms API beta

Google Forms enables easy creation and distribution of forms, surveys, and quizzes. Forms is used for a wide variety of use cases across business operations, customer management, event planning and logistics, education, and more. With the Google Forms API Beta announcement, developers were able to provide programmatic access for managing forms and acting on responses, empowering developers to build powerful integrations on top of Forms.

Google Workspace Marketplace updates

We made many updates to the Google Workspace Marketplace to improve both the user and developer experience. We added updates to the application detail page that included pricing and when the listing was last updated. The homepage also saw improvements with various curated categories by the Google team under Editor’s Choice. Finally, we launched the marketplace badges for developers to promote their published applications on websites and marketing channels. Oh, and we also had a logo update if you hadn’t noticed.

Google Workspace Marketplace Badges for application promotion

Farewell 2021 and here’s to welcoming in 2022

2021 brought us many innovations to the Google Workspace Platform to help developers address the needs of their users and it also brought more empowerment to knowledge workers to build the solutions they needed with our no-code and low-code platforms. These are just the highlights for the Google Workspace Platform and we look forward to more innovation in 2022. To keep up with all the news about the Platform, please subscribe to our newsletter.

Google Developer Group Spotlight: A conversation with software developer Aditi Soni

Posted by Manoranjan Padhy - Developer Relations Lead, India

Six years ago, Aditi Soni was new to computers and programming when she learned about Google Developer Groups (GDG) and Women Techmakers (WTM) from a senior at her university, the Shri Vaishnav Institute of Technology and Science, Indore. Then, everything changed when she joined a Google Developer Group in Indore, the largest city in Central India, which began as a 16th century trading hub.

“Initially, it was extremely overwhelming for me to be in that space, where so many accomplished professionals were present,” Aditi says of her first experiences attending GDG Indore. “I was very hesitant to go and have a conversation with them.”

But Aditi felt determined. Her friend Aditya Sharma taught her C and C++, and she practiced her programming skills on her smartphone, using tools like the C4droid Android app, because she didn’t have a laptop. By the time she got a laptop, she was off and running. Aditi began teaching herself Android development and landed an internship after her second year of college.

image of Aditi standing at a podium

“I consider myself as an opportunity grabber,” Aditi writes in a post on her Medium blog. “ I never miss a single chance to become a better version of myself. I used to attend all community meetups and did not miss a single one.”

All her hard work paid off. In 2017, she became a Women Techmakers lead in Indore and took her first flight on an airplane to the WTM Leads Summit in Bangalore. The same year, she became a Microsoft Student Partner and attended Google Developer Days India. In 2018, Aditi earned the Google India Udacity Android Developers Nanodegree Scholarship, as one of the top 150 students from India, and graduated with a Bachelor’s of Engineering degree in computer science. In 2019, Women Techmakers awarded Aditi a travel grant to Madrid, Spain to attend the Firebase Summit.

image Aditi at Firebase Summit 2019

Using the experience of being a woman in tech to encourage others to pursue STEM careers

Now, Aditi is a full-time software developer at Consultadd Incorporation, India, and a Women Techmakers Ambassador, and a GDG organizer for her local chapter in Pune. She contributes to the community as an organizer, speaker, and mentor.

“We organize monthly technical meetups to empower women and provide them with a platform to achieve their goals,” Aditi explains. “Being able to help others feels like I am giving it back to the community.”

Aditi says GDG and WTM have helped her develop technical skills and have also positively impacted her personal life.

“I had significant life experiences because of the Google Developer Group and Women Techmakers communities, including my first flight, my first hands-on experience with Google's trending technologies, and one-on-one interaction with Googlers and many great personalities while attending global summits,” she says. “All these things have helped me to be a person who can guide and help others and share my knowledge and experiences with hundreds of professionals today.”

Aditi describes herself as a community enthusiast, using her platform to encourage other women and students to pursue careers in technology, even if they’re brand-new to the field. She also enjoys mentoring new programmers.

“I am passionate about making an impact on others’ lives by sharing my journey and experiences and helping people face hurdles like mine with courage and confidence,” she says. “I enjoy helping people who are struggling to learn to code or who want to switch their careers to tech.”

image of Aditi presenting in a classroom

Supporting budding developers

Aditi acknowledges the adage, “Change is not easy,’’ especially when preparing for a career in technology.

“You may try very hard, give up so many times, and go through all that frustration, but remember not to quit,” she advises. “The moment you feel like quitting is the moment you need to keep pushing and get your reward.”

She has specific suggestions for making it easier to build new tech skills, too.

“Before learning a specific technology, understand yourself,” she suggests. “What works for you? What's your learning process? Then look for the appropriate resources. It can be a simple one-page tutorial or a full-fledged course. Everything is easy when the basics are clear and the foundation is strong.”

Aditi plans to continue contributing to the tech community in India and around the world, by sharing her insight, connecting with new people, and developing new technical skills. She recently welcomed a new member into her family–a baby girl–and she is growing her own regional tech community and providing so much to others in her area and the STEM field.

Know someone with a powerful story? Nominate someone in your community to be featured, or share your own personal stories with us through the story submission form!

Google Play Games beta launches on PC in Korea, Taiwan, and Hong Kong

Posted by Arjun Dayal, Group Product Manager, Google Play Games

 Google Play Games Logo

In December, we announced that Google Play Games will be coming to PCs. As part of our broader goal to make our products and services work better together, this product strives to meet players where they are and give them access to their games on as many devices as possible. We're excited to announce that we’ve opened sign-ups for Google Play Games as a beta in Korea, Taiwan, and Hong Kong.

Introduction video of Google Play Games beta

Users participating in the beta can play a catalog of Google Play games on their Windows PC via a standalone application built by Google. We’re excited to announce that some of the most popular mobile games in the world will be available at launch, including Mobile Legends: Bang Bang, Summoners War, State of Survival: The Joker Collaboration, and Three Kingdoms Tactics, which delight hundreds of millions of players globally each month.

Google Play Games beta PC application

This product brings the best of Google Play to more laptops and desktops, enabling immersive and seamless gameplay sessions between a phone, tablet, Chromebook, and Windows PC. Players can easily browse, download, and play their favorite mobile games on their PCs, while taking advantage of larger screens with mouse and keyboard inputs. No more losing your progress or achievements when switching between devices, it just works with your Google Play Games profile! Play Points can also be earned for Google Play Games activity on PCs.

Google Play Games gameplay on multiple devices including a phone, PC, and tablet.

We’re thrilled to expand our platform for players to enjoy their favorite Android games even more. To sign up for future announcements, or to access the beta in Korea, Taiwan, and Hong Kong, please go to g.co/googleplaygames. If you’re an Android developer looking to learn more about Google Play Games, please express interest on our developer site. We’ll have more to share on future beta releases and regional availability soon.

Windows is a trademark of the Microsoft group of companies.
Game titles may vary by region.

Make Payments with Google Pay and Firebase

Posted by Stephen McDonald, Developer Relations Engineer, Google Pay

Connect Multiple Payment Gateways with Google Pay and Firebase

We recently launched a series of open source samples demonstrating the server-side integration between Google Pay and a variety of Payment Service Providers (PSPs). These samples also show how to create a unified interface for integrating multiple PSPs, making integrations as easy as possible by reducing the time investment in integrating multiple APIs and client libraries.

A recent study by 451 Research showed that for merchants with over 50% of sales occurring online, 69% of them used multiple PSPs. We first demonstrated with the aforementioned samples how you can implement a consistent interface to multiple PSPs, streamlining your codebase while also providing more flexibility for the future. We've now taken this one step further and brought this unified PSP interface to the Firebase platform, by way of a Firebase Extension for Google Pay, making it easier than ever to integrate Google Pay with one or more PSPs.

Google Pay Firebase Extension

Firebase Extensions are open source pre-packaged bundles of code that developers can easily pull into their apps, and are designed to increase productivity, and provide extended functionality to your apps without the need to research, write, or debug code on your own. Following this line, the Google Pay Firebase Extension brings the unified PSP interface to developers' Firebase apps.

With the Google Pay Firebase Extension installed, you can pass a payment token from the Google Pay API to your Cloud Firestore database. The extension will listen for a request written to the path defined during installation, and then send the request to the PSP's API. It will then write the response back to the same Firestore node.

Open Source

Like all Firebase Extensions, the Google Pay Firebase Extension is entirely open source, so you can modify the code yourself to change the functionality as you see fit, or even contribute your changes back via pull requests - the sky's the limit.

Furthermore, as the extension is backed by the aforementioned PSP samples project, the same set of PSPs are supported. Want to see your favorite PSP supported? Head on over to the PSP samples project which contains instructions for adding it.

Summing it up

Whether you're new to Google Pay or Firebase, or an existing user of either, the new Google Pay extension is designed to save you even more time and effort when integrating Google Pay and any number of Payment Service Providers with your application.

Get started with the extension today in the Firebase console.

What do you think? Follow us on Twitter for the latest updates @GooglePayDevs

Do you have any questions? Let us know in the comments below or tweet using #AskGooglePayDevs.

Google Play Games beta launches on PC in Korea, Taiwan, and Hong Kong

Posted by Arjun Dayal, Group Product Manager, Google Play Games

Image of Google Play Games logo

In December, we announced that Google Play Games will be coming to PCs. As part of our broader goal to make our products and services work better together, this product strives to meet players where they are and give them access to their games on as many devices as possible. Today we're excited to announce that we’re opening sign-ups for Google Play Games as a beta in Korea, Taiwan, and Hong Kong.

Introduction video of Google Play Games beta

Users participating in the beta can play a catalog of Google Play games on their Windows PC via a standalone application built by Google. We’re excited to announce that some of the most popular mobile games in the world will be available at launch, including Mobile Legends: Bang Bang, Summoners War, State of Survival: The Joker Collaboration, and Three Kingdoms Tactics, which delight hundreds of millions of players globally each month.

Google Play Games beta PC application

This product brings the best of Google Play to more laptops and desktops, enabling immersive and seamless gameplay sessions between a phone, tablet, Chromebook, and Windows PC. Players can easily browse, download, and play their favorite mobile games on their PCs, while taking advantage of larger screens with mouse and keyboard inputs. No more losing your progress or achievements when switching between devices, it just works with your Google Play Games profile! Play Points can also be earned for Google Play Games activity on PCs.

Google Play Games gameplay on multiple devices including a phone, PC, and tablet

We’re thrilled to expand our platform for players to enjoy their favorite Android games even more. To sign up for future announcements, or to access the beta in Korea, Taiwan, and Hong Kong, please go to g.co/googleplaygames. If you’re an Android developer looking to learn more about Google Play Games, please express interest on our developer site. We’ll have more to share on future beta releases and regional availability soon.

Windows is a trademark of the Microsoft group of companies.
Game titles may vary by region.

Let’s Build Solutions! Solution Challenge 2022: Solve for the United Nations’ Sustainable Development Goals Using Google Technologies

Posted by Erica Hanson, Global Senior Program Manager, Google Developer Student Clubs

Solution Challenge 2022 banner image

Have you ever thought about building an application or tool that solves a problem your community faces? Or perhaps you’ve felt inspired to build something that can help improve the lives of those you care about. The year ahead brings more opportunities for helping each other and giving back to our communities.

With that in mind, we invite students around the world to join the Google Developer Student Clubs 2022 Solution Challenge! Where students from around the world are invited to solve for one of the United Nations' Sustainable Development Goals using Google technologies.

About the United Nations’ Sustainable Development Goals

Created by the United Nations in 2015 to be achieved by 2030, the 17 Sustainable Development Goals (SDGs) agreed upon by all 193 United Nations Member States aim to end poverty, ensure prosperity, and protect the planet.

If you’re new to the Solution Challenge, it is an annual competition that invites university students to develop solutions for real world problems using one or more Google products or platforms.

This year, see how you can use Android, Firebase, TensorFlow, Google Cloud, Flutter, or any of your favorite Google technologies to promote employment for all, economic growth, and climate action, by building a solution for one or more of the UN Sustainable Development Goals.

What winners of the Solution Challenge receive

Participants will receive specialized prizes at different stages:

  • Top 50 teams - Receive customized mentorship from Googlers and experts to take solutions to the next level, a branded T-shirt, and a certificate.
  • Top 10 finalists - Receive additional mentorship, a swag box, and the opportunity to showcase solutions to Googlers and developers all around the world during the virtual 2022 Solution Challenge Demo Day live on YouTube.
  • Contest Finalists - In addition to the swag box, each individual from the additional seven recognized teams will receive a Cash Prize of $1,000 per student. Winnings for each qualifying team will not exceed $4,000.
  • Top 3 winners - In addition to the swag box, each individual from the top 3 winning teams will receive a Cash Prize of $3,000 and a feature on the Google Developers Blog. Winnings for each qualifying team will not exceed $12,000.

How to get started on the Solution Challenge

There are four main steps to joining the Solution Challenge and getting started on your project:

  1. Register at goo.gle/solutionchallenge and join a Google Developer Student Club at your college or university. If there is no club at your university, you can join the closest one through the event platform.
  2. Select one or more of the United Nations 17 Sustainable Development Goals to solve for.
  3. Build a solution using Google technology.
  4. Create a demo and submit your project by March 31, 2022.

Resources from Google for Solution Challenge participants

Google will provide Solution Challenge participants with various resources to help students build strong projects for their contest submission.

  • Live online sessions with Q&As
  • Mentorship from Google, Google Developer Experts, and the Google Developer Student Club community
  • Curated codelabs designed by Google Developers
  • Access to Design Sprint guidelines developed by Google Ventures
  • and more!

When are winners announced?

Once all the projects are submitted by the March 31st, 2022 deadline, judges will evaluate and score each submission from around the world using the criteria listed on the website.

From there, winning solutions will be announced in three rounds.

Round 1 (April): The Top 50 teams will be announced.

Round 2 (June): After the top 50 teams submit their new and improved solutions, 10 finalists will be announced.

Round 3 (July): In the finale, the top 3 grand prize winners will be announced live on YouTube during the 2022 Solution Challenge Demo Day.

With a passion for building a better world, savvy coding skills, and a little help from Google technology, we can’t wait to see the solutions students create.

Learn more and sign up for the 2022 Solution Challenge, here.

“Disapproved Ads Auditor” tool

Posted by Nir Kalush, Dvir Kalev, Chen Yoveg, Elad Ben-David

Ads Review Tool

This tool flags (and optionally deletes) policy violating ads across your accounts. Advertisers can learn from the output to ensure their ads are compliant with Google Ads Policies at all times.

Business Challenge:

Advertisers operating at scale need a solution to holistically review policy violating ads across their accounts so they can ensure compliance with Google’s Ad Policies. As Google introduces more policies and enforcement mechanisms, advertisers need to continue checking their accounts to ensure they comply with Google’s Ads Policies.

Solution Overview:

“Disapproved Ads Auditor” is a tool that enables advertisers to review at scale all disapproved ads across their Google Ads accounts. This view allows advertisers to proactively audit their accounts , analyze the ad disapprovals holistically and identify learnings to minimize and reduce submission of potentially policy violating ads.

The tool is based on a Python script, which can be run in either of the following modes:

  • “Audit Mode”- export an output of disapproved ads across your accounts
  • “Remove Mode” - delete currently disapproved ads and log details.

There are a few output files (see here) which are saved locally under the “output” folder and there is an optional feature to export on BigQuery for further data analysis (“Disapproved Ads Auditor” dataset).

Skills Required:

Google Products Used:

  • Google Ads
  • BigQuery

Estimated time to implement the solution: ~2h

Implementation instructions: View on github

Impact:

“Disapproved Ads Auditor” tool automates auditing ads across your accounts to provide you insights into non compliant ads. You can take the learnings from the output to ensure ads are compliant with Google Ads Policies and avoid creating non compliant ads. Moreover, you can optionally remove disapproved ads.

Prediction Framework, a time saver for Data Science prediction projects

Posted by Álvaro Lamas, Héctor Parra, Jaime Martínez, Julia Hernández, Miguel Fernandes, Pablo Gil

Acquiring high value customers using predicted Lifetime Value, taking specific actions on high propensity of churn users, generating and activating audiences based on machine learning processed signals…All of those marketing scenarios require of analyzing first party data, performing predictions on the data and activating the results into the different marketing platforms like Google Ads as frequently as possible to keep the data fresh.

Feeding marketing platforms like Google Ads on a regular and frequent basis, requires a robust, report oriented and cost reduced ETL & prediction pipeline. These pipelines are very similar regardless of the use case and it’s very easy to fall into reinventing the wheel every time or manually copy & paste structural code increasing the risk of introducing errors.

Wouldn't it be great to have a common reusable structure and just add the specific code for each of the stages?

Here is where Prediction Framework plays a key role in helping you implement and accelerate your first-party data prediction projects by providing the backbone elements of the predictive process.

Prediction Framework is a fully customizable pipeline that allows you to simplify the implementation of prediction projects. You only need to have the input data source, the logic to extract and process the data and a Vertex AutoML model ready to use along with the right feature list, and the framework will be in charge of creating and deploying the required artifacts. With a simple configuration, all the common artifacts of the different stages of this type of projects will be created and deployed for you: data extraction, data preparation (aka feature engineering), filtering, prediction and post-processing, in addition to some other operational functionality including backfilling, throttling (for API limits), synchronization, storage and reporting.

The Prediction Framework was built to be hosted in the Google Cloud Platform and it makes use of Cloud Functions to do all the data processing (extraction, preparation, filtering and post-prediction processing), Firestore, Pub/Sub and Schedulers for the throttling system and to coordinate the different phases of the predictive process, Vertex AutoML to host your machine learning model and BigQuery as the final storage of your predictions.

Prediction Framework Architecture

To get involved and start using the Prediction Framework, a configuration file needs to be prepared with some environment variables about the Google Cloud Project to be used, the data sources, the ML model to make the predictions and the scheduler for the throttling system. In addition, custom queries for the data extraction, preparation, filtering and post-processing need to be added in the deploy files customization. Then, the deployment is done automatically using a deployment script provided by the tool.

Once deployed, all the stages will be executed one after the other, storing the intermediate and final data in the BigQuery tables:

  • Extract: this step will, on a timely basis, query the transactions from the data source, corresponding to the run date (scheduler or backfill run date) and will store them in a new table into the local project BigQuery.
  • Prepare: immediately after the extract of the transactions for one specific date is available, the data will be picked up from the local BigQuery and processed according to the specs of the model. Once the data is processed, it will be stored in a new table into the local project BigQuery.
  • Filter: this step will query the data stored by the prepare process and will filter the required data and store it into the local project BigQuery. (i.e only taking into consideration new customers transactionsWhat a new customer is up to the instantiation of the framework for the specific use case. Will be covered later).
  • Predict: once the new customers are stored, this step will read them from BigQuery and call the prediction using Vertex API. A formula based on the result of the prediction could be applied to tune the value or to apply thresholds. Once the data is ready, it will be stored into the BigQuery within the target project.
  • Post_process: A formula could be applied to the AutoML batch results to tune the value or to apply thresholds. Once the data is ready, it will be stored into the BigQuery within the target project.

One of the powerful features of the prediction framework is that it allows backfilling directly from the BigQuery user interface, so in case you’d need to reprocess a whole period of time, it could be done in literally 4 clicks.

In summary: Prediction Framework simplifies the implementation of first-party data prediction projects, saving time and minimizing errors of manual deployments of recurrent architectures.

For additional information and to start experimenting, you can visit the Prediction Framework repository on Github.

Prediction Framework, a time saver for Data Science prediction projects

Posted by Álvaro Lamas, Héctor Parra, Jaime Martínez, Julia Hernández, Miguel Fernandes, Pablo Gil

Acquiring high value customers using predicted Lifetime Value, taking specific actions on high propensity of churn users, generating and activating audiences based on machine learning processed signals…All of those marketing scenarios require of analyzing first party data, performing predictions on the data and activating the results into the different marketing platforms like Google Ads as frequently as possible to keep the data fresh.

Feeding marketing platforms like Google Ads on a regular and frequent basis, requires a robust, report oriented and cost reduced ETL & prediction pipeline. These pipelines are very similar regardless of the use case and it’s very easy to fall into reinventing the wheel every time or manually copy & paste structural code increasing the risk of introducing errors.

Wouldn't it be great to have a common reusable structure and just add the specific code for each of the stages?

Here is where Prediction Framework plays a key role in helping you implement and accelerate your first-party data prediction projects by providing the backbone elements of the predictive process.

Prediction Framework is a fully customizable pipeline that allows you to simplify the implementation of prediction projects. You only need to have the input data source, the logic to extract and process the data and a Vertex AutoML model ready to use along with the right feature list, and the framework will be in charge of creating and deploying the required artifacts. With a simple configuration, all the common artifacts of the different stages of this type of projects will be created and deployed for you: data extraction, data preparation (aka feature engineering), filtering, prediction and post-processing, in addition to some other operational functionality including backfilling, throttling (for API limits), synchronization, storage and reporting.

The Prediction Framework was built to be hosted in the Google Cloud Platform and it makes use of Cloud Functions to do all the data processing (extraction, preparation, filtering and post-prediction processing), Firestore, Pub/Sub and Schedulers for the throttling system and to coordinate the different phases of the predictive process, Vertex AutoML to host your machine learning model and BigQuery as the final storage of your predictions.

Prediction Framework Architecture

To get involved and start using the Prediction Framework, a configuration file needs to be prepared with some environment variables about the Google Cloud Project to be used, the data sources, the ML model to make the predictions and the scheduler for the throttling system. In addition, custom queries for the data extraction, preparation, filtering and post-processing need to be added in the deploy files customization. Then, the deployment is done automatically using a deployment script provided by the tool.

Once deployed, all the stages will be executed one after the other, storing the intermediate and final data in the BigQuery tables:

  • Extract: this step will, on a timely basis, query the transactions from the data source, corresponding to the run date (scheduler or backfill run date) and will store them in a new table into the local project BigQuery.
  • Prepare: immediately after the extract of the transactions for one specific date is available, the data will be picked up from the local BigQuery and processed according to the specs of the model. Once the data is processed, it will be stored in a new table into the local project BigQuery.
  • Filter: this step will query the data stored by the prepare process and will filter the required data and store it into the local project BigQuery. (i.e only taking into consideration new customers transactionsWhat a new customer is up to the instantiation of the framework for the specific use case. Will be covered later).
  • Predict: once the new customers are stored, this step will read them from BigQuery and call the prediction using Vertex API. A formula based on the result of the prediction could be applied to tune the value or to apply thresholds. Once the data is ready, it will be stored into the BigQuery within the target project.
  • Post_process: A formula could be applied to the AutoML batch results to tune the value or to apply thresholds. Once the data is ready, it will be stored into the BigQuery within the target project.

One of the powerful features of the prediction framework is that it allows backfilling directly from the BigQuery user interface, so in case you’d need to reprocess a whole period of time, it could be done in literally 4 clicks.

In summary: Prediction Framework simplifies the implementation of first-party data prediction projects, saving time and minimizing errors of manual deployments of recurrent architectures.

For additional information and to start experimenting, you can visit the Prediction Framework repository on Github.