Tag Archives: library

Build sophisticated search features with AppSearch

Posted by Dan Saadati, Software Engineer, and Hanaa ElAzizi, Technical Program Manager

Introducing AppSearch in Jetpack, now available in Alpha. AppSearch is an on-device search library which provides high performance and feature-rich full-text search functionality.

With AppSearch, your application can:

  • Offer offline search capabilities as AppSearch data lives completely on-device.
  • Have lower latency for indexing and querying over large data sets compared to SQLite, due to lower I/O use.
  • Provide relevant search results with built-in scoring strategies, such as BM25F.
  • Provide multi-language support for text search.
  • Issue a single query to retrieve data of multiple data types compared to issuing one query per data type in SQLite.

In AppSearch, you need to create a database to manage structured data, called “documents”. You then define what the structure looks like using “schema types”. For instance, you can model a message as a schema type with properties such as subject, body, and sender.

Documents that are added to your database can be queried over. Querying for “body:fruit” will retrieve all documents with the term “fruit” in the “body” of the Message.

Diagram illustrating the indexing of grocery list items in AppSearch, and searching for those items later

Diagram illustrating indexing and searching within AppSearch

To showcase how an application might integrate AppSearch, take this example of a grocery list application. Users can add grocery items to their list to refer to when they’re out shopping. Since AppSearch offers multi-language support by default, users can also include specialty ingredients for their global recipes. Users add an item by typing in the name and selecting the store and category it belongs to. The user can search by item name and select filters for store or category. AppSearch will return matching results for the application to display.

Ready to dive into using AppSearch to enrich your app’s search functionality? Check out the AppSearch guide and start using it in your app.

Help us make the library better: give us feedback on things you like, and issues or features you would like to see. If you find a bug or issue, feel free to file an issue.

Cloud Spanner Emulator Reaches 1.0 Milestone!

The Cloud Spanner emulator provides application developers with the full set of APIs, including the full breadth of SQL and DDL features that can be run locally for prototyping, development and testing. This offline emulator is free and improves developer productivity for customers. Today, we are happy to announce that Cloud Spanner emulator is generally available (GA) with support for Partitioned APIs, Cloud Spanner client libraries, and SQL features.

Since Cloud Spanner emulator’s beta launch in April, 2020, we have seen strong adoption of the local emulator from customers of Cloud Spanner. Several new and existing customers adopted the emulator in their development & continuous test pipelines. They noticed significant improvements in developer productivity, speed of test execution, and error-free applications deployed to production. We also added several features in this release based on the valuable feedback we received from beta users. The full list of features is documented in the GitHub readme.

Partition APIs

When reading or querying large amounts of data from Cloud Spanner, it can be useful to divide the query into smaller pieces, or partitions, and use multiple machines to fetch the partitions in parallel. The emulator now supports Partition Read, Partition Query, and Partition DML APIs.

Cloud Spanner client libraries

With the GA launch, the latest versions of all the Cloud Spanner client libraries support the emulator. We have added support for C#, Node.js, PHP, Python, Ruby client libraries and the Cloud Spanner JDBC driver. This is in addition to C++, Go and Java client libraries that were already supported with the beta launch. Be sure to check out the minimum version for each of the client libraries that support the emulator.

Use the Getting Started guides to try the emulator with the client library of your choice.

SQL features

Emulator now supports the full set of SQL features provided by Cloud Spanner. Some of the notable additions being support for SQL functions JSON_VALUE, JSON_QUERY, CEILING, POWER, CHARACTER_LENGTH, and FORMAT. We now also support untyped parameter bindings in SQL statements which are used by our client libraries written in languages with dynamic typing e.g., Python, PHP, Node.js and Ruby.

Using Emulator in CI/CD pipelines

You may now point the majority of your existing CI/CD to the Cloud Spanner emulator instead of a real Cloud Spanner instance brought up on GCP. This will save you both cost and time, since an emulator instance comes up instantly and is free to use!

What’s even better is that you can bring up multiple instances in a single execution of the emulator, and of course multiple databases. Thus, tests that interact with a Cloud Spanner database can now run in parallel since each of them can have their own database, making tests hermetic. This can reduce flakiness in unit tests and reduce the number of bugs that can make their way to continuous integration tests or to production.

In case your existing CI/CD architecture assumes the existence of a Cloud Spanner test instance and/or test database against which the tests run, you can achieve similar functionality with the emulator as well. Note that the emulator doesn’t come up with a default instance or a default database as we expect users to create instances and databases as required in their tests for hermeticity as explained above. Below are two examples of how you can bring up an emulator with a default instance or database: 1) By using a docker image or 2) Programmatically.

Starting Emulator from Docker

The emulator can be started using Docker on Linux, MacOS, and Windows. As a prerequisite, you would need to install Docker on your system. To bring up an emulator with a default database/instance, you can execute a shell script in your docker file to do so. Such a script would make RPC calls to CreateInstance and CreateDatabase after bringing up the emulator server. You can also look at this example on how to put this together when using docker.
Run Emulator Programmatically

You can bring up the emulator binary in the same process as your test program. Then you can then create a default instance/database in your ‘Setup’ and clean up the same when the tests are over. Note that the exact procedure for bringing up an ‘in-process’ service may vary with the client library language and platform of your choice.

Other alternatives to start the emulator, including pre-built linux binaries, are listed here.
Try it now

Learn more about Google Cloud Spanner emulator and try it out now.

By Asheesh Agrawal, Google Open Source

Expanding our Differential Privacy Library

All developers have a responsibility to treat data with care and respect. Differential privacy helps organizations derive insights from data while simultaneously ensuring that those results do not allow any individual's data to be distinguished or re-identified. This principled approach supports data computation and analysis across many of Google’s core products and features.

Last summer, Google open sourced our foundational differential privacy library so developers and organizations around the world can benefit from this technology. Today, we’re announcing the addition of Go and Java to our library, an end-to-end solution for differential privacy: Privacy on Beam, and new tools to help developers implement this technology effectively.

We’ve listened to feedback from our developer community and, as of today, developers can now perform differentially private analysis in Java and Go. We’re working to bring these two libraries to full feature parity with C++.

We want all developers to have access to differential privacy, regardless of their level of expertise. Our new Privacy on Beam framework captures years of Googler developer experience and efficiency improvements in a comprehensive and easy-to-use solution that handles computation end-to-end. Built on Apache Beam, Privacy on Beam can reduce implementation mistakes, and take care of all the steps that are essential to differential privacy, including noise addition, partition selection, and contribution bounding. If you’re new to Apache Beam or differential privacy, our codelab can get you started.

Tracking privacy budgets is another challenge developers face when implementing differential privacy. So, we’re also releasing a new Privacy Loss Distribution tool for tracking privacy budgets. With this tool, developers can maintain an accurate estimate of the total cost to user privacy for collections of differentially private queries, and better evaluate the overall impact of their pipelines. Privacy Loss Distribution supports widely used mechanisms (such as Laplace, Gaussian, and Randomized response) and can scale to hundreds of compositions.

We hope these new languages, tools, and features unlock differential privacy for even more developers. Continue to share your stories and suggestions with us at [email protected]—your feedback will help inform our future differential privacy launches and updates.

Acknowledgements

Software Engineers: Yurii Sushko, Daniel Simmons-Marengo, Christoph Dibak, Damien Desfontaines, Maria Telyatnikova
Research Scientists: Pasin Manurangsi, Ravi Kumar, Sergei Vassilvitskii, Alex Kulesza, Jenny Gillenwater, Kareem Amin


By: Miguel Guevara, Mirac Vuslat Basaran, Sasha Kulankhina, and Badih Ghazi – Google Privacy Team and Google Research

Pigweed: A collection of embedded libraries

We’re excited to announce Pigweed, an open source collection of embedded-targeted libraries, or as we like to call them, modules. Pigweed modules are built to enable faster and more reliable development on 32-bit microcontrollers.

Pigweed is in early development and is not suitable for production use at this time.

Getting Started with Pigweed

As of today, the source is available for everyone at pigweed.googlesource.com under an Apache 2.0 license. Instructions on getting up and running can be found in the README.

See Pigweed in action

Pigweed offers modules that address a wide range of embedded developer needs. These highlight how Pigweed can accelerate embedded development across the entire lifecycle:
  • Setup: Get started faster with simplified setup via a virtual environment with all required tools
  • Development: Accelerated edit-compile-flash-test cycles with watchers and distributed testing
  • Code Submission: Pre-configured code formatting and integrated presubmit checks

SETUP

A classic challenge in the embedded space is reducing the time from running git clone to having a binary executing on a device. Oftentimes, an entire suite of tools is needed for non-trivial production embedded projects. For example, your project likely needs:
  • A C++ compiler for your target device, and also for your host
  • A build system (or three); for example, GN, Ninja, CMake, Bazel
  • A code formatting program like clang-format
  • A debugger like OpenOCD to flash and debug your embedded device
  • A known Python version with known modules installed for scripting
  • … and so on
Below is a system with Python 2.7, clang-format, and no ARM compiler. The bootstrap script in Pigweed’s pw_env_setup module, sets up the current shell to have access to a standardized set of tools—Python 3.8, clang-format, and an ARM compiler among them. All of this is done in a virtual environment so the system’s default environment remains unmodified.

DEVELOPMENT

In typical embedded development, adding even a small change involves the following additional manual steps:
  • Re-building the image
  • Flashing the image to a device
  • Ensuring that the change works as expected
  • Verifying that existing tests continue to pass
This is a huge disparity from web development workflows where file watchers are prevalent—you save a file and instantly see the results of the change.

Pigweed’s pw_watch module solves this inefficiency directly, providing a watcher that automatically invokes a build when a file is saved, and also runs the specific tests affected by the code changes. This drastically reduces the edit-compile-flash-test cycle for changes.



In the demo above, the pw_watch module (on the right) does the following:
  • Detects source file changes
  • Builds the affected libraries, tests, and binaries
  • Flashes the tests to the device (in this case a STM32F429i Discovery board)
  • Runs the specific unit tests
There’s no need to leave your code editor—all of this is done automatically. You can save additional time by using the pw_target_runner module to run tests in parallel across multiple devices.

CODE SUBMISSION

When developing code as a part of a team, consistent code is an important part of a healthy codebase. However, setting up linters, configuring code formatting, and adding automated presubmit checks is work that often gets delayed indefinitely.

Pigweed’s pw_presubmit module provides an off-the-shelf integrated suite of linters, based on tools that you’ve probably already used, that are pre-configured for immediate use for microcontroller developers. This means that your project can have linting and automatic formatting presubmit checks from its inception.

And a bunch of other modules

There are many modules in addition to the ones highlighted above...
  • pw_tokenizer – A module that converts strings to binary tokens at compile time. This enables logging with dramatically less overhead in flash, RAM, and CPU usage.
  • pw_string – Provides the flexibility, ease-of-use, and safety of C++-style string manipulation, but with no dynamic memory allocation and a much smaller binary size impact. Using pw_string in place of the standard C functions eliminates issues related to buffer overflow or missing null terminators.
  • pw_bloat – A module to generate memory reports for output binaries empowering developers with information regarding the memory impact of any change.
  • pw_unit_test – Unit testing is important and Pigweed offers a portable library that’s broadly compatible with Google Test. Unlike Google Test, pw_unit_test is built on top of embedded friendly primitives; for example, it does not use dynamic memory allocation. Additionally, it is to port to new target platforms by implementing the test event handler interface.
  • pw_kvs – A key-value-store implementation for flash-backed persistent storage with integrated wear levelling. This is a lightweight alternative to a file system for embedded devices.
  • pw_cpu_exception_armv7m – Robust low level hardware fault handler for ARM Cortex-M; the handler even has unit tests written in assembly to verify nested-hardware-fault handling!
  • pw_protobuf – An early preview of our wire-format-oriented protocol buffer implementation. This protobuf compiler makes a different set of implementation tradeoffs than the most popular protocol buffer library in this space, nanopb.

Why name the project Pigweed?

Pigweed, also known as amaranth, is a nutritious grain and leafy salad green that is also a rapidly growing weed. When developing the project that eventually became Pigweed, we wanted to find a name that was fun, playful, and reflective of how we saw Pigweed growing. Teams would start out using one module that catches their eye, and after that goes well, they’d quickly start using more.

So far, so good ?

What’s next?

We’re continuing to evolve the collection and add new modules. It’s our hope that others in the embedded community find these modules helpful for their projects.

By Keir Mierle and Mohammed Habibulla, on behalf of the Pigweed team

Introducing Oboe: A C++ library for low latency audio

Posted by Don Turner, Developer Advocate, Android Audio Framework

This week we released the first production-ready version of Oboe - a C++ library for building real-time audio apps. Oboe provides the lowest possible audio latency across the widest range of Android devices, as well as several other benefits.

Single API

Oboe takes advantage of the improved performance and features of AAudio on Oreo MR1 (API 27+) whilst maintaining backward compatibility (using OpenSL ES) on API 16+. It's kind of like AndroidX for native audio.

Diagram showing the underlying audio API which Oboe will use

Less code to write and maintain

Using Oboe you can create an audio stream in just 3 lines of code (vs 50+ lines in OpenSL ES): 

AudioStreamBuilder builder;
AudioStream *stream = nullptr;
Result result = builder.openStream(&stream);

Other benefits

  • Convenient C++ API (uses the C++11 standard)
  • Fast release process: supplied as a source library, bug fixes can be rolled out in days, quite a bit faster than the Android platform release cycle
  • Less guesswork: Provides workarounds for known audio bugs and has sensible default behaviour for stream properties, such as sample rate and audio data formats
  • Open source and maintained by Google engineers (although we welcome outside contributions)

Getting started

Take a look at the short video introduction:

Check out the documentation, code samples and API reference. There's even a codelab which shows you how to build a rhythm-based game.

If you have any issues, please file them here, we'd love to hear how you get on.

Money made easily with the new Google Play Billing Library

Posted by Neto Marin, Developer Advocate

Many developers want to make money through their apps, but it's not always easy to deal with all the different types of payment methods. We launched the Google Play In-app Billing API v3 in 2013, helping developers offer in-app products and subscriptions within their apps. Year after year, we've added features to the API, like subscription renewal, upgrades and downgrades, free trials, introductory pricing, promotion codes, and more.

Based on your feedback, we’re pleased to announce the Play Billing Library - Developer Preview 1. This library aims to simplify the development process when it comes to billing, allowing you to focus your efforts on implementing logic specific to your app, such as application architecture and navigation structure. The library includes several convenient classes and features for you to use when integrating your Android apps with the In-app Billing API. The library also provides an abstraction layer on top of the Android Interface Definition Language (AIDL) service, making it easier for you to define the interface between your app and the In-app Billing API.

Easy to get started and easy to use

Starting with Play Billing Library Developer Preview release, the minimum supported API level is Android 2.2 (API level 8), and the minimum supported In-app Billing API is version 3.

In-app billing relies on the Google Play Store, which handles the communication between your app and Google's Play billing service. To use Google Play billing features, your app must request the com.android.vending.BILLING permission in your AndroidManifest.xml file.

To use the library, add the following dependency in your build.gradle file: 

dependencies {
    ...
    compile 'com.android.billingclient:billing:dp-1'
}

After this quick setup process, you're ready to start using the Play Billing Library in your app and can connect to the In-app Billing API, query for available products, start the purchase flow, and more.

Sample updated: Trivial Drive V2

With a new library comes a refreshed sample! To help you to understand how to implement in-app billing in your app using the new Play Billing Library, we've rewritten the Trivial Drive sample from the ground up.

Since we released Trivial Drive back in 2013, many new features, devices, and platforms have been added to the Android ecosystem. To reflect this evolution, the Trivial Drive v2 sample now runs on Android TV and Android Wear.

Give it a try!

Before integrating within your app, you can try the Play Billing Library with the codelab published during Google I/O 2017: Buy and Subscribe: Monetize your app on Google Play.

In this codelab, you will start with a simplified version of Trivial Drive V2 that lets users to "drive" and then you will add in-app billing to it. You'll learn how to integrate purchases and subscriptions as well as the best practices for developing reliable apps that handle purchases.

If you are looking for a step-by-step guide about how to sell in-app products from your app using the Play Billing Library, check out our new training class, explaining how to prepare your application, add products for purchase, start purchase flow and much more.

We want your feedback

We look forward to hearing your feedback about this new library. Visit the Play Billing Library site, the library reference, and the new version of the Trivial Drive sample. If you have issues or questions, file a bug report on the Google Issue Tracker, and for issues and suggestions on the sample, contact us on the Trivial Drive issues page.

For technical questions on implementation, library usage, and best practices, you can use the tags google-play and play-billing-library on Stackoverflow or visit the community pages on our Google+ page.