Tag Archives: Gmail APIs

Increased account security via OAuth 2.0 token revocation

Originally posted on Google Apps Developers Blog

Posted by Michael Winser, Product Lead, Google Apps and Wesley Chun, Developer Advocate, Google Apps

Last week, we clarified the expectations and responsibilities when accessing Google user data via OAuth 2.0. Today, we’re announcing that in order to better protect users, we are increasing account security for enterprise Gmail users effective October 5, 2016. At this time, a new policy will take effect whereby users in a Google Apps domain, while changing their passwords on or after this date, will result in the revocation of the OAuth 2.0 tokens of apps that access their mailboxes using Gmail-based authorization scopes. Please note that users will not notice any specific changes on this date and their applications will continue to work. It is only when a user changes their password from that point moving forward that their Gmail-related tokens become invalid.

Developers should modify their applications to handle HTTP 400 or 401 error codes resulting from revoked tokens and prompt their users to go through the OAuth flow again to re-authorize those apps, such that they can access the user’s mailbox again (additional details below). Late last year, we announceda similar, planned change to our security policy that impacted a broader set of authorization scopes. We later decidednot to move forward with that change for Apps customers and began working on a less impactful update as described above.

What is a revoked token?

A revoked OAuth 2.0 token no longer provides access to a user’s resources. Any attempt to use a revoked token in API calls will result in an error. Any existing token strings will no longer have any value and should be discarded. Applications accessing Google APIs should be modified to handle failed API calls.

Token revocation itself is not a new feature. Users have always been able to revoke access to applications in Security Checkup, and Google Apps admins have the ability to do the same in the Admin console. In addition, tokens that were not used for extended periods of time have always been subject to expiration or revocation. This change in our security policy will likely increase the rate of revoked tokens that applications see, since in some cases the process will now take place automatically.

What APIs and scopes are impacted?

To achieve the security benefits of this policy change with minimal admin confusion and end-user disruption, we’ve decided to limit its application to mail scopes only and to exclude Apps Script tokens. Apps installed via the Google Apps Marketplace are also not subject to the token revocation. Once this change is in effect, third-party mail apps like Apple Mail and Thunderbird―as well as other applications that use multiple scopes that include at least one mail scope―will stop accessing data upon password reset until a new OAuth 2.0 token has been granted. Your application will need to detect this scenario, notify the user that your application has lost access to their account data, and prompt them to go through the OAuth 2.0 flow again.

Mobile mail applications are also included in this policy change. For example, users who use the native mail application on iOS will have to re-authorize with their Google account credentials when their password has been changed. This new behavior for third-party mail apps on mobile aligns with the current behavior of the Gmail apps on iOS and Android, which also require re-authorization upon password reset.

How can I determine if my token was revoked?

Both short-lived access tokens and long-lived refresh tokens will be revoked when a user changes their password. Using a revoked access token to access an API or to generate a new access token will result in either HTTP 400 or 401 errors. If your application uses a library to access the API or handle the OAuth flow, then these errors will likely be thrown as exceptions. Consult the library’s documentation for information on how to catch these exceptions. NOTE: because HTTP 400 errors may be caused by a variety of reasons, expect the payload from a 400 due to a revoked token to be similar to the following: 

{
  "error_description": "Token has been revoked.", 
  "error": "invalid_grant"
}

How should my application handle revoked tokens?

This change emphasizes that token revocation should be considered a normal condition, not an error scenario. Your application should expect and detect the condition, and your UI should be optimized for restoring tokens.

To ensure that your application works correctly, we recommend doing the following:

  • Add error handling code around API calls and token refreshes that can detect revoked tokens.
  • Upon detecting a revoked token, disable any application features that rely on Google API access until the user can re-authorize your application. For example, suspend any recurring background jobs that sync data with a Google API which may be affected.
  • Notify the user that access has been revoked and prompt them to re-authorize access to their resources.
    • If your app interacts directly with the user, you will need to prompt the user to re-authorize, i.e., send an email to the user and/or show them an alert the next time they open your application.
    • However, if your app runs independently of the user, say a background app that uses the Gmail API, you'll need to notify the user through email or some other mechanism.
    • Provide a streamlined UI for re-authorizing access. Avoid having users navigate through your application to find the original setting.
    • Note that revoked tokens will result in similar error messages regardless of how the token was revoked. Your messaging should not assume that the token was revoked due to a password change.

If your application uses incremental authorization to accrue multiple scopes in the same token, you should track which features and scopes a given user has enabled. The end result is that if your app requested and obtained authorization for multiple scopes, and at least one of them is a mail scope, that token will be revoked, meaning you will need to prompt your user to re-authorize for all scopes originally granted.

Many applications use tokens to perform background or server-to-server API calls. Users expect this background activity to continue reliably. Since this policy change also affects those apps, this makes prompt notification requesting re-authorization even more important.

What is the timeline for this change?

To summarize, properly configured applications should be expected to handle invalid tokens in general, whether they be from expiration, non-existence, and revocation as normal conditions. We encourage developers to make any necessary changes to give their users the best experience possible. The policy change is planned to take effect on October 5, 2016.

Please see this Help Center article and FAQ for more details and the full list of mail scopes. Moving forward, any additional scopes to be added to the policy will be communicated in advance. We will provide those details as they become available.

Increased account security via OAuth 2.0 token revocation

Posted by Michael Winser, Product Lead, Google Apps and Wesley Chun, Developer Advocate, Google Apps

Last week, we clarified the expectations and responsibilities when accessing Google user data via OAuth 2.0. Today, we’re announcing that in order to better protect users, we are increasing account security for enterprise Gmail users effective October 5, 2016. At this time, a new policy will take effect whereby users in a Google Apps domain, while changing their passwords on or after this date, will result in the revocation of the OAuth 2.0 tokens of apps that access their mailboxes using Gmail-based authorization scopes. Please note that users will not notice any specific changes on this date and their applications will continue to work. It is only when a user changes their password from that point moving forward that their Gmail-related tokens become invalid.

Developers should modify their applications to handle HTTP 400 or 401 error codes resulting from revoked tokens and prompt their users to go through the OAuth flow again to re-authorize those apps, such that they can access the user’s mailbox again (additional details below). Late last year, we announceda similar, planned change to our security policy that impacted a broader set of authorization scopes. We later decidednot to move forward with that change for Apps customers and began working on a less impactful update as described above.

What is a revoked token?

A revoked OAuth 2.0 token no longer provides access to a user’s resources. Any attempt to use a revoked token in API calls will result in an error. Any existing token strings will no longer have any value and should be discarded. Applications accessing Google APIs should be modified to handle failed API calls.

Token revocation itself is not a new feature. Users have always been able to revoke access to applications in Security Checkup, and Google Apps admins have the ability to do the same in the Admin console. In addition, tokens that were not used for extended periods of time have always been subject to expiration or revocation. This change in our security policy will likely increase the rate of revoked tokens that applications see, since in some cases the process will now take place automatically.

What APIs and scopes are impacted?

To achieve the security benefits of this policy change with minimal admin confusion and end-user disruption, we’ve decided to limit its application to mail scopes only and to exclude Apps Script tokens. Apps installed via the Google Apps Marketplace are also not subject to the token revocation. Once this change is in effect, third-party mail apps like Apple Mail and Thunderbird―as well as other applications that use multiple scopes that include at least one mail scope―will stop accessing data upon password reset until a new OAuth 2.0 token has been granted. Your application will need to detect this scenario, notify the user that your application has lost access to their account data, and prompt them to go through the OAuth 2.0 flow again.

Mobile mail applications are also included in this policy change. For example, users who use the native mail application on iOS will have to re-authorize with their Google account credentials when their password has been changed. This new behavior for third-party mail apps on mobile aligns with the current behavior of the Gmail apps on iOS and Android, which also require re-authorization upon password reset.

How can I determine if my token was revoked?

Both short-lived access tokens and long-lived refresh tokens will be revoked when a user changes their password. Using a revoked access token to access an API or to generate a new access token will result in either HTTP 400 or 401 errors. If your application uses a library to access the API or handle the OAuth flow, then these errors will likely be thrown as exceptions. Consult the library’s documentation for information on how to catch these exceptions. NOTE: because HTTP 400 errors may be caused by a variety of reasons, expect the payload from a 400 due to a revoked token to be similar to the following: 

{
  "error_description": "Token has been revoked.", 
  "error": "invalid_grant"
}

How should my application handle revoked tokens?

This change emphasizes that token revocation should be considered a normal condition, not an error scenario. Your application should expect and detect the condition, and your UI should be optimized for restoring tokens.

To ensure that your application works correctly, we recommend doing the following:

  • Add error handling code around API calls and token refreshes that can detect revoked tokens.
  • Upon detecting a revoked token, disable any application features that rely on Google API access until the user can re-authorize your application. For example, suspend any recurring background jobs that sync data with a Google API which may be affected.
  • Notify the user that access has been revoked and prompt them to re-authorize access to their resources.
    • If your app interacts directly with the user, you will need to prompt the user to re-authorize, i.e., send an email to the user and/or show them an alert the next time they open your application.
    • However, if your app runs independently of the user, say a background app that uses the Gmail API, you'll need to notify the user through email or some other mechanism.
    • Provide a streamlined UI for re-authorizing access. Avoid having users navigate through your application to find the original setting.
    • Note that revoked tokens will result in similar error messages regardless of how the token was revoked. Your messaging should not assume that the token was revoked due to a password change.

If your application uses incremental authorization to accrue multiple scopes in the same token, you should track which features and scopes a given user has enabled. The end result is that if your app requested and obtained authorization for multiple scopes, and at least one of them is a mail scope, that token will be revoked, meaning you will need to prompt your user to re-authorize for all scopes originally granted.

Many applications use tokens to perform background or server-to-server API calls. Users expect this background activity to continue reliably. Since this policy change also affects those apps, this makes prompt notification requesting re-authorization even more important.

What is the timeline for this change?

To summarize, properly configured applications should be expected to handle invalid tokens in general, whether they be from expiration, non-existence, and revocation as normal conditions. We encourage developers to make any necessary changes to give their users the best experience possible. The policy change is planned to take effect on October 5, 2016.

Please see this Help Center article and FAQ for more details and the full list of mail scopes. Moving forward, any additional scopes to be added to the policy will be communicated in advance. We will provide those details as they become available.

Gmail API: New endpoints for settings

Posted by Brandon Jewett-Hall, Software Engineer, Gmail and Wesley Chun, Developer Advocate, Google Apps

If you've been pining for a way to update your users' email signatures with a different inspirational quote each day or enable "Out of Office" auto-replies when their calendars are marked as busy, then you're in luck. Today, we're extending the Gmail APIwith new endpoints for managing settings. These new endpoints cover the following features:

  • Filters
  • Forwarding addresses and auto-forwarding
  • IMAP and POP settings
  • Send-as aliases
  • Signatures
  • Vacation responder

With this update, we're equipping developers with some new tools that have never been available in any previous Google API. These include the ability to:

  • Retrieve and update signatures for send-as aliases
  • Configure forwarding to external addresses
  • Configure send-as aliases that send mail through external providers
  • Use HTML in vacation messages
  • Manipulate settings for gmail.com accounts

Over time we'll continue to expand the API with additional settings features such as support for managing mailbox delegates, so stay tuned for more announcements.

Get started now

Most of the settings endpoints are available for any Google Apps or Gmail account, but a few sensitive operations, such as modifying send-as aliases or forwarding, are restricted to service accounts with domain-wide authority. See the reference docsfor more details and to get started.

What about the existing Email Settings API in the Admin SDK?

This update to the Gmail API effectively replaces the older Email Settings API, so we're also announcing its deprecation today and will turn it down fully on July 7, 2017. We've put together a migration guide to assist clients in porting their existing integrations over to their newer counterparts in the Gmail API. If you have any issues, check out the gmail-api tag on StackOverflow.

We look forward to seeing what you build with these new features in the Gmail API!

Gmail API Push notifications: don’t call us, we’ll call you

If your app needs to keep up with changes in a Gmail mailbox, whether it's new mail being received or a label being added, you are likely familiar with periodically polling the Gmail API's history feed for updates. If you find this tedious, we have good news for you. Today, at Google I/O 2015, we're launching push notifications for the Gmail API.

Just subscribe to a Gmail mailbox and whenever a change occurs, the Gmail API will instantly notify your server using webhooks or another Cloud Pub/Sub API notification method. No polling required.

Check out the developers’ guide at https://developers.google.com/gmail/api/guides/push to get started. We can’t wait to see what you build.

Posted by Eric DeFriez, Gmail Extensibility Team. As a technical lead for Gmail APIs, Eric works to make it easy for developers to build on top of Gmail.

Retiring the Email Migration API

Posted by Wesley Chun, Developer Advocate, Google Apps

Last summer, we launched the new Gmail API, giving developers more flexible, powerful, and higher-level access to programmatic email management, not to mention improved performance. Since then, it has been expanded to replace the Google Apps Admin SDK's Email Migration API (EMAPI v2). Going forward, we recommend developers integrate with the Gmail API.

EMAPI v2 will be turned down on November 1, 2015, so you should switch to the Gmail API soon. To aid you with this effort, we've put together a developer’s guide to help you migrate from EMAPI v2 to the Gmail API. Before you do that, here’s your final reminder to not forget about these deprecations including EMAPI v1, which are coming even sooner (April 20, 2015).

Updates on Authentication for Gmail IMAP, POP, and SMTP

We’d like to highlight some recent and upcoming changes around authentication for Gmail IMAP, POP, and SMTP.

Additional Scrutiny for Password Authentication 
As previously announced, Google has begun increasing the security checks that occur when logging in with a user’s Google password. This includes access via Gmail IMAP, POP, and SMTP-MSA. It does not apply when authenticating with OAuth 2.0 via the XOAUTH2 mechanism.

If the checks detect anything suspicious about a password login attempt, our servers may deny login and return an error message requesting that the user first login to Google through a web browser. They may also require the user to explicitly enable “Less Secure Apps” on their account. Applications that perform password authentication to IMAP, POP, or SMTP are examples of "Less Secure Apps".

We strongly encourage developers to use OAuth 2.0 (via the XOAUTH2 mechanism for IMAP, POP, and SMTP) in order to better protect their users.

XOAUTH support ends May 5, 2015 
The OAuth 1.0 XOAUTH authentication mechanism for Gmail IMAP and SMTP-MSA is deprecated and will stop being supported on May 5, 2015. Developers must migrate to XOAUTH2 in order to continue authenticating to Gmail after that date. You can migrate existing users without their intervention by following the instructions in this migration guide. Instructions for developing your XOAUTH2 code are in the XOAUTH2 documentation.

Posted by Jamie Nicolson, Gmail Software Engineer

Introducing the new Gmail API

For a while now, many of you have been asking for a better way to access data to build apps that integrate with Gmail. While IMAP is great at what it was designed for (connecting email clients to email servers in a standard way), it wasn’t really designed to do all of the cool things that you have been working on, which is why this week at Google I/O, we’re launching the beta of the new Gmail API.

Designed to let you easily deliver Gmail-enabled features, this new API is a standard Google API, which gives RESTful access to a user’s mailbox under OAuth 2.0 authorization. It supports CRUD operations on true Gmail datatypes such as messages, threads, labels and drafts.

As a standard Google API, you make simple HTTPS calls and get your responses in JSON, XML or Google Protobuf formats. You can also make these calls from standard web languages like Java and Python without using a TCP socket, which means the API is accessible from many cloud environments that couldn’t support IMAP.

In contrast to IMAP, which requires access to all of a user’s messages for all operations, the new API gives fine-grained control to a user’s mailbox. For example, if your app only needs to send mail on behalf of a user and does not need to read mail, you can limit your permission request to send-only.

To keep in sync, the API allows you to query the inbox change history, thereby avoiding the need to do “archaeology” to figure out what changed.

Finally, a huge benefit is speed. While there’s still some tuning to be done (“beta” - remember?), results from our tests and feedback from pre-release developers suggest that the new Gmail API is delivering dramatic performance improvements over IMAP for web application use cases.

Check out the launch video and get started with samples, tutorials, and API references at https://developers.google.com/gmail/api/. We can’t wait to see what you build.

Posted by Eric DeFriez, Gmail Extensibility Team. As a technical lead for Gmail APIs, Eric works to make it easy for developers to build on top of Gmail.

Email address validation with Actions in the Inbox and Mandrill

We launched Actions in the Inbox at Google I/O 2013 as a quick way for users to get things done directly from Gmail. Integrating with this technology only requires adding some markup to an email to define what the message is about and what actions the user can perform.

We support a variety of action types covering common scenarios such as adding a movie to a queue, product reviews, or other even pre-defined requests. Especially popular with senders is the One-Click Action to validate a user’s email address, as shown below:


If you are using Mandrill, the email infrastructure service from MailChimp, writing a Python app to send one of those emails, only takes a few lines of code! Take a look at this example:

import mandrill

# Replace with your own values
API_KEY = 'YOUR_API_KEY'
FROM_ADDRESS = 'YOUR_FROM_ADDRESS'
TO_ADDRESS = 'YOUR_TO_ADDRESS'
SUBJECT = 'Please validate your email address'

HTML_CONTENT = """
<html>
  <body>
    <script type='application/ld+json'>
    {
      "@context": "http://schema.org",
      "@type": "EmailMessage",
      "action": {
        "@type": "ConfirmAction",
        "name": "Confirm Registration",
        "handler": {
          "@type": "HttpActionHandler",
          "url": "https://mydomain.com/validate?id=abc123"
        }
      }
    }
    </script>
    <p>Please click on this link to validate your email address:</p>
    <p><a href="https://mydomain.com/validate?id=abc123">https://mydomain.com/validate?id=abc123</a></p>
  </body>
</html>
"""

# Instantiate the Mandrill client with your API Key
mandrill_client = mandrill.Mandrill(API_KEY)

message = {
 'html': HTML_CONTENT,
 'subject': SUBJECT,
 'from_email': FROM_ADDRESS,
 'to': [{'email': TO_ADDRESS}],
}

result = mandrill_client.messages.send(message=message)

To run this app, just replace the API key with the credentials from your Mandrill account and configure the sender and recipient addresses. You should also edit the HTML content to provide your action handler URL, and customize the messaging if you want.

You can use Actions in the Inbox to reduce the friction and increase the conversion rate. Please note that you are not limited to validating email addresses: you can also review products and services, reply to event invitations, and much more.

For more information, please visit our documentation at https://developers.google.com/gmail/actions. You can also ask questions on Stack Overflow, with the tag google-schemas.

Claudio Cherubino   profile | twitter | blog

Claudio is an engineer in the Gmail Developer Relations team. Prior to Google, he worked as software developer, technology evangelist, community manager, consultant, technical translator and has contributed to many open-source projects. His current interests include Google APIs, new technologies and coffee.