Tag Archives: Gemini

Use Gemini in the side panel of Workspace apps to generate images in seven additional languages

What’s changing

Following the recent announcement of being able to use Gemini in the side panel of Workspace apps in seven additional languages, we’re excited to announce that image generation is now supported in the following languages as well: 
  • French 
  • German 
  • Italian 
  • Japanese 
  • Korean 
  • Portuguese 
  • Spanish 
using gemini to generate an image of coffee in Spanish


Additional details 

  • Although Gemini is generally available in the side panel of Docs, Sheets, Drive, and Gmail in these languages, users with Gemini alpha features enabled may still see the "Alpha" badge as we continue rolling out more alpha features 
  • Image generation of people is not supported at this time in these additional languages. 
  • Gemini in the side panel of Slides is only available in English at this time. 

Getting started 

  • Admins: There is no admin control for this feature. 
  • End users: You can access the side panel by clicking on “Ask Gemini” (spark button) in the top right corner of Docs, Sheets, Drive, and Gmail on the web. Visit the Help Center to learn more about collaborating with Gemini in the side panel of Docs, Sheets, Drive, and Gmail. Note that for accessing other Gemini for Google Workspace features that are supported in English only, you will need to set your Google Account language to English

Rollout pace 


Availability 

Image Generation in the side panel of Gmail only is available for Google Workspace: 
  • Business Starter 
  • Enterprise Starter 

Image Generation in the side panel of Docs, Sheets, Drive, and Gmail is available for Google Workspace: 
  • Business Standard and Plus 
  • Enterprise Standard and Plus 
  • Customers with the Gemini Education or Gemini Education Premium add-on 
  • Google One AI Premium 

Anyone who previously purchased these add-ons will also receive this feature: 
  • Gemini Business* 
  • Gemini Enterprise* 
*As of January 15, 2025, we’re no longer offering the Gemini Business and Gemini Enterprise add-ons for sale. Please refer to this announcement for more details. 


Resources 

Production-ready generative AI on Android with Vertex AI in Firebase

Posted by Thomas Ezan – Sr. Developer Relation Engineer (@lethargicpanda)

Gemini can help you build and launch new user features that will boost engagement and create personalized experiences for your users.

The Vertex AI in Firebase SDK lets you access Google’s Gemini Cloud models (like Gemini 1.5 Flash and Gemini 1.5 Pro) and add GenAI capabilities to your Android app. It became generally available last October which means it's now ready for production and it is already used by many apps in Google Play.

Here are tips for a successful deployment to production.

Implement App Check to prevent API abuse

When using the Vertex AI in Firebase API it is crucial to implement robust security measures to prevent unauthorized access and misuse.

Firebase App Check helps protect backend resources (like Vertex AI in Firebase, Cloud Functions for Firebase, or even your own custom backend) from abuse. It does this by attesting that incoming traffic is coming from your authentic app running on an authentic and untampered Android device.

A flow diagram illustrating App Check, with green lines depicting 'User Request' going through App Check to 'Backend'. A red line depicting 'Bad Request' is being blocked by App Check.
Firebase App Check ensures that only legitimate users access your backend resources

To get started, add Firebase to your Android project and enable the Play Integrity API for your app in the Google Play console. Back in the Firebase console, go to the App Check section of your Firebase project to register your app by providing its SHA-256 fingerprint.

Then, update your Android project’s Gradle dependencies with the App Check library for Android:

dependencies {
    // BoM for the Firebase platform
   implementation(platform("com.google.firebase:firebase-bom:33.7.0"))

    // Dependency for App Check
    implementation("com.google.firebase:firebase-appcheck-playintegrity")
}

Finally, in your Kotlin code, initialize App Check before using any other Firebase SDK:

Firebase.initialize(context)
Firebase.appCheck.installAppCheckProviderFactory(
    PlayIntegrityAppCheckProviderFactory.getInstance(),
)

To enhance the security of your generative AI feature, you should implement and enforce App Check before releasing your app to production. Additionally, if your app utilizes other Firebase services like Firebase Authentication, Firestore, or Cloud Functions, App Check provides an extra layer of protection for those resources as well.

Once App Check is enforced, you’ll be able to monitor your app’s requests in the Firebase console.

An area chart of the Apps Check metrics page in Firebase console, showing the percentages of verified and unverified requests over several days. Numerical breakdowns of verified (51%) and unverified requests (49%) are shown.
App Check metrics page in the Firebase console

You can learn more about App Check on Android in the Firebase documentation.

Use Remote Config for server-controlled configuration

The generative AI landscape evolves quickly. Every few months, new Gemini model iterations become available and some models are removed. See the Vertex AI in Firebase Gemini models page for details.

Because of this, instead of hardcoding the model name in your app, we recommend using a server-controlled variable using Firebase Remote Config. This allows you to dynamically update the model your app uses without having to deploy a new version of your app or require your users to pick up a new version.

You define parameters that you want to control (like model name) using the Firebase console. Then, you add these parameters into your app, along with default "fallback" values for each parameter. Back in the Firebase console, you can change the value of these parameters at any time. Your app will automatically fetch the new value.

Here's how to implement Remote Config in your app:

// Initialize the remote configuration by defining the refresh time
val remoteConfig: FirebaseRemoteConfig = Firebase.remoteConfig
val configSettings = remoteConfigSettings {
    minimumFetchIntervalInSeconds = 3600
}
remoteConfig.setConfigSettingsAsync(configSettings)

// Set default values defined in your app resources 
remoteConfig.setDefaultsAsync(R.xml.remote_config_defaults)

// Load the model name
val modelName = remoteConfig.getString("model_name")

Read more about using Remote Config with Vertex AI in Firebase.

Gather user feedback to evaluate impact

As you roll out your AI-enabled feature to production, it's critical to build feedback mechanisms into your product and allow users to easily signal whether the AI output was helpful, accurate, or relevant. For example, you can incorporate interactive elements such as thumb-up and thumb-down buttons and detailed feedback forms within the user interface. The Material Icons in Compose package provides ready to use icons to help you implement it.

You can easily track the user interaction with these elements as custom analytics events by using Google Analytics logEvent() function:

Row {
   Button (
      onClick = {
         firebaseAnalytics.logEvent("model_response_feedback") {
            param("feedback", "thumb_up")
         }
      }
   ) {
      Icon(Icons.Default.ThumbUp, contentDescription = "Thumb up")
   },
   Button (
      onClick = {
         firebaseAnalytics.logEvent("model_response_feedback") {
            param("feedback", "thumb_down")
         }
      }
   ) {
      Icon(Icons.Default.ThumbDown, contentDescription = "Thumb down")
   }
}

Learn more about Google Analytics and its event logging capabilities in the Firebase documentation.

User privacy and responsible AI

When you use Vertex AI in Firebase for inference, you have the guarantee that the data sent to Google won’t be used by Google to train AI models (see Vertex AI documentation for details).

It's also important to be transparent with your users when they're engaging with generative AI technology. You should highlight the possibility of unexpected model behavior.

Finally, users should have control within your app over how their activity related to AI model interactions is stored and deleted.

You can learn more about how Google is approaching Generative AI responsibly in the Google Cloud documentation.

Generate charts and valuable insights using Gemini in Google Sheets

What’s changing 

Building data visualizations and identifying trends in data is a critical business function, however it can be time consuming and oftentimes overwhelming. To improve upon this experience, we’re making it easier for Sheets users, ranging from small business owners to marketing analysts, to visualize and analyze data using Gemini in Sheets. 

Starting today, Gemini in Sheets can perform actions or answer questions to address a wide variety of scenarios. Specifically, you can ask Gemini questions about your data and Gemini can provide valuable insights, such as contextual trends, patterns, and correlations between different variables in your data. Gemini will also generate charts based on your data that you can insert as static images over cells in your spreadsheet. For example: 
  • A marketing manager analyzing campaign performance can ask "Provide some insights on my top 3 performing channels by conversion rate" and can receive a detailed breakdown with supporting visualizations. 
  • A small business owner managing cash flow can ask "Predict my net income for the next quarter based on historical data" to proactively identify potential shortfalls. 
  • A financial analyst investigating inventory trends can ask "Identify any anomalies in inventory levels for Product X" to uncover potential supply chain issues or unusual demand patterns. 
Ask Gemini in Sheets to share insights and trends from the data.
Ask Gemini in Sheets to share insights and trends from the data.

Ask Gemini in Sheets to visualize data with a chart or bar graph.
Ask Gemini in Sheets to visualize data with a chart or bar graph.



Who’s impacted 

End users 


Why you’d use it 

This update reflects Gemini in Sheets’ enhanced intelligence as it now yields valuable outputs by transforming your requests into Python code, executing on them and then analyzing the results to perform multi-layered analysis. As a result, you’re able to do things like identify top-performing products and then instantly dive into their sales trends — all without writing a single line of code. 


Additional details 

  • For the most accurate results, ensure your data is in a consistent format, with clear headers and no missing values. 
  • When asking Gemini in Sheets to analyze your data, be as specific as possible in your prompts to ensure Gemini understands your request. 
  • For complex analysis, Gemini in Sheet's performance is most consistent in files below 1 million cells. 
  • For some simpler questions about your data, Gemini in Sheets may still provide answers using formulas instead of Python code. 
  • Charts are generated as static images and can be easily inserted into your sheet. 
  • Charts are not linked to your sheet data, so they won't update automatically if your data changes. 
  • Once inserted, charts cannot be modified. If you need to make changes, you'll need to generate a new chart. 

Getting started 

Rollout pace 

Availability 

Available for Google Workspace:
  • Business Standard and Plus 
  • Enterprise Standard and Plus 
  • Customers with the Gemini Education or Gemini Education Premium add-on 
  • Google One AI Premium 
Anyone who previously purchased these add-ons will also receive this feature: 
  • Gemini Business* 
  • Gemini Enterprise* 
*As of January 15, 2025, we’re no longer offering the Gemini Business and Gemini Enterprise add-ons for sale. Please refer to this announcement for more details. 

Resources 

Generate vocabulary lists using Gemini in Google Classroom

What’s changing 

Since May of last year, education institutions have been leveraging new and powerful ways of working, teaching and learning with Gemini for Google Workspace. This includes products and features such as the Gemini app, Gemini Learning Tools Interoperability (LTI™), Gemini in Google Vids and Gemini in Classroom.

In continuing our efforts to ensure AI helps educators save time, create captivating learning experiences, and inspire creativity, we’re introducing a new way to generate vocabulary lists with the help of Gemini. 

Starting today, educators can generate custom word lists that are tailored to specific grade levels and topics identified by the educator. Specifically, the list will provide the word’s grammatical category, a clear definition, and a relevant example sentence to help students understand and use the vocabulary effectively. This functionality is accessible via the “Gemini Education” tab in Google Classroom. 

creating a vocab list using gemini in classroom


Who’s impacted 

Admins and end users 


Why it’s important 

This update helps educators create vocabulary lists tailored to specific grade levels and topics while simultaneously empowering students to master vocabulary and deepen their understanding of language – all right within Google Classroom. 


Getting started 


Rollout pace 


Availability 

Available for Google Workspace customers with these add-ons: 
  • Gemini Education and Education Premium 

Resources 

Use Gemini to interact with PDFs in Google Drive’s overlay file previewer

What’s changing

Earlier this year, we introduced the ability to use Gemini in Google Drive to interact with PDFs. To improve your viewing experience when reviewing PDFs from Drive, we’re excited to announce that Gemini in the side panel of Drive is now integrated into Drive's overlay file previewer. 

As a result, you can seamlessly switch between multiple files while leveraging AI capabilities using Gemini in Drive to do things like: 
  • Admins: To access Gemini in the side panel of Workspace apps, users need to have smart features and personalization turned on. Admins can turn on default personalization setting for their users in the Admin console. 
  • End users:
    • To access this feature, double-click on a PDF from the Google Drive file list and click on "Ask Gemini" (star button) in the top right corner. 
    • Note: When Gemini initially launched in Workspace, PDFs viewed in Drive opened in a new browser tab to allow interaction with the Gemini side panel. With this update, the default behavior will open a file in the overlay file previewer. If you prefer for PDFs to open in a new tab by default, you can update your PDF opening default behavior in your Drive settings. If you previously set a preferred PDF opening default behavior in your Drive setting, your default open behavior will remain the same. 
    • Visit the Help Center to learn more about using Gemini in Drive to work with PDFs. 

Rollout pace 


Availability 

Available for Google Workspace customers with these add-ons: 
  • Gemini Business 
  • Gemini Enterprise 
  • Gemini Education 
  • Gemini Education Premium 
  • Google One AI Premium 

Resources