Tag Archives: Gemini

Google Workspace Updates Weekly Recap – April 12, 2024

2 New updates

Unless otherwise indicated, the features below are available to all Google Workspace customers, and are fully launched or in the process of rolling out. Rollouts should take no more than 15 business days to complete if launching to both Rapid and Scheduled Release at the same time. If not, each stage of rollout should take no more than 15 business days to complete.


Address access permissions for Google Drive embeds in Google Sites 
When adding embedded content from Google Drive into a Google Site, such as a PDF, document or presentation, site editors will now be prompted to address potential access permissions. The notification will also appear when site editors are publishing the site or sharing it with other site collaborators and viewers. This will ensure other site collaborators or viewers have permission to edit or view embedded Drive content when collaborating on a site. | Rolling out to Rapid Release domains now; launch to Scheduled Release domains planned for April 25, 2024. | Available to Google Workspace customers, Google Workspace Individual subscribers, and users with personal Google accounts. | Learn more about adding Google files, videos, website content, & more.
Address access permissions for Google Drive embeds in Google Sites

Track usage for Gemini for Workspace users in the Admin console
We recently announced the Gemini Business add-on which provides a subset of generative AI features, subject to monthly usage limits. Gemini Business customers can now check a user’s Gemini limit status in the admin console. For Gemini Enterprise and Gemini Business customers, admins can check their user’s last Gemini usage date as well. | Gemini usage and limit status reports are now available. | Learn more about Usage limits in Gemini for Google Workspace.




Previous announcements

The announcements below were published on the Workspace Updates blog earlier this week. Please refer to the original blog posts for complete details.


Introducing the AI Meetings and Messaging for Google Workspace add-on 
As we continue to expand our Gemini for Google Workspace offerings, we're excited to introduce the AI Meetings and Messaging add-on, which will help you have richer meetings and foster more meaningful collaboration. | Learn more about the AI Meetings and Messaging add-on. 

Introducing a new AI Security add-on for Google Workspace  
The AI Security add-on will give customers access to the AI Classification capability in Google Drive. AI Classification allows IT teams to automatically and continuously identify, classify, and label sensitive files across the organization. | Learn more about the AI Security add-on. 

Control your users’ access to new Gemini for Google Workspace features before general availability
We’re introducing a new setting in the Admin console which will give Gemini customers the ability to test Gemini for Google Workspace alpha features before they become generally available. Specifically, admins will be able to turn on alpha features for all Gemini provisioned Workspace users or for a subset of Gemini users in a particular Organizational Unit (OU) or Group. | Learn more about accessing Gemini for Google Workspace features. 

Protect sensitive admin actions with multi-party approvals 
To protect our customers from malicious actors taking sensitive admin actions, we’re launching multi-party approvals where one admin must approve certain sensitive actions initiated by another. | Learn more about multi-party approvals.

Changes to displaying the “deprovisioned” status for Google Meet hardware devices 
We are removing the “deprovisioned” state from the Admin console. You’ll no longer see devices in this state from the device status page (Devices > Google Meet Hardware > Devices), nor will you be able to filter for those labels. | Learn more about statuses for Google Meet hardware devices.



For a recap of announcements in the past six months, check out What’s new in Google Workspace (recent releases).   




Gemini 1.5 Pro Now Available in 180+ Countries; With Native Audio Understanding, System Instructions, JSON Mode and More

Posted by Jaclyn Konzelmann and Megan Li - Google Labs

Grab an API key in Google AI Studio, and get started with the Gemini API Cookbook

Less than two months ago, we made our next-generation Gemini 1.5 Pro model available in Google AI Studio for developers to try out. We’ve been amazed by what the community has been able to debug, create and learn using our groundbreaking 1 million context window.

Today, we’re making Gemini 1.5 Pro available in 180+ countries via the Gemini API in public preview, with a first-ever native audio (speech) understanding capability and a new File API to make it easy to handle files. We’re also launching new features like system instructions and JSON mode to give developers more control over the model’s output. Lastly, we’re releasing our next generation text embedding model that outperforms comparable models. Go to Google AI Studio to create or access your API key, and start building.


Unlock new use cases with audio and video modalities

We’re expanding the input modalities for Gemini 1.5 Pro to include audio (speech) understanding in both the Gemini API and Google AI Studio. Additionally, Gemini 1.5 Pro is now able to reason across both image (frames) and audio (speech) for videos uploaded in Google AI Studio, and we look forward to adding API support for this soon.


screen grab of a clooege professor using Gemini 1.5 Pro to create a quiz based on their latest lecture video in Google AI Studio
You can upload a recording of a lecture, like this 117,000+ token lecture from Jeff Dean, and Gemini 1.5 Pro can turn it into a quiz with an answer key. Video sped up for demo purposes.

Gemini API Improvements

Today, we’re addressing a number of top developer requests:

1. System instructions: Guide the model’s responses with system instructions, now available in Google AI Studio and the Gemini API. Define roles, formats, goals, and rules to steer the model's behavior for your specific use case.

image showing where System Instructions is located in Google AI Studio
Set System Instructions easily in Google AI Studio

2. JSON mode: Instruct the model to only output JSON objects. This mode enables structured data extraction from text or images. You can get started with cURL, and Python SDK support is coming soon.

3. Improvements to function calling: You can now select modes to limit the model’s outputs, improving reliability. Choose text, function call, or just the function itself.


A new embedding model with improved performance

Starting today, developers will be able to access our next generation text embedding model via the Gemini API. The new model, text-embedding-004, (text-embedding-preview-0409 in Vertex AI), achieves a stronger retrieval performance and outperforms existing models with comparable dimensions, on the MTEB benchmarks.

table showing Gecko: Versativel Text Embeddings Distilled from Large Language Models
'Text-embedding-004' (aka Gecko) using 256 dims output outperforms all larger 768 dim output models on MTEB benchmarks

These are just the first of many improvements coming to the Gemini API and Google AI Studio in the next few weeks. We’re continuing to work on making Google AI Studio and the Gemini API the easiest way to build with Gemini. Get started today in Google AI Studio with Gemini 1.5 Pro, explore code examples and quickstarts in our new Gemini API Cookbook, and join our community channel on Discord.

Introducing the AI Meetings and Messaging for Google Workspace add-on

This announcement was part of Google Cloud Next ‘24. Visit the Workspace Blog to learn more about the next wave of innovations in Workspace, including enhancements to Gemini for Google Workspace.

What’s changing

As we continue to expand our Gemini for Google Workspace offerings, we're excited to introduce the AI Meetings and Messaging add-on, which will help you have richer meetings and foster more meaningful collaboration.


At launch, the AI Meetings and Messaging add-on will give customers access to Google Meet features such as studio look, studio lighting, studio sound, and take notes for me (coming soon in Alpha) allowing customers to have more effective and efficient meetings. In the future, AI Meetings and Messaging will also provide access to Gemini features in Google Chat features such as on-demand conversation summaries and automatic translation of messages.


Who’s impacted

Admins


Why it’s important

The AI Meetings and Messaging add-on, along with the new AI Security add-on also announced at Google Cloud Next ‘24, give our customers more ways to work with AI that best suits the needs of their organization. The AI Meetings and Messaging add-on can help enhance collaboration across Meet and Chat with a variety of features such as:

  • Generative backgrounds in Google Meet
  • Studio look, studio sound, and studio lighting in Google Meet
  • Real time translated captions in Google Meet
  • Take notes for me in Google Meet (coming soon in alpha) 
  • And upcoming features like:
    • Translate for me in Google Meet and Chat for automatic language detection and translation 
    • Adaptive audio in Google Meet for synchronized audio and no feedback when multiple users join a meeting from a room using only their laptops
    • Screenshare watermark in Google Meet to help discourage the copying and unauthorized distribution of shared content
    • On-demand conversation summaries in the home view of Google Chat to get you caught up quickly

Visit our Help Center for a complete list of features available for the AI Meetings and Messaging add-on. Keep an eye on the Workspace Updates blog for new feature launches in the future.


Additional details

Some announced Meet and Chat features for this add-on will be available later this year. More details on timing will be shared in the coming months here on the Workspace Updates blog. This announcement on the Workspace Updates blog has more information about how to enable alpha testing for your end users.


Getting started

Availability

The AI Meetings and Messaging add-on is available for the following Google Workspace Editions:
  • Business Starter, Standard, and Plus
  • Enterprise Starter, Standard, and Plus
  • Frontline Starter and Standard
  • Enterprise Essentials, Essentials Plus
  • Nonprofits

Resources


Introducing a new AI Security add-on for Google Workspace

This announcement was part of Google Cloud Next ‘24. Visit the Workspace Blog to learn more about the next wave of innovations in Workspace, including enhancements to Gemini for Google Workspace.


What’s changing

As we continue to expand our Gemini for Google Workspace offerings, we're excited to introduce the AI Security add-on for Google Workspace customers. 

At launch, the AI Security add-on will give customers access to the AI Classification capability in Google Drive. AI Classification allows IT teams to automatically and continuously identify, classify, and label sensitive files across the organization. This capability is powered with privacy-preserving AI models that can be uniquely trained for the specific needs of your organization. Classified files can then be protected with existing data loss prevention (DLP) controls. 

Who’s impacted

Admins

Why it matters

Drive Labels enable Workspace Administrators to up-level their security posture by closely monitoring activity on labeled files, and using labels as a vehicle for data loss prevention and lifecycle management policies. The challenge with label-based policies is that they are only effective on files that are correctly identified and labeled. Further, labeling files placed a considerable manual burden on Admins.

This is where AI Classification can help. By training models on customer-identified examples of content that match their data classification definitions, AI Classification can evaluate files where text can be extracted to see if it should be labeled.  This enables organizations to achieve label coverage at a scale and accuracy that is very difficult to accomplish through traditional means and manual Admin intervention. Once labeled, the organization's data can be protected by fine-grained security policies. 


Availability

The AI Security add-on is available for the following Google Workspace Editions:
  • Business Standard and Plus
  • Enterprise Standard and Plus
  • Enterprise Essentials and Essentials Plus
  • Frontline Starter and Standard
  • Google Workspace for Nonprofits 

Resources


Control your users’ access to new Gemini for Google Workspace features before general availability

This announcement was part of Google Cloud Next ‘24. Visit the Workspace Blog to learn more about the next wave of innovations in Workspace, including enhancements to Gemini for Google Workspace.


What’s changing

We’re introducing a new setting in the Admin console which will give Gemini customers the ability to test Gemini for Google Workspace alpha features before they become generally available. Specifically, admins will be able to turn on alpha features for all Gemini provisioned Workspace users or for a subset of Gemini users in a particular Organizational Unit (OU) or Group.

To configure Gemini access features, go to Account settings > Gemini for Google Workspace



Who’s impacted

Admins and end users


Why it matters

As our Gemini for Workspace offerings continue to evolve, you may consider allowing your users to test Gemini features in alpha. This will give your users a head start on leveraging our latest AI features and provide Google with helpful feedback to improve Gemini features before they’re generally available. Alpha features get the same robust data protection standards that come with all Google Workspace services.

Getting started

        Please consider the following before configuring alpha access for your users:
    • Your users will receive all Gemini for Workspace alpha features — it is not possible to enable a subset of features or opt-out of specific features. 
    • Features will appear in alpha as soon as they are available — there is no advanced notice of these features appearing for Gemini  for Workspace alpha provisioned users.
    • As these features are not yet generally available, we will not offer full support for these features. Alpha features get the same robust data protection standards that come with all Google Workspace services.
    • You can also help us improve Gemini for Workspace by allowing users at your organization to provide feedback via research studies and surveys. 
Additionally, we strongly recommend that you and your users sign up for the Google Workspace alpha community page. Subscribing to this page will help users stay on top of the latest Gemini for Workspace alpha features. You can also ask questions about the features on this page.

Rollout pace


Availability

Gemma Family Expands with Models Tailored for Developers and Researchers

Posted by Tris Warkentin – Director, Product Management and Jane Fine - Senior Product Manager

In February we announced Gemma, our family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models. The community's incredible response – including impressive fine-tuned variants, Kaggle notebooks, integration into tools and services, recipes for RAG using databases like MongoDB, and lots more – has been truly inspiring.

Today, we're excited to announce our first round of additions to the Gemma family, expanding the possibilities for ML developers to innovate responsibly: CodeGemma for code completion and generation tasks as well as instruction following, and RecurrentGemma, an efficiency-optimized architecture for research experimentation. Plus, we're sharing some updates to Gemma and our terms aimed at improvements based on invaluable feedback we've heard from the community and our partners.


Introducing the first two Gemma variants


CodeGemma: Code completion, generation, and chat for developers and businesses

Harnessing the foundation of our Gemma models, CodeGemma brings powerful yet lightweight coding capabilities to the community. CodeGemma models are available as a 7B pretrained variant that specializes in code completion and code generation tasks, a 7B instruction-tuned variant for code chat and instruction-following, and a 2B pretrained variant for fast code completion that fits on your local computer. CodeGemma models have several advantages:

  • Intelligent code completion and generation: Complete lines, functions, and even generate entire blocks of code – whether you're working locally or leveraging cloud resources. 
  • Enhanced accuracy: Trained on 500 billion tokens of primarily English language data from web documents, mathematics, and code, CodeGemma models generate code that's not only more syntactically correct but also semantically meaningful, helping reduce errors and debugging time. 
  • Multi-language proficiency: Your invaluable coding assistant for Python, JavaScript, Java, and other popular languages. 
  • Streamlined workflows: Integrate a CodeGemma model into your development environment to write less boilerplate, and focus on interesting and differentiated code that matters – faster.
image of streamlined workflows within an exisitng AI dev project with CodeGemma integrated
This table compares the performance of CodeGemma with other similar models on both single and multi-line code completion tasks. Learn more in the technical report.

Learn more about CodeGemma in our report or try it in this quickstart guide.


RecurrentGemma: Efficient, faster inference at higher batch sizes for researchers

RecurrentGemma is a technically distinct model that leverages recurrent neural networks and local attention to improve memory efficiency. While achieving similar benchmark score performance to the Gemma 2B model, RecurrentGemma's unique architecture results in several advantages:

  • Reduced memory usage: Lower memory requirements allow for the generation of longer samples on devices with limited memory, such as single GPUs or CPUs. 
  • Higher throughput: Because of its reduced memory usage, RecurrentGemma can perform inference at significantly higher batch sizes, thus generating substantially more tokens per second (especially when generating long sequences). 
  • Research innovation: RecurrentGemma showcases a non-transformer model that achieves high performance, highlighting advancements in deep learning research. 
graph showing maximum thoughput when sampling from a prompt of 2k tokens on TPUv5e
This chart reveals how RecurrentGemma maintains its sampling speed regardless of sequence length, while Transformer-based models like Gemma slow down as sequences get longer.

To understand the underlying technology, check out our paper. For practical exploration, try the notebook, which demonstrates how to finetune the model.


Built upon Gemma foundations, expanding capabilities

Guided by the same principles of the original Gemma models, the new model variants offer:

  • Open availability: Encourages innovation and collaboration with its availability to everyone and flexible terms of use. 
  • High-performance and efficient capabilities: Advances the capabilities of open models with code-specific domain expertise and optimized design for exceptionally fast completion and generation. 
  • Responsible design: Our commitment to responsible AI helps ensure the models deliver safe and reliable results. 
  • Flexibility for diverse software and hardware:  
    • Both CodeGemma and RecurrentGemma: Built with JAX and compatible with JAX, PyTorch, , Hugging Face Transformers, and Gemma.cpp. Enable local experimentation and cost-effective deployment across various hardware, including laptops, desktops, NVIDIA GPUs, and Google Cloud TPUs.  
    • CodeGemma: Additionally compatible with Keras, NVIDIA NeMo, TensorRT-LLM, Optimum-NVIDIA, MediaPipe, and availability on Vertex AI. 
    • RecurrentGemma: Support for all the aforementioned products will be available in the coming weeks.

Gemma 1.1 update

Alongside the new model variants, we're releasing Gemma 1.1, which includes performance improvements. Additionally, we've listened to developer feedback, fixed bugs, and updated our terms to provide more flexibility.


Get started today

These first Gemma model variants are available in various places worldwide, starting today on Kaggle, Hugging Face, and Vertex AI Model Garden. Here's how to get started:

We invite you to try the CodeGemma and RecurrentGemma models and share your feedback on Kaggle. Together, let's shape the future of AI-powered content creation and understanding.

Android Studio uses Gemini Pro to make Android development faster and easier

Posted by Sandhya Mohan – Product Manager, Android Studio

As part of the next chapter of our Gemini era, we announced we were bringing Gemini to more products. Today we’re excited to announce that Android Studio is using the Gemini 1.0 Pro model to make Android development faster and easier, and we’ve seen significant improvements in response quality over the last several months through our internal testing. In addition, we are making this transition more apparent by announcing that Studio Bot is now called Gemini in Android Studio.

Gemini in Android Studio is an AI-powered coding assistant which can be accessed directly in the IDE. It can accelerate your ability to develop high-quality Android apps faster by helping generate code for your app, providing complex code completions, answering your questions, finding relevant resources, adding code comments and more — all without ever having to leave Android Studio. It is available in 180+ countries and territories in Android Studio Jellyfish.

If you were already using Studio Bot in the canary channel, you’ll continue experiencing the same helpful and powerful features, but you’ll notice improved quality in responses compared to earlier versions.

Ask Gemini your Android development questions

Gemini in Android Studio can understand natural language, so you can ask development questions in your own words. You can enter your questions in the chat window ranging from very simple and open-ended ones to specific problems that you need help with.

Here are some examples of the types of queries it can answer:

    • How do I add camera support to my app?
    • Using Compose, I need a login screen with the following: a username field, a password field, a 'Sign In' button, a 'Forgot Password?' link. I want the password field to obscure the input.
    • What's the best way to get location on Android?
    • I have an 'orders' table with columns like 'order_id', 'customer_id', 'product_id', 'price', and 'order_date'. Can you help me write a query that calculates the average order value per customer over the last month?
Moving image demonstrating a conversation in Android Studio

Gemini in Android Studio remembers the context of the conversation, so you can also ask follow-up questions, such as “Can you give me the code for this in Kotlin?” or “Can you show me how to do it in Compose?”

Code faster with AI powered Code Completions

Gemini in Android Studio can help you be more productive by providing you with powerful AI code completions. You can receive suggestions of multi-line code completions, suggestions for how to do comments for your code, or how to add documentation to your code.

Moving image demonstrating code completion in Android Studio

Designed with privacy in mind

Gemini in Android Studio was designed with privacy in mind. Gemini is only available after you log in and enable it. You don’t need to send your code context to take advantage of most features. By default, Gemini in Android Studio’s chat responses are purely based on conversation history, and you control whether you want to share additional context for customized responses. You can update this anytime in Android Studio > Settings at a granular project level. We also have a custom way for you to opt out certain files and folders through an .aiexclude file. Much like our work on other AI projects, we stick to a set of AI Principles that hold us accountable. Learn more here.

image of share settings in Android Studio

Build a Generative AI app using the Gemini API starter template

Not only does Android Studio use Gemini to help you be more productive, it can also help you take advantage of Gemini models to create AI-powered features in your applications. Get started in minutes using the Gemini API starter template available in the canary release – channel for Android Studio – under File > New Project > Gemini API Starter. You can also use the code sample available at File > Import Sample > Google Generative AI sample.

The Gemini API is multimodal, meaning it can support image and text inputs. For example, it can support conversational chat, summarization, translation, caption generation etc. using both text and image inputs.

image of starter templates in Android Studio

Try Gemini in Android Studio

Gemini in Android Studio is still in preview, but we have added many feature improvements — and now a major model update — since we released the experience in May 2023. It is currently no-cost for developers to try out. Now is a great time to test it and let us know what you think, before we release this experience to stable.


Stay updated on the latest by following us on LinkedIn, Medium, YouTube, or X. Let's build the future of Android apps together!

Tune Gemini Pro in Google AI Studio or with the Gemini API

Posted by Cher Hu, Product Manager and Saravanan Ganesh, Software Engineer for Gemini API

The following post was originally published in October 2023. Today, we've updated the post to share how you can easily tune Gemini models in Google AI Studio or with the Gemini API.


Last year, we launched Gemini 1.0 Pro, our mid-sized multimodal model optimized for scaling across a wide range of tasks. And with 1.5 Pro this year, we demonstrated the possibilities of what large language models can do with an experimental 1M context window. Now, to quickly and easily customize the generally available Gemini 1.0 Pro model (text) for your specific needs, we’ve added Gemini Tuning to Google AI Studio and the Gemini API.


What is tuning?

Developers often require higher quality output for custom use cases than what can be achieved through few-shot prompting. Tuning improves on this technique by further training the base model on many more task-specific examples—so many that they can’t all fit in the prompt.


Fine-tuning vs. Parameter Efficient Tuning

You may have heard about classic “fine-tuning” of models. This is where a pre-trained model is adapted to a particular task by training it on a smaller set of task-specific labeled data. But with today’s LLMs and their huge number of parameters, fine-tuning is complex: it requires machine learning expertise, lots of data, and lots of compute.

Tuning in Google AI Studio uses a technique called Parameter Efficient Tuning (PET) to produce higher-quality customized models with lower latency compared to few-shot prompting and without the additional costs and complexity of traditional fine-tuning. In addition, PET produces high quality models with as little as a few hundred data points, reducing the burden of data collection for the developer.


Why tuning?

Tuning enables you to customize Gemini models with your own data to perform better for niche tasks while also reducing the context size of prompts and latency of the response. Developers can use tuning for a variety of use cases including but not limited to:

  • Classification: Run natural language tasks like classifying your data into predefined categories, without needing tons of manual work or tools.
  • Information extraction: Extract structured information from unstructured data sources to support downstream tasks within your product.
  • Structured output generation: Generate structured data, such as tables, quickly and easily.
  • Critique Models: Use tuning to create critique models to evaluate output from other models.

Get started quickly with Google AI Studio


1. Create a tuned model

It’s easy to tune models in Google AI Studio. This removes any need for engineering expertise to build custom models. Start by selecting “New tuned model” in the menu bar on the left.

moving image showing how to create a tuned model in Google AI Studio by opening 'New Tuned Model' from the menu

2. Select data for tuning

You can tune your model from an existing structured prompt or import data from Google Sheets or a CSV file. You can get started with as few as 20 examples and to get the best performance, we recommend providing a dataset of at least 100 examples.

moving image showing how to select data for tuning in Google AI Studio by importing data

3. View your tuned model

View your tuning progress in your library. Once the model has finished tuning, you can view the details by clicking on your model. Start running your tuned model through a structured or freeform prompt.

moving image showing how to view your tuned model in Google AI Studio by importing data

4. Run your tuned model anytime

You can also access your newly tuned model by creating a new structured or freeform prompt and selecting your tuned model from the list of available models.

moving image demonstrating what it looks like to run your tuned model in Google AI Studio after importing data

Tuning with the Gemini API

Google AI Studio is the fastest and easiest way to start tuning Gemini models. You can also access the feature via the Gemini API by passing the training data in the API request when creating a tuned model. Learn more about how to get started here.

We’re excited about the possibilities that tuning opens up for developers and can’t wait to see what you build with the feature. If you’ve got some ideas or use cases brewing, share them with us on X (formerly known as Twitter) or Linkedin.