Tag Archives: Gemini

Gaze Link Wins Best Android App in Gemini API Developer Competition

Posted by Thomas Ezan – Sr Developer Relation Engineer (@lethargicpanda)

We're excited to announce Gaze Link as the winner of the Best Android App for our Gemini API Developer Competition!

This innovative app demonstrates the potential of the Gemini API in providing a communication system for individuals with Amyotrophic lateral sclerosis (ALS) who develop severe motor and verbal disabilities, enabling them to type sentences with only their eyes.

About Gaze Link

Gaze Link uses Google’s Gemini 1.5 Flash model to predict the user’s intended sentence based on a few key words and the context of the conversation.

For example if the context is “Is the room temperature ok?” and the user replies “hot AC two” the app will leverage Gemini to generate the full sentence “I am hot, can you turn the AC down by two degrees?”.

The Gaze Link team took advantage of Gemini 1.5 Flash multilingual capabilities to let the app generate sentences in English, Spanish and Chinese, the three languages currently supported by the app.

We were truly impressed by the Gaze Link app. The team used the Gemini API combined with ML Kit Face Detection to empower individuals with ALS providing them with a powerful communication system that is both accessible and affordable.

With Gemini 1.5 Flash currently supporting 38 languages, it is possible for Gaze Link to add support for more languages in the future. In addition, the model’s multimodal abilities could enable the team to enhance the user experience by integrating image, audio and video to augment the context of the conversation.

Build with the Gemini API

The result of the integration of the Gemini API in Gaze Link is inspiring. If you are working on an Android app today, we encourage you to learn about the Gemini API capabilities to see how you can successfully add generative AI to your app and delight your users.

To get started, go to the Android AI documentation!

Use Gemini in Google Docs to create unique images for your documents

What’s changing 

Following our introduction of uploading full-bleed cover images in Google Docs and AI-generated images in Google Slides with help me visualize, we’re launching the ability to create unique inline images and full bleed cover images using Gemini in Docs. 

These new image generation capabilities benefit from our latest image generation model, Imagen 3, our highest quality, text-to-image model. Now you can create photorealistic images of people, landscapes, and more with even better detail than before.

To create inline images for numerous use cases (e.g. a restaurant menu, marketing campaign brief, or promotional flier), simply type your prompt into Create an image, and Gemini in Docs will generate the image based on your instruction. Plus, you can decide the aspect ratio you want the image to be and choose a style like photography, water color and more. 

create an image in Docs with Gemini

You can further customize your document by creating a unique cover image using Gemini in Docs, which can add flavor to a résumé, stylize an invitation, or personalize a client pitch. 

create a cover image with gemini

Who’s impacted 

End users 


Why you’d use it 

The ability to generate unique images with Gemini in Docs empowers everyone, regardless of artistic skill, to create differentiated and visually compelling content. Now you can communicate ideas more effectively, without having to tirelessly search for the perfect image. 


Additional details 

Cover images can only be added to documents set to pageless mode, allowing the image to span the entire width of the document. To set up the more immersive experience of pageless mode, go to File > Page setup > Pageless or Format > Switch to Pageless format. 


Getting started 

  • Admins: There is no admin control for this feature. 
  • End users: 
    • To create a cover image, you can either go to Insert > Cover image > Help me create an image, or Type “@” followed by Cover image > Help me create an image. 
    • To add an image, go to Insert > Image > Help me create an image > type in an image description (optionally select a style for your image) > click Create > then, you will see generated options > select the one you like and it will be added to your document. 
    • Visit the Help Center to learn more about creating images with Gemini in Google Docs and adding a cover image to your document. 

Rollout pace  

  • Rapid Release domains: Extended rollout (potentially longer than 15 days for feature visibility) starting on November 15, 2024
  • Scheduled Release domains: Gradual rollout (up to 15 days for feature visibility) starting on December 16, 2024

Availability 

Available for Google Workspace customers with these add-ons: 
  • Gemini Business 
  • Gemini Enterprise 
  • Gemini Education 
  • Gemini Education Premium 
  • Google One AI Premium 

Resources

Reference your Google Calendar using Gemini in the side panel of Gmail

What’s changing 

Since its introduction earlier this year, Gemini in the side panel of Gmail has helped many users generate, find and respond to information in their inbox. In addition, Gemini in Gmail has the ability to connect with other Workspace apps like Docs, Sheets, Slides and Drive, helping users complete tasks without ever leaving Gmail. 


Today, we’re excited to add Google Calendar to the list of apps that Gemini in Gmail connects with. Users can now ask Gemini in Gmail to perform calendar related actions or answer questions about their calendars such as: 
  • When is my [first event] next week? 
  • Create a [30 min] calendar event for my weekly [yoga class] every [Monday and Wednesday] at [9 AM]. 
  • Create a [1h] event for [lunch] [tomorrow] at [noon]. 
Using Gemini in the side panel of Gmail to ask Calendar to set up a personal yoga event


Who’s impacted 

End users 


Why you’d use it 

Via Gmail, Gemini can now help you: 
  • Create calendar events. 
  • Learn additional details about the existing events in your Calendar. 

Additional details

At this time, Gemini in Gmail cannot: 
  • Add or remove guests from events or create events by extracting information from an email. 
  • Pull up attachments or RSVPs related to a meeting. 
  • Find the best time to meet with other people or find time slots when you’re available to meet. 
  • Create tasks, an out of office, focus times or set working locations. 
  • Add or manage meeting rooms. 
When these features do become available, we will provide an update via the Workspace Updates Blog. 


Getting started 

  • Admins: To access Gemini in the side panel of Workspace apps, users need to have smart features and personalization turned on. Admins can turn on default personalization setting for their users in the Admin console. 
  • End users:
    • On web, you can access Gemini in the Gmail side panel by clicking on “Ask Gemini” icon in the top right corner of Gmail. 
    • This feature is only available in English at this time. 
    • While not required, just like gemini.google.com, you can invoke Calendar by adding "@Calendar" at the beginning of your prompt in the side panel of Gmail to explicitly call on Calendar. 
    • Visit the Help Center to learn more about collaborating with Gemini in Gmail. 

Rollout pace 

Availability 

Available for Google Workspace customers with these add-ons: 
  • Gemini Business 
  • Gemini Enterprise 
  • Gemini Education 
  • Gemini Education Premium 
  • Google One AI Premium 

Resources 

Upload Google Docs and other file types to Gem instructions

What’s changing 

Beginning today, you can upload reference documents, including Google Docs and Google Sheets via Google Drive, when creating Gems. By adding reference documents to your Gems instructions you can further tailor Gems to meet your specific needs. 


These file types include: 
  • Document and text files, such as TXT, DOC, DOCX, PDF, RTF, DOT, DOTX, HWP, HWPX and Google Docs 
  • Data Files, such as XLS, XLSX, CSV, TSV and Google Sheets


Who’s impacted

Admins and end users


Why you’d use it

Gems help you leverage the power of Gemini more efficiently and in a way that’s customized to your needs  by minimizing repetitive prompting. You can create Gems to complete specific goals, tasks, and workflows. This update further enhances this functionality by allowing you to create Gems that anchor to specific files, referencing these files as a source for more relevant responses. For example, you can upload:
  • Corporate style guides that instantly creates or refines brand-aligned copy for websites, email subject lines, or social media posts.
  • Project files to create a project specific assistant 
  • Internal documents which can be referenced at any time, such as how to best provide customer service based on a handbook or other best practices
  • HR documents to easily reference and find answers to common questions
  • Product specs and instruction manuals for quick troubleshooting
  • Customer feedback to easily identify trends and areas of improvement
  • Education standards to plan lessons and create content that align with your school’s needs
  • Grant research and notes to help accelerate the research process and drive innovation 

Additional details

  • Uploading files when creating a Gem will be available starting in English.
  • Gems will automatically recognize the latest updates to any Google Docs or Sheets added to the instructions. Other uploaded file content will need to be re-uploaded if any changes are made to the underlying file. 

Getting started

Expanding access to the Gemini app for teen students in education

What’s changing

Google Workspace for Education admins can now turn on the Gemini app with added data protection as an additional service for their teen users (ages 13+ or the applicable age in your country) in the following languages and countries. With added data protection, chats are not reviewed by human reviewers or otherwise used to improve AI models. The Gemini app will be a core service in the coming weeks for Education Standard and Plus users, including teens, 


We created this Gemini experience for teens after consulting with child development and learning science experts - including conducting learning focused red teaming, an industry standard testing process to find vulnerabilities in generative AI - and testing with youth advisory panels to ensure we’re bringing this technology to students responsibly.

Gemini can help teen students learn responsibly and confidently in an AI-first future.



Who’s impacted

Admins and end users


Why it’s important

Google is committed to making AI helpful for everyone, in the classroom and beyond. We want to both prepare teens with the skills and tools they need to thrive in the future where GenAI exists and help them learn how this technology can be used to unlock creativity and facilitate learning. Expanding access to the Gemini app to younger students provides guided support to help them learn more confidently with in-the-moment assistance, practice materials, and real-time feedback. 


Additional details

Upon accessing the Gemini app for the first time, teens will be guided through an AI Literacy video, endorsed by ConnectSafely and Family Online Safety Institute (FOSI)--with tips on how to use generative AI responsibly. And according to Common Sense Media, the Gemini teen experience is considered “low risk” and is designated as “designed for kids”.

“Google's Gemini, which is currently the only major platform to create a specific experience for teens, includes additional CSAM (child sexual abuse material) safeguards, content policies, and in-product experiences that help teens understand the limitations of generative AI." - Common Sense Media


Getting started


Rollout pace

  • Available now.

Availability

  • Available for Google Workspace Education Fundamentals, Standard, Plus and the Teaching & Learning Upgrade

Resources


Google Vids is now available for Google Workspace for Education, providing easy video creation for teaching and learning

What’s changing

Earlier this year, we announced Google Vids would soon empower educators and students to easily create and collaborate with video. Today, we’re excited to announce the general availability of Google Vids for Education Plus and Gemini for Workspace customers.

Vids can help deepen learning opportunities, up-level instruction, and inspire the next generation of storytellers in a variety of scenarios, for example: 
  • Educators can support diverse learning styles and help students retain information with instructional videos and deep dives on complex concepts. 
  • Students can work on video projects and review recorded lessons. 
  • Education leaders can easily incorporate video in their school newsletters and create internal training videos for staff. 
Similar to other productivity tools like Docs, Sheets, and Slides, Vids includes a simple, easy-to-use interface and the ability to collaborate in real-time and share projects securely from your browser. You can easily get started with a variety of templates organized by use cases to create a new video. From there, you can add motion with animations, transitions, and photo effects, and customize your video using the Vids royalty free education-friendly stock content library or your own media directly from Google Drive or Google Photos. Vids is also integrated with Workspace tools like Google Classroom, empowering educators to easily assign video projects or post video explainers right to their students. As detailed here, Google Vids is a core Workspace Service covered under the Google Workspace for Education terms of service. 

Users 18 years and older with a Gemini Education or Education Premium add-on will also have access to Gemini in Vids, giving them the ability to produce polished videos even faster with the help of AI. ‘Help me create’ can generate a fully editable initial storyboard using a prompt and a doc from your Google Drive. After choosing a style, Gemini pieces together your video draft with suggested scenes, including recommended stock media, text, scripts per scene, and even a background music track. 


Getting started 

  • Admins: 
  • End users: 
    • To start, open Google Vids and choose an option to create a new video. You can create a new video or make a copy of an existing video for editing. 
    • You can use Vids on your computer with the two most recent versions of these browsers: 
    • Dive into Google Vids with our free, self-paced online training course, designed specifically for educators and available now in the Google for Education Learning Center. You can also leverage this one pager to get a quick summary of what you can do with Vids. 
    • Currently, Vids is available in the user’s local language, but the AI features (e.g. Help me create a video, read-along teleprompter) and templates are only available in English at this time. 
    • Visit the Help Center to learn more about getting started with Google Vids. 
    • Note: Create and copy access is available for Education Plus and Gemini for Google Workspace customers, while view and collaborate access is available to anyone with a Google Workspace for Education license. 

Rollout pace 

Availability 

Available for Google Workspace: 
  • Education Plus 
  • Customers with a Gemini Education or Education Premium add-on 

Resources 

Announcing general availability of Google Vids: Our new AI-powered video creation app for work to help tell stories across your organization

What’s changing 

Earlier this year, we announced Google Vids, the newest productivity app in our suite of Google Workspace products. Vids is an AI-powered video creation app for work designed to help teams in customer service, learning and development, project ops and marketing tell more engaging stories at work through video. 

Today, we’re excited to announce the general availability of Google Vids for select Workspace editions. Here’s how you can use it:
  • Customer service teams are using Vids to provide better customer support, supplementing help center articles with videos. 
  • Leaders are creating vids to share company-wide updates and announcements. 
  • Learning and development teams are using Vids to deliver employee training and education at scale. 
  • Marketers are using vids inside their organizations to share recaps on their campaigns and events. 
  • Project management teams are building vids to share meeting recaps, business updates and report share outs. 
With Vids, you can easily build videos by tapping into the power of Gemini to help you get to a first draft faster using a set of generative AI capabilities. ‘Help me create’ can generate a fully editable initial storyboard using a prompt and a doc from your Google Drive. After choosing a style, Gemini pieces together your video draft with suggested scenes, including recommended stock media, text, scripts per scene, and even a background music track. 
                   





You can also start with a variety of use-cased based templates to create a new video. From there, you can add motion with animations, transitions, and photo effects, and customize your video using the Vids royalty free stock content library or your own media directly from Google Drive or Google Photos.

With ‘Help me create’, Gemini suggests scripts to get you started or you can write your own. You can easily add a voiceover by either choosing from one of Gemini’s preset AI voices or by recording your own voice using a teleprompter with Gemini read-along and rolling features that help you record your message with confidence. You can also use the Vids’ in-product recording studio to: 
  • Add a video recording of yourself. 
  • Add a screen recording with narration. 
  • Add a screen recording with video recording. 
  • Add audio-only recording.




Similar to other productivity tools like Docs, Sheets, and Slides, Vids includes a simple, easy-to-use interface and the ability to collaborate in real-time and share projects securely from your browser. 

In summary, Vids enables you to produce engaging videos that help scale your message, build connections and break through the noise across your organization.


Who’s impacted 

End users 


Why you’d use it 

We know that creating and editing video has felt challenging because in the past it has required special skills or licensing, but with Google Vids the power of storytelling is in your hands. Now teams can build and tell compelling stories at work, scaling and landing their message to engage coworkers in an effective way. 


Additional details 

While previously only available to Gemini for Google Workspace customers, with this launch Google Vids is expanding to select Workspace subscription plans. 

Users will have full access to AI-powered features* in Google Vids like "Help me create" and AI voiceovers until at least December 31, 2025. Usage limits for these AI-powered features may apply beginning in 2026. We'll notify you before any change goes into effect. 


To learn more about Vids features and capabilities for Google Workspace for Education customers, refer to this Workspace Updates Blog. 


Getting started

  • Admins: Vids will be ON by default and can be disabled at the organizational level. Visit the Help Center to learn more about turning Vids on or off for users and new core services coming to Google Workspace. 
  • End users: 
    • To start, open Google Vids and choose an option to create a new video. You can create a new video or make a copy of an existing video for editing. Visit the Help Center to learn more about getting started with Google Vids.
    • You can use Vids on your computer with the two most recent versions of these browsers: 
    • You can access our always-on resources in the Google Workspace Learning Center to learn what you can do with Google Vids. 
    • Currently, Vids is available in the user’s local language, but the AI features (e.g. Help me create, read-along teleprompter) are only available in English at this time. 
    • Note: View and collaborate access is available to anyone with a Google Workspace account, while create and copy access is available for select editions of Google Workspace and Gemini for Google Workspace customers. 

Rollout pace 


Availability 

Available for Google Workspace: 
  • Business Standard and Plus 
  • Enterprise Standard and Plus 
  • Essentials, Enterprise Essentials and Enterprise Essentials Plus 
  • Education Plus* 
  • Customers with a Gemini Business, Enterprise, Education* or Education Premium* add-on 
*Please refer to the following Workspace Updates Blog to learn more about Vids availability for Google Workspace for Education customers. 

Resources 

Now generally available: use Gemini in the side panel of Workspace apps in seven additional languages

 What’s changing

Beginning today, all users* can use Gemini in the side panel of Google Docs, Google Sheets, Google Drive, and Gmail,  in seven additional languages:

  • French
  • German
  • Italian
  • Japanese
  • Korean
  • Portuguese
  • Spanish

Gemini in the side panel of Gmail in Spanish



With Gemini in the side panel of your Workspace apps, you can get help summarizing, analyzing, and generating content by utilizing insights gathered from your emails, documents, and more—all without switching applications or tabs. Check out our original announcements for Gemini in the side panel of Docs, Sheets, and Drive, and Gmail for even more information.

*Available for Workspace customers with a Gemini Business, Gemini Enterprise, Gemini Education, and Gemini Education Premium add-on.

Additional details

  • While Gemini in the side panel of Docs, Sheets, Drive, and Gmail, is generally available in these additional languages - as we keep rolling out more Gemini alpha features in the side panel, end users with Gemini alpha features turned on may continue to see the “Alpha” badge. 
  • Image generation is not supported yet in these additional languages.
  • Gemini in the side panel of Slides is only available in English at this time.

Getting started

  • Admins: There is no admin control for this feature.
  • End users: You can access the side panel by clicking on “Ask Gemini” (spark button) in the top right corner of Docs, Sheets, Drive, and Gmail on the web. Visit the Help Center to learn more about collaborating with Gemini in Docs, Sheets, Drive, and Gmail. Note that for accessing other Gemini for Google Workspace features that are supported in English only, you will need to set your Google Account language to English.

Rollout pace



Availability

Available for Google Workspace customers with a:
  • Gemini Business or Enterprise add-on
  • Gemini Education or Education Premium add-on


Resources



Google Workspace Updates Weekly Recap – November 1, 2024

4 New updates

Unless otherwise indicated, the features below are available to all Google Workspace customers, and are fully launched or in the process of rolling out. Rollouts should take no more than 15 business days to complete if launching to both Rapid and Scheduled Release at the same time. If not, each stage of rollout should take no more than 15 business days to complete.

Apply black & white filter to Google Drive scans on Android devices 
In August, we announced that you can now save files scanned in the Google Drive Android app as a .JPEG. This week, we’re excited to introduce an additional scanning option that gives you the ability to apply a black & white filter on your document scans. This new filter helps enhance texts and other important elements, ensuring they are sharply defined when compared to the background region. | Rolling out now to Rapid Release and Scheduled Release domains. | Available to all Google Workspace customers, Workspace Individual Subscribers, and users with personal Google accounts. | Visit the Help Center to learn more about scanning files with your mobile device.
Apply black & white filter to Google Drive scans on Android devices

AI Classification now supports Field Selection for Model Training 
When AI Classification first launched, Labels eligible for model training needed to have a single field of either a badge or option-list field type, and Labels with multiple fields were ineligible. Now, customers that use AI Classification will be able to select which badge or option list field they would like to train a model for after identifying the target label. Once trained and enabled, the AI model will automatically apply the label and will only populate the selected field. | Roll out to Rapid Release and Scheduled Release domains is complete. | Available to customers with the AI Security add-on, Gemini Enterprise add-on, and Gemini Education Premium. | Visit the Help Center to learn more about Label Google Drive files automatically using AI classification. 


Reducing noise from unfollowed threads in Google Chat
In order to make it easier to identify which unread threads are most relevant to you, we’re reducing the noise by removing visual cues from threads that you do not follow in Google Chat. Starting this week, new activity, such as unread messages from threads you do not follow, will no longer bold and appear at the top of your conversation lists. | Rolling out now to Rapid Release and Scheduled Release domains on web and mobile at an extended rollout pace (potentially longer than 15 days for feature visibility). | Available to all Google Workspace customers. 


Introducing a better filter by condition experience for tables in Google Sheets 
Tables in Google Sheets will now provide users with a smarter filter by condition experience. Sheets offers 21 options for users to filter by condition, such as “Date is” or “Text ends with”. However, we know there are scenarios in which certain filters might not be applicable based on the type of data in a spreadsheet. Based on the set column type, users will now only see relevant filter by condition options. For example, if your column type is set to number, the filter options will be number-based only. | Roll out to Rapid Release and Scheduled Release domains is complete. | Available to all Google Workspace customers, Workspace Individual Subscribers, and users with personal Google accounts. | Visit the Help Center to learn more about sorting & filtering your data.
Introducing a better filter by condition experience for tables in Google Sheets



Previous announcements

The announcements below were published on the Workspace Updates blog earlier this week. Please refer to the original blog posts for complete details.


Refine emails faster with updates to the “Polish” shortcut in Gmail 
We’re expanding the Help me write shortcut to web and introducing a Polish shortcut on web and mobile that helps you refine emails even faster. | Learn more about email shortcuts in Gmail. 

Google Classroom now supports exporting missing and excused grades to select Student Information Systems (SIS)
Teachers can now include missing and excused grades when exporting grades to their Student Information Systems (SIS). | Learn more about exporting missing and excused grades to select Student Information Systems (SIS). 

New density setting in Google Chat 
To give users more control over how they see information in Google Chat, we’re introducing a new setting that allows you to control the visual density of screen elements. Choose between “Comfortable” or “Compact” on chat.google.com. | Learn more about density settings in Chat.

Context Aware Access insights and recommendations are now generally available
We’re making it easier to apply context-aware access (CAA) policies with new insights and recommendations. We’ll proactively surface potential security gaps and suggest pre-built CAA levels which admins can deploy to remediate the security gaps. | Learn more Context Aware Access insights.

FedRAMP High authorization for Gemini for Workspace 
As recently announced, we submitted our package to obtain FedRAMP High authorization for Gemini for Workspace, including the Gemini app. A FedRAMP High certification assures federal agencies in the United States that a cloud service provides the highest level of protection for their most sensitive data, enabling them to confidently leverage cloud technologies for critical operations. | Learn more about FedRAMP High authorization for Gemini.

Gemini in the side panel of Google Chat is now available
We’re expanding Gemini in Chat to help users collaborate more effectively in their spaces, group messages and direct messages. | Learn more about Gemini in the side panel of Chat. 

Data classifications labels for Gmail are now available in open beta
In addition to Google Drive, we’re expanding data classification labels to now include Gmail. Classification labels are used to classify and audit content according to organizational guidelines (“Sensitive”, “Confidential”, etc.) and apply policies, such as data loss prevention (DLP) rules, to protect sensitive information in email messages. Classification labels will be available when using Gmail on the web – support for Gmail on mobile devices will be introduced in the coming months. | Learn more about the beta for data classifications labels for Gmail.


Completed rollouts

The features below completed their rollouts to Rapid Release domains, Scheduled Release domains, or both. Please refer to the original blog posts for additional details.


Rapid Release Domains: 
Scheduled Release Domains: 
Rapid and Scheduled Release Domains: 
    For a recap of announcements in the past six months, check out What’s new in Google Workspace (recent releases).