Google Developers

EmbeddingGemma 2: How natively multimodal embeddings work

Published 2026-11-17 · Duration 5:01

Summary

EmbeddingGemma 2 is a lightweight, natively multimodal embedding model designed to map diverse data types—including text, code, images, video, and audio—into a single, unified embedding space. The model achieves this through modality-specific encoders feeding into a bidirectional transformer, which is then refined using a two-stage training process involving distillation from Gemini Embedding 2 and finetuning with contrastive and spread-out loss objectives. Key efficiency features include Matryoshka Representation Learning and quantization down to INT4, enabling fast deployment on edge devices.

Download summary

Key takeaways

  1. Unified Multimodal Representation

    EmbeddingGemma 2 natively understands interleaved sequences of different modalities (text, code, images, video, audio) to create a single unified representation, allowing, for example, a video to be searched using its audio clip.

  2. Two-Stage Training Regime 4:34

    The model is initialized from a pre-trained Gemma 4 decoder model, then learns from the larger teacher, Gemini Embedding 2, via distillation, followed by finetuning using contrastive and spread-out loss objectives.

  3. Efficiency and Deployment 3:37

    The model supports Matryoshka Representation Learning to nest embeddings of decreasing size, and utilizes quantization aware training to compress the footprint down to INT4, making it suitable for resource-constrained environments.

More digests on Multimodal AI

Technical details

  • Input Processing Pipeline 103s

    Raw inputs pass through modality-specific encoders. The vision encoder processes images/video frames into patches, and the audio encoder converts sound waveforms into spectrograms, which are then sliced into 40 millisecond segments before being projected into a shared space.

  • Core Architecture 141s

    A bidirectional transformer contextualizes and blends information across modalities. The intermediate output is collapsed into a single embedding via three steps: Mean pooling, a dense layer projection, and normalization onto the unit sphere.

  • Loss Functions 224s

    Fineluning employs contrastive loss (where positive pairs are pulled closer and negative pairs are pushed apart) and spread out loss (to ensure embeddings are dispersed and robust to quantization). Same tower negatives are also used to prevent modality bias.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.