Introducing Genie 4
Genie 4 is a major world model leap, introducing photorealism, audio modality, and Gemini integration. It enables real-time, interactive simulations for robotics, film pre-vis, and education.
Read articleTopic
Multimodal AI
All digests tagged Multimodal AI
Suggested route
Topic journey
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.
Start here
Building an AI-Powered Personal Companion Mobile App - Gerald Versluis - NDC Copenhagen 2026 Learn how to build an AI-powered companion app using .NET MAUI. This session covers integrating LLMs, handling multimodal input (voice/image), and architecting for model agnosticism across platforms.Build context
Access all Gemini models with the Interactions API The Gemini Interactions API is now generally available! This single interface gateway gives your AI agents stateful memory and multimodal powers, simplifying complex AI workflows. Video Has No Memory. Here's How We Built One. — James Le, TwelveLabs Video AI systems lack memory; this talk details building a durable 'memory layer' by treating video as a spatial temporal volume. The solution uses a Context Graph to enable deep reasoning over vast corpora. Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber Learn how Uber built closed-loop evaluation systems for multimodal agents. Discover strategies for countering reward hacking and scaling AI enhancements while preserving authenticity at 90 billion run rate.Recent signals
Introducing Genie 4 Genie 4 is a major world model leap, introducing photorealism, audio modality, and Gemini integration. It enables real-time, interactive simulations for robotics, film pre-vis, and education. The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft This workshop compares GPT-5.5, Kimi, DeepSeek, and Mistral across complex scenarios like RAG, tool calling, and vision. Learn how to use ASSERT and DSPy to automate model evaluation and optimize prompts for portability. Ask the Experts: Nemotron 3.5 Super VL | Nemotron Labs Nemotron 3.5 Super VL is a multimodal open model built for autonomous agents. Learn how its 120B parameter architecture, optimized with NVFP4 and trained via distillation and mid-training, enables superior performance in agentic and vision-language tasks.
The Model Swap Workshop: GPT-5.5 vs Kimi vs DeepSeek vs Mistral — Pamela Fox, Microsoft
This workshop compares GPT-5.5, Kimi, DeepSeek, and Mistral across complex scenarios like RAG, tool calling, and vision. Learn how to use ASSERT and DSPy to automate model evaluation and optimize prompts for portability.
Read article
Ask the Experts: Nemotron 3.5 Super VL | Nemotron Labs
Nemotron 3.5 Super VL is a multimodal open model built for autonomous agents. Learn how its 120B parameter architecture, optimized with NVFP4 and trained via distillation and mid-training, enables superior performance in agentic and vision-language tasks.
Read article
Build Visual AI Agents With NVIDIA Nemotron 3.5 Super VL
Nemotron 3.5 Super VL is an open-source, multimodal LLM (120B total, 12B active parameters) that powers specialized agents. It can analyze dashboards and scanned documents to propose code-based cost optimizations.
Read article
📷 Searching for images from text with EmbeddingGemma 2
Discover how EmbeddingGemma 2 enables advanced, offline multimodal search on mobile devices. Use natural language to find photos and specific moments in videos using semantic similarity.
Read article
From World Models to Working Machines: Physical AI in the Real World
Physical AI is moving from the lab to industrial sites. Success requires a closed-loop data flywheel, combining multimodal data, advanced simulation, and edge computing to solve the sim-to-real gap.
Read article
Top 3 new model launches at Gemini Audio at Night
Discover the top 3 audio model launches from Gemini Audio at Night, including personalized TTS, 100+ language live translation, and multimodal function calls for developers.
Read article
Introducing EmbeddingGemma 2: An open model for natively multimodal embeddings
EmbeddingGemma 2 is a lightweight, open model for natively multimodal embeddings. It maps text, images, video, and audio into a single unified embedding space, enabling powerful, fully offline search and RAG pipelines on edge devices.
Read article
New frontier AI models, TypeSafe’s Jev AI, & NASA’s IBM collab
Explore the frontier of AI efficiency: From Claude Opus 5.5 and GPT-6 Sol to TypeSafe's Jev AI, learn how structured, calibrated models are revolutionizing automation and specialized scientific computing.
Read article
From VLM/VLA's to Embodied Agents — Armen Aghajanyan, Perceptron AI
Perceptron introduces Embodied Foundation Models, unifying perception, reasoning, and control. By using data sparse Mixture of Experts, they solve context bloat and achieve a new scaling law: 10x video pretraining substitutes for 10x less teleop data.
Read article
Gemma 4 12B: The Encoder-Free Model Explained
Gemma 4 12B is an encoder-free model that processes audio and image inputs directly, bypassing the need for large, dedicated encoders like the Conformer or vision Transformers. This significantly improves efficiency and reduces time-to-first-token.
Read article
Understand the Gemma 4 model family
The Gemma 4 family offers five multimodal models, including E2B/E4B for on-device use, the encoder-free 12B, the MoE 26B, and the 31B dense model.
Read article
Multimodal Collaborative Agents for Next-Gen Commerce — Nidhi Kaushik Vyas, Google DeepMind
Multimodal collaborative agents must move beyond search bar wrappers to handle 'fuzzy intent.' The system uses a Discovery $\rightarrow$ Research $\rightarrow$ Response loop, prioritizing information gain by analyzing context and adapting output formats.
Read article
SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind
SOTA generative media models are launching new APIs like NanoBanana 2 Lite and Gemini Omni Flash. The discussion highlighted that language is an insufficient intermediate representation for complex sensory data, pointing toward the necessity of unified 'World Models' and robust field testing.
Read article
Stanford CS547 HCI Seminar | Spring 2026 | Show It or Tell It? Text, Visualization, and Combination
This seminar explores the interplay between text and data visualization, detailing how Multimodal LLMs process combined inputs using cross-attention mechanisms.
Read article
Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber
Learn how Uber built closed-loop evaluation systems for multimodal agents. Discover strategies for countering reward hacking and scaling AI enhancements while preserving authenticity at 90 billion run rate.
Read article
Video Has No Memory. Here's How We Built One. — James Le, TwelveLabs
Video AI systems lack memory; this talk details building a durable 'memory layer' by treating video as a spatial temporal volume. The solution uses a Context Graph to enable deep reasoning over vast corpora.
Read article
Access all Gemini models with the Interactions API
The Gemini Interactions API is now generally available! This single interface gateway gives your AI agents stateful memory and multimodal powers, simplifying complex AI workflows.
Read article
Building an AI-Powered Personal Companion Mobile App - Gerald Versluis - NDC Copenhagen 2026
Learn how to build an AI-powered companion app using .NET MAUI. This session covers integrating LLMs, handling multimodal input (voice/image), and architecting for model agnosticism across platforms.
Read article