A research team led by Google has introduced StreamVibe, a new video agent designed to decide for itself when to speak up during complex tasks. Researchers from Google, Stanford, Oxford, and several U.S. universities built the assistant to watch users cook or assemble furniture and offer help only when necessary. With StreamVibe, the team developed a video agent capable of understanding vague requests such as assisting users if they run into difficulties, moving beyond simple commands like notifying someone when water boils.
Existing video assistants often struggle because they are built for clear-cut commands, making it difficult for them to pick up on unspoken intentions, track completed steps, or adapt when users change instructions midway through. To measure and address this problem, the team compiled a benchmark of about 4,100 examples from over 183 hours of everyday life and software recordings using Gemini 2.5 Flash and human evaluators. From there, they trained StreamVibe-VL, a model based on Alibaba's Qwen3.5-9B that decides every second whether to stay quiet or speak, supported by reinforcement learning and a separate memory system to track pending requests.
According to the researchers, StreamVibe-VL improves response timing and quality by 10.7 percent over other models, while its memory system boosts proactive interaction by another 12.3 percent and reduces unnecessary interruptions. Although specialized training helps StreamVibe-VL beat many open-source models, the team found that performance across all tested systems still plummets in continuous operation over longer videos as errors compound. Why it matters: This research highlights the current limitations of video AI in maintaining long-term context and timing, marking a step toward more attentive and