Google is set to launch a groundbreaking update to its Gemini AI assistant, enabling it to process live video feeds and screen sharing capabilities. The feature, previously showcased as Project Astra, will debut on Android devices later this March.
The enhanced Gemini Live will allow users to simply point their camera at objects or share their screen, creating a more intuitive way to interact with the AI assistant. Rather than describing situations through text, users can show Gemini what they're looking at and ask questions about what it sees in real-time.
Google demonstrated this technology's potential at their I/O event, where Gemini Live analyzed various objects, including computer code, speaker systems, and whiteboard diagrams. The AI even displayed contextual memory by recalling the location of items from earlier in the demonstration.
The upcoming feature will be exclusively available to Gemini Advanced subscribers, who pay $20 monthly for access to Google's most sophisticated AI models. This subscription requirement reflects the substantial processing power needed to handle continuous video streams compared to text-based interactions.
While Google anticipates this feature will increase operating costs and deepen their losses in the AI sector, the company appears focused on gaining market share against competitors like OpenAI. Currently, Gemini's monthly active users remain in the tens of millions, trailing behind OpenAI's user base.
The integration of video capabilities marks Google's push to transform Gemini into what they call a "true assistant." Users will be able to leverage the technology for various practical applications, from exploring new locations to receiving fashion advice while online shopping.
This advancement represents a notable expansion of Gemini's existing capabilities, which currently include processing text, images, and documents, though with inconsistent video handling abilities.