AI developers have been trying to crack the mystery of digital personal assistants for a while now, offering a service that’s smart, easy to use, and always available. Gemini Live, announced at Made by Google earlier this week, is Google’s latest attempt at doing just that. So I gave the AI a 24-hour test drive to see how close it comes to being truly useful.
While I don’t usually chat directly with AI assistants, I simply ask them to set timers while I cook, I wanted to see what benefit an open conversation with an assistant like Gemini could bring. And after this day of testing, I’m at least confident in the value of talking to an AI this way, even if I’m less confident in some of the answers it gives at the moment.
While my experiences with Gemini Live are far from a formal test of its capabilities, the breadth of questions I was asked give us a good impression of what it does well and what it doesn't. So I'm confident that Gemini Live will be a good addition to the Gemini package, and perhaps reason enough for some free users to upgrade to Gemini Advanced for $20 per month. Even if it doesn't quite meet all of its goals.
Thursday afternoon — The setup
Gemini Live is part of the Gemini Advanced subscription, but while it's rolling out as I write this, it's not available to all users yet. Luckily, I got a Google Pixel 9 Pro XL to try it out. If you're interested in learning more about the phone, you can check out our hands-on review of the Google Pixel 9 Pro and Pro XL , as we'll be focusing exclusively on Gemini Live here.
Another issue is that you currently have to set your Gemini language to American English to use it. Fortunately, even after doing this, I was still able to choose a British voice for Gemini Chat, named “Capella,” from the ten available. All of them sound fairly natural, just varying levels of enthusiasm and pitch. Even when you start asking questions, it’s rare to get a particularly glaring mispronunciation or oddly worded sentence.
Thursday evening – Back home
Once everything was set up, my first major interaction with Gemini Chat was asking for directions to get home. Gemini Live didn't initially tell me what it had found once I told it my preferred mode of transportation and confirmed the stations I wanted to travel between. After a long wait, I then prompted it to tell me what it had found and it described the route to me.
I probably would have gone home with the itinerary. However, the journey would not have been the most pleasant. Gemini misidentified one of the train lines and one of the stations, forgot to note that one of my changes would technically require walking between two stations, and then appeared to make up a train out of thin air. Which is quite strange because Gemini claimed to have checked the Transport for London website for its information.
This is more of an issue with the underlying AI model than Gemini Live, but the fact that it has an authoritative voice (with a British accent to boot) suggests that a route could have led someone less familiar with London's public transport system to get lost. It seems like you'd be better off sticking to Google Maps for this sort of thing.
Friday morning – Press briefing
The next day, I asked Gemini to fill me in on the latest news of the day while I got ready for work. With just one question, he was able to tell me a lot about the changing presenters of Good Morning Britain and This Morning, as well as a brief reference to the recent knife attack in Leicester Square. But when I asked about tech news, things got weirder.
Google Gemini first told me that Microsoft had announced a Surface Duo 3 — a device that has not been confirmed and has in fact been rumored to be canceled for months. The PS5 Slim is real, but it came out last fall, and we can assume she's referencing last month's Crowdstrike outage with her latest comment.
I then asked Gemini Live to delve into the iPhone rumors, but initially his answers all concerned the currently available iPhone 15 lineup. Upon further prodding, he outlined some rumors about the iPhone 16 camera, but not in great detail.
Friday Mid-Morning — Brewing Guide
After a few hours of work, it was time for a coffee break, so I tried to ask Gemini Live to guide me through preparing a V60 pourover.
I was hoping to get step-by-step instructions from the AI, but the problem here is that you have to continually ask or interrupt Gemini Live to force it to give its answers in step form. However, it managed to keep the conversation going, offering convincing answers despite the transcript showing that it initially misunderstood my prompts.
In terms of knowledge, Gemini was a mixed bag. It offered enthusiast-level advice, like filtering my water before boiling it. The overall recipe, while simple, resulted in a drinkable cup. But Gemini Live also gave me a suggested coffee weight in tablespoons of beans rather than grams or ounces, which isn’t a typical measurement when you’re brewing coffee. But with an additional prompt, I was able to get an amount in grams.
Friday noon — discussion of the fights
Having some free time for lunch, I had a quick chat with Gemini Live about Street Fighter 6, the game I'm playing the most right now. He correctly named this year's Evo 2024 champion for SF6, as well as his opponent, but again, he didn't give many initial details.
I then got to the topic of training tips (I tend to rely too much on certain moves), where I got some suggestions on how to rethink my approach in matches. This is easier said than done when your opponent is throwing fireballs at you, but it was still a valid tip.
I also tried to get some advice on where to find in-person meetups, but that didn't work as well. I tried checking the official website for details, but found that there was nothing outside of Capcom's official tournaments. I then found a Facebook group nearby, but couldn't give me a link to access it later in the transcript.
Friday Afternoon – Writing Tips
“Imagine having a real assistant available, ready to help you at any time, without having to type or send a photo. That’s the idea behind Gemini Life. With Gemini Life, you can talk to your AI assistant like you would with a friend. I’m going to spend 24 hours testing it and see if it lives up to the dream of a personal assistant.”
Gemini Live Final Intro Suggestion
As a final task for Gemini, I decided to do some meta, and no, we're not talking about Llama 3. I asked him to help me write the introduction to this article.
After noticing that Gemini didn’t give me much detail in my previous responses, I was surprised by how much more willing Gemini was to suggest specific wording. When I asked it to include more information or change its angle, it responded logically. And as Google proudly pointed out during its Made by Google demo, Gemini Live is able to handle interruptions and adjust its responses on the fly.
This is the best part of Gemini Live, because repeating an idea out loud feels perfectly natural, even when you’re speaking in a light waveform on your phone. Ultimately, I wrote the introduction to this article from scratch. But you can probably see echoes of her final suggestion if you scroll up to compare it to what she gave me.
Google Gemini Live: Final Thoughts
You might assume from reading this article that I don’t think highly of Gemini Live, but that’s not entirely true. The worst of my criticisms are directed at the Gemini Advanced model that runs it, as it seems to misunderstand what it’s looking for in several test scenarios. Oddly enough, a recent Gemini vs. Gemini Advanced showdown we ran shows that I might have been better off sticking with the base Gemini.
On the other hand, Gemini Live itself has been very impressive. Being able to have an ongoing conversation with a chatbot, provided you’re prepared to be specific and interrupt if the conversation gets off track, seems like a much better way to interact than through text or images. You can ask follow-up questions to regular digital assistants, but it’s still not as seamless as Gemini Live has proven. And it’s that seamlessness that makes it practical, helping to answer questions and provide advice not only hands-free, but also eyes-free, allowing you to focus on something else while you and the chatbot talk.
The big question of how this compares to the upcoming ChatGPT Voice remains, especially since Gemini Live relies on interpreting speech as text before providing its response, while ChatGPT Voice can process speech directly. But even with the usual AI caveats, it seems like Google is on the right track in pursuing the dream of a digital personal assistant.