There's a reason Google moved up the date of its Made by Google event to August, though the company has never provided an official explanation. It's clear that Google wants the Pixel 9's early launch to steal the spotlight from the upcoming iPhone 16.
To do this, Google has shown off a wide range of AI features in the Pixel 9, including Add Me to insert someone into a photo after the fact, and Call Notes to record and summarize phone calls. And the Google Pixel Screenshots feature can help you extract details using natural language queries.
All of this is powered by the new Tensor G4 chip, which was explicitly designed to run Google’s most advanced AI models. In fact, it’s the first processor capable of running Gemini Nano with multimodality, meaning the Pixel 9, Pixel 9 Pro, Pixel 9 Pro XL, and Pixel 9 Pro Fold can all understand text, images, and audio.
To get a deeper look at Google's secret AI weapon, I spoke with Jesse Seed, group product manager for Google Silicon, and Zach Gleicher, product manager for Google DeepMind, about what the Tensor G4 chip can do and what sets it apart.
Watch on
What makes the Tensor G4 chip stand out in a sea of smartphones?
Seed of Jesse: I think the biggest innovation we did this year was being the first silicon and the first phone to run Gemini Nano with multimodality. And that opens up some really interesting use cases, one of which is Pixel Screenshots. It's really handy if you're trying to remember things.
Another feature that’s not related to the Gemini Nano model but that I really like is the Add Me feature. Those of us who are the photographers in our family or on our team will definitely appreciate being able to go back and dynamically add the photographer. And this is something that we’ve been working on a lot, refining over 15 different machine learning models, and also using Google’s Augmented Reality SDK.
How did you manage to integrate something as advanced as Gemini Nano onto a phone?
Zach Gleicher: At DeepMind, we collaborate with a lot of teams across Google, and we want to make sure that we're building Gemini models that meet the needs of all of Google's products. As we were building Gemini in collaboration with Android and Pixel, we realized that there was this need for on-device models. We saw that as a challenge because on the server side, everyone wanted more performant and potentially larger models. And we, on the other side, had all these interesting constraints that didn't exist before in terms of memory constraints, power consumption, etc.
So, in partnership with the Tensor and Pixel team, we were able to come together and understand what the key use cases were for these on-device models, what the constraints were for these on-device models, and we actually co-developed a model together. It was a really exciting experience and created something that was so capable of addressing those use cases.
For someone who hasn't upgraded their phone in 3-4 years, what's going to make it stand out with the G4 chip?
Seed: So it's really important for us to improve what we call the fundamentals like power and performance. The Tensor G4, which is our fourth-generation chip, is our most efficient and highest-performing chip. So we think users will see that in their day-to-day experiences like web performance or web browsing, and also in app launches and overall speed of the UI. I think it's a really smooth experience. You'll see that in web performance that's 20% faster on average and app launches that's 17% faster.
And what about gaming performance, because that's really important these days for people buying a new phone?
Seed: So, in our testing, we saw improved peak performance and sustained performance in both games and mainstream games running on the platform.
How does the Tensor G4 contribute to battery life?
Seed: We've improved power efficiency across many everyday use cases, so tasks like capturing video, taking photos, scrolling through social media—all use less power than the previous generation.
All of this contributes to that 20% extra battery life that you saw mentioned in the keynote. So the Tensor G4 contributes to almost 20% more battery life and achieves that.
What AI features Gemini enables on the Pixel 9 phones are you most excited about?
Identical: One of the main reasons the Tensor and Pixel teams come to us for on-device use cases is better reliability. So, because you don’t have to rely on an internet connection, the experience can be reliable and work wherever you are.
Another thing we think about is privacy. If developers don't want data to actually leave the device and be processed entirely on the device, this is possible with an on-device LLM.
In terms of the AI features that I'm excited about, the Pixel screenshots are really cool. I think it really shows how we're able to get these multimodal features that work on devices that can work like you can see in the demos. It was really fast, low latency, but it's also a very capable model. And all of this information and data is stored locally on your device and can be processed locally. So we're really excited that Gemini nano can enable experiences like this.
I think we're seeing increasing interest in summary and smart reply use cases.
How is Pixel Screenshots different from Windows Recall, which has attracted some attention due to privacy concerns?
Seed: One way to protect user privacy is to have an in-device model. So the analysis that's done on that screenshot doesn't leave the device. That's one way we can address that privacy issue.
I think the other thing is empowering users to decide what they want to do, like how they want to use something like Gemini. And what use cases they feel comfortable interacting with and what use cases they don't feel comfortable interacting with. So I think it really comes down to user choice. But in the case [of] Pixel screenshots in particular, which are an entirely on-device use case.
We're going to do all the usual tests with the Tensor G4, but the AI era is also changing things. How do you envision performance with this chip?
Seed: I think it really comes down to the actual use cases. How does this device actually perform in the hand? So I think things like how fast web browsing is, how fast apps launch, how fast and responsive the UI is, these are all everyday use cases. Those are good standard things to consider.
And what about from an AI perspective? When does a Pixel phone pass your test in terms of performance?
Identical: When we thought about benchmarks for LLM and Gemini, and Gemini Nano in particular, we saw that the industry put a lot of emphasis on academic benchmarks. And academic benchmarks like MMLU are great because they provide a common metric. But they could be gamified and people could optimize for them. And they might not reflect what you’re really interested in.
For an on-device model, we don't really care about it knowing about history issues. We think that's probably a better use case for a server-side model. What we care about are use cases like synthesis.
We also have to consider constraints like battery consumption. We have to make sure that the model performs well and doesn't consume too much battery. And that the latency is also good. So we partner with the Tensor team to profile our models, as we design them together to make sure that we end up with an architecture that performs well.
Seed: It's not just traditional performance metrics, it's also quality metrics. So if you're looking at things like the quality of the responses the model provides, or even things like the quality of the photo, that's what's going to be more interesting to real-world users than the numbers on the side of a box.