- | 9:00 am
Google’s new AI projects aren’t ready for the masses yet. Good!
Google DeepMind is showing off an AI assistant that sees and a Chrome extension that can browse the web on its own. They’re for testing purposes only—and that makes sense.
Few tech-industry traditions are as time-honored as vaporware: stuff that gets publicly demoed well before it’s ready to ship. In some cases, the companies in question are just slower to finish their work than they’d expected. Other times, they’re strategically drumming up enthusiasm for something new and shiny to distract customers from competitive offerings. Either way, any gratification involved is delayed, assuming the product ever ships at all—which is not a given.
The high-stakes intensity of the current battle of the tech giants for AI supremacy has led to countless launches that remain vaporous for at least a while, a dynamic I wrote about back in May. So it’s no shock that two new Google creations, Project Astra and Project Mariner, aren’t shipping products. For now, Google DeepMind, the company’s AI research arm, is only making them available to a small pool of hand-selected “trusted testers.” In fact, the “Project” in their names indicates that they’re showcases for work in progress rather than actual products.
And yet, dismissing them as mere vaporware feels unfair. Google is being quite clear about its goals for Astra and Mariner—which is to get a better feel for how people might use new forms of AI before springing them on millions or billions of unprepared humans. Particularly given some of the travails the company has had with AI features that were seemingly undertested before release, it’s the responsible thing to do.
Both projects fall into general AI categories also being ardently pursued by other companies. Astra, which Google first demoed at its I/O developer conference in May, is the company’s vision of a next-generation AI assistant—not an inflexible and limited piece of software like Google Assistant, or a text-centric chatbot like the Gemini app, but a helper that listens, speaks, and sees your world. It’s roughly akin to the version of ChatGPT Advanced Voice Mode that OpenAI unveiled in May—though that product’s camera-enabled features are still vaporware as I write this. (Maybe that will change before OpenAI’s current 12-day advent calendar of “Shipmas” announcements is over.)
Project Mariner, meanwhile, is a Chrome extension that can use websites for you, typing and clicking on its own to accomplish tasks you’d otherwise perform yourself. It’s in the same conceptual zip code as Anthropic’s “Computer Use” feature, which debuted as part of its Claude large language model in October and lets that chatbot control apps. Both are steps toward one of the tech industry’s biggest current obsessions: agentic AI that can work more independently on your behalf.
What Google learns from Astra and Mariner could matter as much to the quality of the experiences it builds as to the raw capabilities of its Gemini large language model—yet another sign that the AI rubber has hit the road. “Academic benchmarks are important, but nowadays, when we say something is best in class, what we mean is, do the users find it best in class?” says Google DeepMind CTO Koray Kavukcuoglu. “The model’s capability has to be merged with the way the application works and is useful. That’s a change for all the researchers.”
That basic reality was reflected in the demos I saw during a recent visit to Google. Running on an Android phone and utilizing its camera, Astra recognized images of paintings, such as Edvard Munch’s The Scream, and answered questions such as, “If I like this, what other artists might I like?” It also scanned the spines and covers of books in a scientific library to help pick among them and read, and summarized two pages of information in a travel book. What it had to say seemed roughly comparable in intelligence to what you might coax out of the Gemini chatbot in a text-based conversation, and wasn’t always dazzling when judged purely by the information it conveyed.
For instance, when I pointed the phone at a shelf of books about hearing and asked Astra to recommend a good introduction to the psychology of hearing, it picked one titled . . . Introduction to the Psychology of Hearing. Shown six bottles of wine and asked which one went best with beef Bourguignon, it rhapsodized about a pinot noir—“a superb pairing!” Even I, a guy who knows nothing about wine, could have figured that out on my own.
Still, Astra’s spoken interface and ability to see the world around it made for a far richer experience than typing prompts into a chatbot. (It might get even richer if Astra eventually runs on AR glasses as well as phones, a scenario Google is working on.) At one point, after the app misunderstood the question about beef Bourguignon—it thought it involved coq au vin—it not only apologized, but did so with an embarrassed half laugh. Maybe that falls well short of OpenAI’s quest to turn the movie Her into everyday life, but it’s an example of simulated humanity we never got from Google Assistant or Siri.
Among the goals of Astra’s controlled testing is to give Google DeepMind’s safety team the opportunity to chime in on exactly how much personality the software should exhibit. “We think a lot about anthropomorphism—what is and isn’t appropriate—because we are not trying to build someone to replace the humans in someone’s life,” says Google DeepMind senior director of responsibility Helen King. Along with that, the team is also assessing such obvious issues as the privacy concerns raised by an AI assistant that sees what you see and has a superhuman photographic memory. For now, Google DeepMind has decided that Astra should only remember the most recent 10 minutes of video it’s captured.
Project Mariner is in an even earlier stage of exploration. In one of the demos I saw, it read a salmon teriyaki recipe in a Google Doc and then complied with the request of director of product management Jaclyn Konzelmann to go off to Safeway’s site, find the necessary vegetables, and place them in a shopping cart. It took several minutes to perform this task and painstakingly explained what it was doing in a pane next to the browser window. For now, Mariner can’t see the shopping process through to actually placing an order, which—considering scenarios like AI getting confused and accidentally buying 10,000 onions, or maybe even doing so on purpose—is probably just as well.
The point of Mariner’s cautious approach, Konzelmann told me, is to err on the side of transparency and avoid potential problems: “We just think it’s really important at this stage of where this research prototype is to keep the human front and center and able to control what’s happening.”
Of course, tech enthusiasts might think it’s kind of cool to have AI help with tasks such as veggie shopping even if it doesn’t save any time. Indeed, King told me that Google’s trusted testers skew more toward AI expertise than the general population, so the company can learn only so much from them. “At the moment, they’ve mostly been those who are familiar [with AI] because we’re in such early stages,” she says. “But as we expand, it’s really important for us to have that mix of civil society and academia—the experts in that as well, and the broader public. Because we want our tools to be able to be used by everyone, not just those who already have that AI literacy.”
Everyone I spoke with during my Google visit emphasized that Astra and Mariner will evolve further as the company learns how outsiders use them. “The whole team is configured in such a way that we can do this kind of exploration quite fast, and that’s the journey we’ve been on,” says Kavukcuoglu. The proof of their value will be in the AI features Google eventually ships. But they do seem promising as a way to make some initial headway.