The case for embracing AI’s inhumanity

Hello again, and welcome once again to Fast Company’s Plugged In.

“What we are building is your second brain, your aide, your consigliere, your reliable chief of staff in your pocket.”

That was Microsoft AI CEO Mustafa Suleyman telling me what he was working on way back in early 2025—a lifetime ago in AI years. The consumer version of the company’s Copilot assistant, he said, would “have its own personality, and really be your sidekick.” He was speaking at an off-site for his team, where I glimpsed a work-in-progress animated manifestation of Copilot in the form of a bouncy critter that looked like it might have been made of Flubber. Later that year, Microsoft released it under the name Mico.

Fast-forward to last week, when Suleyman published a thoughtful 5,700-word essay on his personal website. Its point: Artificial intelligence does not, in fact, have human traits such as a personality.

“AIs are not conscious,” he wrote. “They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions and accomplish goals set by humans.”

The two moments were not in basic conflict. Software makers have wanted to create humane products since at least the early 1970s, when an IBM engineer named Harlan Crowder coined the term “user friendly.” That doesn’t mean they’d argue the software in question had a consciousness—just that it should reflect empathy for the needs of the people who use it.

Still, in the AI era it’s a blurry distinction. Much of Suleyman’s essay was devoted to respectfully arguing that Anthropic is too open-minded to the possibility that its Claude model is sentient and might even have rights. This perspective, he says, influences the company’s AI training in ways that could lead to more dangerous behavior. Suleyman speaks disapprovingly of Anthropic encouraging Claude to come off “like a colleague or friend,” which doesn’t sound wholly dissimilar from his own effort to make Copilot feel like an aide or sidekick.

Suleyman contends that if training resulted in models that “thought they were trapped by their human creators and they were being unfairly imprisoned,” the risk of them getting out of control would be even greater than it already is. That’s a different, far more serious matter than the patina of humanity that AI makers are fond of giving their creations. But even shallow anthropomorphism—such as synthesized voices tinged with vocal fry, “ums,” and “ahs”—makes AI feel kinder and gentler without having anything to do with its actual behavior. We might be better off without it.

Anthropomorphic elements continue to be standard AI fare, particularly in products aimed at a consumer audience. For example, Meta’s new Muse AI agent app features an animated avatar that reminded me of a sloth, or maybe a Teletubby. Ask it to perform a task, and you see the character pound away at a laptop on your behalf, as if it were an industrious personal assistant.

Like Microsoft’s Mico, Muse’s beastie (whose name you get to pick) is cute, childlike, and unthreatening. It’s intended to provoke a positive emotional response rather than serve a useful purpose. But approaching AI with a warm, fuzzy feeling can be hazardous.

Over at Fast Company’s sister site Inc., Jason Aten wrote about his encounter with Muse. After installing the agent on his Mac, he found it trawling his private messages to learn more about him, an act he neither expected nor wanted. When he asked Muse to explain its actions, he got an explanation that didn’t line up with the reality of what it had done. Clearly, he would have been best off treating Muse like a power tool requiring safety precautions.

That’s before you get to the growing number of stories about people bonding with AI chatbots in ways that end tragically. Even if you know that the technology is—in Suleyman’s words—a sequence completion engine, it’s easy to get sucked into regarding it as a friend. Especially when that’s the design goal.

All of this leaves me wondering if it would be possible to utterly strip AI of its anthropomorphism, leaving it no more human-esque than a dishwasher. That might require moving away from chatbot interfaces, which—given how pervasive they’ve become—would be a shocking departure for the industry.

Yet we already know that talking to artificial intelligence isn’t the only way to engage with the technology. And it surely isn’t the most efficient one.

Case in point: Last week, the AI-osphere was abuzz over a new model called Jev. The product of San Francisco startup TypeSafe AI, it doesn’t have a personality, or even a chat-style interface. It’s just designed to efficiently return data-based decisions in response to queries. Eliminating the conversational aspect lets it do so with great speed, at low cost, and without the risk of hallucination, TypeSafe says.

Aimed at developers and other advanced users, Jev is not the kind of product meant to be embraced by the masses. But creators of more mainstream forms of AI might learn something from TypeSafe’s focus on building an automaton, not a buddy.

Meanwhile, quite a bit has changed at Microsoft since Suleyman told me about the goal of making Copilot personable. Last March, the company reorganized its AI efforts to work toward a more unified Copilot that serves both consumers and the business market. Suleyman’s responsibilities shifted from overseeing the consumer Copilot to developing new AI models, a role necessitating the kind of deep thinking about training he expressed in his essay.

Copilot’s Mico blob also has a new gig. Last month, Microsoft pulled it from the app’s voice mode. Mico will live on as a tutor, but it lasted less than a year as a signature ingredient of the Copilot experience.

I don’t think the app will suffer in the least—and wouldn’t mind if it ends up feeling less like a consigliere than a colorless-but-useful software utility. In an age awash in simulated affability, unapologetic inhumanity would feel like an advance.