Some of the most brilliant minds in tech believe it’s only a matter of time before artificial intelligence reaches a new kind of limitless self-awareness, but AI may need to become more aware of its current limitations first.

In recent days, a chorus of Facebook and Instagram users have noticed a new AI-powered feature is being prominently integrated into the main search window of Meta-owned apps. “Meta AI,” which runs on the company’s supercharged Llama 3 large language model (LLM), is essentially a digital assistant that lets you enter your search queries and interact with a natural language interface.

Need recipes for a dinner party? Outfit suggestions? Information about beta blockers? Simply tap the little magnifying glass on Instagram or Facebook and you will be prompted to “ask Meta AI anything.” Meta announced last week that the feature would be rolling out across its core family of apps: Facebook, Instagram, Messenger, and WhatsApp.

Of course, UX changes of this magnitude are not always universally embraced on social media platforms, so the first question many users are likely to ask when they see this new feature is, How the heck do I disable it? A quick search of online forums, including Reddit and X (formerly Twitter), reveals that users are indeed asking that very question, with many expressing frustration that an AI-powered chatbot has been thrust directly into the nucleus of yet another popular app.

Alas, it looks like it’s here to stay. Reached for comment by Fast Company, a Meta spokesperson confirmed that there is no way to opt out of the feature, although the company emphasizes that regular search results continue to appear along with the AI-generated ones.

“Meta AI aims to be a helpful assistant and is in the search bar to assist with your questions,” the company said in an emailed statement. “You can’t disable it from this experience, but you can search how you normally would engage with a variety of results.”

That’s all well and good, but someone apparently forgot to tell Meta AI. As some Reddit users have pointed out, the chatbot will sometimes claim it can be disabled or deleted when prompted a certain way, and it will sometimes even lay out a set of inaccurate instructions to achieve this.

Fast Company confirmed that the AI sometimes gets confused about how integrated it truly is. When we instructed it to “disable Meta AI and return to the regular search on Instagram,” it first said that wasn’t possible, but when we followed up a second time and said “turn it off,” it responded with steps that it said would return us to the “classic Instagram,” although the steps it suggested did not appear to exist in our version of the app.

Asked about the erroneous instructions, Meta referred Fast Company to the Meta AI help page, which explains that the chatbot’s responses “may not be accurate or appropriate, and should not be used to make important decisions.” A similar disclaimer is shown to users the first time they engage with the AI, Meta says.

The question of how useful this chatbot will ultimately be, or the extent to which users of legacy apps will tolerate being shoehorned into new AI-powered features in service of Big Tech’s ever-growing AI arms race, remains to be seen. As Google recently learned with the regrettable rollout of its Gemini image generator, the headlines generated by these tools aren’t always positive.

But Google has so far kept its generative AI tools separate from its signature search products, aka the products that make Google most of its money. What happens when AI tools are integrated into familiar products to such an extent that they disrupt the user experience before consumers are ready to accept them? What if consumers are never ready to accept them? That LLMs can produce inaccurate answers isn’t exactly a revelation at this point. That they may never produce impressive profits might be.