Most people who check whether AI recommends their business do it one of two ways. They open ChatGPT and type a question a customer might ask. Or they use a tool that asks ChatGPT for them, through the API.
It is natural to assume those are the same thing. They are both ChatGPT. They are not the same thing.
What we saw
In October we pointed our own product at a well-known outdoor clothing brand and asked seven questions a buyer might ask. Things like which brands make durable rain shells for year-round hiking, and how insulated jackets compare for warmth and weight.
We asked each question two ways. We asked in the ChatGPT app, the way a person would, and through the API, the way most tools do.
The app named the brand in 4 of the 7 answers. The API named it in 2.
Seven questions is a small sample, and I would not stake much on the exact numbers. The bigger gap was in what they cited. The app pointed at the brand’s own website in 5 of the 7 answers. The API pointed at it in none.
Why they differ
The app is not just a model. It is a model plus a search step, plus whatever the app decides to show, plus instructions you never see. The API gives you the model and whatever you bolt on yourself. When a tool asks the API “what are the best rain jackets”, it is asking a different question of a different system, even if the words are identical.
So if a tool tells you that AI recommends you 30% of the time, the first thing to ask is which AI. Is it the one your customers open on their phone, or the one developers call from a server?
Why this matters more than it seems
A year ago the question was whether you ranked on Google. You could check that yourself in ten seconds. You searched, you scrolled, you saw where you were.
AI answers are harder to check, for three reasons.
First, they vary. Ask the same question twice and you can get a different list. One check tells you very little. You need many answers, over days, to see a pattern.
Second, they are private. Nobody else sees the answer your customer saw. There is no results page to screenshot.
Third, as above, there is more than one ChatGPT. Measure the wrong one and you can be confident and wrong at the same time.
What to do about it
If you want to know whether AI recommends you, measure what your customers actually see. That means asking the apps themselves, not just the APIs, and asking the questions buyers ask rather than your brand name.
Then look at what the answers cite. When an AI recommends a competitor, it usually leans on a page, such as a comparison, a review, or a buying guide. Those pages are the lever. You cannot argue with a model, but you can make sure the pages it reads describe you accurately, and you can publish the page it is missing.
And check again next week. The answers move.
We built Behavior to do this. It asks the questions your buyers ask in the ChatGPT and Gemini apps and in Google’s AI answers, and through the APIs of models such as GPT, Claude, and Gemini. Then it shows who gets named and why. But even if you never use it, try the experiment yourself. Ask the app and ask the API the same question about your market. The difference is the point.
Run the free check to see what Google AI Overviews and AI Mode say about your brand. A free account adds ChatGPT.