Why is my site not showing in AI search? Two failures

This is usually not a ranking problem. Two unrelated failures produce the same symptom, and the repair for one does nothing for the other. Budgets get committed before anyone establishes which failure is in play.

Which of the two failures is yours?

• Failure one is supply. A crawler requested your page and received something it could not read, so none of your text entered the index that the answer is assembled from
• Failure two is demand. The query you tested does not produce an AI answer for anybody, so there is no answer to be missing from

The order matters. Diagnosing supply when the real answer is demand is how a budget disappears into a problem that was never there.

What does a crawler actually get when it fetches your page?

We fetched the homepage of 30 Tamil Nadu engineering colleges as raw HTML with JavaScript disabled. Five came back without usable content. What counts as usable is our judgment, not a published standard.

"Two homepages in thirty returned nothing an AI crawler could quote, and nobody viewing them in a browser would ever see it."

Google's JavaScript SEO documentation says why this still matters: server side rendering "is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript".

Want this run on your own site? Here is what a free AI visibility audit covers.

Do the AI crawlers run JavaScript?

We read the current crawler documentation from three publishers on 10 August 2026 and recorded one thing: does the publisher state whether its crawler executes JavaScript?

• Google states that its renderer does, using an evergreen version of Chromium, and its generative AI guide adds that Google can process content within JavaScript as long as it is not blocked
• OpenAI documents four agents, their user agent strings and their published IP ranges, and says nothing about JavaScript execution
• Perplexity documents two agents, their IP ranges and firewall guidance, and also says nothing about JavaScript execution

Read OpenAI's crawler page yourself. You will find user agent strings and published IP ranges, which is what a network team needs, and no statement either way about rendering.

An absent statement is not proof that a crawler cannot render. It means there is nothing to rely on. If your text only exists after a script runs, your visibility rests on undocumented behaviour.

Are you blocked without knowing it?

Blocking GPTBot and blocking OAI-SearchBot are different decisions. OpenAI documents GPTBot as the agent for foundation model training and OAI-SearchBot as the agent for search, and states that sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers, though can still appear as navigational links".

Your firewall is the second place to look, and nobody in marketing has the login. Perplexity's own crawler documentation tells site owners they may need to explicitly whitelist its bots so they can reach the content.

The third is newer. Google's generative AI optimisation guide now states that "in addition to the technical requirements for Search, a site must be included in Search generative AI features in Search Console to be eligible for display in generative AI features on Google Search".

"Three separate switches, owned by three separate people, and not one of them reports a failure to anybody."

Is there an AI answer to be absent from?

Before repairing anything, confirm the answer exists. In Ahrefs SERP data for 160 Indian marketing queries, AI Overviews appeared on 74 of 80 informational searches. Of the 80 commercial searches in the same pull, 30 carried a city name, and none of those 30 returned an AI Overview.

"If your buyers search with a city name attached, absence from an AI Overview is the normal state of that result, not a defect in your site."

Record two things for every query you test:

• whether an AI answer appears at all, in the exact words a buyer would type
• whether it still appears once you add your city name to the same query

That segmentation, its method and its limits are set out in our analysis of which query types AI Overviews actually take.

What we would check, in order

• 1

  Fetch your own homepage and two money pages as raw HTML with JavaScript disabled, and read what returns

• 2

  Open robots.txt and confirm OAI-SearchBot and PerplexityBot are not disallowed alongside GPTBot

• 3

  Ask whoever owns the firewall whether AI crawler user agents are being challenged or blocked

• 4

  Confirm the site is included in Search generative AI features in Search Console

• 5

  Only then test the queries, in the exact words a buyer would use, and record whether an AI answer appears at all

Steps one to four cost an afternoon and no money. Step five is the one that decides whether the other four were worth doing, which is also why we put technical checks ahead of content work on a first engagement.

What this cannot tell you

• The fetch covers 30 engineering college homepages in one Indian state. It is evidence about how that sector builds websites and nothing more. A retailer on a hosted platform will look different
• The SERP figures are Ahrefs' record of what its crawler saw, not our own searches, and they cover marketing services queries in India only
• We have not measured the thing that would settle the argument, which is whether repairing server side rendering changes how often a site gets cited. That needs a before and after on the same set of sites, and we have not run it

The full metadata breakdown from the same fetch sits in our teardown of Tamil Nadu college homepages. It records what was missing across all 30, in aggregate, with the named results held back.