Can AI crawlers read React (create-react-app / Vite SPA) sites?
Client-rendered
AI crawlers see almost nothing by default.
A plain React SPA is close to invisible to AI crawlers: the served HTML is a near-empty div and everything else arrives via JavaScript they do not run.
What is actually happening
A Vite or CRA build ships an index.html containing a single mount point and a script tag. The browser fills it in; GPTBot, ClaudeBot and PerplexityBot do not. They read the shell, find nothing, and index nothing.
13 of the major AI crawlers — including GPTBot, OAI-SearchBot, ChatGPT-User — fetch your HTML and parse it without running a JavaScript engine. Anything your page adds after hydration is not part of what they read.
What to do on React (create-react-app / Vite SPA)
- 01Prerender the routes at build time — vite-plugin-ssr, react-snap or a headless-Chrome prerender step writes real HTML per route.
- 02For a content-heavy site, migrate the public pages to a framework that server-renders and keep React for the app behind the login.
- 03If prerendering is not possible, serve a server-rendered version to crawlers specifically. This is not cloaking as long as the content matches exactly.
- 04Verify with `curl`, never with the browser — view-source is the only view that matches what a crawler gets.
What trips people up
- react-helmet sets the title after hydration, so it never reaches a non-rendering crawler.
- Google can render JavaScript, which is why a site can rank in Google Search and still be absent from AI answers. The two are not the same pipeline.
- Adding structured data via JavaScript has the same problem — inject it server-side or not at all.
Where the files go on React (create-react-app / Vite SPA)
Put llms.txt and robots.txt in the /public folder so they are copied to the build output root.
Stop guessing where your site sits
The baseline above is the platform. The free scan measures your actual site: how many words a crawler reads, whether your pages come back as empty shells, and which crawlers your robots.txt lets in.
Scan my React (create-react-app / Vite SPA) site