Can AI see your dealership website? We tested 71 dealer sites.
We checked the robots.txt of 71 dealership and dealer-group websites for 10 major AI crawlers, then tested whether a server-side reader could load each homepage at all. Here is what we found.
Key findings
- 27 of 71 sites (38%) hard-blocked our server-side reader even with realistic browser headers. The blocking happens in bot-protection firewalls, not robots.txt.
- Only 0 of 71 sites (0%) fully disallow a major AI crawler in robots.txt. Dealers are not opting out of AI on purpose; their platforms are blocking readers for them.
- 1 of 71 sites (1.4%) soft-blocked: fine for browsers, blocked or challenged for a generic bot user agent.
- Major public retail groups: 0 of 13 (0%) block at least one AI crawler in robots.txt, versus 0 of 58 (0%) of everyday dealer sites.
- 31 of 71 sites (43.7%) were fully open to both request types in the reader test.
All figures come from a single automated survey run on June 12, 2026. Robots.txt is a voluntary standard: a Disallow rule blocks compliant crawlers, but it is not a technical wall, and assistants doing user-driven browsing may still fetch a page. Full methodology below.
Do dealership websites block AI crawlers?
Many do: 0 of the 71 dealer and dealer-group sites we surveyed (0%) fully disallow at least one major AI crawler in robots.txt. That means a compliant bot like GPTBot or ClaudeBot that requests those sites is told not to read any page.
Major public retail groups
0 of 13 sites from national retailers (AutoNation, Lithia, Penske, CarMax, Carvana, and peers) block at least one AI crawler in robots.txt.
Everyday dealer sites
0 of 58 single-store and private regional group sites block at least one AI crawler in robots.txt. We aggregate these sites rather than naming them.
To be precise about what this measures: robots.txt is an instruction file that reputable crawlers choose to obey. A full Disallow for GPTBot keeps OpenAI's compliant crawler out, but it does not physically prevent access, and it does not affect tools that ignore the standard. The risk is one-sided: the bots most likely to respect the rule are exactly the ones that power mainstream AI answers.
Which AI bots do dealer sites block most?
GPTBot tops the list: 0 of 71 sites (0%) fully disallow it. The table below shows every AI crawler we checked, sorted by how often it is fully blocked in robots.txt.
| AI crawler | Fully blocked |
|---|---|
| GPTBot | 0 (0%) |
| OAI-SearchBot | 0 (0%) |
| ChatGPT-User | 0 (0%) |
| ClaudeBot | 0 (0%) |
| anthropic-ai | 0 (0%) |
| PerplexityBot | 0 (0%) |
| Google-Extended | 0 (0%) |
| CCBot | 0 (0%) |
| meta-externalagent | 0 (0%) |
| Applebot-Extended | 0 (0%) |
"Fully blocked" means a robots.txt group explicitly names the bot with Disallow: /. "Partial rules" means the bot is named with narrower Disallow paths. "No rule" means the bot is never named, so only the site's default (*) rules apply.
Do the big dealer groups block AI?
In this survey, none of the 13 national retail sites fully disallowed the AI crawlers we checked. These are factual readings of public robots.txt files; any of them can change at any time.
| Site | AI crawlers fully blocked (robots.txt) |
|---|---|
| AutoNation autonation.com | None of the bots checked |
| Lithia Motors (corporate) lithia.com | None of the bots checked |
| Driveway (Lithia retail) driveway.com | None of the bots checked |
| Group 1 Automotive group1auto.com | None of the bots checked |
| Penske Cars (retail) penskecars.com | None of the bots checked |
| Penske Automotive (corporate) penskeautomotive.com | None of the bots checked |
| Sonic Automotive (corporate) sonicautomotive.com | None of the bots checked |
| EchoPark (Sonic retail) echopark.com | None of the bots checked |
| Asbury Automotive asburyauto.com | None of the bots checked |
| Hendrick Automotive hendrickcars.com | None of the bots checked |
| Ken Garff kengarff.com | None of the bots checked |
| CarMax carmax.com | None of the bots checked |
| Carvana carvana.com | None of the bots checked |
Why does AI blocking cost dealers leads?
When an AI assistant cannot read your website, it cannot cite your inventory, your hours, or your offers in its answers. Shoppers increasingly ask ChatGPT, Perplexity, and Google's AI features questions like "best Honda dealer near me with a lease special." If your site is unreadable to those systems, the answer can only be built from competitors, third-party listings, and stale data.
Uncitable inventory
A blocked crawler never sees your VDPs, so your actual cars and prices cannot show up in AI shopping answers.
Third parties answer instead
AI tools fall back on aggregators and review sites that are readable, and those sources control how your store is described.
A growing blind spot
Nobody can say exactly how many sales AI answers influence today, but the share is growing, and blocking removes you from that channel by default.
We state this as risk, not certainty: blocking a crawler in robots.txt does not guarantee you vanish from every AI answer, since assistants can use other sources and user-driven browsing. But it stacks the odds against you in a channel where being the citable source is the whole game.
How do I check my own site?
Start with your robots.txt: open yourdealership.com/robots.txt in a browser and look for User-agent lines naming GPTBot, ClaudeBot, PerplexityBot, CCBot, or Google-Extended followed by Disallow: /. If you see that pair, compliant AI crawlers are being turned away from your entire site.
Instant demo (free)
Our instant demo reads your live website the same way an AI reader would and shows you what comes back.
Run the Instant DemoFull AI compatibility analysis
Our companion study scored 100 dealer websites on AI access and structured data. See the methodology and where the industry stands.
Read the AI Compatibility StudyHow do I fix it?
Allow reputable AI crawlers in robots.txt, make sure your CDN or firewall is not challenging every non-browser request, and give AI something worth reading: complete structured data for your business, hours, and inventory.
1. Allowlist reputable AI crawlers
Remove blanket Disallow rules for GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, and Google-Extended unless you have a specific reason to keep them. Your website provider controls this file; ask them what is in it and why.
2. Check your WAF and CDN settings
In our survey, 27 of 71 sites (38%) blocked a server-side reader regardless of robots.txt, usually via challenge pages. Verified AI crawlers can typically be allowed in Cloudflare, Akamai, and similar tools without opening the door to scrapers.
3. Ship structured data AI can use
Access is only half the problem. LocalBusiness, AutoDealer, and inventory schema tell AI systems what you sell, where you are, and when you are open. Savvy Dealer websites ship AI-readable by default: open to reputable crawlers, with structured data built in.
The full dataset
All 71 sites in the sample. Major public retail groups are named because their robots.txt files are public corporate policy. Everyday dealer sites are anonymized; we publish their results in aggregate only.
| Site | AI bots fully blocked | Reader test |
|---|---|---|
| AutoNation | 0 of 10 | Hard block |
| Lithia Motors (corporate) | 0 of 10 | Unreachable |
| Driveway (Lithia retail) | 0 of 10 | Open |
| Group 1 Automotive | 0 of 10 | Hard block |
| Penske Cars (retail) | 0 of 10 | Hard block |
| Penske Automotive (corporate) | 0 of 10 | Hard block |
| Sonic Automotive (corporate) | 0 of 10 | Open |
| EchoPark (Sonic retail) | 0 of 10 | Soft block |
| Asbury Automotive | 0 of 10 | Hard block |
| Hendrick Automotive | 0 of 10 | Open |
| Ken Garff | 0 of 10 | Hard block |
| CarMax | 0 of 10 | Open |
| Carvana | 0 of 10 | Hard block |
| Dealer site 01 | 0 of 10 | Open |
| Dealer site 02 | 0 of 10 | Open |
| Dealer site 03 | 0 of 10 | Open |
| Dealer site 04 | 0 of 10 | Open |
| Dealer site 05 | 0 of 10 | Open |
| Dealer site 06 | 0 of 10 | Open |
| Dealer site 07 | 0 of 10 | Open |
| Dealer site 08 | 0 of 10 | Open |
| Dealer site 09 | 0 of 10 | Hard block |
| Dealer site 10 | 0 of 10 | Unreachable |
| Dealer site 11 | 0 of 10 | Open |
| Dealer site 12 | 0 of 10 | Unreachable |
| Dealer site 13 | 0 of 10 | Open |
| Dealer site 14 | 0 of 10 | Open |
| Dealer site 15 | 0 of 10 | Open |
| Dealer site 16 | 0 of 10 | Open |
| Dealer site 17 | 0 of 10 | Open |
| Dealer site 18 | 0 of 10 | Hard block |
| Dealer site 19 | 0 of 10 | Hard block |
| Dealer site 20 | 0 of 10 | Unreachable |
| Dealer site 21 | 0 of 10 | Open |
| Dealer site 22 | 0 of 10 | Open |
| Dealer site 23 | 0 of 10 | Unreachable |
| Dealer site 24 | 0 of 10 | Hard block |
| Dealer site 25 | 0 of 10 | Hard block |
| Dealer site 26 | 0 of 10 | Open |
| Dealer site 27 | 0 of 10 | Open |
| Dealer site 28 | 0 of 10 | Hard block |
| Dealer site 29 | 0 of 10 | Open |
| Dealer site 30 | 0 of 10 | Open |
| Dealer site 31 | 0 of 10 | Open |
| Dealer site 32 | 0 of 10 | Hard block |
| Dealer site 33 | 0 of 10 | Hard block |
| Dealer site 34 | 0 of 10 | Hard block |
| Dealer site 35 | 0 of 10 | Hard block |
| Dealer site 36 | 0 of 10 | Unreachable |
| Dealer site 37 | 0 of 10 | Unreachable |
| Dealer site 38 | 0 of 10 | Unreachable |
| Dealer site 39 | 0 of 10 | Open |
| Dealer site 40 | 0 of 10 | Open |
| Dealer site 41 | 0 of 10 | Hard block |
| Dealer site 42 | 0 of 10 | Hard block |
| Dealer site 43 | 0 of 10 | Hard block |
| Dealer site 44 | 0 of 10 | Open |
| Dealer site 45 | 0 of 10 | Unreachable |
| Dealer site 46 | 0 of 10 | Hard block |
| Dealer site 47 | 0 of 10 | Unreachable |
| Dealer site 48 | 0 of 10 | Hard block |
| Dealer site 49 | 0 of 10 | Hard block |
| Dealer site 50 | 0 of 10 | Hard block |
| Dealer site 51 | 0 of 10 | Unreachable |
| Dealer site 52 | 0 of 10 | Open |
| Dealer site 53 | 0 of 10 | Hard block |
| Dealer site 54 | 0 of 10 | Open |
| Dealer site 55 | 0 of 10 | Open |
| Dealer site 56 | 0 of 10 | Hard block |
| Dealer site 57 | 0 of 10 | Hard block |
| Dealer site 58 | 0 of 10 | Unreachable |
Methodology
On June 12, 2026, Savvy Dealer ran an automated survey against 74 candidate dealership and dealer-group websites; 71 resolved and form the sample. The sample mixes 13 consumer-facing sites from major public or national retail groups with 58 everyday franchise dealer sites (single stores and private regional groups) across regions and website platforms.
Test 1: robots.txt policy
We fetched https://site/robots.txt (8 second timeout) and parsed user-agent groups for 10 AI crawler tokens: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, anthropic-ai, PerplexityBot, Google-Extended, CCBot, meta-externalagent, Applebot-Extended. A bot is counted as "fully blocked" only when a group explicitly names it with Disallow: /. Narrower rules count as partial. Unreadable or missing robots.txt counts as no restriction.
Test 2: reader access
We requested each homepage twice over HTTPS (10 second timeout, one request per second, no retries): once with a realistic Chrome desktop header set, once with a generic bot user agent ("Mozilla/5.0 AppleWebKit/537.36 (compatible; SavvyReader/1.0)"). We recorded status codes and known challenge-page signatures (Cloudflare, Akamai, DataDome, Imperva, PerimeterX, and generic denial pages) on 200 responses. Open means both succeeded; soft block means only the bot UA was blocked; hard block means both were blocked or challenged.
Limitations
- Robots.txt is voluntary compliance. It blocks well-behaved crawlers; it does not technically prevent access, and assistants doing user-driven browsing may still fetch pages.
- Our server-side reader is a proxy for AI crawler access, not identical to any specific bot. Real AI crawlers come from known IP ranges and may be treated differently (better or worse) by bot-management tools.
- This is a single snapshot of 71 sites on one date. Robots.txt files and firewall rules change; any site listed here may behave differently today.
- The sample is hand-built to span major groups, regions, and platforms, but it is not a random sample of all U.S. dealerships, so percentages describe this sample, not the whole industry.
Questions about the data, or believe your site is misclassified? Email support@savvydealer.com and we will review it promptly.
Find out what AI sees on your site
Run the free instant demo to see your website the way an AI reader does, or talk to us about a site that ships AI-readable from day one.