Agent Access

What Agent Access measures

Agent Access tests whether a live AI assistant can fetch and parse your pages on demand — not whether AI training crawlers can read you (that's Training Access). Live agents are what users interact with when they ask ChatGPT to "go check this site" or use Perplexity for research.

Unlike Training Access, Agent Access doesn't check robots.txt — live agents ignore it. Live AI assistants (ChatGPT browse, Perplexity, Gemini) all use real headless browsers and render JavaScript fully, so the question is never “can they run your code” — it's whether the fetch succeeds at all, returns content quickly enough, and isn't blocked by infrastructure between them and your page.

What happens when Agent Access is low

When live agents can't reliably fetch your site, every real-time AI query about you silently fails. The user asks ChatGPT "what does AcmeCorp offer?" or "go check acmecorp.com" and gets back vague answers from the model's training data — or worse, the wrong answer pulled from a competitor or an outdated source. You're effectively invisible in real-time AI workflows, regardless of how good your training-data presence is.

This hits hardest in agentic discovery (a user delegating research to AI), live-grounded answers in ChatGPT and Perplexity, and any flow where the assistant needs current-state info from your domain. Training-data recovery takes months; Agent Access recovers as soon as the underlying problem is fixed.

What tends to break Agent Access

A few patterns show up over and over in failed agent fetches. None of them are about JavaScript itself — they're about whether the agent gets a usable response back at all.

• cookie Consent overlays that block the page. A “you must accept cookies” modal covering the viewport blocks content for AI agents the same way it does for users with assistive tech — they get the modal, not the page underneath.
• slow_motion_video Slow response times. Live agents work on a tight budget. If your TTFB is multi-second or your page takes 10+ seconds to be useful, the fetch times out and the agent moves on. Often invisible to humans because we wait; agents don't.
• login Redirect mazes and auth walls. Multi-hop locale redirects, login gates on content pages, and infinite paywall loops all derail fetches before they reach anything useful.
• dns Aggressive bot challenges. Cloudflare's “Under Attack” mode, CAPTCHA gates, and strict rate limits can silently serve a challenge page to AI bot traffic. The agent sees a challenge instead of your content.

How to improve your Agent Access score

The fixes here are mostly infrastructure-level, not content-level. Priority order:

• cookie_off Don't gate content behind consent. Show the cookie banner as a non-blocking strip at the bottom of the viewport. Content stays visible to humans and to agents. (The GDPR doesn't require blocking content — it requires informed consent, which a non-blocking banner provides.)
• speed Cut response time. Push TTFB under 1 second via a CDN, edge caching, and avoiding expensive synchronous work on hot paths. Vercel, Cloudflare Pages, and Netlify all give you sub-100ms TTFB out of the box.
• timeline Audit your redirect chains. Aim for 0 redirects on your homepage; max 1 (e.g., http → https). Use curl -L -I https://yoursite.com to walk every hop.
• verified Whitelist known AI bot UAs in your WAF. Cloudflare, Fastly, and AWS WAF all support known-bot allowlists. If yours doesn't have AI agents whitelisted, your challenges fire on traffic that you'd actually want to let through.
• refresh Re-run the free access check after each fix. Agent Access reflects live state immediately — no caches involved.

Agent Access vs Training Access vs Extractability

Three CORE dimensions overlap on rendering / accessibility but test different things. Here's when each matters.

• smart_toy Training Access — checks robots.txt allowance for 10 AI training crawlers. Different bots, different policy layer. Free.
• support_agent Agent Access (this page) — checks live-agent fetchability via JS, cookies, speed, redirects. Free, homepage-only.
• code Extractability — deeper paid dim. Scores schema, semantic HTML, alt text, content structure, OG/meta, and consent barriers across 8–12 sampled pages, not just the homepage.

Frequently asked questions

• Why doesn't Agent Access check robots.txt? expand_more
  Live AI agents (ChatGPT browse, Perplexity, Gemini live grounding) fetch pages on behalf of a user — they're not indexers. Per the robots.txt spec (the Robots Exclusion Protocol), REP applies to indexer crawlers, not user-agents. So Agent Access focuses on whether the fetch succeeds end-to-end — whether the agent receives usable content quickly, isn't blocked by infrastructure, and isn't sent through redirect mazes — rather than reading a policy file the agent ignores anyway.
• What's the relationship between Agent Access and Extractability? expand_more
  They overlap but answer different questions. Agent Access is a free homepage-only check: can a live AI agent reach and use this page right now? Extractability is a paid dim that looks at how reliably AI can pull specific facts out of your pages across the site — you can have a perfectly reachable homepage (high Agent Access) and still get vague, low-confidence citations because facts aren't structured for clean extraction (low Extractability).
• My site is a React/Vue/Angular SPA. Will Agent Access pass? expand_more
  Yes — live AI agents (ChatGPT browse, Perplexity, Gemini) all use real headless browsers that render JavaScript fully. SPAs with client-side hydration are not a problem on their own. Agent Access is about whether the fetch succeeds end-to-end (response time, anti-bot infrastructure, redirect chains, consent overlays). Your framework choice is largely independent.
• Can I have a high Agent Access but low Training Access? expand_more
  Yes — this is common. Example: a fast site with a permissive cookie strip scores high on Agent Access. But if its robots.txt blocks GPTBot or ClaudeBot, Training Access fails. Live agents will still reach you (they ignore robots.txt) but you won't show up in AI training data, which hurts AI Brand Recognition long-term. Both halves matter.