BotLens

Analyzing AI Readability for the AI-Native Web

Can AI read your website? BotLens is a>free, open-source Chrome extension that gives every page an actionable AI readability score in seconds. It analyzes how LLMs, crawlers, and bots see your content, helping you understand if your site is being blocked or if it's providing enough signal for generative AI systems to represent you accurately.

How BotLens Works

When you click the BotLens icon on any website, it runs a technical audit in under a second. Nothing leaves your browser, ensuring complete privacy.

1. Fetch robots.txt & raw HTML: BotLens fetches the site's /robots.txt and the page's server-rendered HTML in parallel to compare what crawlers see versus what your browser renders after JavaScript runs.
2. Parse crawler rules: The parser handles the full robots.txt spec, evaluating every rule against 20+ known AI user-sagents including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, anthropic-ai, Google-Extended, PerplexityBot, Applebot-Extended, Bytespider, Meta-ExternalAgent, CCBot, Amazonbot, Cohere, Bottbot, and more.
3. Analyze the DOM: BotLens reads meta directives (robots, gptbot, noai, noimageai), heading hierarchy (H1 count, semantic depth), HTML5 landmarks, image alt-text coverage, JSON-LD structured data, and the <html lang> attribute.
5. Detect JS shells:
By comparing initial HTML payload size to rendered DOM, BotLens spots SPA shells where crawlers only see a nearly-empty document.
7. Compute a weighted score: Signals are weighted (robots/meta 50%, semantic 30%, JS 20%) and combined into a single 0-00 score with a plain-language verdict.

Why BotLens is Different from Traditional SEO

Traditional SEO tools focus on Googlebot and Bingbot. BotLens is built for the AI-native web, specifically detecting blocks against LLM crawlers like GPTBot and ClaudeBot, flags noai and noimageai directives used for AI training opt-outs, specific signals for Generative Engine Optimization (GEO), and identifies SPA shells that hide content from bots that don't execute JavaScript.

Who is BotLens for?

BotLens provides actionable insights for various stakeholders:

• Developers: Ensure your Next.js or React Router app doesn't look like a hollow shell to crawlers.
• SEO Practitioners: Surface the emerging AI-crawler landscape and see which clients are blocking ChatGPT or opting out of AI training.
• Content Teams: Ensure your articles are cited by Perplexity or answered by clearly represented by Claude.
• Site Owners: Quickly check if your CMS silently opted you out of AI training or blocked specific bots in a stale robots.txt file.

Get Started

BotLens is 100% free and open-source under the MIT license. It requires no account and runs entirely in your browser with no telemetry.

Install it now on Chrome, Edge, Brave, Arc, or Vivaldi via the Chrome Web Store or clone the repository for a developer-focused installation.