Skip to content
Web Crawler Specification

About PlainsightBot

PlainsightBot is an automated web accessibility scanner operated by EveryVisitor. It verifies web pages against WCAG 2.2 Level AA accessibility standards on behalf of verified site owners and their digital agencies.

User Agent Identification

Every HTTP request initiated by our scanner identifies itself using our standard user-agent string:

Mozilla/5.0 (compatible; PlainsightBot/1.0; +https://plainsight.io/bot)

During headless browser rendering passes (via Chromium), the user agent additionally includes standard headless browser identification while preserving the +https://plainsight.io/bot token.

Politeness & Crawl Frequency

Max Rate
1 req / sec

Our scanner enforces a polite default cap of 1 request per second per host.

Scope Cap
150–250 pages

Scans only audit representative template samples, never recursive infinite crawls.

Schedule
Weekly

Scans run once per week during customer-configured off-peak hours.

robots.txt Compliance

PlainsightBot strictly respects the standard Robots Exclusion Protocol (RFC 9309). Before crawling, our engine fetches and parses your /robots.txt file. Any path disallowed for PlainsightBot or for all robots (*) is immediately excluded from scanning.

# Example: Disallow PlainsightBot from crawling admin and checkout paths
User-agent: PlainsightBot
Disallow: /checkout/
Disallow: /cart/
Disallow: /account/

Mandatory Domain Ownership Verification

PlainsightBot does not blindly scan arbitrary web properties on the internet. Before any weekly recurring scan or multi-page audit can proceed, the requesting organization must prove ownership through one of three verifiable methods:

  1. DNS TXT Record: Adding a unique verification token to the domain's DNS records.
  2. Root Verification File: Placing a static /plainsight-verify.txt file in the website root directory.
  3. Platform App Install: Installing our verified Shopify App or WordPress Plugin with OAuth handshake.

Zero Form Submissions & Safe Execution

PlainsightBot performs passive, non-destructive DOM analysis. It never submits forms, never adds items to payment gateways, never creates test orders, and never attempts vulnerability penetration testing. It purely extracts public HTML, executes client-side axe-core accessibility audits, and calculates contrast ratios.

Need to Allowlist or Report an Issue?

If your Web Application Firewall (Cloudflare, AWS WAF, Fastly) is blocking PlainsightBot and preventing scheduled scans, or if you believe PlainsightBot is crawling your site in error, please reach out to our network team:

Your session has ended

Sign in again to carry on. Anything you typed on this page is kept.

Signed in another way? Use another method (opens a new tab)