Crawler identity
You found this in your access log.
LegibilityBot requested one page from your site. It was almost certainly because someone entered your domain into the free checker at legibility.io, which reads a single page and reports whether a machine can extract typed facts from it.
User-Agent: LegibilityBot/0.1 (+https://legibility.io/about/bot)
§01 · behaviour
What it does.
- One request, to your homepage
- A single GET. It does not crawl your site, follow internal links, or come back on a schedule unless someone checks your domain again.
- No headless browser
- It does not execute your JavaScript. That is deliberate: an answer engine's crawler often does not either, so running one would flatter the result and measure the wrong thing.
- No proxy, no residential IP
- The capability to route around a refusal exists and is switched off. A site that refuses a plain request is recorded as having refused, which is the finding.
- Nothing is submitted
- It does not post forms, accept cookies, log in, or touch anything behind an account.
- It reads the markup, not the people
- What gets recorded is whether structured data was present and parseable. No personal data is collected from your pages, and no buyer data exists in this product at all.
§02 · how to block it
If you would rather it did not.
Add this to your robots.txt. It is checked before the page is requested, so a disallow means the page is never fetched at all.
User-agent: LegibilityBot Disallow: /
A site that blocks us is recorded as having chosen to, and that is published as a finding rather than treated as an obstacle to get around. We do not retry from a different address, and we do not interpret a refusal as a technical fault.
To remove a page we have already read, file a takedown. We honour it within 24 hours.
§03 · what it records
One verdict, from a closed list.
A page is readable when typed facts can be extracted from it without guessing, which in practice means JSON-LD or OpenGraph. When it is not readable, the reason is one of these and nothing else. There is no free-text judgement of your site anywhere in this product.
- blocked
- The site refused the request. This is a choice the site made.
- js_shell
- The page arrived, but its content is assembled by JavaScript. A crawler that does not run scripts sees an empty frame.
- no_structured_data
- The page is readable as text but carries no structured data, so a machine has to guess what any of it means.
- not_a_product
- The page was read successfully, but it is not a product page.
- low_confidence
- The page was read, but the extraction did not clear the 0.7 trust gate, so it is not safe to rely on.
- timeout
- The page did not answer in time.
- robots_disallowed
- robots.txt asks machines not to read this path. Recorded as a data point, never circumvented.
- error
- The result was inconclusive. We do not know, and we would rather say so.