Search
robots.txt is not blocking pages you want found
seo.blocked-by-robots
Why this matters
A page disallowed in robots.txt is a page search engines will not read. That is often deliberate — admin screens, carts, internal search results — but it is worth confirming none of them matter. The serious case is a page that appears in your sitemap and is blocked at the same time: the sitemap says index this, robots.txt says do not look, the block wins, and the contradiction usually means one of the two files is out of date.
Who fixes it
You can, usually
Roughly how long
Minutes
Care needed
Test before and after
How to fix it
Decide which file is right. If the page should be found, remove the Disallow rule; if it should not, take it out of the sitemap.
How we score it
Failing this check takes up to 12 points off your search score. It is a fact about your site rather than a measurement, so it reads the same on every scan until you change something.
Does your site pass this one?
This check runs on every scan, along with the other 106. Free, no account, and you see the evidence for each result.
Check my siteOther search checks
- Canonical tags point somewhere validA canonical tag tells Google which address is the real one for a page. When it points somewhere wrong — another domain, a dead page, or the wrong protocol — Google follows it anyway, and the page you wanted indexed can be dropped in favour of one that does not exist.
- Content has been updated recentlySome content is meant to sit still, and age alone is not a fault. But prices, opening hours and staff pages that have not been touched in over a year are usually wrong by now — and a visitor who acts on the wrong opening hours does not come back to check whether the rest was right.
- Descriptions are a sensible lengthLong descriptions get truncated mid-sentence; very short ones waste the space you are given.
- Each page has exactly one title tagA page can only have one title. When there are several, the browser and Google each pick one — and they do not always pick the same one, so what you see in the tab may not be what appears in search.
- Each page has one main headingSeveral top-level headings makes the structure of the page ambiguous. Worth checking rather than urgent — modern HTML permits it, and Google handles it, but it usually signals that headings are being chosen for their size.
- Every page has a main headingThe h1 is the headline of the page. Without one, both visitors skimming and search engines parsing the page have nothing telling them what it is about.