Every check ยท Search
robots.txt
What it checks
Whether robots.txt says Disallow: / to every user agent.
Why it matters
That tells every search engine to crawl nothing.
How to fix it
- Change Disallow: / to Disallow: with nothing after it, or list only the paths to keep out.
User-agent: *
Disallow:Check a site for this and everything else: snag.page. More on search: Page title Meta description noindex Sitemap Main heading Canonical link Missing pages