Check whether your robots.txt exists, accidentally blocks crawlers, and points search engines at your sitemap.
This tool does a read-only fetch of https://your-domain/robots.txt and reports three things: whether the file exists, whether it blocks major crawlers (a Disallow: / under User-agent: *, Googlebot, or Bingbot), and whether it references a Sitemap:. It never writes to your site.
robots.txt is the first file a crawler reads. A single stray Disallow: / — often left over from a staging site — can quietly de-index your whole domain, which also starves an AdSense site of the traffic it needs. The opposite mistake is having no Sitemap: line, so crawlers have to discover your pages the slow way.
For most content sites you want robots.txt present, not blocking Googlebot or Bingbot from /, and pointing at your sitemap (e.g. Sitemap: https://your-domain/sitemap.xml). Blocking specific admin paths is fine; blocking the whole site is almost never what you intended.
A NicheAudit scan runs the whole structural checklist against your site — thin and duplicate content, leftover AI/template boilerplate, missing required pages, broken links, ads.txt, images, and more — read-only, with the evidence for each. It won't promise approval; it shows you the concrete list.
Run a free scan →