robots.txt और साइटमैप चेकर

See what search engines are allowed to crawl, and whether your sitemap is present and readable.

और गहराई सेआप आगे क्या पता लगाना चाहते हैं?
🤖 See exactly what crawlers can access🗺️ Check DNS before blaming indexing🔎 Inspect page metadata

शब्दों की व्याख्या

robots.txtSearch EngineसाइटमैपWeb CrawlerGoogle Search ConsoleIndexing

अक्सर पूछे जाने वाले प्रश्न

What does "Disallow: /" do?

It tells all crawlers not to crawl the entire site. It is a common cause of pages disappearing from Google after a staging site goes live.

Where should my sitemap be?

At /sitemap.xml, and referenced in robots.txt with a Sitemap: line.