SEO tools / Links, crawling and indexing
Robots.txt checker and tester
Read a robots.txt and test whether a URL is allowed for a bot.
More links, crawling and indexing tools
URL redirect checkerFollow every redirect hop of a URL with its status code.
404 page checkerDoes the site answer missing pages with a real 404?
Broken link checkerCheck every link on a page and find the broken ones.
Sitemap checkerFind and validate a site's XML sitemap.
Internal links checkerList every internal link on a page.
External links checkerList every outgoing link on a page.
Unsafe external links checkerFind target="_blank" links without rel="noopener".
Indexability checkerCan search engines index this URL? Checks status, robots.txt, noindex and canoni...
How to use the robots.txt checker and tester
Read a robots.txt and test whether a URL is allowed for a bot. Enter the page address above and press the button. Results appear on this page and you can share the link.
Visitors can run 30 checks a day. Create a free account for projects, site audits, rank tracking and saved keyword lists.