Robots.txt Generator
Generate a robots.txt file with allow/disallow rules and a sitemap link.
How to use Robots.txt Generator
- 1Choose whether to allow all crawlers or restrict specific paths.
- 2Add any disallowed paths, one per line.
- 3Add your sitemap URL, then copy the result.
Why use this tool
- Produces a standards-compliant robots.txt.
- Includes a sitemap directive automatically when provided.
Frequently asked questions
Where do I put the robots.txt file?
It must be placed at the root of your domain, e.g. https://example.com/robots.txt, for search engines to find it.