robots.txt
Generate a robots.txt with allow, disallow, and sitemap.
Processed in this browser. Nothing is sent to a server.
Result
How do I use robots.txt?
Provide the three fields. Empty allows the root.
What are the limits of robots.txt?
It does not publish the file.
Related tools
- HTTP status explainerExplain a pasted code. It does not request the URL.
- Meta tagsGenerate title, description, and canonical.
- Meta analyzerRead title, description, canonical, and robots from pasted HTML.
- Open GraphGenerate the og meta properties.
- Twitter cardGenerate a summary_large_image card.
- XML sitemapGenerate a urlset from one URL per line.