Robots.txt Generator

Free Tier A

Generate a robots.txt file with custom crawl rules, disallow paths, and sitemap URL.

User-agent

Default Rule

Allow all (recommended) Disallow all

Disallow Paths (one per line)

Allow Paths (one per line, overrides disallow)

Sitemap URL

Crawl Delay (seconds, optional)